Scalable Perturbation Learning for Online Self-Supervised Learning in Echo State Networks

Kantaro Fujiwara; Taiki Yamada

Scalable Perturbation Learning for Online Self-Supervised Learning in Echo State Networks

T0 review · 0 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For echo state networks, a self-supervised loss splits into an input-sized learnable part and a fixed offline part, so perturbation learning can scale with input dimension rather than reservoir dimension.

desk verdict A clean, well-derived variance-reduction trick for perturbation learning in ESNs; the math holds up and the scaling claim is real, though the experiments are narrower than the abstract suggests. read the letter →

arxiv 2607.06079 v2 pith:VCEOH7GQ submitted 2026-07-07 cs.LG cs.NE

Taiki Yamada , Kantaro Fujiwara This is my paper · ORCID

classification cs.LGcs.NE

keywords echostatenetworksself-supervisedlearningperturbationonlinereservoircomputingvariancescalingorthogonaldecompositionscalarfeedback

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper shows that online self-supervised learning in echo state networks (ESNs)—where the reservoir learns to reconstruct its own next state, and hence the input, without labels—does not actually need to live in the full reservoir space. Its loss separates into a low-dimensional component driven by the input and a redundant component fixed by the reservoir's static wiring. Only the input-driven component has to be learned online; the residual can be removed by a fixed transformation S. Perturbing only the n_in-dimensional component keeps the scalar-feedback advantage of perturbation learning while cutting perturbation-induced variance from O(n_r²) (weight perturbation) or O(n_r) (node perturbation) to O(n_in). If correct, this resolves a basic tension between self-supervised adaptation, online updates, and memory-efficient scalar feedback in large reservoirs.

What carries the argument

The load-bearing object is the orthogonal decomposition Eq. (2.33) of the self-supervised cost into an online term ℓ~_sSL and an offline residual. The online term is evaluated through the projected variable ỹ_t=(W_in)^+W_dyn r_t, which lives in R^{n_in} (the input dimension) rather than R^{n_r}, so the perturbation source for the update rule (2.39) is an n_in-dimensional random vector ξ̃ instead of an n_r×n_r matrix (weight perturbation) or an n_r-dimensional vector (node perturbation). The companion static map S, defined by projecting W_dyn onto Im(W_in) for the W_dyn-dependent part and replacing the orthogonal part with W_rec, eliminates the residual term without using reservoir data; beca

What would settle it

Fix n_in=2, choose a large reservoir (say n_r=1000) with tanh activation, run the proposed rule (Eq. 2.39) for 10^5 steps, apply S, and compare (i) the measured SNR of the update with the theoretical O(n_in) prediction across n_r, and (ii) the final S(W_dyn) against the batch optimum W_dyn^*=⟨σ^{-1}(r_{t+1}) r_t^T⟩⟨r_t r_t^T⟩^{-1}. If the SNR decays as n_r grows, or the final parameter systematically deviates from W_dyn^*, the scaling claim fails.

Watch

Extended reading notes

Core claim

The central claim is Eq. (2.33): the self-supervised loss ½‖W_dyn r_t − σ^{-1}(r_{t+1})‖² decomposes orthogonally into an online term ℓ~_sSL that depends on W_dyn only through the n_in-dimensional variable ỹ_t=(W_in)^+W_dyn r_t, plus a residual ½‖Π⊥(W_dyn)‖² governed by Π⊥(W_dyn)=(I−W_in(W_in)^+)(W_dyn−W_rec). Only the online term needs data; the residual is removed by the static map S(W_dyn)=W_in(W_in)^+W_dyn+(I−W_in(W_in)^+)W_rec, which commutes with online updates. Perturbing ỹ_t with an n_in-dimensional noise vector therefore yields a scalar-feedback update whose variance scales as O(n_in), and applying S reproduces the full self-supervised optimum.

Load-bearing premise

The argument depends on σ being invertible (so σ^{-1}(r_{t+1}) is a valid target), on W_in having full column rank, and on the reservoir-state covariance being invertible; if any of these fails, the equality between reduced online learning plus static S and the original self-supervised optimum is not established.

Editorial extensions

If this is right

Perturbation-induced update variance becomes O(n_in) instead of O(n_r²) (weight perturbation) or O(n_r) (node perturbation), so large reservoirs no longer force a vanishing learning rate or heavy averaging for scalar-feedback rules.
The learned W_dyn reaches the same optimum as full self-supervised SGD once the static map S is applied, even though the online trajectory only optimizes the reduced loss.
The online error path needs only a single global scalar loss difference plus n_in-dimensional injection, so error-feedback memory and routing costs stop growing with reservoir size.
A hardware variant replaces the pseudoinverse injection by W_in, delivering the perturbation through the existing input pathway; convergence and stationary point are preserved up to a positive-definite preconditioner.
When n_in approaches n_r, the variance advantage over node perturbation fades and vanishes at n_in=n_r under orthonormal W_in.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The strategy is probably not limited to ESNs: any self-supervised loss induced by a fixed, full-rank, low-dimensional embedding of the target admits the same split—an online part in the embedding's image and an offline residual that a static projection can remove. The paper argues this as a design principle but only demonstrates it for input reconstruction.
Because the perturbation is injected through W_in, the rule is naturally a three-factor local rule (presynaptic activity × postsynaptic perturbation × global scalar); this points to a direct physical implementation in analog or neuromorphic hardware, beyond what the paper's numerical simulations show.
The invertibility of σ is the real boundary: leaky ReLU, ELU, softplus, and tanh fit the theory, while ReLU does not, so the practical claim is about strictly monotone reservoirs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this if you care about local learning rules for reservoir computing. The core claim—that the self-supervised ESN loss splits into an n_in-dimensional online part and a redundant static part, and that perturbing only the online part cuts perturbation variance from O(n_r^2) or O(n_r) to O(n_in) while a static map S recovers the full optimum—is correct. I checked the decomposition in Eq. 2.33 and the surrounding algebra in Appendix A; the derivation is clean and the assumptions (full-column-rank W_in, invertible sigma, invertible reservoir-state covariance) are stated up front. The proposed update is a sensible extension of node perturbation to the reduced loss, and the static map S is a neat trick that makes the orthogonal residual disappear without touching the online dynamics.

What's new: the orthogonal decomposition, the static map S, and the O(n_in) perturbation rule. The self-supervised objective itself is from the authors' own prior work, but that is disclosed and not a problem. The variance scaling argument is careful: they claim O(n_in) for the dimension-dependent factor, which is the right kind of claim, and the SNR experiment in Figure 3A shows the practical benefit. I also appreciate the explicit discussion of the physical-implementation variant in Section 4.3, which replaces the pseudoinverse injection with W_in and notes the preconditioning effect—honest about the trade-off.

Soft spots, in proportion. The numerical study is narrow: one task (input reconstruction), one reservoir type, and no code shipped despite the reproducibility claim in Appendix C. The claim of 'exactly reproducible' from seeds is not verifiable without the code, and that is a real but minor gap. The assumptions, while standard, do exclude ReLU and require invertible sigma; that is a genuine limitation for some hardware and they acknowledge it. The SNR comparison uses a method-specific reference gradient—disclosed in Appendix C—and is not the central evidence, since the learning-curve comparisons use a common projected loss.

Who this is for: researchers working on reservoir computing, neuromorphic hardware, and local/perturbation-based learning. They will get a concrete, low-variance update rule with a clear geometric picture. I would send this to a serious referee—it deserves careful review rather than desk rejection—and I would cite it in work on perturbation learning, though the lack of code makes me hesitate to build directly on the numerics.

Referee Report

0 major / 4 minor

Summary. The paper studies online self-supervised learning of an n_r × n_r dynamical readout W_dyn in an echo state network, where the target is σ^{-1}(r_{t+1}) and the loss is quadratic (Eq. 2.29). Its central contribution is an orthogonal decomposition of this loss (Eq. 2.33): the full loss separates into a reduced loss that depends on W_dyn only through an n_in-dimensional projected variable ˜y_t = (W_in)^+ W_dyn r_t, plus a residual term that depends on W_dyn only through the orthogonal complement of Im(W_in). The paper then proposes a scalar-feedback perturbation rule (Eq. 2.39) that perturbs only the n_in-dimensional component, reducing the perturbation-dimension factor in the variance from O(n_r^2) (weight perturbation) or O(n_r) (node perturbation) to O(n_in). A static map S (Eq. 2.42) zeroes the orthogonal residual without affecting the reduced loss, so the online update restricted to Im(W_in) plus S recovers the full self-supervised optimum. Numerical experiments verify the predicted SNR scaling, learning-rate stability, and learning curves.

Significance. If the claims hold, this is a genuinely useful contribution: it identifies an exploitable low-dimensional structure in a recurrent self-supervised objective and turns it into a concrete reduction in perturbation dimension, which is directly relevant to hardware and neuromorphic implementations. The strengths are that the orthogonal decomposition is derived cleanly and self-contained in Appendix A, the variance formulas (Eqs. 2.11 and 2.18) are standard and exact for quadratic losses, and the assumptions — full-column-rank W_in, invertible σ, invertible reservoir covariance — are stated explicitly in Sections 2.2 and 4.4. The numerical study is carefully designed and includes comparisons with SGD, weight perturbation, and node perturbation. The main caveats are local: the SNR definition in Eq. (C.3) uses method-specific reference gradients, and the claimed exact reproducibility in Appendix C is not backed by released code or listed seed values. Neither issue undermines the central algebraic derivation.

minor comments (4)

[Appendix C, first paragraph] The text states that random numbers were drawn from NumPy 'with explicitly specified seeds, so that every experiment is exactly reproducible.' However, the manuscript does not list any seed values or provide code. The reproducibility claim is therefore overstated; please provide the seeds/code or rephrase the claim.
[Section 3, Eq. (C.3)] The SNR in Eq. (C.3) is defined relative to a method-specific reference gradient: for Ours the reference is the projected gradient P_in e_t r_t^T, while for WP and NP it is the full gradient e_t r_t^T. This is a reasonable choice for isolating variance, but the figure title 'Gradient estimate accuracy' and the text 'lower SNRs than for SGD' should be qualified so that readers do not interpret the SNR values as directly comparable across methods in an absolute sense.
[Abstract and §2.3] The wording 'offline component determined by the fixed ESN parameters' is imprecise. The residual term in Eq. (2.33), ½‖Π⊥(W_dyn)‖²_{I, r_t r_t^T}, depends on the reservoir state r_t and on the current W_dyn; what is 'offline' is the fact that it can be zeroed by the fixed static map S without using dynamic data. Please rephrase to avoid suggesting that the residual term itself is independent of the data.
[Appendix B, Eqs. (B.10)–(B.11)] The notation Cov(vec(Ξ))^{+} and Cov(ζ)^{+} is ambiguous. Please state explicitly that + denotes the Moore–Penrose inverse, or use a less overloaded notation. This is a presentation issue that does not affect the main argument.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the core decomposition is proved in-paper and empirically verified, not fitted.

full rationale

The derivation chain is self-contained. Eq. (2.33) is proved in Appendix A directly from the Pythagorean decomposition with P_in = W_in(W_in)+ and the ESN identity sigma^{-1}(r_{t+1}) = W_in s_t + W_rec r_t. The reduced loss l_tilde depends on W_dyn only through y_tilde = (W_in)^+ W_dyn r_t by Eqs. (2.34)-(2.36), so the proposed update (2.39) uses exactly n_in perturbation coordinates and is an unbiased estimator of the reduced-loss SGD gradient for the quadratic loss (central difference is exact), giving the O(n_in) dimension factor. The static map S in Eq. (2.42) zeroes Pi_perp(W_dyn) by construction while leaving l_tilde unchanged (Eqs. (2.43)-(2.44)), so applying it to the online solution recovers the full self-supervised optimum; this is a proven algebraic property, not a fitted equivalence. The numerical experiments measure SNR from the defined rules and use empirically selected learning rates for each algorithm; no constant is fitted to force the scaling. The self-citation [31] supplies the self-supervised objective, but Eq. (2.25) rederives the P relation inside the paper, so the citation is provenance rather than the load-bearing inference. The explicitly stated assumptions (invertible sigma, full-column-rank W_in, covariance invertibility) in Sec. 2.2 and 4.4 are genuine restrictions, not disguised forms of the conclusion. No circular step was found.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central variance-scaling result is self-contained linear algebra; no constants are fitted to data. The method assumes the ESN setup from the authors' prior work [31]: full-column-rank W_in, invertible σ, and invertible reservoir-state covariance, all stated and discussed in §4.4. The experimental hyperparameters (learning rates, α, ρ) are standard choices and do not enter the derivation.

free parameters (3)

Per-algorithm learning rate η = SGD: 3.56e-3; WP: 1.61e-5; NP: 1.22e-4; Ours: 1.98e-1
Chosen by minimizing the median normalized projected loss over 50 realizations in Fig 3C; used for Fig 3D. Not part of the variance-scaling proof.
Perturbation scale α = 1
Hand-set for all perturbation rules; central-difference variance formulas are α-independent for the quadratic losses used here.
Reservoir spectral radius ρ = 0.9
Standard ESN hyperparameter in all simulations; not used in the theoretical derivation.

assumptions (5)

domain assumption W_in has full column rank (n_in ≤ n_r), so (W_in)^+ W_in = I_nin.
Stated in §2.2 and §4.4; needed for Q, the reduced-loss metric, and the injection identity; nearly free for random W_in when n_r > n_in.
domain assumption Activation σ is invertible on its range.
Assumed in §2.2; required to define target σ^{-1}(r_{t+1}) and to keep the problem linear; excludes ReLU, includes tanh/ELU/softplus.
domain assumption Reservoir-state covariance ⟨r_t r_t^T⟩_{r_t} is invertible.
Invoked for uniqueness of optimal solutions in Eqs. 2.20 and 2.23 and for Eq. 2.25; standard persistence-of-excitation condition.
standard math Perturbation variables have zero mean, unit variance, and bounded fourth moment.
Assumed in Eqs. 2.8, 2.15, and 2.40 to derive variance formulas; satisfied by Gaussian perturbations used in experiments.
domain assumption The self-supervised loss is quadratic in W_dyn and in y_t = W_dyn r_t.
Follows from Eqs. 2.29 and 2.33; this makes central-difference perturbation updates equal exact directional derivatives for any α.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Scalable Perturbation Learning for Online Self-Supervised Learning in Echo State Networks." pith.science (2026). https://pith.science/paper/VCEOH7GQ

@misc{pith2026260706079,
  author       = {Pith},
  title        = {Pith review of: Scalable Perturbation Learning for Online Self-Supervised Learning in Echo State Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VCEOH7GQ}},
  note         = {Machine review of arXiv:2607.06079}
}

read the original abstract

Intelligent systems should not only solve tasks but also adapt under real-world constraints. Autonomous adaptation via self-supervised learning, sequential adaptation via online learning, and memory-efficient implementation via perturbation-based learning are important requirements for such systems. However, these requirements are generally in tension for high-dimensional systems, because perturbation-based learning suffers from variance that grows with the dimension of the perturbed variables. In this study, we focus on echo state networks (ESNs), where this tension naturally arises in large reservoirs. We propose a perturbation-based learning rule for online self-supervised learning in ESNs. The proposed rule is derived from an orthogonal decomposition of the self-supervised learning cost, which separates an input-dependent component from a redundant component determined by the fixed ESN parameters. By perturbing only the input-dependent component, the effective perturbation dimension is reduced from the reservoir dimension to the input dimension. Thus, the proposed method preserves self-supervised adaptation, online learning, and scalar-feedback perturbation learning, while avoiding reservoir-size-dependent variance growth. This suggests a design principle for scalable and hardware-compatible learning: online learning should be restricted to the dynamically necessary low-dimensional component of the objective.

Figures

Figures reproduced from arXiv: 2607.06079 by the authors.

**Figure 1.** Overview of the proposed learning rule. (A) The input reconstruction task on an echo [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

**Figure 2.** Performance evaluation of the proposed perturbation-based online learning algo [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

Figures from the paper (2 more)

**Figure 2.** Figure 2: Numerically computed learning orbits of the proposed rule, illustrating the geometric [PITH_FULL_IMAGE:figures/full_fig_p018_2.png]

**Figure 3.** Figure 3: Performance evaluation of the proposed perturbation-based online learning algo [PITH_FULL_IMAGE:figures/full_fig_p021_3.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 1 canonical work pages

[1]

Large-Scale Machine Learning with Stochastic Gradient Descent, in: Lechevallier, Y., Saporta, G

Bottou, L., 2010. Large-Scale Machine Learning with Stochastic Gradient Descent, in: Lechevallier, Y., Saporta, G. (Eds.), Proceedings of COMP- STAT’2010, Physica-Verlag HD, Heidelberg. pp. 177–186. doi:10.1007/ 978-3-7908-2604-3_16

2010
[2]

(Eds.), 2013

Casazza, P.G., Kutyniok, G. (Eds.), 2013. Finite Frames: The- ory and Applications. Applied and Numerical Harmonic Analy- sis, Birkhäuser, Boston. URL:https://link.springer.com/10.1007/ 978-0-8176-8373-3, doi:10.1007/978-0-8176-8373-3

work page doi:10.1007/978-0-8176-8373-3 2013
[3]

Méthode générale pour la résolution des systèmes d’équations simultanées

Cauchy, A.L., 1847. Méthode générale pour la résolution des systèmes d’équations simultanées. Comptes rendus hebdomadaires des séances de l’Académie des sciences 25, 536–538
[4]

Infor- mation Processing Capacity of Dynamical Systems

Dambre, J., Verstraeten, D., Schrauwen, B., Massar, S., 2012. Infor- mation Processing Capacity of Dynamical Systems. Scientific Reports 2, 514. URL:https://www.nature.com/articles/srep00514, doi:10. 1038/srep00514

2012
[5]

Gradient learning in spiking neural networks by dynamic perturbation of conductances

Fiete, I.R., Seung, H.S., 2006. Gradient learning in spiking neural networks by dynamic perturbation of conductances. Physical Review Letters 97, 048104. doi:10.1103/PhysRevLett.97.048104

doi:10.1103/physrevlett.97.048104 2006
[6]

Summed Weight Neuron Pertur- bation: An O(N) Improvement Over Weight Perturbation, in: Advances in Neural Information Processing Systems, Morgan- Kaufmann

Flower, B., Jabri, M., 1992. Summed Weight Neuron Pertur- bation: An O(N) Improvement Over Weight Perturbation, in: Advances in Neural Information Processing Systems, Morgan- Kaufmann. URL:https://proceedings.neurips.cc/paper/1992/ hash/996a7fa078cc36c46d02f9af3bef918b-Abstract.html

1992
[7]

Theoria motus corporum coelestium in sectionibus coni- cis solem ambientium

Gauss, C.F., 1809. Theoria motus corporum coelestium in sectionibus coni- cis solem ambientium. Frid. Perthes et I. H. Besser, Hamburg. 38
[8]

Learning Dynamics by Reservoir Computing (In Memory of Prof

Hara, M., Kokubu, H., 2022. Learning Dynamics by Reservoir Computing (In Memory of Prof. Pavol Brunovský). Journal of Dynamics and Differ- ential Equations URL:https://doi.org/10.1007/s10884-022-10159-w, doi:10.1007/s10884-022-10159-w

doi:10.1007/s10884-022-10159-w 2022

Show all 35 references

[9]

Memory and Information Processing in Neuro- morphic Systems

Indiveri, G., Liu, S.C., 2015. Memory and Information Processing in Neuro- morphic Systems. Proceedings of the IEEE 103, 1379–1397. URL:https: //ieeexplore.ieee.org/document/7159144, doi:10.1109/JPROC.2015. 2444094

2015
[10]

Weight Perturbation: An Optimal Archi- tecture and Learning Technique for Analog VLSI Feedforward and Re- current Multilayer Networks

Jabri, M., Flower, B., 1991. Weight Perturbation: An Optimal Archi- tecture and Learning Technique for Analog VLSI Feedforward and Re- current Multilayer Networks. Neural Computation 3, 546–565. URL: https://ieeexplore.ieee.org/document/6796476, doi:10.1162/neco. 1991.3.4.546

1991
[11]

echo state

Jaeger, H., 2001. The “echo state” approach to analysing and training recurrent neural networks. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 148, 13

2001
[12]

Adaptive Nonlinear System Identification with Echo State Networks, in: Advances in Neural Information Processing Systems, MIT Press

Jaeger, H., 2002. Adaptive Nonlinear System Identification with Echo State Networks, in: Advances in Neural Information Processing Systems, MIT Press. URL:https://proceedings.neurips.cc/paper_files/paper/ 2002/hash/426f990b332ef8193a61cc90516c1245-Abstract.html

2002
[13]

Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication

Jaeger, H., Haas, H., 2004. Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science 304, 78–

2004
[15]

Nouvelles méthodes pour la détermination des or- bites des comètes

Legendre, A.M., 1805. Nouvelles méthodes pour la détermination des or- bites des comètes. F. Didot, Paris
[16]

Backpropagation and the brain

Lillicrap, T.P., Santoro, A., Marris, L., Akerman, C.J., Hinton, G., 2020. Backpropagation and the brain. Nature Reviews Neuroscience 21, 335–346. URL:https://www.nature.com/articles/s41583-020-0277-3, doi:10. 1038/s41583-020-0277-3

2020
[17]

On- DeviceTrainingUnder256KBMemory, in: AdvancesinNeuralInformation Processing Systems, Curran Associates, Inc

Lin, J., Zhu, L., Chen, W.M., Wang, W.C., Gan, C., Han, S., 2022. On- DeviceTrainingUnder256KBMemory, in: AdvancesinNeuralInformation Processing Systems, Curran Associates, Inc.. pp. 22941–22954. URL: https://proceedings.neurips.cc/paper_files/paper/2022/hash/ 90c56c77c6df45fc...

2022
[18]

Reservoir observers: Model-free inference of unmeasured variables in chaotic systems

Lu, Z., Pathak, J., Hunt, B., Girvan, M., Brockett, R., Ott, E., 2017. Reservoir observers: Model-free inference of unmeasured variables in chaotic systems. Chaos: An Interdisciplinary Journal of Nonlinear Science 27, 041102. URL:https://aip.scitation.org/doi/full/10.1063/1. 4...

2017 doi
[19]

Reservoir computing approaches to recurrent neural network training

Lukoševičius, M., Jaeger, H., 2009. Reservoir computing approaches to recurrent neural network training. Computer Science Review 3, 127–

2009
[20]

Real-Time Com- puting Without Stable States: A New Framework for Neural Compu- tation Based on Perturbations

Maass, W., Natschläger, T., Markram, H., 2002. Real-Time Com- puting Without Stable States: A New Framework for Neural Compu- tation Based on Perturbations. Neural Computation 14, 2531–2560. URL:https://direct.mit.edu/neco/article/14/11/2531-2560/6650, doi:10.1162/089976602760...

2002 doi
[21]

Neuromorphic electronic systems

Mead, C., 1990. Neuromorphic electronic systems. Proceedings of the IEEE 78, 1629–1636. URL:https://ieeexplore.ieee.org/document/58356, doi:10.1109/5.58356

1990 doi
[22]

Con- tinual lifelong learning with neural networks: A review

Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S., 2019. Con- tinual lifelong learning with neural networks: A review. Neural Networks 113, 54–71. URL:https://www.sciencedirect.com/science/article/ pii/S0893608019300231, doi:10.1016/j.neunet.2019.01.012

2019 doi
[23]

Model-Free Predic- tion of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach

Pathak, J., Hunt, B., Girvan, M., Lu, Z., Ott, E., 2018. Model-Free Predic- tion of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. Physical Review Letters 120, 024102. URL:https: //link.aps.org/doi/10.1103/PhysRevLett.120.024102, doi:10.1103/...

2018 doi
[24]

Some Theorems in Least Squares

Plackett, R.L., 1950. Some Theorems in Least Squares. Biometrika 37, 149–157. URL:https://www.jstor.org/stable/2332158, doi:10.2307/ 2332158

1950
[25]

Scaling forward gradient with local losses, in: The Eleventh International Conference on Learning Representations

Ren, M., Kornblith, S., Liao, R., Hinton, G., 2023. Scaling forward gradient with local losses, in: The Eleventh International Conference on Learning Representations. URL:https://openreview.net/forum?id= JxpBP1JM15-

2023
[26]

A Stochastic Approximation Method

Robbins, H., Monro, S., 1951. A Stochastic Approximation Method. The Annals of Mathematical Statistics 22, 400–407. URL:https:// projecteuclid.org/journals/annals-of-mathematical-statistics/ volume-22/issue-3/A-Stochastic-Approximation-Method/10.1214/ aoms/1177729586.full, doi...

1951
[27]

Generating coherent patterns of activity fromchaoticneuralnetworks

Sussillo, D., Abbott, L.F., 2009. Generating coherent patterns of activity fromchaoticneuralnetworks. Neuron63, 544–557. doi:10.1016/j.neuron. 2009.07.018. 41

2009 doi
[28]

Recent advances in physical reservoir computing: A review

Tanaka, G., Yamane, T., Héroux, J.B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., Hirose, A., 2019. Recent advances in physical reservoir computing: A review. Neural Networks 115, 100–

2019
[29]

Learning curves for stochastic gradi- ent descent in linear feedforward networks

Werfel, J., Xie, X., Seung, H.S., 2005. Learning curves for stochastic gradi- ent descent in linear feedforward networks. Neural Computation 17, 2699–

2005
[30]

Numerical Evaluation of a Weakly Supervised Filtering Method Based on Echo State Networks

Yamada, T., Katori, Y., Fujiwara, K., 2025. Numerical Evaluation of a Weakly Supervised Filtering Method Based on Echo State Networks. Pro- ceedingsoftheISCIEInternationalSymposiumonStochasticSystemsThe- ory and its Applications 2026, 109–113. doi:10.5687/sss.2026.109

2025 doi
[31]

UnsupervisedLearninginEcho StateNetworksforInputReconstruction

Yamada, T., Katori, Y., Fujiwara, K., 2026. UnsupervisedLearninginEcho StateNetworksforInputReconstruction. NeuralComputation38, 198–227. URL:https://doi.org/10.1162/NECO.a.38, doi:10.1162/NECO.a.38

2026 doi
[32]

Weight versus Node Perturbation Learning in Temporally Extended Tasks: Weight Per- turbation Often Performs Similarly or Better

Züge, P., Klos, C., Memmesheimer, R.M., 2023. Weight versus Node Perturbation Learning in Temporally Extended Tasks: Weight Per- turbation Often Performs Similarly or Better. Physical Review X 13, 021006. URL:https://link.aps.org/doi/10.1103/PhysRevX.13. 021006, doi:10.1103/Ph...

2023 doi
[80]

URL:https://www.science.org/doi/10.1126/science.1091277, doi:10.1126/science.1091277

doi
[123]

URL:https://www.sciencedirect.com/science/article/pii/ S0893608019300784, doi:10.1016/j.neunet.2019.03.005

2019 doi
[149]

URL:https://www.sciencedirect.com/science/article/pii/ S1574013709000173, doi:10.1016/j.cosrev.2009.03.005

2009 doi
[2718]

doi:10.1162/089976605774320539

doi

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Large-Scale Machine Learning with Stochastic Gradient Descent, in: Lechevallier, Y., Saporta, G

Bottou, L., 2010. Large-Scale Machine Learning with Stochastic Gradient Descent, in: Lechevallier, Y., Saporta, G. (Eds.), Proceedings of COMP- STAT’2010, Physica-Verlag HD, Heidelberg. pp. 177–186. doi:10.1007/ 978-3-7908-2604-3_16

2010

[2] [2]

(Eds.), 2013

Casazza, P.G., Kutyniok, G. (Eds.), 2013. Finite Frames: The- ory and Applications. Applied and Numerical Harmonic Analy- sis, Birkhäuser, Boston. URL:https://link.springer.com/10.1007/ 978-0-8176-8373-3, doi:10.1007/978-0-8176-8373-3

work page doi:10.1007/978-0-8176-8373-3 2013

[3] [3]

Méthode générale pour la résolution des systèmes d’équations simultanées

Cauchy, A.L., 1847. Méthode générale pour la résolution des systèmes d’équations simultanées. Comptes rendus hebdomadaires des séances de l’Académie des sciences 25, 536–538

[4] [4]

Infor- mation Processing Capacity of Dynamical Systems

Dambre, J., Verstraeten, D., Schrauwen, B., Massar, S., 2012. Infor- mation Processing Capacity of Dynamical Systems. Scientific Reports 2, 514. URL:https://www.nature.com/articles/srep00514, doi:10. 1038/srep00514

2012

[5] [5]

Gradient learning in spiking neural networks by dynamic perturbation of conductances

Fiete, I.R., Seung, H.S., 2006. Gradient learning in spiking neural networks by dynamic perturbation of conductances. Physical Review Letters 97, 048104. doi:10.1103/PhysRevLett.97.048104

doi:10.1103/physrevlett.97.048104 2006

[6] [6]

Summed Weight Neuron Pertur- bation: An O(N) Improvement Over Weight Perturbation, in: Advances in Neural Information Processing Systems, Morgan- Kaufmann

Flower, B., Jabri, M., 1992. Summed Weight Neuron Pertur- bation: An O(N) Improvement Over Weight Perturbation, in: Advances in Neural Information Processing Systems, Morgan- Kaufmann. URL:https://proceedings.neurips.cc/paper/1992/ hash/996a7fa078cc36c46d02f9af3bef918b-Abstract.html

1992

[7] [7]

Theoria motus corporum coelestium in sectionibus coni- cis solem ambientium

Gauss, C.F., 1809. Theoria motus corporum coelestium in sectionibus coni- cis solem ambientium. Frid. Perthes et I. H. Besser, Hamburg. 38

[8] [8]

Learning Dynamics by Reservoir Computing (In Memory of Prof

Hara, M., Kokubu, H., 2022. Learning Dynamics by Reservoir Computing (In Memory of Prof. Pavol Brunovský). Journal of Dynamics and Differ- ential Equations URL:https://doi.org/10.1007/s10884-022-10159-w, doi:10.1007/s10884-022-10159-w

doi:10.1007/s10884-022-10159-w 2022