Learning dynamical systems with hit-and-run random feature maps

Georg A. Gottwald; Pinak Mandal

REVIEW 4 major objections 5 minor 1 cited by

Learning dynamical systems with hit-and-run random feature maps

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Modified random feature maps forecast chaotic dynamical systems as well as or better than reservoir computers and LSTMs, with model sizes one to three orders of magnitude smaller, in dimensions from 3 to 512.

desk verdict L63 and L96 results are solid and worth taking seriously, but the dimension-512 KS claim rests on an ad hoc noise level that makes the 'single hyperparameter' pitch overstate the case. read the letter →

arxiv 2501.06661 v2 pith:PWXE7UTB submitted 2025-01-11 cs.LG physics.data-anstat.MEstat.ML

Pinak Mandal , Georg A. Gottwald This is my paper · ORCID

classification cs.LGphysics.data-anstat.MEstat.ML MSC 37M1068T07

keywords randomfeaturemapschaosforecastinghit-and-runsamplingskipconnectionsdeeprandomizednetworkslocalizationLorenz-96Kuramoto-Sivashinsky

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that random feature maps, the simplest feedforward surrogates in which only the output layer is trained by linear regression, can be turned into excellent forecasters of chaotic dynamical systems. The authors modify classical random feature maps in three ways: internal weights are sampled by a hit-and-run algorithm so that the tanh features explore its nonlinear middle region rather than its saturated or linear parts; a skip connection makes the map learn the tendency u_{n+1} minus u_n instead of the full propagator; and several such units are chained into a deep architecture with sequentially trained layers. For high-dimensional systems, a localization scheme trains a single small map on local patches and replicates it across the state, which avoids the curse of dimensionality. On Lorenz-63, Lorenz-96, and a 512-point discretization of the Kuramoto-Sivashinsky equation, the models reach mean valid-prediction times of 12.0, 7.3, and 5.0 Lyapunov times, respectively, matching or beating published reservoir-computing and LSTM results at a fraction of the model size, while still reproducing the invariant measures over long runs. If correct, this makes random feature maps a competitive, near-hyperparameter-free alternative to reservoir computers for model-free forecasting.

What carries the argument

The workhorse is a hit-and-run sampler that draws each internal row (win, bin) so that for all training data the pre-activation win dot u plus bin falls in the good interval [L0, L1] = [0.4, 3.5] of the tanh function, outside the saturated region |z| >= 3.5 and outside the almost-linear region |z| <= 0.4. Around that, three structural modifications carry the results: (i) a skip connection that learns the tendency u_{n+1} minus u_n, i.e., a forward-Euler-style residual map; (ii) a deep stacking of B units, each trained with its own ridge regression on the augmented input formed by u_n and the previous unit's output, which preserves the state as part of every unit's input and lets depth increase model size without increasing GPU memory; (iii) localization, which splits the state into Ng = D/G patches of size G and learns one small map from (2I+1)G inputs per patch, exploiting translational symmetry with tensor operations so a single trained unit is replicated across the whole system. For ill-conditioned training data, such as the Kuramoto-Sivashinsky trajectory matrix with condition number near $10^{15}$, a small amount of added Gaussian noise with standard deviation $10^{-3}$ is used to keep the learned output weights well-conditioned.

What would settle it

Take a chaotic system with known long-range or global coupling, such as a globally coupled map lattice or Lorenz-96 with a nonlocal coupling kernel, train the same LocalDeepSkip configuration with N = $10^{5}$, and measure the mean VPT; if the localization assumption is load-bearing, forecast skill should collapse well below the reported 7.3 and below the non-localized baseline. A second check: retrain the Lorenz-63 models with uniformly random internal weights instead of hit-and-run samples, keeping everything else fixed, and see whether the mean VPT drops materially; if it does not, the claimed contribution of the good-region sampling collapses.

Watch

Extended reading notes

Core claim

The central claim is that a random feature map, if its fixed internal weights are chosen so that the training data are mapped into the nonlinear but non-saturated region of the tanh activation, forecasts better than the same architecture with naive random weights, and that stacking such maps with skip connections plus localizing them makes the resulting surrogate match or exceed the forecast skill of reservoir computers and LSTMs. The paper states this as: modified random feature maps provide excellent forecasting skill for both single trajectory forecasts as well as long-time estimates of statistical properties, for a range of chaotic dynamical systems with dimensions up to 512, with only the ridge penalty beta requiring tuning. Concretely, the best models achieve E[VPT] = 12.0 on Lorenz-63, E[VPT] = 7.3 on the 40-dimensional Lorenz-96 system, and E[VPT] = 5.0 on a 512-point discretization of the Kuramoto-Sivashinsky equation, at model sizes roughly 1.1 to 2.7 orders of magnitude smaller than the reservoir or recurrent baselines they are compared against. The paper also claims that for sufficiently large width, increasing depth improves forecasting more than increasing width at fixed model size, and that localization is what makes the high-dimensional cases work.

Load-bearing premise

The localized models rest on the assumption that, at the sampling time used, each variable's next value depends only on its own small patch and the 2I neighboring patches; any system with long-range or global coupling would hide essential inputs from the model and the reported high-dimensional results would not transfer.

Editorial extensions

If this is right

On Lorenz-63 the best DeepSkip model reaches E[VPT] = 12.0 Lyapunov times, matching the 11.8 to 12.0 of the best reservoir computer at roughly one twelfth of its model size.
On 40-dimensional Lorenz-96, localized deep models reach E[VPT] = 7.3, above the localized reservoir computer's 6.5 to 6.8, with a model about twenty times smaller.
On the 512-dimensional Kuramoto-Sivashinsky equation, a localized deep model trained on mildly noised data reaches E[VPT] = 5.0 versus 4.8 for the localized reservoir computer, with a model about 480 times smaller.
Only the regularization hyperparameter beta needs tuning, since the internal weights are sampled once by the hit-and-run algorithm with fixed boundaries L0 = 0.4 and L1 = 3.5.
For fixed model size, deeper networks train up to an order of magnitude faster, and depth only pays off once the width is large enough that forecast skill has saturated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the good-region sampling heuristic is what carries the gain, identical improvements should appear with other saturating activations such as sigmoid or error function once the linear and saturated thresholds are rescaled; this is a cheap experiment the paper does not run.
The localization assumption restricts the method to short-range interactions; on a system with known long-range coupling the interaction length I would have to grow with the state dimension, and the reported speed-ups would erode, so testing the same architectures on a globally coupled map would delineate where the localization advantage ends.
Because the deep units are trained sequentially, each on the residual error of the previous unit, the architecture has a natural streaming variant in which later units could be retrained or appended online without touching earlier units, a direction the paper does not explore.

Share X Bluesky LinkedIn Reddit HN

Formalized claims in Lean

Claim #1: The central claim is that a random feature map, if its fixed internal weights are chosen so that the training data are mapped into the nonlinear but non-saturated region of the tanh activation, forecasts better than the same architecture with naive random weights, and that stacking such maps with skip connections plus localizing them makes the resulting surrogate match or exceed the forecast skill

/-- @claim 1 The central claim is that a random feature map, if its fixed internal weights are chosen so that the training data are mapped into the nonlinear but non-saturated region of the tanh activation, forecasts better than the same architecture with naive random weights, and that stacking such maps with skip connections plus localizing them makes the resulting surrogate match or exceed the forecast skill -/
def central_claim : Prop :=

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, the L63 and L96 forecast results are genuinely good: 500 realizations per configuration, full tables, open code, and consistent gains over published RC/LSTM baselines at one to three orders of magnitude smaller model size. That part is convincing. Second, the KS result, which is the only dimension-512 evidence for the central claim, is fragile in exactly the way the stress-test says. LocalDeepSkip on clean data gets E[VPT]=0.5; add Gaussian noise of standard deviation 10^-3 to the training data and it jumps to 5.0, which is only marginally better than the Vlachas et al. RC baseline at 4.8, with a 0.9 standard deviation across the 500-realization distribution. The noise level is doing real work—it controls the condition number of the learned output weights—and the paper gives no sensitivity analysis over sigma. The authors are honest that the noise placement is unsettled, but the abstract's promise of a single hyperparameter simply does not cover this situation. Width, depth, localization scheme, and the noise level are all hand-chosen, and the localization assumption itself (future state depends on a local patch) is plausible but only tested on three systems with short-range interactions.

What is actually new: the combination of hit-and-run initialized random features with skip connections, deep stacking, and localization is a useful engineering contribution. The empirical study is unusually thorough for this literature: full tables in the appendix, training times, condition numbers, and statistical variation over 500 realizations. The hit-and-run method comes from the authors' prior paper, but here it is benchmarked against an independent Bayesian approach and against published baselines, so there is no circular reasoning. The code and data are public, which makes the reproducible claims checkable.

Where I would push back: the headline claim should be reworded to say that the method requires tuning one regularization parameter plus a small number of architectural choices, and the KS section needs a sensitivity analysis over the noise level before the state-of-the-art claim at dimension 512 can be taken at face value. That is a load-bearing flaw for one of the three test systems, but the L63 and L96 results stand on their own.

This paper deserves a serious referee. The empirical work is extensive, the method is clearly described, and the flaws are fixable with a sensitivity analysis and a more careful statement of the hyperparameter claim. I would not desk-reject it.

Referee Report

4 major / 5 minor

Summary. The manuscript studies random feature maps (RFMs) for forecasting chaotic dynamical systems, combining a data-informed hit-and-run initialization of tanh internal weights with skip connections, sequential deep stacking, and localization. It reports mean valid prediction times E[VPT]=12.0 for Lorenz-63, 7.3 for Lorenz-96, and 5.0 for a 512-dimensional Kuramoto-Sivashinsky discretization, with model sizes one to three orders of magnitude smaller than the RC/LSTM baselines compared, and it also reports Wasserstein distances for invariant measures. The authors argue that their method effectively requires tuning only a single hyperparameter.

Significance. If the results hold, the paper makes a strong practical case that simple feed-forward random feature surrogates can replace recurrent architectures for chaotic forecasting at a fraction of the model size. The empirical protocol is unusually thorough: 500 realizations per configuration, detailed appendix tables, open code and forecast data, and comparisons with published baselines. The main caveats are that the dimension-512 claim depends on a hand-chosen noise level and that the 'single hyperparameter' framing overstates what was actually selected; these issues are fixable but need to be addressed before the central claims can be accepted in their current form.

major comments (4)

[Section 3.3 / Table 8] The dimension-512 KS result is load-bearing for the 'dimensions up to 512' claim and depends entirely on noise injection. LocalDeepSkip trained on clean data achieves E[VPT]=0.5 (std 0.1); after adding zero-mean Gaussian noise with standard deviation 10^-3, the same architecture reaches E[VPT]=5.0 (std 0.9), which is only 0.2 above the Vlachas et al. RC baseline of 4.8. No sensitivity analysis over the noise magnitude is reported, and Section 2.6 justifies the chosen value only as 'indistinguishable by eye' while leaving open whether the noise should instead be placed on Phi. Since the reported condition numbers (~950-1350) are consistent with conditioning controlled by 1/sigma through the smoothed-analysis mechanism cited in Section 2.6, the 5.0 value may be a tuned point rather than a plateau. Please provide a sigma sweep (for example 0, 3e-4, 1e-3, 3e-3, 1e-2) with the corresponding VPT and condition numbers, and temper the D=512 state-of-the-art claim if the performance is not robust across that sweep.
[Abstract / Section 4] The claim that 'effectively need to tune only a single hyperparameter' is contradicted by the experimental protocol. The width Dr, depth B, localization scheme (G,I), and for KS the noise standard deviation sigma are all selected per system, sometimes via the exploratory sweeps in Appendix 8.3 and Table 8, while beta is optimized by grid search. The constants L0 and L1 are asserted to be insensitive, but no detailed sensitivity study is shown. Please either restrict the 'single hyperparameter' wording to the regularization parameter after the architecture is fixed, or supply sensitivity analyses demonstrating plateaus in Dr, B, (G,I), and sigma.
[Section 2.5] The localization scheme is built on a conditional-independence/local-interaction assumption: for sufficiently small sampling time, the future state of a local region depends only on that region and 2I neighboring regions. This physical assumption is not validated beyond the three test problems, all of which have local coupling and, for L96 and KS, translational symmetry that allows a single unit to be replicated across the domain. The paper should state explicitly that the high-dimensional results are for translationally invariant, locally coupled systems, and ideally test at least one non-translationally-invariant or long-range-coupled system before claiming that localization generally mitigates the curse of dimensionality.
[Tables 2-3 / Sections 3.2-3.3] The 'state-of-the-art' comparisons are not fully controlled. Table 2 compares L96 with F=10 against Platt et al.'s F=8 result, relying on an external claim that forecast skill is similar for F=8 and F=10, and Table 3's winning KS model includes an extra noise-injection ingredient that the baseline does not use. Tables 1-3 also mix values of N, Delta t, and epsilon across rows. Please state explicitly which comparisons are controlled and, where feasible, run the same baseline under identical training conditions so the reader can separate architectural gains from protocol differences.

minor comments (5)

[Appendix 8.3] The localization-scheme selection in Figures 15 and 16 is based on only 5 samples per beta value; the main text should state that these are crude estimates, not the 500-realization statistics reported elsewhere.
[References / Tables 2-3] Tables 2 and 3 cite 'Vlachas et al. (2022) [8]' but reference [8] is Vlachas et al., Neural Networks 126, 191-217 (2020); the year should be corrected or a separate 2022 reference supplied.
[Section 2.6 / Table 7] The sentence 'We only apply noise when dealing with ill-conditioned training data U' should be reconciled with the LocalDeepSkipN row in Table 7, where noise is applied to the well-conditioned L96 data and gives no improvement; please clarify why that row is included.
[Figure 8] The legend for Figure 8 is missing; the curves labeled in the caption need a visual legend or explicit line-style description so the reader can identify 'RFM (Ours)', 'SkipRFM (Ours)', 'RFM (L+S)', and 'rhs (L+S)'.
[Title page] The running header 'RANDOM FEA TURE MAPS' contains a spurious space in 'FEATURE'; this should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all headline forecast metrics are computed on held-out test data against independent published baselines; the only self-citation (hit-and-run initialization [20]) is fully specified in the paper and benchmarked against the independent Bayesian method [13].

full rationale

The paper's central claims are empirical: the E[VPT] values for Lorenz-63, Lorenz-96, and Kuramoto-Sivashinsky are evaluated by running the learned surrogate autonomously on unseen test trajectories (Eqs. 10-11), and are compared with published reservoir-computing, LSTM, and ESN baselines from independent groups [8, 13, 14, 29]. The architectural modifications (skip connections, deep stacking, localization) are defined by explicit constructive equations (Eqs. 5-9) rather than derived from the paper's own outputs, so there is no self-definitional reduction. The hit-and-run internal-weight sampler is attributed to the authors' prior work [20], which is a genuine self-citation; however, the algorithm is reproduced in full in Appendix 8.1, and its forecasting value is demonstrated in this paper against the independent Bayesian random-feature framework [13] (Fig. 8), so the self-citation is not load-bearing in the circularity sense. The one methodological point that invites scrutiny is the fixed Gaussian noise level (sigma = 1e-3) used for the KS training data: the reported E[VPT] jumps from 0.5 to 5.0 when this noise is added (Table 8), and no sensitivity analysis over sigma is given. This is a robustness/reproducibility concern about a regularization choice, not a circularity: the VPT values are still measured on noise-free test data, and the noise level was not fitted to the test trajectories. Therefore the derivation chain is self-contained and no circular step can be exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method rests on several hand-chosen parameters beyond the claimed single hyperparameter, and on domain assumptions about locality, noise-free data, and small sampling times.

free parameters (6)

Regularization parameter β = Varies per model, e.g., 4.96e-09 for L63 DeepSkip
Tuned via grid search for each architecture; controls ridge regression (Eq. 4).
Width Dr = 512 to 16,384 depending on system
Chosen by hand; increasing width improves skill until saturation.
Depth B = 1 to 32
Chosen by hand; depth improves L63 and large-Δt performance but can hurt L96 when width is insufficient.
Localization scheme (G, I) = (2,2) for L96, (8,1) for KS
Selected based on experiments in Appendix 8.3; defines the locality assumption and input dimension.
Noise level for KS training data = σ = 10^-3
Added to ill-conditioned data to control condition number; chosen 'by eye' and critical for the KS forecast skill.
tanh region thresholds L0, L1 = L0=0.4, L1=3.5
Constants defining 'good' features; the paper notes they could be considered hyperparameters but observes robustness.

assumptions (4)

domain assumption Observations are complete and noise-free
Stated in Section 2.1; the paper explicitly restricts to this setting and does not test partial/noisy observations.
domain assumption Local conditional independence for small Δt
Section 2.5 assumes the future local state depends only on itself and 2I neighbors; if violated, localized models lose essential inputs.
domain assumption Tendency map approximates an Euler step
Skip connection learns un+1 - un and is interpreted as one Euler step (Eq. 5); accuracy degrades for larger Δt.
standard math Universal approximation of random feature maps
Invoked via citations [21-24] to justify that RFMs can represent the propagator; no specific error bounds are provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning dynamical systems with hit-and-run random feature maps." pith.science (2026). https://pith.science/paper/PWXE7UTB

@misc{pith2026250106661,
  author       = {Pith},
  title        = {Pith review of: Learning dynamical systems with hit-and-run random feature maps},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWXE7UTB}},
  note         = {Machine review of arXiv:2501.06661}
}

read the original abstract

We show how random feature maps can be used to forecast dynamical systems with excellent forecasting skill. We consider the tanh activation function and judiciously choose the internal weights in a data-driven manner such that the resulting features explore the nonlinear, non-saturated regions of the activation function. We introduce skip connections and construct a deep variant of random feature maps by combining several units. To mitigate the curse of dimensionality, we introduce localization where we learn local maps, employing conditional independence. Our modified random feature maps provide excellent forecasting skill for both single trajectory forecasts as well as long-time estimates of statistical properties, for a range of chaotic dynamical systems with dimensions up to 512. In contrast to other methods such as reservoir computers which require extensive hyperparameter tuning, we effectively need to tune only a single hyperparameter, and are able to achieve state-of-the-art forecast skill with much smaller networks.

Figures

Figures reproduced from arXiv: 2501.06661 by the authors.

**Figure 1.** Illustration of the types of features produced by a tanh-activation function, motivating the choice of the internal weights and biases (Win, bin). Here and elsewhere L0 = 0.4 and L1 = 3.5. for a dynamical system with the vector field F˜∆t(un) = 1 ∆t Wtanh (Winun + bin). (6) Note that the map F∆t is learned from data with a fixed and specified value of the sampling time ∆t. When applying the learned map to forecast u… view at source ↗

**Figure 2.** Schematic of the deep architecture DeepSkip with depth B = 3. The symbols ∥, = and + denote concatenation, identity operation and addition (skip connection), respectively. to as DeepRFM. Similarly, when the unit is a SkipRFM, the corresponding deep model is referred to as DeepSkip. We found empirically that using augmented states (8), such that each unit has the state un as part of its input, rather than using y (ℓ… view at source ↗

**Figure 3.** Schematic of a localized architecture. In this example, the local state dimension is G = 3 and the interaction length is I = 2. time step. Concretely, we assume that the state of a local region at time tn+1 depends on the state of the same local region as well as on its 2I neighbouring local regions at time tn, where I is called the interaction length. The pair (G, I) defines a localization scheme [PITH_FULL_IMAGE:… view at source ↗

Figures from the paper (16 more)

**Figure 4.** Figure 4: shows a sample forecast of a DeepSkip model which is accurate up to approximately VPT ≈ 19 Lyapunov time units. However, there is a significant variability in the VPT due to the sensitivity to initial conditions of the chaotic L63 system [PITH_FULL_IMAGE:figures/full_…

**Figure 5.** Figure 5: Kernel density plots of VPT for the L63 system (12) for (N, ∆t, ε) = (5 × 104 , 0.01, 0.3). For RFM and SkipRFM, log2 (Dr) is indicated on the top of the plots. For DeepSkip, (log2 (Dr), B) is indicated on the top of each plot. The ∗-symbol indicates the model with the…

**Figure 6.** Figure 6: Mean VPT for DeepSkip as a function of depth B for constant model size S for the L63 system (12). Along each curve the model size S = (3D + 1)DrB remains constant and the width Dr decreases with depth. Source Model log10(model size) E[VPT] N ∆t ε Akiyama et al. (2022) …

**Figure 7.** Figure 7: Kernel density plots of VPT for the L63 system (12) for (N, ∆t, ε) = (5 × 104 , 0.1, 0.3). Sizes of the models are S = 114, 688, S = 114, 688 and S = 114, 560, from left to right, with, Dr = 16, 384, Dr = 16, 384 and Dr = 716 respectively. The DeepSkip model has depth …

**Figure 8.** Figure 8: Mean validity time τf as a function of the sampling time ∆t for the L63 system (12) with Dr = 200. Each model was trained on a time-series spanning T = 1, 000 time units and hence the length of the training data N decreases with increasing ∆t = T /N. Averages for our m…

**Figure 9.** Figure 9: Marginal densities for x, y and z estimated from the invariant measures of the L63 system (12) and of the surrogate models RFM, SkipRFM and DeepSkip. The respective Wasserstein W2 distances are indicated at the top of the kernel density plots. We used the best models m…

**Figure 10.** Figure 10: shows the distribution of VPT for these models for N = 105 , ∆t = 0.01 and ε = 0.5. We choose a localization scheme with (G, I) = (2, 2); see Appendix 8.3 for different localization schemes and general guidelines for selecting an optimal localization scheme. The posit…

**Figure 11.** Figure 11: Mean VPT for LocalDeepSkip as a function of depth B for constant model size S and localization scheme (G, I) = (2, 2) for the L96 system (14). Along each curve the model size S = (Dˆ + G + 1)DrB with Dˆ = 2G(I + 1), remains constant and the width Dr decreases with dep…

**Figure 12.** Figure 12: Marginal densities for a component of the L96 system, estimated from the invariant measures of the original L96 system (14) and of various surrogate models. We employ translational symmetry and use all 40 components to estimate the densities. The respective Wasserstei…

**Figure 13.** Figure 13: Kernel density plots of VPT for the KS system (15) for (N, ∆t, ε) = (105 , 0.25, 0.5) for various localized surrogate models. For all models (log2 (Dr), B) is indicated on the top of each plot. The ∗-symbol indicates the model with the best mean VPT within each archi…

**Figure 14.** Figure 14: Marginal densities for a component of the invariant measure of the KS system, estimated from the original KS system (15) and from various surrogate models. We employ translational symmetry and use all 512 components to estimate the densities. The respective Wasserstei…

**Figure 15.** Figure 15: Estimates of the mean VPT as a function of the regularization hyperparameter β for different localization schemes for the L96 system. The models depicted here are LocalSkip with Dr = 4, 096. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

**Figure 16.** Figure 16: Estimates of the mean VPT as a function of the regularization hyperparameter β for different localization schemes for the KS equation for various localized random feature models. While choosing a localization scheme, practitioners should consider several factors such …

**Figure 17.** Figure 17: Mean VPT as a function of the depth B with fixed width Dr = 1, 024 (left) and of the width Dr with fixed depth B = 2 (right) for the 3-dimensional L63 system (12). We show DeepSkip surrogate models with the data taken from [PITH_FULL_IMAGE:figures/full_fig_p026_17.png]

**Figure 18.** Figure 18: Mean VPT as a function of the depth B with fixed width Dr = 1, 024 (left) and of the width Dr with fixed depth B = 2 (right) for the 40-dimensional L96 system (12). We show LocalDeepSkip2,2 surrogate models with the data taken from [PITH_FULL_IMAGE:figures/full_fig_p…

**Figure 19.** Figure 19: Average training time in seconds as a function of depth B. The left and right panels show results for DeepSkip and LocalDeepSkip taken from Tables 4 and 7, respectively. Along each curve the model size S remains constant and the width Dr decreases with depth. 27 [PIT…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Machine-Precision Prediction of Low-Dimensional Chaotic Systems from Noise-Free Data
nlin.CD 2025-07 conditional novelty 6.0 of 10

Polynomial regression with 512-bit arithmetic reaches machine-precision forecasting of low-dimensional chaotic systems from noise-free data, far exceeding previous valid prediction times.

Reference graph

Works this paper leans on

56 extracted references · 45 canonical work pages · cited by 1 Pith paper

[1]

Bi, K. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 533–538 (2023)

2023
[2]

Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, 1416–1421 (2023). URL https: //www.science.org/doi/abs/10.1126/science.adi2336

doi:10.1126/science.adi2336 2023
[3]

Price, I. et al. Probabilistic weather forecasting with machine learning. Nature (2024)

work page 2024
[4]

& Schmidhuber, J

Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Computation 9, 1735–1780 (1997). URL https://doi. org/10.1162/neco.1997.9.8.1735

doi:10.1162/neco.1997.9.8.1735 1997
[5]

& Markram, H

Maass, W., Natschl¨ ager, T. & Markram, H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14, 2531–2560 (2002)

work page 2002
[6]

A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ”echo state network” approach

Jaeger, H. A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ”echo state network” approach. GMD-Report 159, German National Research Institute for Computer Science (2002)

work page 2002
[7]

& Haas, H

Jaeger, H. & Haas, H. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004)

work page 2004
[8]

Vlachas, P.-R. et al. Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks 126, 191–217 (2020)

2020

Show all 56 references

[9]

& Gu´ ery-Odelin, D

Bompas, S., Georgeot, B. & Gu´ ery-Odelin, D. Accuracy of neural networks for the simulation of chaotic dynamics: Precision of training data vs precision of the algorithm. Chaos: An Interdisciplinary Journal of Nonlinear Science 30, 113118 (2020). URL https://doi.org/10.1063/5.0021264

2020 doi
[10]

& Ott, E

Pathak, J., Hunt, B., Girvan, M., Lu, Z. & Ott, E. Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical Review Letters 120, 024102 (2018)

2018
[11]

& Gigan, S

Rafayelyan, M., Dong, J., Tan, Y., Krzakala, F. & Gigan, S. Large-scale optical reservoir computing for spatiotemporal chaotic systems prediction. Physical Review X 10, 041037 (2020)

2020
[12]

& Fischer, I

Nakajima, K. & Fischer, I. Reservoir Computing (Springer, 2021)

2021
[13]

Levine, M. E. & Stuart, A. M. A framework for machine learning of model error in dynamical systems. Comm. Amer. Math. Soc. 2, 283–344 (2022)

2022
[14]

A., Penny, S

Platt, J. A., Penny, S. G., Smith, T. A., Chen, T.-C. & Abarbanel, H. D. A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics. Neural Networks 153, 530–552 (2022)

2022
[15]

J., Bollt, E., Griffith, A

Gauthier, D. J., Bollt, E., Griffith, A. & Barbosa, W. A. S. Next generation reservoir computing. Nature Communications 12, 5564 (2021)

2021
[16]

Bollt, E. On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to V AR and DMD.Chaos: An Interdisciplinary Journal of Nonlinear Science 31, 013108 (2021). URL https://doi.org/10.1063/5.0024890

2021 doi
[17]

& Recht, B

Rahimi, A. & Recht, B. Random features for large-scale kernel machines. In Platt, J. C., Koller, D., Singer, Y. & Roweis, S. T. (eds.) Advances in Neural Information Processing Systems 20 , 1177–1184 (Curran Associates, Inc., 2008). URL http://papers.nips.cc/paper/3182-random-...

2008
[18]

Gottwald, G. A. & Reich, S. Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations. Chaos: An Interdisciplinary Journal of Nonlinear Science 31, 101103 (2021). URL https: //doi.org/10.1063/5.0066080. 17

2021 doi
[19]

Nelsen, N. H. & Stuart, A. M. The random feature model for input-output maps between Banach spaces. SIAM Journal on Scientific Computing 43, A3212–A3243 (2021). URL https://doi.org/10.1137/20M133957X

2021 doi
[20]

& Gottwald, G

Mandal, P. & Gottwald, G. A. On the choice of the non-trainable internal weights in random feature maps. arXiv preprint arXiv:2408.03626 (2024)

2024 arXiv
[21]

Approximation by superposition of a sigmoidal function

Cybenko, G. Approximation by superposition of a sigmoidal function. Math. Contr., Sign., and Syst. 2, 303–314 (1989)

1989
[22]

& Sandberg, I

Park, J. & Sandberg, I. Universal approximation using radial-basis-function networks. Neural Computation 3, 246–257 (1991)

1991
[23]

Universal approximation bounds for superposition of a sigmoidal function

Barron, A. Universal approximation bounds for superposition of a sigmoidal function. IEEE Trans. on Inform. Theory 39, 930–945 (1993)

1993
[24]

& Recht, B

Rahimi, A. & Recht, B. Uniform approximation of functions with random bases. In 2008 46th Annual Allerton Conference on Communication, Control, and Computing , 555–561 (2008)

2008
[25]

Dunbar, O. R. A., Nelsen, N. H. & Mutic, M. Hyperparameter optimization for randomized algorithms: a case study on random features. Statistics and Computing 35, 56 (2025). URL https://doi.org/10.1007/s11222-025-10587-w

2025 doi
[26]

& Sun, J

He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. In Leibe, B., Matas, J., Sebe, N. & Welling, M. (eds.) Computer Vision – ECCV 2016 , 630–645 (Springer International Publishing, Cham, 2016)

2016
[27]

& Gallicchio, C

Ceni, A. & Gallicchio, C. Residual Echo State Networks: Residual recurrent neural networks with stable dynamics and fast learning. Neurocomputing 597, 127966 (2024). URL https://www.sciencedirect.com/science/article/pii/ S0925231224007379

2024
[28]

A proposal on machine learning via dynamical systems

E, W. A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics 5, 1–11 (2017)

2017
[29]

& Tanaka, G

Akiyama, T. & Tanaka, G. Computational efficiency of multi-step learning echo state networks for nonlinear time series prediction. IEEE Access 10, 28535–28544 (2022)

2022
[30]

& Zhang, J

Ding, S., Zhang, N., Xu, X., Guo, L. & Zhang, J. Deep extreme learning machine and its application in EEG classification. Mathematical Problems in Engineering 2015, 129021 (2015). URL https://onlinelibrary.wiley.com/doi/abs/10.1155/ 2015/129021

2015
[31]

& Mian, A

Uzair, M., Shafait, F., Ghanem, B. & Mian, A. Representation learning with deep extreme learning machines for efficient image set classification. Neural Computing and Applications 30, 1211–1223 (2018)

2018
[32]

& Reich, S

Gottwald, G. & Reich, S. Localized Schr¨ odinger bridge sampler. arXiv:2409.07968 (2024)

2024 arXiv
[33]

Gottwald, G. A. & Reich, S. Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation. Physica D: Nonlinear Phenomena 423, 132911 (2021)

2021
[34]

S., Queiruga, A

Krishnapriyan, A. S., Queiruga, A. F., Erichson, N. B. & Mahoney, M. W. Learning continuous models for continuous physics. Communications Physics 6, 319 (2023). URL https://doi.org/10.1038/s42005-023-01433-4

2023 doi
[35]

& Hassibi, B

Bosch, D., Panahi, A. & Hassibi, B. Precise asymptotic analysis of deep random feature models. In Neu, G. & Rosasco, L. (eds.) Proceedings of Thirty Sixth Conference on Learning Theory, vol. 195 of Proceedings of Machine Learning Research, 4132–4179 (PMLR, 2023). URL https://p...

2023
[36]

Schapire, R. E. & Freund, Y. Boosting: Foundations and Algorithms (The MIT Press, 2012). URL https://doi.org/10. 7551/mitpress/8291.001.0001

2012
[37]

& Myoung, S.-H

Kim, I.-C. & Myoung, S.-H. Text categorization using hybrid multiple model schemes. In R. Berthold, M., Lenz, H.-J., Bradley, E., Kruse, R. & Borgelt, C. (eds.) Advances in Intelligent Data Analysis V , 88–99 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2003)

2003
[38]

Spielman, D. A. & Teng, S.-H. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. Journal of the ACM (JACM) 51, 385–463 (2004)

2004
[39]

R., Byeon, W., Wan, Z

Vlachas, P. R., Byeon, W., Wan, Z. Y., Sapsis, T. P. & Koumoutsakos, P. Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 474, 20170844 (2018). U...

2018 doi
[40]

Feydy, J. et al. Interpolating between optimal transport and mmd using Sinkhorn divergences. In The 22nd International Conference on Artificial Intelligence and Statistics , 2681–2690 (PMLR, 2019)

2019
[41]

Lorenz, E. N. Deterministic nonperiodic flow. Journal of the Atmospheric Sciences 20, 130–141 (1963)

1963
[42]

& L¨ udge, K

K¨ oster, F., Patel, D., Wikner, A., Jaurigue, L. & L¨ udge, K. Data-informed reservoir computing for efficient time-series prediction. Chaos: An Interdisciplinary Journal of Nonlinear Science 33 (2023)

2023
[43]

Lorenz, E. N. Predictability: A problem partly solved. In Proc. Seminar on predictability , vol. 1 (Reading, 1996)

1996
[44]

Penny, S. G. et al. Integrating recurrent neural networks with data assimilation for scalable data-driven state estimation. Journal of Advances in Modeling Earth Systems 14, e2021MS002843 (2022)

2022
[45]

Diffusion-induced chaos in reaction systems

Kuramoto, Y. Diffusion-induced chaos in reaction systems. Progress of Theoretical Physics Supplement 64, 346–367 (1978)

1978
[46]

Nonlinear analysis of hydrodynamic instability in laminar flames—i

Sivashinsky, G. Nonlinear analysis of hydrodynamic instability in laminar flames—i. derivation of basic equations. In Dynamics of Curved Fronts , 459–488 (Elsevier, 1988)

1988
[47]

& Trefethen, L

Kassam, A.-K. & Trefethen, L. N. Fourth-order time-stepping for stiff PDEs. SIAM Journal on Scientific Computing 26, 1214–1233 (2005)

2005
[48]

Keller, R. T. & Du, Q. Discovery of dynamics using linear multistep methods. SIAM Journal on Numerical Analysis 59, 429–455 (2021)

2021
[49]

& Zhou, C

Du, Q., Gu, Y., Yang, H. & Zhou, C. The discovery of dynamics via linear multistep methods and deep learning: Error estimation. SIAM Journal on Numerical Analysis 60, 2014–2045 (2022). 18

2022
[50]

Cheap robust learning of data anomalies with analytically solvable entropic outlier sparsification

Horenko, I. Cheap robust learning of data anomalies with analytically solvable entropic outlier sparsification. Proceed- ings of the National Academy of Sciences 119, e2119659119 (2022). URL https://www.pnas.org/doi/abs/10.1073/pnas. 2119659119

2022 doi
[51]

Horenko, I. et al. On cheap entropy-sparsified regression learning. Proceedings of the National Academy of Sciences 120, e2214972120 (2023). URL https://www.pnas.org/doi/abs/10.1073/pnas.2214972120

2023 doi
[52]

& O’Kane, T

Groom, M., Bassetti, D., Horenko, I. & O’Kane, T. J. On the comparative utility of entropic learning versus deep learning for long-range ENSO prediction. Artificial Intelligence for the Earth Systems 3, 240009 (2024). URL https://journals. ametsoc.org/view/journals/aies/3/4/AI...

2024
[53]

Saad, F. et al. Scalable spatiotemporal prediction with Bayesian neural fields. Nature Communications 15 (2024)

2024
[54]

Detecting strange attractors in turbulence

Takens, F. Detecting strange attractors in turbulence. In Dynamical systems and turbulence, Warwick 1980 (Coventry 1979/1980), vol. 898 of Lecture Notes in Math. , 366–381 (Springer, Berlin, 1981)

1980
[55]

& Schreiber, T

Kantz, H. & Schreiber, T. Nonlinear Time Series Analysis (Cambridge University Press, Cambridge, 1997). 19

1997
[56]

One-shot hit-and-run algorithm to draw internal weights for random feature maps

Appendix 8.1. One-shot hit-and-run algorithm to draw internal weights for random feature maps. Algorithm 1 Hit-and-run sampling for a row of the internal augmented matrix Win|bin 1: Input: data U = [ u1, u2, · · ·, uN ]. Boundaries for the good range of the tanh-function L0,1....

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Bi, K. et al. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 533–538 (2023)

2023

[2] [2]

Lam, R. et al. Learning skillful medium-range global weather forecasting. Science 382, 1416–1421 (2023). URL https: //www.science.org/doi/abs/10.1126/science.adi2336

doi:10.1126/science.adi2336 2023

[3] [3]

Price, I. et al. Probabilistic weather forecasting with machine learning. Nature (2024)

work page 2024

[4] [4]

& Schmidhuber, J

Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Computation 9, 1735–1780 (1997). URL https://doi. org/10.1162/neco.1997.9.8.1735

doi:10.1162/neco.1997.9.8.1735 1997

[5] [5]

& Markram, H

Maass, W., Natschl¨ ager, T. & Markram, H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14, 2531–2560 (2002)

work page 2002

[6] [6]

A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ”echo state network” approach

Jaeger, H. A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ”echo state network” approach. GMD-Report 159, German National Research Institute for Computer Science (2002)

work page 2002

[7] [7]

& Haas, H

Jaeger, H. & Haas, H. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004)

work page 2004

[8] [8]

Vlachas, P.-R. et al. Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks 126, 191–217 (2020)

2020