Pith. sign in

REVIEW 2 major objections 4 minor 188 references

This paper proves that finite-size Neural ODEs can approximate Morse-Smale multistable systems and normally hyperbolic continuous attractors over the infinite time horizon: for any ε, δ > 0, all but a δ-fraction of initial conditions stay w

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Neural ODEs can approximate Morse-Smale and continuous-attractor dynamical systems over infinite time in an ε-δ sense, provided limit-cycle periods are matched exactly.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection First infinite-horizon multistable approximation results for Neural ODEs, with a solid fixed-point theorem, a clever limit-cycle construction, and a load-bearing gap in the continuous-attractor theorem that needs real work. the 2 major comments →

arxiv 2602.08640 v3 pith:66URBFPP submitted 2026-02-09 math.DS q-bio.NC

Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees

Abel Sagodi , Il Memming Park This is my paper
classification math.DS q-bio.NC MSC 37C1037C2068T07
keywords Neural ODEsuniversal approximationMorse-Smale systemsinfinite-time horizonmultistabilitylimit cyclescontinuous attractorsε-δ closeness
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural ODEs—differential equations whose vector fields are neural networks—have long been known to approximate arbitrary smooth flows on finite windows, but those guarantees degrade or fail for systems with multiple attractors. This paper tries to remove that barrier for a broad, structurally stable class. It claims that a finite-size Neural ODE can track trajectories forever, with error below ε for every initial condition outside a set of volume δ, whenever the target is Morse-Smale with hyperbolic fixed points or hyperbolic limit cycles. For limit cycles it adds exact period matching, because any period mismatch causes unbounded phase drift. For normally hyperbolic continuous attractors—line attractors, ring attractors—it claims the same ε-δ guarantee by tiling the manifold with a dense grid of discrete attractors. A short corollary converts ε-δ closeness into a bound on time-averaged Lp error, linking the topological guarantee to losses used in training.

Core claim

The central claim is a universal approximation theorem for infinite-horizon dynamics: for any target in the class F_FP (Morse-Smale systems whose ω-limit sets are hyperbolic fixed points), and any ε, δ > 0, there exists a finite-size Neural ODE whose flow is ε-δ close to the target on [0, ∞). The same is claimed for F_LC—Morse-Smale systems with hyperbolic limit cycles—provided the hypothesis class supports bump functions so that each cycle's period can be corrected exactly. For F_CA, systems with normally hyperbolic continuous attractors, the claim is achieved by discretizing the attractor: a C1-close proxy with a finite skeleton of hyperbolic fixed points or limit cycles forming an ε/3- or

What carries the argument

The load-bearing machinery is ε-δ closeness, a metric measuring the volume of initial conditions whose supremum-in-time trajectory error exceeds ε; it induces the topology of convergence in measure on the space of flows and is metrizable via the Ky Fan metric. On top of that, the proofs use four tools: (i) structural stability of Morse-Smale flows to confine basin mismatch to a thin separatrix layer of measure below δ; (ii) a transient/asymptotic split bound, Kν/λ, using exponential contraction near hyperbolic attractors; (iii) localized scalar multiplication via bump functions to correct each limit cycle's period exactly, with diagonal dominance of the period Jacobian ensuring that correcti

Load-bearing premise

The continuous-attractor theorems rest on an unproven existence assertion: a C1-arbitrarily-close proxy must exist whose hyperbolic fixed points or limit cycles form a prescribed dense skeleton with spacing ε/3 or ε/8 of the original normally hyperbolic manifold; generic hyperbolicity gives isolated hyperbolic sets, not a skeleton of prescribed spacing, and the paper supplies no construction.

What would settle it

For a normally hyperbolic disk of fixed points, compute the infimal C1 distance to any perturbation whose hyperbolic fixed points form an ε/3-net of the disk. If this infimum fails to vanish as ε→0, the tiling claim in the continuous-attractor theorem is false; if it vanishes, the proof's Step 1 has a constructive basis.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • For any Morse-Smale multistable system with hyperbolic fixed points, a finite Neural ODE can reproduce all trajectories on [0, ∞) within ε except for initial conditions in a δ-measure set; this extends the reach of neural dynamical models beyond globally stable, fading-memory systems.
  • For hyperbolic limit cycles, exact period matching eliminates unbounded phase drift: trajectories remain within a bounded distance of the true cycle forever, not just for a few periods.
  • For normally hyperbolic continuous attractors, trajectories stay ε-close forever even though the approximating system has only discrete attractors; the cost is a bounded discretization error controlled by the tiling resolution.
  • ε-δ closeness over infinite time implies time-averaged Lp error and mean-squared error are bounded by ε^p + δ·D^p, so the topological guarantee transfers to the loss functions used in practical training.
  • Because the approximation is direct in the same state space rather than an embedding, the attractor geometry and basin structure are preserved, making learned models mechanistically interpretable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Extension: The ε-δ metric deliberately sacrifices all information inside the δ-layer—saddle-region timing, exact separatrix geometry, and homoclinic structure. Two networks with identical ε-δ certificates can therefore disagree arbitrarily about trajectories near separatrices, which is precisely the regime where decision-making circuits operate; the paper flags the layer but not this consequence.
  • Extension: The continuous-attractor proof is only as strong as its Step 1: generic hyperbolicity is invoked where a prescribed dense skeleton is needed, and no construction is supplied. A computational check—computing the minimal C1 distance from a flat normally hyperbolic manifold to a system with an ε-net of hyperbolic attractors—would either supply the missing construction or expose a gap.
  • Extension: Because exact period matching multiplies the vector field by a bump function, it changes flow speed but not the cycle's geometry or the zero set. This suggests a trainable phase-field parameterization: constraining the network family so that the period is an explicit output, rather than a post-hoc correction, would make the infinite-time guarantee more robust to weight perturbations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces an ε-δ approximation criterion for infinite-horizon trajectory closeness: the volume of initial conditions for which the supremum error exceeds ε is less than δ. It claims universal approximation theorems for Neural ODEs for three target classes: (i) Morse-Smale systems with hyperbolic fixed points, (ii) Morse-Smale systems with hyperbolic limit cycles, using exact period matching by localized vector-field scaling, and (iii) normally hyperbolic continuous attractors, handled by a discretization/tiling strategy. It also proves a temporal generalization bound: if two flows are ε-δ close, then the time-averaged L^p error is at most ε^p + δD^p. The main proofs are in Appendices C–E; Appendix F reviews the literature.

Significance. If fully established, these results would be a substantial step beyond finite-time universal approximation and beyond the monostable/fading-memory infinite-time results. The ε-δ metric is well motivated for multistable systems, and Theorem 6 is a clean and useful bridge to training loss. The period-correction lemmas in Appendix D are largely coherent under the stated bump-function assumptions. However, the central claim for continuous attractors (Theorem 5) depends on an unproven Kupka-Smale-based construction, and the fixed-point theorem (Theorem 3) has a rigor gap in the valid-set argument. Both are fixable but currently prevent the paper from being accepted as a complete proof of its advertised claims.

major comments (2)
  1. [Appendix E, Theorem 5 Step 1] The proof asserts that by the Kupka-Smale Theorem there exists a C^1-arbitrarily-close proxy f† whose fixed points/limit cycles are hyperbolic and form an ε/3- or ε/8-net of the continuous attractor M, with alternating attractors and saddles, and with controlled stable manifolds. Kupka-Smale genericity gives hyperbolicity and transversality for generic perturbations, but it gives no control over the number, spacing, or sink/saddle alternation of the periodic orbits, nor a construction of a skeleton at prescribed mesh size. This assertion is load-bearing: without such a proxy, neither Theorem 3 nor Theorem 4 can be applied to f†, and the entire continuous-attractor claim collapses. The gap is a missing construction or transversality-counting argument, not a demonstrated falsehood, but it must be supplied.
  2. [Theorem 3, full proof Step 3 vs. Lemma 11] The full proof of Theorem 3 takes V = X \ (E_basin ∪ S) and claims that this set has uniform separation η' > 0 from the separatrices. Lemma 11, however, requires exactly inf_{x∈V} dist(x, ∂BoA(A)) ≥ η > 0. E_basin is the set of initial conditions whose basin assignment differs between f and f̂; it is contained in a tubular neighborhood of the separatrices but does not, in general, contain a full neighborhood of S. Thus V may contain points arbitrarily close to the separatrix but not in E_basin, and Lemma 11 cannot be applied as written. The proof can be repaired by defining V as the complement of an η'-neighborhood of S and using vol(N_η'(S)) → 0, but the current text needs to be corrected.
minor comments (4)
  1. [Throughout] There are numerous typographical issues: 'T o' for 'To', 'fail to to a true', inconsistent spacing, and duplicated words. These should be corrected.
  2. [Appendix D, Lemma 18] The proof of Lemma 18 states C^0 closeness of the approximating bump functions, but the hypothesis class has the C^1 UAP. The argument should use C^1 approximation to control the Jacobian off-diagonal bounds, not C^0 alone.
  3. [Remark after Theorem 3] The remark claims vol(E_basin) ≤ C_S ||f - f̂||_C1 without proof. This linear scaling is not used for the main theorem (Lemma 12 already suffices via continuity of measure), so it should be labeled as a heuristic or moved to limitations.
  4. [Appendix E, Case 2 Step 4] The tiling-error bound sup_t dist(γ_θ(t), γ̂_i(t)) ≤ ε/8 + L_H η_base is not justified solely by the ε/8-net property; for two same-period periodic orbits the same-time distance depends on phase alignment. The decomposition should either prove phase alignment or replace this step with a phase-aware estimate.

Circularity Check

0 steps flagged

No circular reduction found: Theorem 6 is a direct implication of the metric, self-citations are contextual, and the main proofs rely on external theorems. The continuous-attractor discretization assumption in Appendix E is an unproven assertion (a proof gap), not a circular step.

full rationale

I walked the main derivation chain (Theorems 3-6 and Appendices C-E). No load-bearing step reduces to the paper's own inputs by construction. Theorem 3's epsilon-delta guarantee is obtained by combining external structural stability (Palis-Smale), stable-manifold measure-zero separatrix control, and the C1 UAP; the target system is not used to define the approximant. Theorem 4's exact period matching is a genuine construction (bump-function scaling / adjoint first variation) with an external invertibility argument. Theorem 6 is definitional: once Def. 9 fixes the bad set to have volume < delta*vol(X), the Lp bound follows by partitioning X into good/bad sets; this is an implication, not a re-derivation of the theorem's conclusion from a fitted parameter. The self-citations ([131], [142], [143]) are contextual, supporting background remarks on continuous attractors and D-type error, not the epsilon-delta proofs. The most serious weakness is Appendix E Step 1 (and the Case 2 analogue): the paper asserts, from Kupka-Smale genericity, the existence of a C1-arbitrarily-close proxy whose hyperbolic fixed points/limit cycles form an epsilon/3- or epsilon/8-net with controlled basins. That assertion is not supplied and is stronger than the quoted Kupka-Smale theorem. This is a correctness gap that could sink Theorem 5, but it is not circularity: the proxy is assumed, not constructed from the conclusion, and no equation of the paper equates the theorem's input with its output. The manuscript's own limitations sections ('existence vs. learnability', 'fragility of period matching') also weaken practical scope but do not indicate circular reasoning.

Axiom & Free-Parameter Ledger

1 free parameters · 8 axioms · 0 invented entities

The proofs depend on standard dynamical-systems theorems plus two paper-specific strengthening assertions that are not fully proved: the linear basin-error scaling constant C_S and the existence of a C1-close proxy with a prescribed ε-net of hyperbolic skeleton. No data are fitted; constants such as λ, K, L_T are target-system properties, not free parameters.

free parameters (1)
  • C_S (basin-error scaling constant) = not estimated
    Section 4.2 Remark asserts vol(E_basin) ≤ C_S·||f−f̂||_C1 without proof and uses it to choose η_vol = δ/C_S; no bound or construction is provided.
axioms (8)
  • standard math Morse-Smale systems are structurally stable (Palis-Smale).
    Invoked in Theorem 3/4 Step 1 to convert C1 closeness into topological equivalence with a homeomorphism close to identity.
  • standard math Stable manifolds of saddles have measure zero and basin boundaries are finite unions of such manifolds.
    Used in Lemma 12 to conclude vol(E_basin) → 0 as the perturbation shrinks.
  • standard math Feedforward networks with smooth activations have the C1 universal approximation property.
    Definition 1 and Theorem 8 (Hornik); the whole hypothesis class depends on this.
  • standard math Fenichel persistence for normally hyperbolic invariant manifolds.
    Theorem 5 uses persistence of M under C1 perturbations with a Hausdorff-distance bound.
  • ad hoc to paper Kupka-Smale density implies existence of a C1-close proxy whose hyperbolic skeleton is an ε/3- or ε/8-net of the attractor.
    Appendix E Step 1: this is the paper's own unproved strengthening of Kupka-Smale; hyperbolicity genericity does not by itself give a prescribed dense skeleton.
  • ad hoc to paper Basin error volume scales linearly with the C1 perturbation: vol(E_basin) ≤ C_S·||f−f̂||_C1.
    Section 4.2 Remark Basin error scaling; stated without proof and used for the main error budget.
  • domain assumption The hypothesis class supports bump functions and scalar multiplication for limit-cycle corrections.
    Theorems 4 and 5 require localized period correction; not true of every neural ODE architecture but achievable with compact-support activations.
  • domain assumption Continuous attractors in F_CA with oscillatory structure are isochronous.
    Definition 13(iv) and the following Remark; without isochrony, the tiling strategy fails due to compounding phase drift.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees." pith.science (2026). https://pith.science/paper/66URBFPP

@misc{pith2026260208640,
  author       = {Pith},
  title        = {Pith review of: Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/66URBFPP}},
  note         = {Machine review of arXiv:2602.08640}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Universal approximation theorems establish the expressive capacity of neural network architectures. For dynamical systems, existing results are limited to finite time horizons or systems with a globally stable equilibrium, leaving multistability and limit cycles unaddressed. We prove that Neural ODEs achieve $\varepsilon$-$\delta$ closeness -- trajectories within error $\varepsilon$ except for initial conditions of measure $< \delta$ -- over the \emph{infinite} time horizon $[0,\infty)$ for three target classes: (1) Morse-Smale systems (a structurally stable class) with hyperbolic fixed points, (2) Morse-Smale systems with hyperbolic limit cycles via exact period matching, and (3) systems with normally hyperbolic continuous attractors via discretization. We further establish a temporal generalization bound: $\varepsilon$-$\delta$ closeness implies $L^p$ error $\leq \varepsilon^p + \delta \cdot D^p$ for all $t \geq 0$, bridging topological guarantees to training metrics. These results provide the first universal approximation framework for multistable infinite-horizon dynamics.

Figures

Figures reproduced from arXiv: 2602.08640 by Abel Sagodi, Il Memming Park.

Figure 1
Figure 1. Figure 1: illustrates the landscape of universal approximation results. Prior infinite￾time theories were limited to fading memory systems (a single global attractor). Our results extend to the full class of Morse-Smale systems and normally hyperbolic continuous attractors143 . non-hyperbolic hyperbolic multi-stability (structurally stable) oscillations marginally stable manifolds infinite-time horizon approximation… view at source ↗
Figure 3
Figure 3. Figure 3: The 1D bistable system (Example 1). (a) Vector field f(x) = x − x 3 with stable x± = ±1 (filled) and unstable x0 = 0 (open); phase line shows basins with per￾turbed separatrix (dashed orange). (b) Trajectories from various initial conditions; orange region marks ICs that switch basins under perturbation ˆf = f − ν cos(2x). (c) Time-averaged L 2 error vs. perturbation size ν: empirical error (black) stays b… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

188 extracted references · 3 canonical work pages

  1. [1]

    Universal approxima- tion of flows of control systems by recurrent neural networks.arXiv preprint arXiv:2304.00352, 2023

    Miguel Aguiar, Amritam Das, and Karl H Johansson. Universal approxima- tion of flows of control systems by recurrent neural networks.arXiv preprint arXiv:2304.00352, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Adapted Wasserstein distance between the laws of SDEs.arXiv preprint arXiv:2209.03243, 2022

    Julio Backhoff-Veraguas, Sigrid Källblad, and Benjamin A Robinson. Adapted Wasserstein distance between the laws of SDEs.arXiv preprint arXiv:2209.03243, 2022

    arXiv 2022

  3. [3]

    An empirical evaluation of generic convolutional and recurrent networks for sequence modeling.arXiv preprint arXiv:1803.01271, 2018

    Shaojie Bai. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling.arXiv preprint arXiv:1803.01271, 2018

    Pith/arXiv arXiv 2018

  4. [4]

    Deep equilibrium models

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equilibrium models. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  5. [5]

    Continuously differentiable exponential linear units

    Jonathan T Barron. Continuously differentiable exponential linear units. arXiv preprint arXiv:1704.07483, 2017

    Pith/arXiv arXiv 2017

  6. [6]

    Neural flows: Efficient alter- native to neural ODEs.Advances in neural information processing systems, 34: 21325–21337, 2021

    Marin Biloš, Johanna Sommer, Syama Sundar Rangapuram, Tim Januschowski, and Stephan Günnemann. Neural flows: Efficient alter- native to neural ODEs.Advances in neural information processing systems, 34: 21325–21337, 2021

    2021

  7. [7]

    Universal time-uniform trajectory approximation for ran- dom dynamical systems with recurrent neural networks.arXiv preprint arXiv:2211.08018, 2022

    Adrian N Bishop. Universal time-uniform trajectory approximation for ran- dom dynamical systems with recurrent neural networks.arXiv preprint arXiv:2211.08018, 2022

    Pith/arXiv arXiv 2022

  8. [8]

    Recurrent neural networks and univer- sal approximation of Bayesian filters

    Adrian N Bishop and Edwin V Bonilla. Recurrent neural networks and univer- sal approximation of Bayesian filters. InInternational Conference on Artificial Intelligence and Statistics, pages 6956–6967. PMLR, 2023

    2023

  9. [9]

    Recurrent neu- ral networks and (time-uniform) universal approximation of Bayesian filters

    Adrian N Bishop, Edwin V Bonilla, and Pierre Del Moral. Recurrent neu- ral networks and (time-uniform) universal approximation of Bayesian filters. IEEE Transactions on Automatic Control, 2026

    2026

  10. [10]

    Fading memory and the problem of approxi- mating nonlinear operators with Volterra series.IEEE Transactions on Circuits and Systems, 32(11):1150–1161, 1985

    Stephen Boyd and Leon Chua. Fading memory and the problem of approxi- mating nonlinear operators with Volterra series.IEEE Transactions on Circuits and Systems, 32(11):1150–1161, 1985

    1985

  11. [11]

    Universal computation and other capabilities of hybrid and continuous dynamical systems.Theoretical computer science, 138(1):67– 100, 1995

    Michael S Branicky. Universal computation and other capabilities of hybrid and continuous dynamical systems.Theoretical computer science, 138(1):67– 100, 1995

    1995

  12. [12]

    Modern Koopman theory for dynamical systems.arXiv preprint arXiv:2102.12086, 2021

    Steven L Brunton, Marko Budišić, Eurika Kaiser, and J Nathan Kutz. Modern Koopman theory for dynamical systems.arXiv preprint arXiv:2102.12086, 2021

    Pith/arXiv arXiv 2021

  13. [13]

    Explanatory models in neuroscience: Part 1– taking mechanistic abstraction seriously.arXiv preprint arXiv:2104.01490, 2021

    Rosa Cao and Daniel Y amins. Explanatory models in neuroscience: Part 1– taking mechanistic abstraction seriously.arXiv preprint arXiv:2104.01490, 2021

    Pith/arXiv arXiv 2021

  14. [14]

    Explanatory models in neuroscience: Part 2– constraint-based intelligibility.arXiv preprint arXiv:2104.01489, 2021

    Rosa Cao and Daniel Y amins. Explanatory models in neuroscience: Part 2– constraint-based intelligibility.arXiv preprint arXiv:2104.01489, 2021. 13 Preprint – Under Review

    Pith/arXiv arXiv 2021

  15. [15]

    Application de la théorie des équations intégrales linéaires aux systèmes d’équations différentielles non linéaires.Acta Mathe- matica, 59(1):63–87, 1932

    T orsten Carleman. Application de la théorie des équations intégrales linéaires aux systèmes d’équations différentielles non linéaires.Acta Mathe- matica, 59(1):63–87, 1932

    1932

  16. [16]

    AntisymmetricRNN: A dynamical system view on recurrent neural networks.arXiv preprint arXiv:1902.09689, 2019

    Bo Chang, Minmin Chen, Eldad Haber, and Ed H Chi. AntisymmetricRNN: A dynamical system view on recurrent neural networks.arXiv preprint arXiv:1902.09689, 2019

    Pith/arXiv arXiv 1902

  17. [17]

    F . C. Chen and H. K. Khalil. Adaptive control of nonlinear systems using neural networks.International Journal of Control, 55(6):1299–1317, 1992

    1992

  18. [18]

    Neural ordinary differential equations.Advances in neural information pro- cessing systems, 31, 2018

    Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations.Advances in neural information pro- cessing systems, 31, 2018

    2018

  19. [19]

    Neural networks for nonlinear dynamic system modelling and identification.International Journal of Control, 56(2): 319–346, 1992

    Sheng Chen and Stephen A Billings. Neural networks for nonlinear dynamic system modelling and identification.International Journal of Control, 56(2): 319–346, 1992

    1992

  20. [20]

    Chen and H

    T. Chen and H. Chen. Approximations of continuous functionals by neural networks with application to dynamic systems.IEEE Transactions on Neural Networks, 4(6):910–918, 1993. doi:10.1109/72.258503

    work page doi:10.1109/72.258503 1993

  21. [21]

    Chen and H

    T. Chen and H. Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dy- namical systems.IEEE Transactions on Neural Networks, 6(4):911–917, 1995. doi:10.1109/72.388949

    work page doi:10.1109/72.388949 1995

  22. [22]

    Recurrent neural networks are universal approximators with stochastic in- puts.IEEE Transactions on Neural Networks and Learning Systems, 2022

    Xiuqiong Chen, Y angtianze T ao, Wenjie Xu, and Stephen Shing-T oung Y au. Recurrent neural networks are universal approximators with stochastic in- puts.IEEE Transactions on Neural Networks and Learning Systems, 2022

    2022

  23. [23]

    Springer, New Y ork, NY, 2006

    Carmen Chicone.Ordinary Differential Equations with Applications. Springer, New Y ork, NY, 2006

    2006

  24. [24]

    A constructive approach for nonlinear system identification using multilayer perceptrons

    Ju-Y eop Choi, Hugh F Van Landingham, and Stanoje Bingulac. A constructive approach for nonlinear system identification using multilayer perceptrons. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 26 (2):307–312, 1996

    1996

  25. [25]

    T ommy WS Chow and Xiao-Dong Li. Modeling of continuous time dynam- ical systems with input by recurrent neural networks.IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 47(4):575–578, 2000

    2000

  26. [26]

    L Chua and D Green. A qualitative analysis of the behavior of dynamic non- linear networks: Steady-state solutions of nonautonomous networks.IEEE Transactions on Circuits and Systems, 23(9):530–550, 1976

    1976

  27. [27]

    Fast and ac- curate deep network learning by exponential linear units (elus)

    Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and ac- curate deep network learning by exponential linear units (elus). InProceed- ings of the 33rd International Conference on Machine Learning, pages 448–456. PMLR, 2016. 14 Preprint – Under Review

    2016

  28. [28]

    The construction of arbitrary stable dynamics in nonlinear neural networks.Neural Networks, 5(1):83–103, 1992

    Michael A Cohen. The construction of arbitrary stable dynamics in nonlinear neural networks.Neural Networks, 5(1):83–103, 1992

    1992

  29. [29]

    On the general theory of fading memory.Archive for Rational Mechanics and Analysis, 29:18–31, 1968

    Bernard D Coleman and Victor J Mizel. On the general theory of fading memory.Archive for Rational Mechanics and Analysis, 29:18–31, 1968

    1968

  30. [30]

    M. F . Danca and G. Chen. Approximation and decomposition of attractors of a hopfield neural network system.Chaos, Solitons & Fractals, 186:115213, 2024

    2024

  31. [31]

    Universal transformers.arXiv preprint arXiv:1807.03819, 2018

    Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers.arXiv preprint arXiv:1807.03819, 2018

    Pith/arXiv arXiv 2018

  32. [32]

    On the approximation of complicated dy- namical behavior.SIAM Journal on Numerical Analysis, 36(2):491–515, 1999

    Michael Dellnitz and Oliver Junge. On the approximation of complicated dy- namical behavior.SIAM Journal on Numerical Analysis, 36(2):491–515, 1999

    1999

  33. [33]

    eXponential FAmily dy- namical systems (XFADS): Large-scale nonlinear gaussian state-space model- ing

    Matthew Dowling, Yuan Zhao, and Il Memming Park. eXponential FAmily dy- namical systems (XFADS): Large-scale nonlinear gaussian state-space model- ing. InAdvances in Neural Information Processing Systems (NeurIPS), December

  34. [34]

    K. Doya. Universality of fully connected recurrent neural networks. T echni- cal Report 1, Dept. of Biology, UCSD, 1993

    1993

  35. [35]

    Acti- vation functions in deep learning: A comprehensive survey and benchmark

    Shiv Ram Dubey, Satish Kumar Singh, and Bidyut Baran Chaudhuri. Acti- vation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 2022

    2022

  36. [36]

    Survey of neural transfer func- tions.Neural computing surveys, 2(1):163–212, 1999

    Włodzisław Duch and Norbert Jankowski. Survey of neural transfer func- tions.Neural computing surveys, 2(1):163–212, 1999

    1999

  37. [37]

    Dudley.Real Analysis and Probability, volume 74 ofCambridge Studies in Advanced Mathematics

    Richard M. Dudley.Real Analysis and Probability, volume 74 ofCambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 2 edition, 2002

    2002

  38. [38]

    Reconstructing computational system dynamics from neural data with recurrent neural net- works.Nature Reviews Neuroscience, 24(11):693–710, 2023

    Daniel Durstewitz, Georgia Koppe, and Max Ingo Thurm. Reconstructing computational system dynamics from neural data with recurrent neural net- works.Nature Reviews Neuroscience, 24(11):693–710, 2023

    2023

  39. [39]

    Stochastic reservoir com- puters.Nature Communications, 16(1):1–11, 2025

    Peter J Ehlers, Hendra I Nurdin, and Daniel Soh. Stochastic reservoir com- puters.Nature Communications, 16(1):1–11, 2025

    2025

  40. [40]

    Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.Neural Networks, 107:3–11, 2018

    Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning.Neural Networks, 107:3–11, 2018

    2018

  41. [41]

    Springer, 2010

    Bard Ermentrout and David Hillel T erman.Mathematical foundations of neu- roscience, volume 35. Springer, 2010

    2010

  42. [42]

    Persistence and smoothness of invariant man- ifolds for flows.Indiana University Mathematics Journal, 21(3):193–226, 1971

    Neil Fenichel and JK Moser. Persistence and smoothness of invariant man- ifolds for flows.Indiana University Mathematics Journal, 21(3):193–226, 1971. 15 Preprint – Under Review

    1971

  43. [43]

    Vers une approche algébrique des systèmes non linéaires en temps discret

    Michel Fliess. Vers une approche algébrique des systèmes non linéaires en temps discret. In A. Bensoussan and J. L. Lions, editors,Analysis and Optimization of Systems, pages 594–603, Berlin, Heidelberg, 1980. Springer Berlin Heidelberg. ISBN 978-3-540-38489-2

    1980

  44. [44]

    Explicit error bounds for Carleman lin- earization.arXiv preprint arXiv:1711.02552, 2017

    Marcelo Forets and Amaury Pouly. Explicit error bounds for Carleman lin- earization.arXiv preprint arXiv:1711.02552, 2017

    Pith/arXiv arXiv 2017

  45. [45]

    Franz and Bernhard Schölkopf

    Marcel O. Franz and Bernhard Schölkopf. A unifying view of Wiener and Volterra theory and polynomial kernel regression.Neural Computation, 18 (12):3097–3118, 2006

    2006

  46. [46]

    On the approximate realization of continuous mappings by neural networks.Neural Networks, 2(3):183–192, 1989

    Ken-Ichi Funahashi. On the approximate realization of continuous mappings by neural networks.Neural Networks, 2(3):183–192, 1989

    1989

  47. [47]

    Approximation of dynamical sys- tems by continuous time recurrent neural networks.Neural Networks, 6(6): 801–806, 1993

    Ken-ichi Funahashi and Yuichi Nakamura. Approximation of dynamical sys- tems by continuous time recurrent neural networks.Neural Networks, 6(6): 801–806, 1993

    1993

  48. [48]

    Springer Science & Business Media, 2012

    Freddy Rafael Garces, Victor Manuel Becerra, Chandrasekhar Kambhampati, and Kevin Warwick.Strategies for feedback linearisation: A dynamic neural network approach. Springer Science & Business Media, 2012

    2012

  49. [49]

    Hierarchical control system design using approximate simulation.Automatica, 45(2):566–571, 2009

    Antoine Girard and George J Pappas. Hierarchical control system design using approximate simulation.Automatica, 45(2):566–571, 2009

    2009

  50. [50]

    Approximate simu- lation relations for hybrid systems.Discrete event dynamic systems, 18(2): 163–179, 2008

    Antoine Girard, A Agung Julius, and George J Pappas. Approximate simu- lation relations for hybrid systems.Discrete event dynamic systems, 18(2): 163–179, 2008

    2008

  51. [51]

    Reservoir computing universality with stochastic inputs.IEEE Transactions on Neural Networks and Learning Systems, 31(1):100–112, 2019

    Lukas Gonon and Juan-Pablo Ortega. Reservoir computing universality with stochastic inputs.IEEE Transactions on Neural Networks and Learning Systems, 31(1):100–112, 2019

    2019

  52. [52]

    Fading memory echo state networks are universal.Neural Networks, 138:10–13, 2021

    Lukas Gonon and Juan-Pablo Ortega. Fading memory echo state networks are universal.Neural Networks, 138:10–13, 2021

    2021

  53. [53]

    Risk bounds for reservoir computing.Journal of Machine Learning Research, 21(240):1–61, 2020

    Lukas Gonon, Lyudmila Grigoryeva, and Juan-Pablo Ortega. Risk bounds for reservoir computing.Journal of Machine Learning Research, 21(240):1–61, 2020

    2020

  54. [54]

    Approximation bounds for random neural networks and reservoir systems.The Annals of Applied Probability, 33(1):28–69, 2023

    Lukas Gonon, Lyudmila Grigoryeva, and Juan-Pablo Ortega. Approximation bounds for random neural networks and reservoir systems.The Annals of Applied Probability, 33(1):28–69, 2023

    2023

  55. [55]

    Echo state networks are uni- versal.Neural Networks, 108:495–508, 2018

    Lyudmila Grigoryeva and Juan-Pablo Ortega. Echo state networks are uni- versal.Neural Networks, 108:495–508, 2018

    2018

  56. [56]

    Lyudmila Grigoryeva and Juan-Pablo Ortega. Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non- homogeneous state-affine systems.Journal of Machine Learning Research, 19(24):1–40, 2018. 16 Preprint – Under Review

    2018

  57. [57]

    Springer, New Y ork, 1983

    John Guckenheimer and Philip Holmes.Nonlinear Oscillations, Dynamical Sys- tems, and Bifurcations of Vector Fields, volume 42 ofApplied Mathematical Sciences. Springer, New Y ork, 1983

    1983

  58. [58]

    Structurally stable heteroclinic cy- cles

    John Guckenheimer and Philip Holmes. Structurally stable heteroclinic cy- cles. InMathematical Proceedings of the Cambridge Philosophical Society, vol- ume 103, pages 189–192. Cambridge University Press, 1988

    1988

  59. [59]

    Neural ordinary differential equa- tion based recurrent neural network model

    Mansura Habiba and Barak A Pearlmutter. Neural ordinary differential equa- tion based recurrent neural network model. In2020 31st Irish signals and systems conference (ISSC), pages 1–6. IEEE, 2020

    2020

  60. [60]

    On the approximation capability of recurrent neural net- works.Neurocomputing, 31(1-4):107–123, 2000

    Barbara Hammer. On the approximation capability of recurrent neural net- works.Neurocomputing, 31(1-4):107–123, 2000

    2000

  61. [61]

    Universal simulation of stable dynam- ical systems by recurrent neural nets

    Joshua Hanson and Maxim Raginsky. Universal simulation of stable dynam- ical systems by recurrent neural nets. InLearning for Dynamics and Control, pages 384–392. PMLR, 2020

    2020

  62. [62]

    Embedding and approxi- mation theorems for echo state networks.Neural Networks, 128:234–247, 2020

    Allen Hart, James Hook, and Jonathan Dawes. Embedding and approxi- mation theorems for echo state networks.Neural Networks, 128:234–247, 2020

    2020

  63. [63]

    Joseph D Hart. Attractor reconstruction with reservoir computers: The ef- fect of the reservoir’s conditional Lyapunov exponents on faithful attractor reconstruction.Chaos: An Interdisciplinary Journal of Nonlinear Science, 34(4), 2024

    2024

  64. [64]

    T alu: A hybrid activation function combining tanh and rectified linear unit to enhance neural networks.arXiv preprint arXiv:2305.04402, 2023

    Md Mehedi Hasan, Md Ali Hossain, Azmain Y akin Srizon, and Abu Sayeed. T alu: A hybrid activation function combining tanh and rectified linear unit to enhance neural networks.arXiv preprint arXiv:2305.04402, 2023

    Pith/arXiv arXiv 2023

  65. [65]

    Liquid time-constant recurrent neural networks as universal approx- imators.arXiv preprint arXiv:1811.00321, 2018

    Ramin M Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, and Radu Grosu. Liquid time-constant recurrent neural networks as universal approx- imators.arXiv preprint arXiv:1811.00321, 2018

    Pith/arXiv arXiv 2018

  66. [66]

    Connecting invariant manifolds and the solution of theC 1 stability andΩ-stability conjectures for flows.Annals of mathematics, pages 81–137, 1997

    Shuhei Hayashi. Connecting invariant manifolds and the solution of theC 1 stability andΩ-stability conjectures for flows.Annals of mathematics, pages 81–137, 1997

    1997

  67. [67]

    On the impact of the activation function on deep neural networks training

    Soufiane Hayou, Arnaud Doucet, and Judith Rousseau. On the impact of the activation function on deep neural networks training. InInternational conference on machine learning, pages 2672–2680. PMLR, 2019

    2019

  68. [68]

    F . Hess, Z. Monfared, M. Brenner, and D. Durstewitz. Generalized teacher forcing for learning chaotic dynamics.arXiv preprint arXiv:2306.04406, 2023

    Pith/arXiv arXiv 2023

  69. [69]

    Computing with dynamic attractors in neural networks.Biosystems, 34(1-3):173–195, 1995

    Morris W Hirsch and Bill Baird. Computing with dynamic attractors in neural networks.Biosystems, 34(1-3):173–195, 1995

    1995

  70. [70]

    Springer Science & Business Media, 2013

    Frank C Hoppensteadt.Analysis and simulation of chaotic systems, volume 94. Springer Science & Business Media, 2013. 17 Preprint – Under Review

    2013

  71. [71]

    K. Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2):251–257, 1991

    1991

  72. [72]

    A proof ofC 1 stability conjecture for three-dimensional flows.Trans- actions of the American Mathematical Society, 342(2):753–772, 1994

    Sen Hu. A proof ofC 1 stability conjecture for three-dimensional flows.Trans- actions of the American Mathematical Society, 342(2):753–772, 1994

    1994

  73. [73]

    Neural autoregressive flows

    Chin-Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive flows. InInternational conference on machine learning, pages 2078–2087. PMLR, 2018

    2078

  74. [74]

    Upper ap- proximation bounds for neural oscillators.arXiv preprint arXiv:2512.01015, 2025

    Zifeng Huang, Konstantin M Zuev, Y ong Xia, and Michael Beer. Upper ap- proximation bounds for neural oscillators.arXiv preprint arXiv:2512.01015, 2025

    Pith/arXiv arXiv 2025

  75. [75]

    Kernel modelling of fading memory systems.arXiv preprint arXiv:2403.11945, 2024

    Y ongkang Huo, Thomas Chaffey, and Rodolphe Sepulchre. Kernel modelling of fading memory systems.arXiv preprint arXiv:2403.11945, 2024

    arXiv 2024

  76. [76]

    On the number of limit cycles in asymmetric neural networks.Journal of Statistical Mechanics: Theory and Experiment, 2019(5): 053402, 2019

    Sungmin Hwang, Viola Folli, Enrico Lanza, Giorgio Parisi, Giancarlo Ruocco, and Francesco Zamponi. On the number of limit cycles in asymmetric neural networks.Journal of Statistical Mechanics: Theory and Experiment, 2019(5): 053402, 2019

    2019

  77. [77]

    On the number of limit cycles in diluted neural networks.Journal of Statistical Physics, 181:2304–2321, 2020

    Sungmin Hwang, Enrico Lanza, Giorgio Parisi, Jacopo Rocchi, Giancarlo Ruocco, and Francesco Zamponi. On the number of limit cycles in diluted neural networks.Journal of Statistical Physics, 181:2304–2321, 2020

    2020

  78. [78]

    Optimal simulation of automata by neural nets

    Piotr Indyk. Optimal simulation of automata by neural nets. InAnnual Sym- posium on Theoretical Aspects of Computer Science, pages 337–348, Berlin, Heidelberg, March 1995. Springer Berlin Heidelberg

    1995

  79. [79]

    Universal approximation property of invertible neu- ral networks.Journal of Machine Learning Research, 24(287):1–68, 2023

    Isao Ishikawa, T akeshi T eshima, Koichi T ojo, Kenta Oono, Masahiro Ikeda, and Masashi Sugiyama. Universal approximation property of invertible neu- ral networks.Journal of Machine Learning Research, 24(287):1–68, 2023

    2023

  80. [80]

    echo state

    Herbert Jaeger. The “echo state” approach to analysing and training recur- rent neural networks-with an erratum note.Bonn, Germany: German National Research Center for Information T echnology GMD T echnical Report, 148(34):13, 2001

    2001

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.