Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Chaos Meets Attention: Transformers for Large-Scale Dynamical Prediction

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A softly unitary forward operator lets a transformer preserve long-term statistics of chaotic flows without matching distributions.

desk verdict Useful benchmark and a plausible architecture, but the advertised von Neumann guarantee does not follow from the stated loss — the paper needs a corrected theory before its main claim can be taken at face value. read the letter →

arxiv 2504.20858 v2 pith:FOXN3FQH submitted 2025-04-29 nlin.CD

Yi He , Yiming Yang , Xiaoyuan Cheng , Hai Wang , Xiao Xue , Boli Chen , Yukun Hu This is my paper · ORCID
classification nlin.CD
keywords chaoticdynamicsergodicityKoopmanoperatorunitaryconstraintturbulentchannelflowKolmogorovtransformerrandomFourierfeatures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a transformer can generate long, stable trajectories of high-dimensional chaotic flows if its learned forward operator is pushed toward unitarity, because the von Neumann mean ergodic theorem says unitary operators on $L^2$ preserve the time averages that define a chaotic attractor's statistics. Instead of matching output distributions to the true attractor with optimal transport, which is intractable at high dimension, the authors add a cheap stochastic-trace penalty that drives $G^T G$ toward the identity in feature space. They also redesign attention as factorized axial mean-max-min (A3M) pooling with random Fourier positional encoding, cutting cost from the product to the sum of the grid dimensions while keeping extreme events. On $256\times256$ Kolmogorov flow and on a new $192\times192\times3$ turbulent-channel-flow dataset, the model reports lower relative $L^2$ error at short horizons and better energy-spectrum, mixing-rate, and KL-divergence agreement over long rollouts than MNO, UNO, MWT, and FactFormer. If right, this makes ergodicity preservation practical for large-scale chaotic systems without distribution matching.

What carries the argument

The load-bearing identity is $G^*G = I_d$: a unitary forward operator $G$ on the $L^2$ feature space preserves norms and inner products, so by the von Neumann mean ergodic theorem time averages of observables converge to their spatial averages, which is exactly the ergodicity the model needs for long rollouts. The paper encodes this as a soft loss $\mathcal{L}_{\mathrm{unitary}}(\hat G) = |\mathbb{E}_v[v^T \hat G^T \hat G v] - 1|$, estimated with a stochastic trace estimator at $O(k d^2)$ cost rather than the $O(d^3)$ Frobenius penalty. The second carrying mechanism is A3M attention, meaning factorized axial attention with separate mean, max, and min pooling heads, plus random Fourier positional encoding; this approximates a Gaussian spatial kernel and cuts attention cost from a product of grid dimensions to their sum while preserving bulk statistics and extreme values.

What would settle it

Track the singular values of the learned $G$ and extend rollouts to 5,000 steps on the test attractor: if $G$ stays nearly unitary while the energy-spectrum or KL-divergence errors grow with rollout length, or if the errors stay flat while $G$ is clearly non-unitary, then the unitary loss is not the operative mechanism preserving the statistics.

Watch

Extended reading notes

Core claim

The central claim is that long-term statistical fidelity in learned chaotic dynamics reduces to a unitary-operator constraint: if the learned feature-evolution operator $G$ is (nearly) unitary, then the von Neumann mean ergodic theorem guarantees that time averages of observables equal phase-space averages, so an autoregressive model rolling $G$ forward will generate trajectories whose invariant statistics match the true attractor. The concrete discovery is that this constraint can be imposed cheaply and effectively inside a transformer: a soft loss $\mathcal{L}_{\mathrm{unitary}}(\hat G) = |\mathbb{E}_v[v^T \hat G^T \hat G v] - 1|$, estimated by stochastic trace sampling, combined with A3M factorized attention, produces lower relative $L^2$ error at short horizons and better energy-spectrum, mixing-rate, and KL-divergence agreement over long rollouts than MNO, UNO, MWT, and FactFormer on Kolmogorov flow at $256\times256$ and on the newly introduced turbulent-channel-flow dataset at $192\times192$ with three velocity channels.

Load-bearing premise

The load-bearing premise is that the learned feature space faithfully represents the physical system's observables, so that forcing the learned operator $G$ to be unitary in that space actually preserves the physical invariant measure; the paper also assumes, without establishing, that turbulent channel flow is ergodic on its attractor.

Editorial extensions

If this is right

  • On the two benchmarks tested, the model out-scores MNO, UNO, MWT, and FactFormer on all five reported metrics, including the energy-spectrum errors ME-APE and ME-LRw, mixing-rate error $\Delta\lambda$, and KL divergence on principal components.
  • The stochastic-trace unitary loss replaces distribution matching, so the framework scales to states with more than $10^5$ grid points, where optimal-transport regularizers become intractable.
  • A3M factorized attention captures both bulk statistics and local extreme events at a cost that grows with the sum of grid dimensions rather than their product, making high-resolution chaotic flow tractable for transformers.
  • The new turbulent-channel-flow benchmark (140k snapshots from 3D lattice-Boltzmann simulations at $Re_\tau=180$) gives the machine-learning community a public high-dimensional ergodic testbed with both short- and long-term metrics.
  • The method is limited to ergodic chaotic systems on uniform grids; non-ergodic dynamics and unstructured meshes are outside the stated scope.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the unitary penalty should also improve long-term statistics if attached to other operator-learning backbones (for example a Fourier or convolutional operator); the paper only demonstrates it inside the full A3M transformer, so the standalone contribution of the loss has not been isolated.
  • A direct test of the theory is to watch the singular-value spectrum of the learned $G$ during rollout: if the spectrum drifts from 1 while the long-term metrics hold, the guarantee is doing less work than claimed, and if the metrics degrade while the spectrum stays unitary, the feature space is not faithfully representing the physical observables.
  • Because unitarity preserves norms exactly, we expect the method to be strongest for second-order statistics such as energy spectra and weaker for higher-order moments; checking velocity structure functions or extreme-event tails would reveal the boundary of the guarantee.
  • The dependence of the random-Fourier bandwidth $\sigma$ on flow scale suggests that a rollout schedule adjusting $\sigma$ as the flow develops could further stabilize statistics on multi-scale turbulence, an extension the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes a transformer architecture for autoregressive long-term prediction of high-dimensional dissipative chaotic systems. It introduces A3M factorized attention with mean/max/min axial pooling and random Fourier positional encoding, and a forward operator G regularized by a 'unitary' loss derived from the von Neumann ergodic theorem. The authors contribute two datasets (a 256×256 Kolmogorov flow dataset and a 140k-snapshot turbulent channel flow dataset) and evaluate against MNO, UNO, MWT, and FactFormer on relative L2 error, ME-APE, ME-LRw, Δλ, and KLD, reporting superior performance on both systems.

Significance. If the results hold, the empirical contribution is valuable: the gains are consistent across two turbulent systems and several metrics, the baselines use official implementations, and the new public datasets are likely useful for the community. The A3M attention and the scalability analysis are meaningful engineering contributions. However, the advertised theoretical guarantee—that the von Neumann ergodic theorem 'ensures' preservation of long-term statistics via the unitary regularizer—is not established as written, so the central claim needs repair rather than being acceptable in its current form.

major comments (4)
  1. [§3.3, Eq. (9)] Equation (9) as displayed is not a unitarity constraint. For v uniform on the unit sphere, E[v^T G^T G v] = tr(G^T G)/d, so Lunitary = |tr(G^T G)/d - 1|. A matrix such as diag(√2,√2,0,0) in d=4 has zero penalty even though G^T G ≠ I and det G = 0. The statement that the regularizer 'constrains Ĝ within SO(d)' is therefore not implied by the displayed loss. Algorithm 1 instead computes (1/B)Σ|q_b - 1|, which estimates E[|v^T G^T G v - 1|], a different and stronger objective; no proof is given that minimizing this penalty drives G to orthogonality. Please reformulate the loss so that it matches Algorithm 1, or provide a supporting argument that the soft penalty indeed yields a nearly unitary operator, or present the term explicitly as a heuristic inductive bias.
  2. [§3.3 and Theorem 2.2] The inference from Eq. (2) and Eq. (9) to 'preservation of long-term statistics' requires G to be the Koopman operator of T acting on L^2(M, μ), unitary with respect to the invariant-measure inner product. The manuscript does not show that the learned latent feature map φ is a Koopman embedding or that the latent Euclidean inner product corresponds to the L^2(M, μ) inner product. Without such a conjugacy, an exactly orthogonal G in latent space does not by itself imply preservation of the physical invariant measure. The von Neumann-based guarantee is therefore not derived; it should either be supported with additional arguments or explicitly softened to an inspiration/inductive bias.
  3. [§4, Tables 1 and 2] The experimental results are reported as means over three seeds without standard deviations or significance tests. Several claimed advantages are small (for example, 1.55% for KF256 at τ=25 and 7.14% for TCF at τ=25), so the 'consistently outperform' claim is not yet fully supported. Please include per-seed spreads or confidence intervals, and state whether the baselines were run on the same seeds as the proposed model.
  4. [§4.1 and §4.2] The paper motivates the approach by arguing that optimal-transport-based methods (Jiang et al., 2024; Schiff et al., 2024) suffer from the curse of dimensionality, but neither method is benchmarked. To support the claim that the proposed method is more scalable and more accurate than OT-based state-of-the-art, either include comparisons on a common benchmark (even at reduced resolution) or explicitly acknowledge that such comparisons are absent and discuss the computational constraints behind that choice.
minor comments (4)
  1. [Eq. (4) and Appendix B] For B ~ N(0, σ^2), the characteristic function gives E[cos(2π b Δ)] = exp(-2π^2 σ^2 ||Δ||^2), not exp(-πσ^2||Δ||^2/2) as written in Eq. (4) and Eq. (12). This affects the stated Gaussian kernel and the interpretation of the bandwidth σ; please correct the formula or redefine the sampling distribution accordingly.
  2. [Theorem 2.2] The notation uses φ for both the observable and its Cesàro limit. Please use, for example, ar{φ} = lim (1/N) Σ G^k φ and state the ergodic case as ar{φ} = ∫ φ dμ, to avoid the current ambiguity.
  3. [Appendix A, Table of Notations] The notation table lists μ as the Lebesgue measure, while the text treats μ as the invariant/ergodic measure on the attractor. These definitions should be reconciled.
  4. [§4.3, Table 4] The qualitative statements about low- and high-frequency spectral performance for different σ would be easier to interpret if the text specified whether ME-APE is averaged over all wavenumbers or over a particular band, and if a quantitative proxy for the attention-map observations in Figure 8 were added.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: long-term statistics are held-out evaluation metrics, not training targets; the self-citations are comparative and non-load-bearing.

full rationale

The paper's claimed derivation chain is not circular. The training objective (Eq. 8) is a one-step feature-space prediction error plus a regularizer; the long-term statistics reported in Tables 1 and 2 (ME-APE, ME-LRw, Δλ, KLD) are computed on test rollouts and are not terms being minimized, so the empirical 'predictions' are not fitted inputs renamed as results. The von Neumann theorem (Neumann, 1932) is standard external mathematics; invoking it to motivate a unitary constraint on the learned forward operator is an inductive-bias argument, not a self-referential derivation. The only self-citations (Cheng et al., 2025 for regularizer comparison; He 2025 and He et al. 2025 for dataset DOIs) are not load-bearing: the central loss and architecture are introduced and derived in this paper, and the cited prior work is not used to forbid alternatives or supply a uniqueness theorem. The stated limitations (ergodic-only systems, uniform-grid dependence) are scope restrictions, not admissions of circularity. There are genuine derivation gaps that belong under correctness risk rather than circularity: Eq. (9) as written equals |tr(G^T G)/d - 1| for uniform unit vectors, i.e., a trace constraint rather than a unitarity constraint, and Algorithm 1 estimates a different expectation; moreover, exact Euclidean unitarity of a latent G does not by itself imply preservation of the physical invariant measure μ. These gaps mean the advertised guarantee is under-derived, but they do not make the paper's results equivalent to its inputs by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method rests on standard Koopman and ergodic theory plus the unproven correspondence between the latent operator and the physical Koopman operator. Two hyperparameters are tuned or unspecified; no new physical entities are introduced.

free parameters (2)
  • kernel bandwidth σ = 8 (for all experiments, chosen from {1,4,8,16,32} via KF128 ablation)
    Controls the decay of the RFF Gaussian kernel; selected to optimize short- and long-term metrics in Table 4, so the reported long-term statistics reflect a tuned hyperparameter.
  • unitary regularization weight λ = not reported
    Coefficient on L_unitary in Equation (8); the paper states λ∈(0,1] but never gives the value used in Tables 1-4, so the reported results are not fully reproducible.
assumptions (4)
  • standard math The Koopman operator of a measure-preserving transformation is unitary on L2.
    Used in Section 3.3 to justify constraining the learned operator to SO(d). This is standard ergodic theory.
  • ad hoc to paper The learned feature-space operator G accurately represents the Koopman operator of the physical chaotic system on its attractor.
    Assumed in Section 3.3; without it, a unitary constraint on G does not imply preservation of the physical invariant measure.
  • domain assumption The considered turbulent systems are ergodic on their attractors.
    Stated in Section 1 and Definition 2.1; ergodicity of turbulent channel flow at Re_tau=180 is assumed but not proven.
  • standard math The random Fourier feature approximation converges sufficiently fast for the chosen m.
    Used in Equations (4)-(5); convergence is asymptotic in m, but the paper does not report m or the resulting approximation error.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chaos Meets Attention: Transformers for Large-Scale Dynamical Prediction." pith.science (2026). https://pith.science/paper/FOXN3FQH

@misc{pith2026250420858,
  author       = {Pith},
  title        = {Pith review of: Chaos Meets Attention: Transformers for Large-Scale Dynamical Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FOXN3FQH}},
  note         = {Machine review of arXiv:2504.20858}
}
abstract

Generating long-term trajectories of dissipative chaotic systems autoregressively is a highly challenging task. The inherent positive Lyapunov exponents amplify prediction errors over time. Many chaotic systems possess a crucial property - ergodicity on their attractors, which makes long-term prediction possible. State-of-the-art methods address ergodicity by preserving statistical properties using optimal transport techniques. However, these methods face scalability challenges due to the curse of dimensionality when matching distributions. To overcome this bottleneck, we propose a scalable transformer-based framework capable of stably generating long-term high-dimensional and high-resolution chaotic dynamics while preserving ergodicity. Our method is grounded in a physical perspective, revisiting the Von Neumann mean ergodic theorem to ensure the preservation of long-term statistics in the $\mathcal{L}^2$ space. We introduce novel modifications to the attention mechanism, making the transformer architecture well-suited for learning large-scale chaotic systems. Compared to operator-based and transformer-based methods, our model achieves better performances across five metrics, from short-term prediction accuracy to long-term statistics. In addition to our methodological contributions, we introduce a new chaotic system benchmark: a machine learning dataset of 140$k$ snapshots of turbulent channel flow along with various evaluation metrics for both short- and long-term performances, which is well-suited for machine learning research on chaotic systems.

Figures

Figures reproduced from arXiv: 2504.20858 by the authors.

Figure 1
Figure 1. The framework of ChaosMeetsAttention [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Time correlation of our long-term predictions of TCF and the baselines [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. TKE error of long-term predictions of TCF. The left-most snapshot indicates the TKE of ground truth trajectories; and the rest snapshots present the absolute percentage error of TKE in 1000-step rollout predictions from baselines and our approach. mance using three distinct metrics: (1) the mean energy ab￾solute percentage error (ME-APE), which quantifies the ab￾solute deviation of predicted energy in each frequency… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Absolute error of short-term KF256 predictions of baselines and ours [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 5
Figure 5. Figure 5: Time correlation of our long-term predictions of KF256 and the baselines. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: TKE error of long-term predictions of KF256. The left-most snapshot indicates the TKE of ground truth trajectories; and the rest snapshots present the absolute percentage error of TKE in 1000-step rollout predictions from baselines and our approach. F.2. Turbulent chan…
Figure 7
Figure 7. Figure 7: Absolute error of short-term TCF predictions of baselines and ours. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Impact of applying random Fourier positional encoding on the attention maps with respect to the kernel bandwidth σ ∈ {1, 4, 8, 16, 32}. From the attention maps of the first and the last block of our implementation, we observed that a small σ ∈ {1, 4} produced overly sm…
Figure 9
Figure 9. Figure 9: Turbulent Channel Flow dataset for illustration 25 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Deep Delta Learning

    cs.LG 2026-01 unverdicted novelty 7.0 of 10

    Replacing additive residual connections with a gated rank-1 delta update that interpolates identity, projection, and reflection slightly improves language modeling and downstream averages in reported 124M/353M runs.

  2. A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series

    cs.LG 2025-11 unverdicted novelty 6.0 of 10

    The Weak Penalty Neural ODE uses a weak form loss to filter noise and learn stable chaotic dynamics from noisy observations.

Reference graph

Works this paper leans on

84 extracted references · 64 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    A., Brenner, M., Wang, Q., and Hoyer, S

    Alieva, A., Kochkov, D., Smith, J. A., Brenner, M., Wang, Q., and Hoyer, S. Machine learning accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences USA, 2021

    2021

  3. [3]

    The metamathematics of ergodic theory

    Avigad, J. The metamathematics of ergodic theory. Annals of Pure and Applied Logic, 157 0 (2-3): 0 64--76, 2009

    2009

  4. [4]

    Lagrangian statistics in fully developed turbulence

    Biferale, L., Boffetta, G., Celani, A., Lanotte, A., and Toschi, F. Lagrangian statistics in fully developed turbulence. Journal of Turbulence, 0 (7): 0 N6, 2006

    2006

  5. [5]

    Bochner, S. et al. Lectures on Fourier integrals, volume 42. Princeton University Press, 1959

    work page 1959

  6. [6]

    and Herrmann, D

    Bousquet, O. and Herrmann, D. On the complexity of learning the kernel matrix. Advances in neural information processing systems, 15, 2002

    work page 2002

  7. [7]

    L., Brunton, B

    Brunton, S. L., Brunton, B. W., Proctor, J. L., Kaiser, E., and Kutz, J. N. Chaos as an intermittently forced linear system. Nature communications, 8 0 (1): 0 19, 2017

    work page 2017

  8. [8]

    L., Budi s i \'c , M., Kaiser, E., and Kutz, J

    Brunton, S. L., Budi s i \'c , M., Kaiser, E., and Kutz, J. N. Modern koopman theory for dynamical systems. arXiv preprint arXiv:2102.12086, 2021

    arXiv 2021

Show all 84 references
  1. [9]

    Choose a transformer: Fourier or galerkin

    Cao, S. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems, 34: 0 24924--24940, 2021

    2021

  2. [10]

    Learning chaos in a linear way

    Cheng, X., He, Y., Yang, Y., Xue, X., Cheng, S., Guiles, D., and Hu, Y. Learning chaos in a linear way. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=Llh6CinTiy

    2025

  3. [11]

    Kinetic energy represented in terms of moments of vorticity and applications

    Cie \'s lak, T., Oleszkiewicz, K., Preisner, M., and Szuma \'n ska, M. Kinetic energy represented in terms of moments of vorticity and applications. Journal of Mathematical Fluid Mechanics, 21 0 (4): 0 53, 2019

    2019

  4. [12]

    P., Fomin, S

    Cornfeld, I. P., Fomin, S. V., and Sinai, Y. G. Ergodic theory, volume 245. Springer Science & Business Media, 2012

    2012

  5. [13]

    Lie group valued koopman eigenfunctions

    Das, S. Lie group valued koopman eigenfunctions. Nonlinearity, 36 0 (5): 0 2149, 2023

    2023

  6. [14]

    A., Brenner, M

    Dresdner, G., Kochkov, D., Norgaard, P., Zepeda-Núñez, L., Smith, J. A., Brenner, M. P., and Hoyer, S. Learning to correct spectral methods for simulating turbulent flows. 2022. doi:10.48550/ARXIV.2207.00556. URL https://arxiv.org/abs/2207.00556

    2022 doi arXiv

  7. [15]

    V., Taskiran, M., Taskiran, Z

    Dudukcu, H. V., Taskiran, M., Taskiran, Z. G. C., and Yildirim, T. Temporal convolutional networks with rnn approach for chaotic time series prediction. Applied soft computing, 133: 0 109945, 2023

    2023

  8. [16]

    and Ruelle, D

    Eckmann, J.-P. and Ruelle, D. Ergodic theory of chaos and strange attractors. Reviews of modern physics, 57 0 (3): 0 617, 1985

    1985

  9. [17]

    and Sapsis, T

    Farazmand, M. and Sapsis, T. P. A variational approach to probing extreme events in turbulent dynamical systems. Science advances, 3 0 (9): 0 e1701533, 2017

    2017

  10. [18]

    and Tsinober, A

    Galanti, B. and Tsinober, A. Is turbulence ergodic? Physics Letters A, 330 0 (3-4): 0 173--180, 2004

    2004

  11. [19]

    Chaos as an interpretable benchmark for forecasting and data-driven modelling

    Gilpin, W. Chaos as an interpretable benchmark for forecasting and data-driven modelling. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreview.net/forum?id=enYjtbjYJrf

    2021

  12. [20]

    Golub, G. H. and Van Loan, C. F. Matrix computations. JHU press, 2013

    2013

  13. [21]

    Efficient token mixing for transformers via adaptive fourier neural operators

    Guibas, J., Mardani, M., Li, Z., Tao, A., Anandkumar, A., and Catanzaro, B. Efficient token mixing for transformers via adaptive fourier neural operators. In International Conference on Learning Representations, 2021

    2021

  14. [22]

    Multiwavelet-based operator learning for differential equations

    Gupta, G., Xiao, X., and Bogdan, P. Multiwavelet-based operator learning for differential equations. Advances in neural information processing systems, 34: 0 24048--24062, 2021

    2021

  15. [23]

    and Mattingly, J

    Hairer, M. and Mattingly, J. C. Ergodicity of the 2d navier-stokes equations with degenerate stochastic forcing. Annals of Mathematics, pp.\ 993--1032, 2006

    2006

  16. [24]

    A survey on vision transformer

    Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., et al. A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45 0 (1): 0 87--110, 2022

    2022

  17. [25]

    Gnot: A general neural operator transformer for operator learning

    Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pp.\ 12556--12569. PMLR, 2023

    2023

  18. [26]

    Benchmark dataset kf256 for chaos meets attention: Transformers for large-scale dynamical prediction, may 2025

    He, Y. Benchmark dataset kf256 for chaos meets attention: Transformers for large-scale dynamical prediction, may 2025. URL https://doi.org/10.5281/zenodo.14801580

    2025 doi

  19. [27]

    Benchmark dataset turbulent channel flow for chaos meets attention: Transformers for large-scale dynamical prediction, May 2025

    He, Y., Xue, X., Hu, Y., Yang, Y., Cheng, X., and Wang, H. Benchmark dataset turbulent channel flow for chaos meets attention: Transformers for large-scale dynamical prediction, May 2025. URL https://rdr.ucl.ac.uk/articles/dataset/Benchmark_dataset_Turbulent_Channel_Flow_for_C...

    2025

  20. [28]

    Holden, A. V. Chaos, volume 461. Princeton University Press, 2014

    2014

  21. [29]

    Sparse learning of dynamical systems in rkhs: An operator-theoretic approach

    Hou, B., Sanjari, S., Dahlin, N., Bose, S., and Vaidya, U. Sparse learning of dynamical systems in rkhs: An operator-theoretic approach. In International Conference on Machine Learning, pp.\ 13325--13352. PMLR, 2023

    2023

  22. [30]

    A lattice B oltzmann subgrid model for high R eynolds number flows

    Hou, S., Sterling, J., Chen, S., and Doolen, G. A lattice B oltzmann subgrid model for high R eynolds number flows. Pattern formation and lattice gas automata, pp.\ 151--166, 1995

    1995

  23. [31]

    Koopman and perron--frobenius operators on reproducing kernel banach spaces

    Ikeda, M., Ishikawa, I., and Schlosser, C. Koopman and perron--frobenius operators on reproducing kernel banach spaces. Chaos: An Interdisciplinary Journal of Nonlinear Science, 32 0 (12), 2022

    2022

  24. [32]

    Johns hopkins turbulence databases, 2023

    JHTDB. Johns hopkins turbulence databases, 2023. URL https://turbulence.pha.jhu.edu/

    2023

  25. [33]

    Y., Orlova, E., and Willett, R

    Jiang, R., Lu, P. Y., Orlova, E., and Willett, R. Training neural operators to preserve invariant measures of chaotic attractors. Advances in Neural Information Processing Systems, 36, 2024

    2024

  26. [34]

    J rgensen, P. E. Ruelle Operators: Functions which Are Harmonic with Respect to a Transfer Operator: Functions which are Harmonic with Respect to a Transfer Operator, volume 720. American Mathematical Soc., 2001

    2001

  27. [35]

    H., Guilhoto, L

    Kissas, G., Seidman, J. H., Guilhoto, L. F., Preciado, V. M., Pappas, G. J., and Perdikaris, P. Learning operators with coupled attention. Journal of Machine Learning Research, 23 0 (215): 0 1--63, 2022

    2022

  28. [36]

    Kloeckner, B. R. Empirical measures: regularity is a counter-curse to dimensionality. ESAIM: Probability and Statistics, 24: 0 408--434, 2020

    2020

  29. [37]

    Eigendecompositions of transfer operators in reproducing kernel hilbert spaces

    Klus, S., Schuster, I., and Muandet, K. Eigendecompositions of transfer operators in reproducing kernel hilbert spaces. Journal of Nonlinear Science, 30: 0 283--315, 2020

    2020

  30. [38]

    Kolesov, A. Y. and Rozov, N. K. On the definition of'chaos'. Russian Mathematical Surveys, 64 0 (4): 0 701, 2009

    2009

  31. [39]

    M., Filippov, A., and Burnaev, E

    Korotin, A., Li, L., Genevay, A., Solomon, J. M., Filippov, A., and Burnaev, E. Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark. Advances in neural information processing systems, 34: 0 14593--14605, 2021

    2021

  32. [40]

    K., Yamaghani, M

    Langeroudi, M. K., Yamaghani, M. R., and Khodaparast, S. Fd-lstm: A fuzzy lstm model for chaotic time-series prediction. IEEE Intelligent Systems, 37 0 (4): 0 70--78, 2022

    2022

  33. [41]

    Straight velocity boundaries in the lattice boltzmann method

    Latt, J., Chopard, B., Malaspinas, O., Deville, M., and Michler, A. Straight velocity boundaries in the lattice boltzmann method. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics, 77 0 (5): 0 056703, 2008

    2008

  34. [42]

    Fourier neural operator for parametric partial differential equations

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020

    2010 arXiv

  35. [43]

    Learning chaotic dynamics in dissipative systems

    Li, Z., Liu-Schiaffini, M., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Learning chaotic dynamics in dissipative systems. Advances in Neural Information Processing Systems, 35: 0 16768--16781, 2022 a

    2022

  36. [44]

    Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations' operator learning. arXiv preprint arXiv:2205.13671, 2022 b

    2022 arXiv

  37. [45]

    Scalable transformer for pde surrogate modeling

    Li, Z., Shu, D., and Barati Farimani, A. Scalable transformer for pde surrogate modeling. Advances in Neural Information Processing Systems, 36, 2024

    2024

  38. [46]

    Lorenz, E. N. and Haman, K. The essence of chaos. Pure and Applied Geophysics, 147 0 (3): 0 598--599, 1996

    1996

  39. [47]

    Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3 0 (3): 0 218--229, 2021

    2021

  40. [48]

    and Gibbons, T

    Madondo, M. and Gibbons, T. Learning and modeling chaos using lstm recurrent neural networks. In Proceedings of the Midwest Instruction and Computing Symposium, Duluth, Minnesota, pp.\ 6--7, 2018

    2018

  41. [49]

    Marchenko, V. A. Nonlinear equations and operator algebras, volume 17. Springer Science & Business Media, 2012

    2012

  42. [50]

    A., Musco, C., Musco, C., and Woodruff, D

    Meyer, R. A., Musco, C., Musco, C., and Woodruff, D. P. Hutch++: Optimal stochastic trace estimation. In Symposium on Simplicity in Algorithms (SOSA), pp.\ 142--155. SIAM, 2021

    2021

  43. [51]

    Koopman operator, geometry, and learning of dynamical systems

    Mezi \'c , I. Koopman operator, geometry, and learning of dynamical systems. Not. Am. Math. Soc, 68 0 (7): 0 1087--1105, 2021

    2021

  44. [52]

    On the difficulty of learning chaotic dynamics with rnns

    Mikhaeil, J., Monfared, Z., and Durstewitz, D. On the difficulty of learning chaotic dynamics with rnns. Advances in Neural Information Processing Systems, 35: 0 11297--11312, 2022

    2022

  45. [53]

    Neumann, J. v. Proof of the quasi-ergodic hypothesis. Proceedings of the National Academy of Sciences, 18 0 (1): 0 70--82, 1932

    1932

  46. [54]

    Two-equation turbulent channel flow, 2023

    NVIDIA. Two-equation turbulent channel flow, 2023. URL https://docs.nvidia.com/deeplearning/modulus/modulus-sym/user_guide/intermediate/two_equation_turbulent_channel.html

    2023

  47. [55]

    Ottino, J. M. et al. Mixing, chaotic advection, and turbulence. Annual Review of Fluid Mechanics, 22 0 (1): 0 207--254, 1990

    1990

  48. [56]

    Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach

    Pathak, J., Hunt, B., Girvan, M., Lu, Z., and Ott, E. Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical review letters, 120 0 (2): 0 024102, 2018

    2018

  49. [57]

    and Recht, B

    Rahimi, A. and Recht, B. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007

    2007

  50. [58]

    A., Ross, Z

    Rahman, M. A., Ross, Z. E., and Azizzadenesheli, K. U-no: U-shaped neural operators. arXiv preprint arXiv:2204.11127, 2022

    2022 arXiv

  51. [59]

    and Dercole, F

    Sangiorgio, M. and Dercole, F. Robustness of lstm neural networks for multi-step forecasting of chaotic time series. Chaos, Solitons & Fractals, 139: 0 110045, 2020

    2020

  52. [60]

    Sapsis, T. P. Statistics of extreme events in fluid flows and waves. Annual Review of Fluid Mechanics, 53 0 (1): 0 85--111, 2021

    2021

  53. [61]

    Y., Parker, J

    Schiff, Y., Wan, Z. Y., Parker, J. B., Hoyer, S., Kuleshov, V., Sha, F., and Zepeda-N \'u \ n ez, L. Dyslim: Dynamics stable learning by invariant measure for chaotic systems. arXiv preprint arXiv:2402.04467, 2024

    2024 arXiv

  54. [62]

    Toward a theory of chaos

    Sengupta, A. Toward a theory of chaos. International Journal of Bifurcation and Chaos, 13 0 (11): 0 3147--3233, 2003

    2003

  55. [63]

    The space of ergodic measures for lorenz attractors

    Shi, Y., Tian, X., and Wang, X. The space of ergodic measures for lorenz attractors. arXiv preprint arXiv:2006.08193, 2020

    2006

  56. [64]

    Shiryaev, A. N. Probability-1, volume 95. Springer, 2016

    2016

  57. [65]

    General circulation experiments with the primitive equations: I

    Smagorinsky, J. General circulation experiments with the primitive equations: I. the basic experiment. Monthly Weather Review, 91 0 (3): 0 99--164, 1963

    1963

  58. [66]

    and Zhou, D.-X

    Smale, S. and Zhou, D.-X. Learning theory estimates via integral operators and their approximations. Constructive approximation, 26 0 (2): 0 153--172, 2007

    2007

  59. [67]

    Ein beitrag zur hydrodynamischen erklarung der turbulenten flussigkeisbewegung

    Sommerfield, A. Ein beitrag zur hydrodynamischen erklarung der turbulenten flussigkeisbewegung. Atti Congr. Int. Math. 4th, 1908

    1908

  60. [68]

    The L attice B oltzmann E quation for F luid D ynamics and B eyond

    Succi, S. The L attice B oltzmann E quation for F luid D ynamics and B eyond . Oxford University Press, 2001

    2001

  61. [69]

    Fourier features let networks learn high frequency functions in low dimensional domains

    Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., and Ng, R. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33: 0 ...

    2020

  62. [70]

    Infinite-dimensional dynamical systems in mechanics and physics, volume 68

    Temam, R. Infinite-dimensional dynamical systems in mechanics and physics, volume 68. Springer Science & Business Media, 2012

    2012

  63. [71]

    and Palatella, L

    Trevisan, A. and Palatella, L. Chaos and weather forecasting: the role of the unstable subspace in predictability and state estimation problems. International Journal of Bifurcation and Chaos, 21 0 (12): 0 3389--3415, 2011

    2011

  64. [72]

    Attention is all you need

    Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  65. [73]

    E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al

    Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17 0 (3): 0 261--272, 2020

    2020

  66. [74]

    o gler, D. The curse of dimension in multi-marginal optimal transport. PhD thesis, Universit \

    V \"o gler, D. The curse of dimension in multi-marginal optimal transport. PhD thesis, Universit \"a t M \"u nchen, 2023

    2023

  67. [75]

    Y., Baptista, R., Boral, A., Chen, Y.-F., Anderson, J., Sha, F., and Zepeda-Nunez, L

    Wan, Z. Y., Baptista, R., Boral, A., Chen, Y.-F., Anderson, J., Sha, F., and Zepeda-Nunez, L. Debias coarsely, sample conditionally: Statistical downscaling through optimal transport and probabilistic diffusion models. In Thirty-seventh Conference on Neural Information Process...

    2023

  68. [76]

    and Yang, Z

    Xiong, J. and Yang, Z. Chaos caused by a topologically mixing map. Technical report, International Centre for Theoretical Physics, 1991

    1991

  69. [77]

    Synthetic turbulence generator for lattice B oltzmann method at the interface between RANS and LES

    Xue, X., Yao, H.-D., and Davidson, L. Synthetic turbulence generator for lattice B oltzmann method at the interface between RANS and LES . Physics of Fluids, 34 0 (5): 0 055118, 2022

    2022

  70. [78]

    Xue, X., Wang, S., Yao, H.-D., Davidson, L., and Coveney, P. V. Physics informed data-driven near-wall modelling for lattice boltzmann simulation of high reynolds number turbulent flows. Communications Physics, 7 0 (1): 0 338, 2024

    2024

  71. [79]

    Emerging opportunities and challenges for the future of reservoir computing

    Yan, M., Huang, C., Bienstman, P., Tino, P., Lin, W., and Sun, J. Emerging opportunities and challenges for the future of reservoir computing. Nature Communications, 15 0 (1): 0 2056, 2024

    2024

  72. [80]

    Random attractors for the stochastic kuramoto-sivashinsky equation

    Yang, D. Random attractors for the stochastic kuramoto-sivashinsky equation. Stochastic analysis and applications, 24 0 (6): 0 1285--1303, 2006

    2006

  73. [81]

    Yeung, P., Zhai, X., and Sreenivasan, K. R. Extreme events in computational turbulence. Proceedings of the National Academy of Sciences, 112 0 (41): 0 12633--12638, 2015

    2015

  74. [82]

    What are srb measures, and which dynamical systems have them? Journal of statistical physics, 108: 0 733--754, 2002

    Young, L.-S. What are srb measures, and which dynamical systems have them? Journal of statistical physics, 108: 0 733--754, 2002

    2002

  75. [83]

    Attractors

    Zelik, S. Attractors. then and now. arXiv preprint arXiv:2208.12101, 2022

    2022 arXiv

  76. [84]

    Projection-based techniques for high-dimensional optimal transport problems

    Zhang, J., Ma, P., Zhong, W., and Meng, C. Projection-based techniques for high-dimensional optimal transport problems. Wiley Interdisciplinary Reviews: Computational Statistics, 15 0 (2): 0 e1587, 2023

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.