Pith. sign in

REVIEW 3 major objections 5 minor 58 references

Autoregressive neural operators for PDEs are stabilized by training against the model's own worst previous rollouts, not only against ground truth.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:02 UTC pith:3B4RTOGS

load-bearing objection Genuinely useful training-time reweighting with solid empirical gains; the long-horizon mechanism story outruns what the 5-step diagnostics can actually support. the 3 major comments →

arxiv 2607.29135 v1 pith:3B4RTOGS submitted 2026-07-31 cs.LG cs.AI

HERO: History-Enriched Rollout Training for Long-Horizon Autoregressive Neural Operators

Jiaquan Zhang , Shuxu Chen , Haifan Meng , Yi Lu , Zhihan Lyu , Fan Mo , Wei Dong , Yang Yang
show 1 more author
Chaoning Zhang
This is my paper
classification cs.LG cs.AI MSC 68T07
keywords HEROautoregressive neural operatorslong-horizon rollout stabilityPDE surrogatestrain-test mismatchmargin-based objectivefailure referenceerror accumulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces HERO, a training objective for neural operators that roll a PDE forward step by step. Because each prediction becomes the next input, small errors compound and eventually destabilize the rollout. HERO keeps standard ground-truth rollout supervision but adds a margin term that forces the current rollout to beat a dynamically selected 'failure reference' drawn from the model's optimization history—usually a periodically frozen older copy, sometimes the current rollout itself or a perturbed-input rollout. The reference is selected by ranking candidates on rollout error, spectral mismatch, energy drift, and error growth, and it enters the loss only as a fixed baseline, so it reweights the existing ground-truth gradient instead of adding a new gradient direction. The paper reports consistent gains in 100- to 200-step accuracy, stable rollout length, and out-of-distribution robustness on nine PDE benchmarks, at no inference-time cost.

Core claim

HERO's central claim is that absolute error from the true trajectory is not enough supervision for long-horizon autoregressive prediction, because it cannot tell whether the operator has escaped failure modes it exhibited earlier in training. The fix is to construct a currently failing trajectory—using a lagged copy of the model, a detached copy of the current rollout, or a perturbed-input rollout—rank candidates by rollout error, spectral mismatch, energy drift, and error growth, pick the worst, and add a smooth margin loss rewarding the current rollout for beating that reference by m. The reference is detached, so the margin term only multiplies the ground-truth rollout gradient by a sigmo

What carries the argument

The central object is the 'failure reference' τ⁻, a detached candidate trajectory chosen at each training step as the strongest measured failure among a lagged-operator rollout, the detached current rollout, and a perturbed-input rollout. The four diagnostics—rollout error, spectral-amplitude discrepancy, energy drift, and positive error growth—are normalized across candidates and averaged into a score; the highest-scoring candidate is the reference. The mechanism that carries the argument is the margin objective ℓ_rel = (1/β) log(1+exp(β[d_cur − d_fail + m])), whose gradient equals σ(β[d_cur − d_fail + m]) ∇ d_cur, so the reference never supplies an independent gradient but reweights the gr

Load-bearing premise

The gain depends on the assumption that the failure rankings computed over the 5-step training rollout are a faithful proxy for what happens at the 100- to 200-step horizons measured at test time; if 5-step diagnostics do not track long-horizon error growth, the selected reference can be the wrong one and the margin signal would not transfer.

What would settle it

Collect a holdout set of checkpoints during HERO training, compute the 5-step diagnostic score and the 100-step rollout error for each, and measure their rank correlation. If the selected 5-step reference is often better (lower long-horizon error) than unselected candidates—or if the correlation is near zero or negative—the transfer premise collapses. A more direct test: train HERO with the diagnostics computed over K=50 instead of K=5; if long-horizon accuracy does not improve, the short-horizon proxy is not carrying the claimed signal.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If HERO is correct, long-horizon rollout stability can be improved without changing the deployed model: the inference-time architecture, parameter count, and cost stay identical to the backbone.
  • The gain is concentrated at long horizons: in the reported ablations, one-step error changes little while 100-step error drops by roughly a third, implying the method targets error accumulation rather than one-step fitting.
  • The advantage transfers across architecture families: the same training objective improves both spectral and attention-based neural operators, suggesting the mechanism is not tied to a particular inductive bias.
  • HERO also improves zero-shot generalization: models trained with the margin objective stay stable under shifted physical parameters and spatial resolutions.
  • The lagged branch and diagnostic selection are load-bearing: ablations show that removing the lagged candidate or freezing the reference recovers only part of the gain, while the relative objective itself contributes nearly all of it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Implicit extension: if 5-step failure diagnostics transfer to 100-step stability, the same margin principle could apply to other autoregressive sequence models—weather forecasting, video prediction, or learned simulation—where rollouts diverge and one-step loss is a poor proxy for long-term quality.
  • Testable extension: a minimal variant that replaces the four diagnostics with a single direct estimate of long-horizon error growth would reveal whether the diagnostic machinery is essential or merely sufficient; if the gain persists with one diagnostic, the method's complexity can be cut substantially.
  • Implicit extension: the covariance decomposition suggests a measurable predictor of when the method helps: compute the correlation between relative activation weights and long-horizon usefulness on a validation set. If that correlation is not positive, the paper's own mechanism predicts no gain, giving practitioners a cheap prescriptive check.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes HERO (History-Enriched Rollout Training), a training-only method for autoregressive neural operators. HERO augments conventional ground-truth rollout regression with a margin-based relative objective against a dynamically selected 'failure reference' trajectory. The reference is chosen from up to three detached candidates (lagged operator rollout, perturbed-input rollout, and the current detached rollout) using four diagnostics computed over the training rollout horizon (rollout error, spectral discrepancy, energy drift, error growth). The authors show analytically that the relative objective induces a bounded, sample-dependent reweighting of the ground-truth rollout gradient rather than an independent gradient direction, and they provide a conditional finite-step entry theorem for a long-horizon risk sublevel set. Experiments on nine PDE benchmarks with FNO and Transolver backbones report consistent improvements in nRMSE@100, geometric-mean rollout error, stable step length, and out-of-distribution robustness at no inference-time cost. The appendix includes full 5-seed statistics, hyperparameter sensitivity studies, and computational cost measurements.

Significance. If the mechanism claimed by the paper is valid, HERO is a practically attractive training strategy: it leaves the inference-time operator unchanged, is architecture-agnostic, and shows consistent gains across a broad set of PDE benchmarks and two structurally different backbones. The paper has notable strengths: the experiments are extensive and include standard deviations in the appendix; the ablation includes a clean 'HERO w/o Relative' control showing that candidate construction alone contributes nothing; the computational-cost table is careful; and the theory section explicitly acknowledges the conditional nature of the finite-step result (B.2.6). The gradient algebra in Eqs. (28)–(40) is correct. However, two load-bearing issues prevent full acceptance. First, the diagnostics that select the failure reference are computed over K=5 rollouts while the paper's claims and motivation concern 100–200-step stability; no evidence is given that K=5 diagnostics predict long-horizon failure, and the theory assumes rather than establishes this transfer. Second, the 'Self-relative' ablation in Table 2 is inconsistent with the method's own equations: as defined in Eq. (11), the self-referen

major comments (3)
  1. [Sec. 4.3 (Eqs. 14–18) and Sec. 5.1] The candidate diagnostics used for failure-reference selection are computed over the training rollout horizon K=5 (set in Sec. 5.1). The motivating analysis in Fig. 2 and the headline metrics concern 100–200-step rollouts, but the paper never shows that the K=5 diagnostics are predictive of long-horizon failure. The theory in Appendix B.2 relies on a positive covariance Cov(α,a) (Prop. B.1, Cor. B.2) or on a uniform descent margin on the long-horizon risk J_L (Theorem B.4, condition 2); it does not derive this covariance or margin from the K=5 selection mechanism. Section B.2.6 states that the covariance is 'empirically accessible,' but no such measurement is reported. Without this link, the selected 'failure reference' may simply be a high-loss K=5 trajectory, and the empirical gain could be interpreted as hard-example reweighting of the K=5 rollout loss rather than the claimed history-
  2. [Sec. 5.3, Eq. (11), Table 2] The 'Self-relative' variant restricts the candidate set to τ_self = sg(τ_θs), the detached current rollout. For a deterministic operator, d_fail = D(τ_self, τ★) = D(τ_θs, τ★) = d_cur. Substituting into the margin loss (23) gives h = d_cur − d_fail + m = m, so the relative loss is constant up to a sample-independent constant, and its gradient is σ(βm)∇d_cur (Eq. (28)). Thus Self-relative should be equivalent to FNO-PF with the total loss scaled by 1+λσ(βm) ≈ 1.01 — essentially a learning-rate rescaling. Yet Table 2 reports Self-relative improving Test GM100 from 1.682 (FNO-PF) to 1.574, a 6.4% gain that a ~1% learning-rate change cannot plausibly explain. This suggests either the self-reference is not actually the current detached rollout (e.g., it might come from a previous iteration), or the reported ablation is affected by an implementation artifact. Because the ablation is the key evi
  3. [Table 2] The component-level ablation in Table 2 reports single point estimates without standard deviations, in contrast to the appendix tables (Tables 9–10) which include 5-seed SDs. The conclusions drawn from Table 2 — for example, that 'HERO w/o Self comes closest overall' or that 'w/o Lag degrades most' — rely on differences that may be within seed noise. Please provide 5-seed means and SDs for the ablation, or at least report the number of seeds and a significance test for the key contrasts (e.g., Full HERO vs. w/o Self, w/o Lag vs. w/o Pert).
minor comments (5)
  1. [Sec. 4.3, stage 3] The diagnostics in Eqs. (14)–(18) are computed over a K=5 rollout, yet the text describes them as measuring 'long-horizon behavior.' This terminology is misleading; suggest renaming them to 'K-step' or 'multi-step' diagnostics and explicitly noting that the training horizon differs from the evaluation horizon.
  2. [Abstract] The phrase 'dynamically selected failure trajectory' should be qualified as a trajectory selected using K-step diagnostics during training, to avoid implying that the selection is based on the 100–200-step horizon that motivates the paper.
  3. [Eq. (20)] The normalization in Eq. (20) adds ε in the denominator, which can produce unstable normalized diagnostics when the range (max − min) is very small. Consider a more standard normalization or an explicit floor.
  4. [Sec. 5.1 and Appendix C.3] The hyperparameter defaults fix K=5 for all tasks. The sensitivity study in C.3 covers s0 and R but not K. Reporting sensitivity to K, or at least a justification for the fixed K=5, would strengthen the interpretation that the method is not sensitive to this choice.
  5. [Fig. 2 caption] Clarify that the historical checkpoints in Fig. 2 are from the HERO training run itself and how the comparison to 'Final HERO' motivates the method. The current caption is ambiguous about what exactly is being compared.

Circularity Check

0 steps flagged

No significant circularity: HERO's gradient reweighting is an algebraic identity, the finite-step theorem is explicitly conditional, and the empirical gains are benchmark-measured rather than derived from fitted constants.

full rationale

The central mechanism of HERO is transparently a bounded reweighting of the ground-truth rollout gradient: because the relative margin loss (Eq. 23) uses the same trajectory discrepancy D as the absolute loss, Eq. (39) gives grad_theta l_H = (1 + lambda_k alpha_k) grad_theta l_tr. This is an algebraic identity that the paper states directly ("the relative objective induces a bounded, sample-dependent reweighting of the current ground-truth rollout gradient"), not a hidden equivalence. Proposition B.1's covariance decomposition is likewise an algebraic identity, and its corollaries follow by substitution. Theorem B.4 is conditional on a uniform long-horizon descent margin (condition 2, Eq. 67), and the paper's own Appendix B.2.6 explicitly limits the claim: "Theorem B.4 is a conditional finite-step entry result rather than an unconditional convergence guarantee" and "applies a standard smoothness-based descent argument once a uniform long-horizon descent margin has been established." This is an honestly scoped limitation, not a circular derivation. The reported improvements are empirical comparisons against matched baselines on fixed PDE benchmarks, with no constants fit to those benchmark results; the HERO hyperparameters are set once and shared across all tasks. The K=5 diagnostics versus the 100/200-step evaluation is a possible evidence gap: Figure 2 motivates with 200-step error landscapes, but the paper does not measure the correlation between the 5-step diagnostic scores used for reference selection and 100-step rollout error. That is a correctness/empirical-support concern, not circularity, because nothing in the loss or selection procedure is defined in terms of the reported test metrics, and the mechanism is not fitted to them. No load-bearing self-citations, imported uniqueness theorems, or ansatz-via-citation steps were found. Therefore no circular steps are identified.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 0 invented entities

The central claim rests on several fixed hyperparameters and two unproven transfer assumptions: that 5-step diagnostics proxy long-horizon stability, and that the covariance between relative activation and long-horizon usefulness is positive. The latter is stated as a condition, not established. No new physical or conceptual entities are introduced.

free parameters (7)
  • margin m = 0.02
    Fixed hyperparameter of the relative loss (Eq. 23); chosen by hand, not fit per task.
  • sharpness beta = 5.0
    Controls transition sharpness of the margin loss.
  • max relative weight lambda_max = 0.02
    Scales the relative loss in the total objective (Eq. 26).
  • rollout horizon K = 5
    Length of both absolute and candidate rollouts; also the horizon over which diagnostics are computed.
  • relative start step s0 = 4000
    Iteration at which relative supervision is enabled.
  • refresh interval R = 2000
    Period for refreshing the lagged operator (Eq. 9).
  • warmup length W = 2000
    Linear warm-up of the relative weight.
axioms (4)
  • standard math Differentiability of the operator and rollout; loss functions are differentiable and gradients integrable.
    Required for gradient-based training and for the theoretical analysis (Sec B.2.1).
  • domain assumption The base operator supports recursive rollout with stop-gradients.
    HERO requires differentiable one-step prediction and recursive rollout (Sec 4.1).
  • domain assumption The selected worst-candidate reference provides a useful curriculum signal: positive covariance between relative activation and long-horizon usefulness.
    The theoretical benefits in Corollaries B.2–B.3 and the finite-step result depend on Cov(α, a) ≥ c > 0; the paper does not verify this empirically despite stating it is measurable (Sec B.2.6).
  • ad hoc to paper The K=5-step rollout error and the four diagnostics are predictive of long-horizon 100/200-step stability.
    The method selects references using K=5 rollouts, while the claim is about long-horizon stability; no analysis links the two. This is a design assumption specific to this paper.

pith-pipeline@v1.3.0-daily-deepseek · 33773 in / 16684 out tokens · 162367 ms · 2026-08-03T13:02:25.887193+00:00 · methodology

0 comments
read the original abstract

Neural operators provide fast surrogates for time-dependent partial differential equations (PDEs) by applying a learned evolution operator recursively to its own predictions, but this autoregressive rollout feeds every prediction error back as input, so local errors accumulate. Existing rollout-training strategies reduce the mismatch between training inputs and self-generated states, yet their supervision still measures only the absolute discrepancy from the ground-truth trajectory. Such supervision is therefore uninformative about whether the operator has overcome the long-horizon failure behaviors it exhibited earlier during optimization. We propose history-enriched rollout training (HERO), which augments conventional absolute trajectory supervision with relative supervision derived from the model's optimization history. HERO ranks detached candidate rollouts from a periodically refreshed lagged operator, the current model, and a perturbed input by rollout error, spectral discrepancy, energy drift, and error growth, and selects the strongest failure trajectory as reference. This reference enters a margin-based objective as a fixed comparison baseline, inducing a bounded, sample-dependent reweighting of the ground-truth rollout gradient rather than an independent gradient direction, which we further analyze theoretically. Experiments on nine PDE benchmarks with spectral and attention-based backbones show that HERO consistently improves long-horizon accuracy, stable rollout length, and out-of-distribution robustness at no inference-time cost. These results indicate that history-enriched relative supervision is effective for stabilizing long-horizon autoregressive prediction.

Figures

Figures reproduced from arXiv: 2607.29135 by Chaoning Zhang, Fan Mo, Haifan Meng, Jiaquan Zhang, Shuxu Chen, Wei Dong, Yang Yang, Yi Lu, Zhihan Lyu.

Figure 1
Figure 1. Figure 1: Short-horizon accuracy versus long-horizon rollout [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Relative rollout-error landscape across training [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the HERO training framework. A lagged operator, refreshed every [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 2 canonical work pages

  1. [1]

    Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks. InAdvances in Neural Information Processing Systems, Vol. 28

    2015

  2. [2]

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. InProceedings of the 26th Annual International Conference on Machine Learning(Montreal, Quebec, Canada)(ICML ’09). Association for Computing Machinery, New York, NY, USA, 41–48. doi:10.1145/1553374.1553380

    arXiv 2009

  3. [3]

    Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. 2023. Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere. InProceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskil...

    2023

  4. [4]

    Worrall, and Max Welling

    Johannes Brandstetter, Daniel E. Worrall, and Max Welling. 2022. Message Passing Neural PDE Solvers. InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022

    2022

  5. [5]

    Steven L Brunton, Bernd R Noack, and Petros Koumoutsakos. 2020. Machine learning for fluid mechanics.Annual review of fluid mechanics52, 1 (2020), 477–508

    2020

  6. [6]

    J.M. Burgers. 1948. A Mathematical Model Illustrating the Theory of Turbulence. Advances in Applied Mechanics, Vol. 1. Elsevier, 171–199. doi:10.1016/S0065- 2156(08)70100-5

    doi:10.1016/s0065- 1948

  7. [7]

    Qianying Cao, Somdatta Goswami, and George Em Karniadakis. 2024. Laplace neural operator for solving differential equations.Nature Machine Intelligence6, 6 (2024), 631–640. doi:10.1038/s42256-024-00844-4

    doi:10.1038/s42256-024-00844-4 2024

  8. [8]

    Shuhao Cao. 2021. Choose a Transformer: Fourier or Galerkin. InAdvances in Neural Information Processing Systems, Vol. 34

    2021

  9. [9]

    Wuyang Chen, Jialin Song, Pu Ren, Shashank Subramanian, Dmitriy Morozov, and Michael W. Mahoney. 2024. Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning. InAdvances in Neural Information Processing Systems, Vol. 37

    2024

  10. [10]

    Courant, K

    R. Courant, K. Friedrichs, and H. Lewy. 1928. Über die partiellen Differen- zengleichungen der mathematischen Physik.Math. Ann.100, 1 (1928), 32–74. doi:10.1007/BF01448839

    doi:10.1007/bf01448839 1928

  11. [11]

    Adolf Fick. 1855. Ueber diffusion.Annalen der physik170, 1 (1855), 59–86

  12. [12]

    Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. 2021. Multiwavelet-based Op- erator Learning for Differential Equations. InAdvances in Neural Information Processing Systems, Vol. 34. 24048–24062. https://proceedings.neurips.cc/paper/ 2021/hash/c9e5c2b59d98488fe1070e744041ea0e-Abstract.html

    2021

  13. [13]

    Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, and Jun Zhu. 2024. DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. InProceed- ings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235), Ruslan Salakhutdi...

    2024

  14. [14]

    Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Song- ming Liu, Ze Cheng, Jian Song, and Jun Zhu. 2023. GNOT: A General Neu- ral Operator Transformer for Operator Learning. InProceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learn- ing Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyungh...

    2023

  15. [15]

    Maximilian Herde, Bogdan Raonić, Tobias Rohner, Roger Käppeli, Roberto Moli- naro, Emmanuel de Bézenac, and Siddhartha Mishra. 2024. Poseidon: Efficient Foundation Models for PDEs. InAdvances in Neural Information Processing Sys- tems, Vol. 37

    2024

  16. [16]

    Greenberg

    Yunfei Huang and David S. Greenberg. 2025. Geometric and Physical Constraints Synergistically Enhance Neural PDE Surrogates. InForty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, Vol. 267

    2025

  17. [17]

    Ruoxi Jiang, Peter Y Lu, Elena Orlova, and Rebecca Willett. 2023. Training neural operators to preserve invariant measures of chaotic attractors. InAdvances in Neural Information Processing Systems, Vol. 36. 27645–27669

    2023

  18. [18]

    Ruoxi Jiang, Xiao Zhang, Karan Jakhar, Peter Y Lu, Pedram Hassanzadeh, Michael Maire, and Rebecca Willett. 2025. Hierarchical implicit neural emulators. In Advances in Neural Information Processing Systems, Vol. 38. 73718–73751

    2025

  19. [19]

    Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang

    George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. 2021. Physics-informed machine learning.Nature Reviews Physics3 (2021), 422–440. doi:10.1038/s42254-021-00314-5

    doi:10.1038/s42254-021-00314-5 2021

  20. [20]

    Smith, Ayya Alieva, Qing Wang, Michael P

    Dmitrii Kochkov, Jamie A. Smith, Ayya Alieva, Qing Wang, Michael P. Bren- ner, and Stephan Hoyer. 2021. Machine learning–accelerated computational fluid dynamics.Proceedings of the National Academy of Sciences118, 21 (2021), e2101784118. doi:10.1073/pnas.2101784118

    doi:10.1073/pnas.2101784118 2021

  21. [21]

    Felix Koehler, Simon Niedermayr, Rüdiger Westermann, and Nils Thuerey. 2024. APEBench: A benchmark for autoregressive neural emulators of PDEs. InAd- vances in Neural Information Processing Systems, Vol. 37. 120252–120310

    2024

  22. [22]

    D. J. Korteweg and G. de Vries. 1895. XLI. On the change of form of long waves advancing in a rectangular canal, and on a new type of long stationary waves.The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science39, 240 (1895), 422–443. arXiv:https://doi.org/10.1080/14786449508620739 doi:10.1080/14786449508620739

    doi:10.1080/14786449508620739

  23. [23]

    Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. 2023. Neural operator: Learning maps between function spaces with applications to pdes.Journal of Machine Learning Research24, 89 (2023), 1–97

    2023

  24. [24]

    Yoshiki Kuramoto. 1978. Diffusion-Induced Chaos in Reac- tion Systems.Progress of Theoretical Physics Supplement64 (02 1978), 346–367. arXiv:https://academic.oup.com/ptps/article- pdf/doi/10.1143/PTPS.64.346/5293041/64-346.pdf doi:10.1143/PTPS.64.346

    work page doi:10.1143/ptps.64.346/5293041/64-346.pdf 1978

  25. [25]

    Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron Courville, and Yoshua Bengio. 2016. Professor Forcing: A New Algorithm for Training Recurrent Networks. InAdvances in Neural Information Processing Systems, Vol. 29

    2016

  26. [26]

    Jiayi Li, Penghao Jiang, Hira Saleem, Zhaonan Wang, Piotr Koniusz, and Flora D Salim. 2026. SGNO: Spectral Generator Neural Operators for Stable Long Horizon PDE Rollouts.arXiv preprint arXiv:2602.18801(2026)

    Pith/arXiv arXiv 2026

  27. [27]

    Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar

  28. [28]

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. 2021. Fourier neural operator for parametric partial differential equations. InInternational Conference on Learning Representations. https://openreview.net/forum?id=c8P9NQVtmnO

    2021

  29. [29]

    Zongyi Li, Miguel Liu-Schiaffini, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar

  30. [30]

    Zijie Li, Dule Shu, and Amir Barati Farimani. 2023. Scalable transformer for pde surrogate modeling. InAdvances in Neural Information Processing Systems, Vol. 36. 28010–28039

    2023

  31. [31]

    Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzadenesheli, and Anima Anandkumar. 2024. Physics-Informed Neural Operator for Learning Partial Differential Equations.ACM/IMS Journal of Data Science1, 3, Article 9 (May 2024), 27 pages. doi:10.1145/3648506

    doi:10.1145/3648506 2024

  32. [32]

    Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brand- stetter. 2023. Pde-refiner: Achieving accurate long rollouts with neural pde solvers. InAdvances in Neural Information Processing Systems, Vol. 36. 67398–67433

    2023

  33. [33]

    Bjoern List, Li-Wei Chen, Kartik Bali, and Nils Thuerey. 2025. Differentiability in unrolled training of neural physics simulators on transient dynamics.Computer Methods in Applied Mechanics and Engineering433 (2025), 117441

    2025

  34. [34]

    Michael McCabe, Peter Harrington, Shashank Subramanian, and Jed Brown. 2023. Towards Stability of Autoregressive Neural Operators.Trans. Mach. Learn. Res. 2023 (2023). https://openreview.net/forum?id=RFfUUtKYOG

    2023

  35. [35]

    Meshalkin and Ia.G

    L.D. Meshalkin and Ia.G. Sinai. 1961. Investigation of the stability of a stationary solution of a system of equations for the plane movement of an incompressible viscous liquid.Journal of Applied Mathematics and Mechanics25, 6 (1961), 1700–

    1961

  36. [36]

    CLMH Navier et al. 1823. Mémoire sur les lois du mouvement des fluides.Mé- moires de l’Académie Royale des Sciences de l’Institut de France6, 1823 (1823), 389–440

  37. [37]

    Battaglia

    Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W. Battaglia

  38. [38]

    Raissi, P

    M. Raissi, P. Perdikaris, and G.E. Karniadakis. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.J. Comput. Phys.378 (2019), 686–707. doi:10.1016/j.jcp.2018.10.045

    doi:10.1016/j.jcp.2018.10.045 2019

  39. [39]

    Bogdan Raonić, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bar- tolucci, Rima Alaifari, Siddhartha Mishra, and Emmanuel de Bézenac. 2023. Con- volutional Neural Operators for robust and accurate learning of PDEs. InAdvances in Neural Information Processing Systems, Vol. 36

    2023

  40. [40]

    Dueben, Sebastian Scher, Jonathan A

    Stephan Rasp, Peter D. Dueben, Sebastian Scher, Jonathan A. Weyn, Said Mouata- did, and Nils Thuerey. 2020. WeatherBench: A Benchmark Data Set for Data- Driven Weather Forecasting.Journal of Advances in Modeling Earth Systems12 (2020), e2020MS002203. doi:10.1029/2020MS002203

    doi:10.1029/2020ms002203 2020

  41. [41]

    Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to Reweight Examples for Robust Deep Learning. InProceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 4334–4343. https://proceedings.mlr.press/v80/ren18a.html Conference ac...

    2018

  42. [42]

    Stephane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A Reduction of Im- itation Learning and Structured Prediction to No-Regret Online Learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 15), Geoffrey Gordon, David Dunson, and Miroslav Dudík (Eds.)...

    2011

  43. [43]

    Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. 2020. Learning to simulate complex physics with graph networks. InInternational conference on machine learning. PMLR, 8459–8468

    2020

  44. [44]

    Parker, Stephan Hoyer, Volodymyr Kuleshov, Fei Sha, and Leonardo Zepeda-Núñez

    Yair Schiff, Zhong Yi Wan, Jeffrey B. Parker, Stephan Hoyer, Volodymyr Kuleshov, Fei Sha, and Leonardo Zepeda-Núñez. 2024. DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems. InForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, Vol. 235. 43649– 43684

    2024

  45. [45]

    Sivashinsky

    G.I. Sivashinsky. 1977. Nonlinear analysis of hydrodynamic instability in laminar flames—I. Derivation of basic equations.Acta Astronautica4, 11 (1977), 1177–1206. doi:10.1016/0094-5765(77)90096-0

    doi:10.1016/0094-5765(77)90096-0 1977

  46. [46]

    Swift and P

    J. Swift and P. C. Hohenberg. 1977. Hydrodynamic fluctuations at the convective instability.Phys. Rev. A15 (Jan 1977), 319–328. Issue 1. doi:10.1103/PhysRevA.15. 319

    work page doi:10.1103/physreva.15 1977

  47. [47]

    Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, and Mathias Niepert. 2022. PDEBENCH: An Extensive Benchmark for Scientific Machine Learning. InAdvances in Neural Information Processing Systems, Vol. 35

    2022

  48. [48]

    Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. 2023. Factorized Fourier Neural Operators. InThe Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=tmIiMPl4IPa

    2023

  49. [49]

    Kiwon Um, Robert Brand, Yun Raymond Fei, Philipp Holl, and Nils Thuerey

  50. [50]

    Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long

  51. [51]

    Zaijun Ye, Chen-Song Zhang, and Wansheng Wang. 2025. Recurrent Neural Operators: Stable Long-Term PDE Prediction.arXiv preprint arXiv:2505.20721 (2025)

    Pith/arXiv arXiv 2025

  52. [52]

    Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. 2020. Symplectic ode-net: Learning hamiltonian dynamics with control. InInternational Conference on Learning Representations. https://openreview.net/forum?id=ryxmb1rKDS A Related Works A.1 Rollout Training with Self-Generated States Neural operators learn mappings between function spaces and provi...

    arXiv 2020

  53. [1705]

    doi:10.1016/0021-8928(62)90149-1

    doi:10.1016/0021-8928(62)90149-1

  54. [2020]

    InAdvances in Neural Information Processing Systems, Vol

    Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde-solvers. InAdvances in Neural Information Processing Systems, Vol. 33. 6111–6122

  55. [2021]

    In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021

    Learning Mesh-Based Simulation with Graph Networks. In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021

    2021

  56. [2022]

    InAdvances in Neural Information Processing Systems, Vol

    Learning chaotic dynamics in dissipative systems. InAdvances in Neural Information Processing Systems, Vol. 35. 16768–16781

  57. [2023]

    http://jmlr.org/papers/v24/23-0064.html

    Fourier Neural Operator with Learned Deformations for PDEs on Gen- eral Geometries.Journal of Machine Learning Research24, 388 (2023), 1–26. http://jmlr.org/papers/v24/23-0064.html

    2023

  58. [2024]

    In International Conference on Machine Learning

    Transolver: A Fast Transformer Solver for PDEs on General Geometries. In International Conference on Machine Learning