Pith. sign in

REVIEW 2 major objections 5 minor 75 references

Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training neural PDE surrogates to predict the temporal derivative instead of the next state, then advancing the solution with a standard ODE integrator, improves rollout accuracy and stability across architectures and equations.

desk verdict A systematic, well-run empirical comparison of derivative-vs-state prediction for neural PDE surrogates; the main claims mostly hold, but the paper overstates novelty and never measures the accuracy of the derivative labels it trains on. read the letter →

arxiv 2412.13074 v2 pith:VN4RMUAO submitted 2024-12-17 cs.LG

Anthony Zhou , Amir Barati Farimani This is my paper · ORCID
classification cs.LG
keywords neuralPDEsurrogatestemporalderivativepredictionODEintegrationstateFourierOperatorautoregressiverollouttime-dependentPDEsscientificmachinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that neural surrogates for time-dependent partial differential equations should not be trained to jump directly from one solution snapshot to the next. Instead, the network should learn the instantaneous rate of change $\partial u/\partial t$ at a given state, and a conventional ODE integrator — Euler, Adams-Bashforth, Heun, or Runge-Kutta — should carry the solution forward at inference. The authors show this simple swap lowers rollout error and improves stability on advection, heat, Kuramoto-Sivashinsky, Burgers, Navier-Stokes, and Kolmogorov-flow benchmarks with both FNO and Unet architectures. Because the model is no longer tied to the dataset's time step, it can be trained on full-resolution simulation output and queried at arbitrary step sizes without retraining. If the results hold, derivative prediction is a cheap, broadly applicable upgrade to the standard state-prediction recipe.

What carries the argument

The central machinery is the derivative-prediction training objective paired with an ODE-integrator inference loop. Training minimizes $L_\theta(u(t_n), t_n, y) = \|F_\theta(u(t_n), t_n) - y\|_2^2$ with $y = \partial u/\partial t|_{t_n}$, where the labels are estimated from the stored trajectory using high-order finite differences and one-sided Richardson extrapolation at the endpoints. At inference the same network feeds its predicted derivative into Forward Euler, Adams-Bashforth, Heun's, or fourth-order Runge-Kutta updates, which is what decouples the learned spatial dynamics from the temporal advancement. The paper also decomposes error by comparing against a numerical oracle that integrates ground-truth derivatives, isolating integrator error from model error.

What would settle it

Train a derivative-prediction surrogate on a PDE with an analytic solution (for example, linear advection with known $u$) using derivative labels computed from progressively coarser saved trajectories while holding the inference step size fixed. If rollout error does not systematically drop below an equivalent state-prediction model trained on the same data once label bias is large, then the method's gain is an artifact of label accuracy rather than of the derivative objective.

Watch

Extended reading notes

Core claim

The central claim is that "by simply changing the training target and introducing numerical integration during inference, neural surrogates can gain accuracy and stability in finely-discretized regimes." Rather than fitting $u(t_{n+1}) = F_\theta(u(t_n))$, the model is trained to satisfy $\frac{\partial u}{\partial t}\big|_{t_n} = F_\theta(u(t_n))$, with labels computed from saved trajectories by finite differences and Richardson extrapolation. At inference an ODE integrator produces $\hat{u}(t_{n+1})$ from the predicted derivative. The paper reports that this framework matches or beats state prediction on every tested PDE and architecture, that higher-order integrators add accuracy and permit larger steps, that models trained on twice the temporal resolution of the data achieve lower rollout error without extra inference cost, and that the benefit comes from a better-conditioned loss surface and from separating the learned spatial update from the integrated temporal update.

Load-bearing premise

The load-bearing premise is that the temporal-derivative labels computed from the saved simulation trajectories are accurate enough to serve as training targets; if the stored time resolution is too coarse, the finite-difference and Richardson-extrapolation labels become biased, and the predicted derivatives inherit that bias, eroding the claimed accuracy advantage.

Editorial extensions

If this is right

  • Training on full-resolution PDE data becomes viable: the same trajectory supplies many more training pairs without pinning the model to a tiny prediction step, and the paper reports lower rollout error on Navier-Stokes when trained on twice the data.
  • Inference can use higher-order integrators or different step sizes without retraining; Adams-Bashforth costs the same as one forward pass per step because past derivative predictions are cached, while Heun and RK4 trade extra model evaluations for larger stable steps.
  • Derivative prediction composes with existing training modifiers: the paper shows pushforward/unrolled training can be applied on top of derivative prediction for modest further gains on complex PDEs, and gains are comparable to or better than scaling model parameters.
  • The framework is limited to explicit time-stepping and time-dependent problems: implicit integrators are not covered, and steady-state or boundary-value problems would fail because the initial derivative does not encode the terminal solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference (editorial): The same decoupling should transfer to other learned dynamical models — for example weather emulators, video prediction, or learned simulators with ODE structure — because the mechanism (a smoother loss surface for changes than for states) is not specific to PDEs.
  • Inference (editorial): The numerical-oracle experiment implies that once model error shrinks, integrator error becomes the binding constraint; pushing accuracy further may require learned correctors, implicit or exponential integrators, or adaptive step-size selection based on predicted derivative magnitude, none of which the paper implements.
  • Inference (editorial): The loss-landscape and noised-trajectory observations suggest derivative-trained models should be more sensitive to input perturbations during inference than state-trained models, because noise is amplified when differencing states; a targeted robustness study could test this directly.
  • Inference (editorial): If label accuracy is the key variable, the method's value depends on the availability of finely saved trajectories; for datasets archived at coarse temporal resolution, re-simulating or upsampling before computing derivative labels may be necessary to realize the advertised gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes an alternative training/inference framework for neural PDE surrogates: instead of predicting the next state u(t_{n+1}) from u(t_n), the network is trained to predict the temporal derivative ∂u/∂t at t_n, and a numerical ODE integrator (Forward Euler, Adams-Bashforth, Heun, RK4) advances the solution in time. The framework is evaluated with FNO and Unet architectures on six time-dependent PDEs (Advection, Heat, Kuramoto-Sivashinsky, Burgers, Navier-Stokes, Kolmogorov flow). The main empirical claims are that derivative prediction improves rollout accuracy and stability compared to direct state prediction, allows flexible time-stepping and integration schemes during inference, and enables training on more finely discretized data. The paper also compares against several training modifications (larger models, pushforward/unrolled training, PDE-Refiner), analyzes error sources via next-step error and a numerical oracle, and reports computational costs against numerical solvers.

Significance. If the central claim holds, this is a simple, architecture-agnostic improvement that could be broadly adopted by the neural surrogate community: it requires only changing the training target and adding an ODE integrator at inference. The paper is unusually thorough in its empirical scope (six PDEs, two architectures, multiple integrators, training and inference modifiers), and it releases code and datasets, which is a strength. The distinction between model error and integrator error in Section 5.2 is a useful contribution. However, the general applicability claim rests on the accuracy of finite-difference derivative labels, which is never directly validated, and the closest existing alternative (residual prediction) is discussed but never benchmarked. These gaps limit the strength of the conclusions as currently stated.

major comments (2)
  1. [Sec. 3.1, Eq. (2); Sec. 4.1, Table 4] The training labels y = du/dt are computed from the saved simulation trajectories using finite differences and Richardson extrapolation, but the paper never measures the error of these labels against true derivatives. The 'Derivative Error' in Table 4 is evaluated against the same finite-difference labels, so it is a training loss rather than a measure of label fidelity. Since the framework's broad applicability depends on the availability of accurate derivative labels, the authors should quantify label error directly (e.g., by comparing to analytic derivatives for Advection or to high-resolution solver outputs for chaotic systems) and show how performance degrades as the snapshot spacing coarsens. Without this, the reported gains may be specific to datasets whose stored temporal resolution happens to be fine enough, and the paper's own Table 3 (2x steps/data) and Figure 6 suggest that label quality is indeed a sensitive factor.
  2. [Sec. 2 (Related Work); Sec. 4 (Results)] The paper identifies residual prediction as the closest alternative and as a special case of derivative prediction with a fixed time step and Forward Euler integration, but no residual prediction baseline is included in any experiment. Given that residual prediction is widely used (e.g., Pfaff et al. 2021, Sanchez-Gonzalez et al. 2020, Stachenfeld et al. 2022, cited in Section 2), the claim that derivative prediction is a broadly beneficial framework requires a direct comparison against this baseline. Such a comparison would isolate the contribution of the higher-order derivative labels and the ODE integrator from the already-known benefits of predicting differences rather than absolute states; without it, the improvement over state prediction in Table 1 could be reinterpreted as a confirmation of residual-learning effects rather than evidence for the proposed framework.
minor comments (5)
  1. [Sec. 3.2, Heun's method] The formula for Heun's method is missing a closing parenthesis: it should read 'u(t_{n+1}) = u(t_n) + (Δt/2)(F_θ(u(t_n), t_n) + F_θ(\tilde{u}(t_{n+1}), t_{n+1}))'.
  2. [Sec. 3.4, Eq. (5)] The text says that initial-condition parameters are sampled from A_j, ω_j, l_j, and φ_j, but ω_j does not appear in Eq. (5); this appears to be a typo for l_j.
  3. [Sec. 4.4 (and elsewhere)] 'CF L' should be 'CFL' (e.g., 'CFL > 1'), and the phrase 'the resolution at which CF L = 1' in Figure 4's caption should be corrected.
  4. [Fig. 5 caption] The caption contains the fragment 'at sampled a, bvalues'; this should read 'sampled at a, b values'.
  5. [Sec. 6 (Conclusion)] The sentence 'this is the first work to broadly apply this framework' is somewhat overstated given the prior derivative-prediction works cited in Section 2 (e.g., Sanchez-Gonzalez et al. 2019, Zeng et al. 2024); consider softening 'first' to 'one of the first to broadly evaluate across architectures and PDEs'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical comparison measured against external ground-truth rollouts, not reduced to its own training inputs.

full rationale

The paper's central claim is empirical: training a neural surrogate to predict temporal derivatives and integrating with an ODE solver improves rollout accuracy relative to state prediction. That claim is evaluated with rollout error (relative L2 error against ground-truth trajectories) and correlation time, both of which are external to the training objective. The derivative labels in Eq. (2) are computed from saved solver trajectories via finite differences and Richardson extrapolation, but the headline results are not measured against those labels; Table 4's 'Derivative Error' is explicitly a held-out evaluation of the training target and is not used as evidence for the main claim. The flexible-time-stepping and 2x-data experiments are also measured against ground-truth trajectories. The paper explicitly credits prior residual/derivative-prediction works (Sanchez-Gonzalez et al., Pfaff et al., Stachenfeld et al., and others), so it is not renaming a known result; it positions itself as a broad empirical study of when derivative prediction helps. Self-citations (e.g., refs. 9, 11, 14, 17, 24, 25, 30, 37, 47) appear in related-work and supporting comments only, such as using [24] to contextualize Unet versus FNO scaling; none is load-bearing for the derivative-prediction claim. The numerical-oracle analysis in Section 5.2 is a diagnostic baseline, not a predicted result. Possible concerns about finite-difference label accuracy are correctness or robustness limitations, not circularity, because the comparison with state prediction is made on held-out ground-truth rollouts and the main finding does not reduce by construction to the training labels.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper introduces no fitted constants and no invented entities; it is an empirical comparison rather than a derivation. The two axioms above are the load-bearing modeling premises: accurate derivative labels and small integrator error relative to model error.

assumptions (2)
  • domain assumption Finite-difference derivative labels computed from the saved trajectory accurately approximate the true temporal derivative.
    In Section 3.1, the training label y in Eq. 2 is set to du/dt and computed with Richardson extrapolation from discretized snapshots. If this approximation is poor at the dataset resolution, derivative prediction loses its accuracy advantage.
  • domain assumption ODE integrator error remains small relative to the learned model error at the step sizes tested.
    Section 5.2 concludes this from the numerical oracle comparison, but the oracle uses future ground-truth states, and the conclusion is empirical and step-size dependent.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates." pith.science (2026). https://pith.science/paper/VN4RMUAO

@misc{pith2026241213074,
  author       = {Pith},
  title        = {Pith review of: Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VN4RMUAO}},
  note         = {Machine review of arXiv:2412.13074}
}
read the original abstract

Neural surrogates for partial differential equations (PDEs) have become popular due to their potential to quickly simulate physics. With a few exceptions, neural surrogates generally treat the forward evolution of time-dependent PDEs as a black box by directly predicting the next state. While this is a natural and easy framework for applying neural surrogates, it can be an over-simplified and rigid framework for predicting physics. In this work, we evaluate an alternate framework in which neural solvers predict the temporal derivative and an ODE integrator forwards the solution in time, which has little overhead and is broadly applicable across model architectures and PDEs. We find that by simply changing the training target and introducing numerical integration during inference, neural surrogates can gain accuracy and stability in finely-discretized regimes. Predicting temporal derivatives also allows models to not be constrained to a specific temporal discretization, allowing for flexible time-stepping during inference or training on higher-resolution PDE data. Lastly, we investigate why this framework can be beneficial and in what situations does it work well.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 16 canonical work pages

  1. [1]

    Elsevier (2012)

    Schiesser, W.E.: The Numerical Method of Lines, Integration of Partial Differential Equations. Elsevier (2012)

    work page 2012

  2. [2]

    Bulletin of the American Mathematical Society (1967)

    Chorin, A.J.: The numerical solution of the navier-stokes equations for an incompressible fluid. Bulletin of the American Mathematical Society (1967)

    work page 1967

  3. [3]

    https://arxiv.org/abs/2010.08895

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A.: Fourier Neural Operator for Parametric Partial Differential Equations (2021). https://arxiv.org/abs/2010.08895

    arXiv 2021

  4. [4]

    Nature Machine Intelligence 3(3), 218–229 (2021) https://doi.org/10.1038/s42256-021-00302-5

    Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E.: Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence 3(3), 218–229 (2021) https://doi.org/10.1038/s42256-021-00302-5

    doi:10.1038/s42256-021-00302-5 2021

  5. [5]

    https://doi.org/10.5555/3648699.3648788

    Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A.: Neural Operator: Learning Maps Between Function Spaces (2024). https://doi.org/10.5555/3648699.3648788 . https://arxiv.org/abs/2108.08481

    work page arXiv 2024

  6. [6]

    https://arxiv.org/abs/2105.14995

    Cao, S.: Choose a Transformer: Fourier or Galerkin (2021). https://arxiv.org/abs/2105.14995

    arXiv 2021

  7. [7]

    Computers & Graphics 103, 201–211 (2022) https://doi.org/10.1016/j.cag.2022.02.004

    Li, Z., Farimani, A.B.: Graph neural network-accelerated lagrangian fluid simulation. Computers & Graphics 103, 201–211 (2022) https://doi.org/10.1016/j.cag.2022.02.004

    doi:10.1016/j.cag.2022.02.004 2022

  8. [8]

    https://arxiv.org/abs/1612.00222

    Battaglia, P.W., Pascanu, R., Lai, M., Rezende, D., Kavukcuoglu, K.: Interaction Networks for Learning about Objects, Relations and Physics (2016). https://arxiv.org/abs/1612.00222

    arXiv 2016

Show all 75 references
  1. [9]

    https://arxiv.org/abs/2205.13671

    Li, Z., Meidani, K., Farimani, A.B.: Transformer for Partial Differential Equations’ Operator Learning (2023). https://arxiv.org/abs/2205.13671

    2023 arXiv

  2. [10]

    https://arxiv.org/abs/2402

    Alkin, B., F¨ urst, A., Schmid, S., Gruber, L., Holzleitner, M., Brandstetter, J.: Universal Physics Trans- formers: A Framework For Efficiently Scaling Neural Operators (2024). https://arxiv.org/abs/2402. 12365

    2024

  3. [11]

    Computer Methods in Applied Mechanics and Engineering 426, 116983 (2024)

    Hemmasian, A., Farimani, A.B.: Multi-scale time-stepping of partial differential equations with transformers. Computer Methods in Applied Mechanics and Engineering 426, 116983 (2024)

    2024

  4. [12]

    AIAA Journal 58(1), 25–36 (2020) https://doi.org/10.2514/ 1.j058291

    Thuerey, N., Weißenow, K., Prantl, L., Hu, X.: Deep learning methods for reynolds-averaged navier–stokes simulations of airfoil flows. AIAA Journal 58(1), 25–36 (2020) https://doi.org/10.2514/ 1.j058291

    2020

  5. [13]

    https://arxiv.org/abs/2209.15616

    Gupta, J.K., Brandstetter, J.: Towards Multi-spatiotemporal-scale Generalized PDE Modeling (2022). https://arxiv.org/abs/2209.15616

    2022 arXiv

  6. [14]

    https://arxiv.org/abs/2410.01153

    Zhou, A., Li, Z., Schneier, M., Jr, J.R.B., Farimani, A.B.: Text2PDE: Latent Diffusion Models for Accessible Physics Simulation (2025). https://arxiv.org/abs/2410.01153

    2025 arXiv

  7. [15]

    https://arxiv.org/abs/2309.01745

    Kohl, G., Chen, L.-W., Thuerey, N.: Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation (2024). https://arxiv.org/abs/2309.01745

    2024 arXiv

  8. [16]

    In: The Thirteenth International Conference on Learning Representations (2025)

    Valencia, M.L., Pfaff, T., Thuerey, N.: Learning distributions of complex fluid simulations with diffusion graph networks. In: The Thirteenth International Conference on Learning Representations (2025). https: //openreview.net/forum?id=uKZdlihDDn

    2025

  9. [17]

    https:// arxiv.org/abs/2305.17560 18

    Li, Z., Shu, D., Farimani, A.B.: Scalable Transformer for PDE Surrogate Modeling (2023). https:// arxiv.org/abs/2305.17560 18

    2023 arXiv

  10. [18]

    https://arxiv.org/abs/2402.02366

    Wu, H., Luo, H., Wang, H., Wang, J., Long, M.: Transolver: A Fast Transformer Solver for PDEs on General Geometries (2024). https://arxiv.org/abs/2402.02366

    2024 arXiv

  11. [19]

    https://arxiv.org/abs/2308.05732

    Lippe, P., Veeling, B.S., Perdikaris, P., Turner, R.E., Brandstetter, J.: PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers (2023). https://arxiv.org/abs/2308.05732

    2023 arXiv

  12. [20]

    https://arxiv

    Brandstetter, J., Worrall, D., Welling, M.: Message Passing Neural PDE Solvers (2023). https://arxiv. org/abs/2202.03376

    2023 arXiv

  13. [21]

    https://arxiv.org/abs/2309.00583

    Li, Z., Kovachki, N.B., Choy, C., Li, B., Kossaifi, J., Otta, S.P., Nabian, M.A., Stadler, M., Hundt, C., Azizzadenesheli, K., Anandkumar, A.: Geometry-Informed Neural Operator for Large-Scale 3D PDEs (2023). https://arxiv.org/abs/2309.00583

    2023 arXiv

  14. [22]

    https://arxiv

    Hao, Z., Su, C., Liu, S., Berner, J., Ying, C., Su, H., Anandkumar, A., Song, J., Zhu, J.: DPOT: Auto- Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training (2024). https://arxiv. org/abs/2403.03542

    2024 arXiv

  15. [23]

    https://arxiv.org/abs/2405.19101

    Herde, M., Raoni´ c, B., Rohner, T., K¨ appeli, R., Molinaro, R., B´ ezenac, E., Mishra, S.: Poseidon: Efficient Foundation Models for PDEs (2024). https://arxiv.org/abs/2405.19101

    2024 arXiv

  16. [24]

    https://arxiv.org/abs/2406.08473

    Zhou, A., Lorsung, C., Hemmasian, A., Farimani, A.B.: Strategies for Pretraining Neural Operators (2024). https://arxiv.org/abs/2406.08473

    2024 arXiv

  17. [25]

    https://arxiv.org/abs/2403

    Zhou, A., Farimani, A.B.: Masked Autoencoders are PDE Learners (2024). https://arxiv.org/abs/2403. 17728

    2024

  18. [26]

    Nature Machine Intelligence 6(10), 1256–1269 (2024) https://doi.org/10.1038/s42256-024-00897-5

    McGreivy, N., Hakim, A.: Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. Nature Machine Intelligence 6(10), 1256–1269 (2024) https://doi.org/10.1038/s42256-024-00897-5

    2024 doi

  19. [27]

    Journal of Computational Physics 378, 686–707 (2019) https://doi.org/10.1016/j.jcp.2018.10.045

    Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: A deep learning frame- work for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 378, 686–707 (2019) https://doi.org/10.101...

    2019 doi

  20. [28]

    https:// arxiv.org/abs/2204.11127

    Rahman, M.A., Ross, Z.E., Azizzadenesheli, K.: U-NO: U-shaped Neural Operators (2023). https:// arxiv.org/abs/2204.11127

    2023 arXiv

  21. [29]

    https://arxiv

    Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., Anandkumar, A.: Physics-Informed Neural Operator for Learning Partial Differential Equations (2023). https://arxiv. org/abs/2111.03794

    2023 arXiv

  22. [30]

    https://arxiv.org/abs/2405.07395

    Li, Z., Zhou, A., Patil, S., Farimani, A.B.: CaF A: Global Weather Forecasting with Factorized Attention on Sphere (2024). https://arxiv.org/abs/2405.07395

    2024 arXiv

  23. [31]

    https://arxiv.org/abs/2202.11214

    Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., Hassanzadeh, P., Kashinath, K., Anandkumar, A.: FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neura...

    2022 arXiv

  24. [32]

    https://arxiv.org/abs/ 2106.06898

    Li, Z., Liu-Schiaffini, M., Kovachki, N., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., Anandkumar, A.: Learning Dissipative Dynamics in Chaotic Systems (2022). https://arxiv.org/abs/ 2106.06898

    2022 arXiv

  25. [33]

    arXiv preprint arXiv:2202.07643 (2022) 19

    Brandstetter, J., Welling, M., Worrall, D.E.: Lie point symmetry data augmentation for neural pde solvers. arXiv preprint arXiv:2202.07643 (2022) 19

    2022 arXiv

  26. [34]

    Computer Methods in Applied Mechanics and Engineering 433, 117441 (2025) https://doi.org/10.1016/j.cma.2024.117441

    List, B., Chen, L.-W., Bali, K., Thuerey, N.: Differentiability in unrolled training of neural physics simulators on transient dynamics. Computer Methods in Applied Mechanics and Engineering 433, 117441 (2025) https://doi.org/10.1016/j.cma.2024.117441

    2025

  27. [35]

    https://arxiv.org/abs/2212.12794

    Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., Merose, A., Hoyer, S., Holland, G., Vinyals, O., Stott, J., Pritzel, A., Mohamed, S., Battaglia, P.: GraphCast: Learning skillful medium-range...

    2023 arXiv

  28. [36]

    https://arxiv.org/abs/2312.15796

    Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T.R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., Lam, R., Willson, M.: GenCast: Diffusion-based ensemble forecasting for medium-range weather (2024). https://arxiv.org/abs/2312.15796

    2024 arXiv

  29. [37]

    https://arxiv.org/abs/2305.08757

    Lorsung, C., Li, Z., Farimani, A.B.: Physics Informed Token Transformer for Solving Partial Differential Equations (2024). https://arxiv.org/abs/2305.08757

    2024 arXiv

  30. [38]

    https://arxiv.org/abs/2010.03409

    Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., Battaglia, P.W.: Learning Mesh-Based Simulation with Graph Networks (2021). https://arxiv.org/abs/2010.03409

    2021 arXiv

  31. [39]

    https://arxiv.org/abs/2002.09405

    Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., Battaglia, P.W.: Learning to Simulate Complex Physics with Graph Networks (2020). https://arxiv.org/abs/2002.09405

    2020 arXiv

  32. [40]

    https://arxiv.org/abs/2112.15275

    Stachenfeld, K., Fielding, D.B., Kochkov, D., Cranmer, M., Pfaff, T., Godwin, J., Cui, C., Ho, S., Battaglia, P., Sanchez-Gonzalez, A.: Learned Coarse Models for Efficient Turbulence Simulation (2022). https://arxiv.org/abs/2112.15275

    2022 arXiv

  33. [41]

    https://arxiv.org/abs/1911.08655

    Wang, R., Kashinath, K., Mustafa, M., Albert, A., Yu, R.: Towards Physics-informed Deep Learning for Turbulent Flow Prediction (2020). https://arxiv.org/abs/1911.08655

    2020 arXiv

  34. [42]

    https://arxiv.org/abs/1909.12790

    Sanchez-Gonzalez, A., Bapst, V., Cranmer, K., Battaglia, P.: Hamiltonian Graph Networks with ODE Integrators (2019). https://arxiv.org/abs/1909.12790

    2019 arXiv

  35. [43]

    https:// arxiv.org/abs/2410.01337

    Zeng, B., Wang, Q., Yan, M., Liu, Y., Chengze, R., Zhang, Y., Liu, H., Wang, Z., Sun, H.: PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems (2024). https:// arxiv.org/abs/2410.01337

    2024 arXiv

  36. [44]

    Behler, J., Parrinello, M.: Generalized neural-network representation of high-dimensional potential- energy surfaces. Phys. Rev. Lett. 98, 146401 (2007) https://doi.org/10.1103/PhysRevLett.98.146401

    2007 doi

  37. [45]

    Journal of Chem- ical Theory and Computation 16(8), 4757–4775 (2020) https://doi.org/10.1021/acs.jctc.0c00355 https://doi.org/10.1021/acs.jctc.0c00355

    Gkeka, P., Stoltz, G., Barati Farimani, A., Belkacemi, Z., Ceriotti, M., Chodera, J.D., Dinner, A.R., Ferguson, A.L., Maillet, J.-B., Minoux, H., Peter, C., Pietrucci, F., Silveira, A., Tkatchenko, A., Trstanova, Z., Wiewiora, R., Leli` evre, T.: Machine learning force fields ...

    2020 doi

  38. [46]

    Chemical Reviews 121(16), 10142–10186 (2021) https:// doi.org/10.1021/acs.chemrev.0c01111 https://doi.org/10.1021/acs.chemrev.0c01111

    Unke, O.T., Chmiela, S., Sauceda, H.E., Gastegger, M., Poltavsky, I., Sch¨ utt, K.T., Tkatchenko, A., M¨ uller, K.-R.: Machine learning force fields. Chemical Reviews 121(16), 10142–10186 (2021) https:// doi.org/10.1021/acs.chemrev.0c01111 https://doi.org/10.1021/acs.chemrev.0...

    2021 doi

  39. [47]

    The Journal of Chemical Physics 156(14), 144103 (2022) https://doi.org/10.1063/5.0083060 https://pubs.aip.org/aip/jcp/article-pdf/doi/10.1063/5.0083060/16539506/144103 1 online.pdf

    Li, Z., Meidani, K., Yadav, P., Barati Farimani, A.: Graph neural networks accelerated molecular dynamics. The Journal of Chemical Physics 156(14), 144103 (2022) https://doi.org/10.1063/5.0083060 https://pubs.aip.org/aip/jcp/article-pdf/doi/10.1063/5.0083060/16539506/144103 1 ...

    2022 doi

  40. [48]

    Proceedings of the National 20 Academy of Sciences 118(21), 2101784118 (2021) https://doi.org/10.1073/pnas.2101784118 https://www.pnas.org/doi/pdf/10.1073/pnas.2101784118

    Kochkov, D., Smith, J.A., Alieva, A., Wang, Q., Brenner, M.P., Hoyer, S.: Machine learning–accelerated computational fluid dynamics. Proceedings of the National 20 Academy of Sciences 118(21), 2101784118 (2021) https://doi.org/10.1073/pnas.2101784118 https://www.pnas.org/doi/p...

    2021 doi

  41. [49]

    Sun, Z., Yang, Y., Yoo, S.: A neural pde solver with temporal stencil modeling (2023)

    2023

  42. [50]

    Proceedings of the National Academy of Sciences 116(31), 15344–15349 (2019) https://doi.org/10.1073/pnas.1814058116 https://www.pnas.org/doi/pdf/10.1073/pnas.1814058116

    Bar-Sinai, Y., Hoyer, S., Hickey, J., Brenner, M.P.: Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences 116(31), 15344–15349 (2019) https://doi.org/10.1073/pnas.1814058116 https://www.pnas.org/doi/pdf/10.107...

    2019 doi

  43. [51]

    Computer Methods in Applied Mechanics and Engineering 420, 116692 (2024) https://doi.org/10.1016/j.cma.2023.116692

    Margenberg, N., Jendersie, R., Lessig, C., Richter, T.: Dnn-mg: A hybrid neural network/finite ele- ment method with applications to 3d simulations of the navier–stokes equations. Computer Methods in Applied Mechanics and Engineering 420, 116692 (2024) https://doi.org/10.1016/...

    2024

  44. [52]

    Journal of the Mechanics and Physics of Solids 158, 104668 (2022) https://doi.org/10.1016/j.jmps.2021.104668

    Liu, B., Kovachki, N., Li, Z., Azizzadenesheli, K., Anandkumar, A., Stuart, A.M., Bhattacharya, K.: A learning-based multiscale method and its application to inelastic impact problems. Journal of the Mechanics and Physics of Solids 158, 104668 (2022) https://doi.org/10.1016/j....

    2022

  45. [53]

    Mistani, P., Aragon-Calvo, M.A., Gibou, F.: Solving inverse-pde problems with physics- aware neural networks

    Pakravan, S., A. Mistani, P., Aragon-Calvo, M.A., Gibou, F.: Solving inverse-pde problems with physics- aware neural networks. Journal of Computational Physics 440, 110414 (2021) https://doi.org/10.1016/ j.jcp.2021.110414

    2021

  46. [54]

    galaxy model fitting

    Aragon-Calvo, M.A., Carvajal, J.C.: Self-supervised learning with physics-aware neural networks – i. galaxy model fitting. Monthly Notices of the Royal Astronomical Society 498(3), 3713–3719 (2020) https://doi.org/10.1093/mnras/staa2228

    2020 doi

  47. [55]

    https://arxiv.org/abs/1806.07366

    Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural Ordinary Differential Equations (2019). https://arxiv.org/abs/1806.07366

    2019 arXiv

  48. [56]

    https://arxiv.org/abs/2002.02798

    Finlay, C., Jacobsen, J.-H., Nurbekyan, L., Oberman, A.M.: How to train your neural ODE: the world of Jacobian and kinetic regularization (2020). https://arxiv.org/abs/2002.02798

    2020 arXiv

  49. [57]

    https://arxiv.org/abs/ 1906.01563

    Greydanus, S., Dzamba, M., Yosinski, J.: Hamiltonian Neural Networks (2019). https://arxiv.org/abs/ 1906.01563

    2019 arXiv

  50. [58]

    https://arxiv.org/abs/2003.04630

    Cranmer, M., Greydanus, S., Hoyer, S., Battaglia, P., Spergel, D., Ho, S.: Lagrangian Neural Networks (2020). https://arxiv.org/abs/2003.04630

    2020 arXiv

  51. [59]

    https://arxiv.org/abs/1907.04490

    Lutter, M., Ritter, C., Peters, J.: Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning (2019). https://arxiv.org/abs/1907.04490

    2019 arXiv

  52. [60]

    Kumar, R., Bhattacharyya, S.N.: One-sided finite-difference approximations suitable for use with richardson extrapolation. J. Comput. Phys. 219, 13–20 (2006)

    2006

  53. [61]

    https://arxiv.org/abs/2006.10739

    Tancik, M., Srinivasan, P.P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoor- thi, R., Barron, J.T., Ng, R.: Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (2020). https://arxiv.org/abs/2006.10739

    2020 arXiv

  54. [62]

    https://arxiv.org/abs/1706.03762

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention Is All You Need (2023). https://arxiv.org/abs/1706.03762

    2023 arXiv

  55. [63]

    https://arxiv.org/abs/ 2210.07182

    Takamoto, M., Praditia, T., Leiteritz, R., MacKinlay, D., Alesiani, F., Pfl¨ uger, D., Niepert, M.: PDEBENCH: An Extensive Benchmark for Scientific Machine Learning (2024). https://arxiv.org/abs/ 2210.07182

    2024 arXiv

  56. [64]

    https://arxiv.org/abs/2412.00568

    Ohana, R., McCabe, M., Meyer, L., Morel, R., Agocs, F.J., Beneitez, M., Berger, M., Burkhart, B., Dalziel, S.B., Fielding, D.B., Fortunato, D., Goldberg, J.A., Hirashima, K., Jiang, Y.-F., Kerswell, R.R., 21 Maddu, S., Miller, J., Mukhopadhyay, P., Nixon, S.S., Shen, J., Watte...

    2024 arXiv

  57. [65]

    https://arxiv.org/abs/2003

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A.: Neural Operator: Graph Kernel Network for Partial Differential Equations (2020). https://arxiv.org/abs/2003. 03485

    2020

  58. [66]

    https://arxiv

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A.: Multipole Graph Neural Operator for Parametric Partial Differential Equations (2020). https://arxiv. org/abs/2006.09535

    2020 arXiv

  59. [67]

    https://arxiv.org/abs/2310.00120

    Kossaifi, J., Kovachki, N., Azizzadenesheli, K., Anandkumar, A.: Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs (2023). https://arxiv.org/abs/2310.00120

    2023 arXiv

  60. [68]

    https://arxiv.org/abs/1505.04597

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmen- tation (2015). https://arxiv.org/abs/1505.04597

    2015 arXiv

  61. [69]

    https://arxiv.org/abs/ 2006.11239

    Ho, J., Jain, A., Abbeel, P.: Denoising Diffusion Probabilistic Models (2020). https://arxiv.org/abs/ 2006.11239

    2020 arXiv

  62. [70]

    https://arxiv.org/abs/1605.07146

    Zagoruyko, S., Komodakis, N.: Wide Residual Networks (2017). https://arxiv.org/abs/1605.07146

    2017 arXiv

  63. [71]

    https://arxiv.org/abs/2306.01776

    Lienen, M., L¨ udke, D., Hansen-Palmus, J., G¨ unnemann, S.: From Zero to Turbulence: Generative Modeling for 3D Flow Simulation (2024). https://arxiv.org/abs/2306.01776

    2024 arXiv

  64. [72]

    https://arxiv.org/abs/2411.00180

    Koehler, F., Niedermayr, S., Westermann, R., Thuerey, N.: APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs (2024). https://arxiv.org/abs/2411.00180

    2024 arXiv

  65. [73]

    https://arxiv.org/abs/2409

    Oommen, V., Bora, A., Zhang, Z., Karniadakis, G.E.: Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modeling (2024). https://arxiv.org/abs/2409. 08477

    2024

  66. [74]

    https://arxiv.org/abs/1712.09913

    Li, H., Xu, Z., Taylor, G., Studer, C., Goldstein, T.: Visualizing the Loss Landscape of Neural Nets (2018). https://arxiv.org/abs/1712.09913

    2018 arXiv

  67. [75]

    https://arxiv.org/abs/2207

    Dresdner, G., Kochkov, D., Norgaard, P., Zepeda-N´ u˜ nez, L., Smith, J.A., Brenner, M.P., Hoyer, S.: Learning to correct spectral methods for simulating turbulent flows (2023). https://arxiv.org/abs/2207. 00556 22

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.