Pith. sign in

REVIEW 3 major objections 6 minor 82 references

Replay training stabilizes hybrid neural-network flow solver

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:29 UTC pith:7TRW772Q

load-bearing objection Replay training genuinely stabilizes DNN-MG and the generalization tests are useful, but the accuracy claims rest on self-consistent metrics; 'robust and stable' is stronger than the evidence certifies. the 3 major comments →

arxiv 2601.16598 v2 pith:7TRW772Q submitted 2026-01-23 math.NA cs.NAphysics.comp-ph

A robust and stable hybrid neural network/finite element method for 2D flows that generalizes to different geometries

Robert Jendersie , Nils Margenberg , Christian Lessig , Thomas Richter This is my paper
classification math.NA cs.NAphysics.comp-ph MSC 65M6065N5576D0568T07
keywords DNN-MGNavier-Stokes equationsfinite element methoddeep learningreplay bufferstabilitygeneralizationcomputational fluid dynamics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the instabilities seen in the hybrid DNN-MG solver—a coarse finite element solve corrected by a neural network—are not inherent to the method but come from a distributional shift between training data and the solver's own evolving state. Retraining on data generated by the hybrid simulation itself, using replay buffers, is claimed to consistently prevent these instabilities and improve every reported accuracy metric. The paper further claims that enlarging the neural network's spatial receptive field, most effectively with a Transformer over local patch neighborhoods, improves accuracy on unseen geometries and at doubled Reynolds numbers without a meaningful performance penalty. If these claims hold, a neural-network-corrected coarse solver can sustain stable long-time Navier-Stokes simulations at roughly 2.2–2.6 times the coarse-grid cost while approaching fine-grid accuracy on a class of 2D flows.

Core claim

On the paper's own terms: DNN-MG computes each time step by solving on a coarse level, prolongating to a fine level, evaluating the fine residual, and letting a neural network predict a defect that is added to the solution; this corrected solution feeds back into the next right-hand side. The paper's central discovery is that this loop can be made stable and accurate by closing the gap between training and inference. The mechanism is replay training: run the hybrid simulation, compute a fine-grid reference for the neural-network-perturbed state at each step, and finetune on the resulting data. In their experiments this replay training consistently prevents instabilities and also improves eve

What carries the argument

The load-bearing mechanism is the defect-correction loop of Algorithm 1: after a coarse solve, the interpolated fine solution's residual is computed, and a neural network predicts a fine-level defect that is added to the interpolant; the corrected fine state then defines the right-hand side for the next coarse solve. Stability comes from replay buffers: a meta-loop that runs the hybrid simulation, records fine-grid reference corrections for the network's own perturbed states, and finetunes the network on these replayed samples plus the original data, without needing a differentiable solver. The architectural comparison centers on the receptive field: patch size M, temporal stacking, and, for

Load-bearing premise

The load-bearing premise is that low divergence and low one-step local error against a fine-grid reference certify a physically correct long-time trajectory; the paper itself notes in its limitations section that for sensitive cases this cannot currently be validated by increasing mesh resolution and reducing the time step.

What would settle it

Run a replay-trained DNN-MG simulation on a case with a sensitive symmetric solution, such as the perturbed nine-square-obstacle geometry, and compare its time-averaged velocity and lift against reference solutions computed on two successively refined meshes with smaller time steps; if the hybrid run keeps low divergence and low local error yet its mean velocity or force functionals drift away from the refined references beyond the coarse baseline, the central accuracy claim would be falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Long-time hybrid simulations can remain stable without backpropagation through time: replay training alone closes the distributional shift, at the price of one extra training stage.
  • Coarse-grid runs corrected this way run about 2.2–2.6 times faster than the fine reference and add only 20–30% over the coarse solve in the tested cases; neural network inference is a minor part of the runtime.
  • Replay-trained networks generalize to unseen obstacle arrangements, perturbed meshes, and doubled Reynolds numbers; local error drops by up to a factor of 16 and mean-velocity error by up to a factor of 5 relative to the coarse solution.
  • Larger receptive fields, whether from a Transformer or a larger patch, improve accuracy on non-local flows; Transformers provide this on unstructured meshes while preserving locality, without significant performance impact.
  • Direct feedback of the network's own corrected fine solution as input destabilizes the run, so the stable configuration deliberately keeps the coarse solve in the loop.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the claims hold, the replay meta-loop is architecture-agnostic and could transfer to other learned-correction PDE solvers; the main open cost is that replay generation is per-network and per-case, so the economics depend on how often the retrained network is reused.
  • Editorial inference: the paper's evidence for fine-level accuracy is mostly divergence, local error, and force functionals over moderate horizons; a more demanding test would compare long trajectories against a reference on two successively refined meshes, a check the paper says is currently impossible for sensitive cases.
  • Editorial inference: the failure of fine-state feedback suggests the next bottleneck is not single-step accuracy but a stable differentiable coupling between solver and network; implementing true backpropagation-through-time in a differentiable solver could unlock the temporal receptive field that the paper found unhelpful as input.
  • Editorial inference: a testable extension would be to train one replay-stabilized network on a family of geometries and measure how far it extrapolates in Reynolds number or obstacle shape before the replay loop must be rerun; the paper's mixed-geometry cases are a start but leave the boundary of applicability unspecified.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports improvements to the hybrid neural-network / finite-element solver DNN-MG for two-dimensional instationary Navier–Stokes equations. The method solves on a coarse level and uses a neural network to predict a defect correction on a finer level. The contributions are: input normalization; noise and rotation data augmentation; a replay-buffer retraining stage that generates reference targets from the hybrid simulation itself; and a comparison of MLP, RNN, Transformer, and larger-patch architectures. The central empirical claims are that replay training 'consistently prevents instabilities and also improves every metric' (Section 6.3, Table 1), that Transformers and enlarged receptive fields improve the accuracy of DNN-MG without a significant performance penalty (Section 7), and that the resulting hybrid simulation runs at roughly 2.2--2.6x the coarse-grid cost while approaching fine-grid accuracy. The paper provides an extensive experimental matrix: four networks per configuration, multiple channel-flow geometries, hold-out validation cases, perturbations of the geometry, and Reynolds-number doubling, with code and trained weights promised on Zenodo [82].

Significance. If the claims hold, DNN-MG would be a practical and reproducible hybrid strategy for long-time 2D Navier–Stokes simulation at substantially reduced cost, with useful guidance on architecture choice and on stabilizing autoregressive NN/FEM coupling. The strengths of the study are its breadth: 4 seeds per architecture, separate train/validation/replay stages, multiple accuracy metrics, explicit runtime profiling, and the public code/weights deposit. The authors also honestly report negative results (fine-velocity feedback in Appendix C, limitations in Section 7.3). However, the central accuracy claims rest on metrics that condition the reference on the NN-perturbed trajectory (Eq. (17) and the replay generation in Section 6.3), and the paper itself states in Section 7.3 that for sensitive cases validation by mesh refinement is currently not possible. This limits the strength of the generalization claims, especially for the chaotic sq9* and ro3* cases.

major comments (3)
  1. [§7.3 and §6.3 / Eq. (17)] The two load-bearing accuracy metrics, the local error τ in Eq. (17) and the replay-training targets in Section 6.3, are computed from a fine-grid reference that is started from the NN-perturbed state. This measures how well the NN corrects the coarse input for the state it is given, but it cannot detect a systematic bias that pushes the whole trajectory onto a different physical branch. Section 7.3 explicitly concedes: 'it is currently not possible to validate a solution by increasing the mesh resolution and using a smaller time step' for sensitive cases. The only independent check provided is the ro1 drag/lift comparison against higher-level references in Section 6.3 and Table 2. No such check is offered for sq9*, ro3*, or the Re-doubling experiments in Section 7.1. Since the title-claim 'generalizes to different geometries' is substantiated mainly by Tables 4–6, this is a load-bearing
  2. [Tables 1–6] All central comparison tables report only 'mean' and 'best' over 4 seeds, without standard deviations, individual values, or a measure of spread. With N=4, the claims that replay training 'consistently prevents instabilities' and that 'Transformer > MLP > RNN' are statistically fragile: the authors themselves report that 2 of 4 Transformers become unstable at Re=200 (Fig. 9), 1 of 4 baseline MLPs fails on sq9*, and 2 of 4 MLP-M1 networks deviate on sq9* (Section 7.1). The absence of spread makes it impossible for the reader to assess whether the reported metric differences are meaningful. Please provide per-seed values in an appendix, or at least standard deviations/ranges, and state the number of seeds used for each table.
  3. [§7 and Table 7 / Fig. 10] The summary claim of Section 7 that Transformers improve accuracy 'without a significant impact on performance' is overstated as stated. Table 7 shows Transformer training time of 39.3 h vs. 24.0 h for MLP, and Fig. 10 shows roughly 12x slower inference, though the simulation runtime is dominated by the solver. More importantly, the accuracy improvement is not uniform: on sq9* (Table 5) the Transformer's mean divergence (5.2428) is worse than both MLP (5.1221) and RNN (5.0278), and only the best value is comparable; on ro3* (Table 6) the Transformer is best on eJd and e_v but worst on mean divergence. The claim should be qualified to the metrics and cases where the improvement actually holds.
minor comments (6)
  1. [Abstract / Title] The abstract uses 'instationary' while the title and Introduction use 'nonstationary'. Please make the terminology consistent throughout.
  2. [§4.4.1] Typo: 'AdamW turned out to be essential essential for stability' should be 'essential for stability'.
  3. [§2] Typo: 'generaly therms' should be 'generally these terms'.
  4. [Eq. (17)] The definition of τ has a double comma in the argument list and the notation is otherwise dense. Please rewrite the argument list for clarity, e.g. τ(\tilde v_{n-1}, \tilde v_n, f_{n-1}, f_n).
  5. [Table 5 / §7.1] In the text after Table 5, 'the training data where J_l ≈0.s0' contains an apparent typo ('0.s0'). Also, the figure captions in Appendix B (Figs. B.14, B.15) refer to 'ro6' where the text says 'sq9*' and 'ro3*'; please correct the captions.
  6. [§5 / B.11–B.15] Several figure captions in Appendix B repeat the same description and do not always match the listed table numbers (e.g., Fig. B.11 is called for Table 1, B.12 for Table 2, etc.). Please align captions with the tables they illustrate.

Circularity Check

0 steps flagged

No significant circularity; central accuracy claims rest on independent fine-grid FEM reference targets, not on the neural network's own outputs.

full rationale

The paper's central claims are empirical: replay training improves DNN-MG stability and accuracy, and Transformer/larger receptive fields improve accuracy. The supervised target at training time is the defect d_n = x_n^{L+J} - \hat{x}_n^{L+J}, computed from a fine-grid FEM reference solution that is independent of the network. During replay, the reference is also obtained by running the fine-grid solver from the network-perturbed state, so the target is still produced by the numerical method, not by the network itself. Metrics such as J_div (Eq. 15) and the local error tau (Eq. 17) are diagnostics evaluated against fine-grid solves of the same right-hand side; they are not training targets and they do not reappear as predicted quantities. Section 7.3's admission that a solution cannot currently be validated by mesh refinement in sensitive cases is a limitation of the evaluation methodology with respect to physical correctness, not a circular reduction of the reported claims. The paper cites its own prior work extensively, but those citations describe the predecessor method and its analysis; the architecture comparisons, replay procedure, and generalization experiments presented here are new and are supported by the reported simulations rather than by the self-citations alone. No equation or fitted parameter is shown to be equivalent by construction to the quantity it is said to predict.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claim rests on standard finite-element convergence assumptions plus empirical premises about single-step supervised training and replay conditioning transferring to long-time hybrid trajectories. No new physical entities are introduced.

free parameters (5)
  • noise strength sigma = 0.02
    Chosen by a sweep on small MLPs (Sec. 6.1, Fig. 4); used in all main training data.
  • noise period c = 16
    Same sweep; probability 1/c per time step of adding noise.
  • patch size M and jump level J = M=0 (and M=1 variant), J=1
    Design choices for spatial receptive field; not fitted to simulation metrics.
  • neural network hyperparameters = MLP hidden 256, layers 17; RNN hidden 150; Transformer token 128, layers 4, ensemble 4, sequence length 128
    Selected by validation loss and training cost; affect accuracy/runtime but are not physical constants.
  • stabilization constant alpha_0 = 0.04
    Standard FEM pressure-gradient stabilization constant (Sec. 3.2); not specific to the ML method.
axioms (5)
  • standard math The equal-order Q2/Q2 finite element pair with local projection stabilization yields a consistent, inf-sup-stable discretization of the Navier-Stokes equations.
    Sec. 3.2; standard FEM theory as cited in [52].
  • domain assumption The fine-grid reference solution is a valid ground truth, and the metrics (divergence, drag/lift, local error) capture physical fidelity.
    Sec. 5; there is no manufactured-solution or grid-convergence validation of the NN-corrected trajectories, and Sec. 7.3 admits this gap.
  • ad hoc to paper One replay pass over hybrid simulation data is sufficient to condition the network on its own error distribution.
    Sec. 6.3; the two-stage replay design is justified empirically, not theoretically, and the paper notes deviations appear only after ~250 steps.
  • domain assumption Local patch predictions with normalized inputs and a 1-ring neighborhood contain enough information to correct non-local Navier-Stokes dynamics on unstructured meshes.
    Sec. 4.1-4.4; tested empirically but not proven; exact fine-velocity feedback inputs failed (Appendix C).
  • ad hoc to paper Noise and rotation augmentation produce training samples that resemble the neural network's own inference errors.
    Sec. 6.1-6.2; an empirical heuristic. The paper itself notes augmented MLPs are far from equivariant.

pith-pipeline@v1.3.0-alltime-deepseek · 30380 in / 10795 out tokens · 119250 ms · 2026-08-03T08:29:45.886785+00:00 · methodology

0 comments
read the original abstract

The deep neural network multigrid solver (DNN-MG) combines a coarse-grid finite element simulation with a deep neural network that corrects the solution on finer grid levels, thereby improving the computational efficiency. In this work, we discuss various design choices for the DNN-MG method and demonstrate significant improvements in accuracy and generalizability when applied to the solution of the nonstationary Navier-Stokes equations. We investigate the stability of the hybrid simulation and show how the neural networks can be made more robust with the help of replay buffers. After an initial single-step training, we run the hybrid simulation for extended periods and compute new reference solutions of the neural network perturbed state for each step. By retraining on this data, the error caused by the neural network over multiple time-steps due to distributional shift can be effectively reduced without the need for a differentiable numerical solver. Furthermore, we compare multiple neural network architectures, including recurrent neural networks and Transformers, and study their ability to utilize more information from an increased temporal and spatial receptive field. Transformers allow us to make use of information from cells outside the predicted patch even with unstructured meshes while maintaining the locality of our approach. This can further improve the accuracy of DNN-MG without a significant impact on performance.

Figures

Figures reproduced from arXiv: 2601.16598 by Christian Lessig, Nils Margenberg, Robert Jendersie, Thomas Richter.

Figure 1
Figure 1. Figure 1: Comparison of the single obstacle case with reference simulations refinement levels 5 and 6 and the NN enhanced [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Design choices regarding patches, which are the basic units for the prediction. For increased clarity, the indicated [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Geometry of two example cases (ro1, so4). Each channel has a parabolic inflow profile [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The divergence Jdiv (15) and time-averaged velocity error ev¯ (16) of NNs trained on noise augmented data. During training data generation the numerical solution is perturbed by adding noise sampled from a normal distribution with mean µ = 0 and standard deviation σ to the velocity components with a probability of c−1 per time-step (c = 1 means every time-step). For reference, the mean magnitude of velocit… view at source ↗
Figure 5
Figure 5. Figure 5: Functionals of the single obstacle case ro1 in the standard orientation and with the whole domain rotated by [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Metrics of NN enhanced simulations compared with the reference simulations for ro9, the 9-round-obstacles case [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Divergence Jdiv on ro1 for 4 MLPs each, taken from n0 = 850 over k = 200 steps for simulations with increasing Reynolds number ℜ. The viscosity ν = 0.001 is the same in all training cases, giving ℜ = 100, and reduced to ν = 0.0005 for the highest Reynolds number. are difficult to evaluate because they lead to symmetric solutions that are instable and highly sensitive. The symmetry is quickly broken by DNN-… view at source ↗
Figure 8
Figure 8. Figure 8: Validation loss of different cases during replay training for 4 NNs of each architecture. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Functionals on ro1 for 4 NNs each, taken from [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Runtime of the simulation for different cases. The bars show the mean from 4 runs and the whiskers indicate the [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 23 canonical work pages · 2 internal anchors

  1. [1]

    S. L. Brunton, J. N. Kutz, Promising directions of machine learning for partial differential equations, Nature Computational Science 4 (7) (2024) 483–494. doi:10.1038/s43588-024-00643-2

    doi:10.1038/s43588-024-00643-2 2024

  2. [2]

    Kimmritz, T

    M. Kimmritz, T. Richter, Parallel multigrid method for finite element simulations of complex flow problems on locally refined meshes, Numerical Linear Algebra with Applications 18 (2011) 615–636

    2011

  3. [3]

    Ahmed, C

    N. Ahmed, C. Bartsch, V. John, U. Wilbrandt, An assessment of some solvers for saddle point problems emerging from the incompressible Navier–Stokes equations 331 (2018) 492–513. doi:10.1016/j.cma.2017.12.004

    work page doi:10.1016/j.cma.2017.12.004 2018

  4. [4]

    A. Ghai, C. Lu, X. Jiao, A comparison of preconditioned krylov subspace methods for large- scale nonsymmetric linear systems, Numerical Linear Algebra with Applications 26 (1) (Oct. 2018). doi:10.1002/nla.2215

    work page doi:10.1002/nla.2215 2018

  5. [5]

    Liebchen, R

    M. Liebchen, R. Jendersie, U. Kaya, C. Lessig, T. Richter, An adaptive finite element multigrid solver using gpu acceleration (May 2024). arXiv:2405.05047, doi:10.48550/ARXIV.2405.05047

    work page doi:10.48550/arxiv.2405.05047 2024

  6. [6]

    Thomas, A

    S. Thomas, A. H. Baker, Efficient gmres+amg on gpus: Composite smoothers and mixedV-cycles, SIAM Journal on Scientific Computing 46 (5) (2024) S246–S270. doi:10.1137/23m1578632

    work page doi:10.1137/23m1578632 2024

  7. [7]

    Munch, T

    P. Munch, T. Heister, L. Prieto Saavedra, M. Kronbichler, Efficient distributed matrix-free multigrid methods on locally refined meshes for fem computations, ACM Trans. Parallel Comput. 10 (1) (Mar. 2023). doi:10.1145/3580314

    doi:10.1145/3580314 2023

  8. [8]

    I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differ- ential equations, Trans. Neur. Netw. 9 (5) (1998) 987–1000. doi:10.1109/72.712178. URLhttps://doi.org/10.1109/72.712178

    doi:10.1109/72.712178 1998

  9. [9]

    Raissi, P

    M. Raissi, P. Perdikaris, G. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686–707. doi:10.1016/j.jcp.2018.10.045

    doi:10.1016/j.jcp.2018.10.045 2019

  10. [10]

    W. E, B. Yu, The deep ritz method: A deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics 6 (1) (2018) 1–12. doi:10.1007/s40304-018- 0127-z

    work page doi:10.1007/s40304-018- 2018

  11. [11]

    L. Lu, X. Meng, Z. Mao, G. Karniadakis, DeepXDE: A Deep Learning Library for Solving Differential Equations, SIAM Review 63 (1) (2021) 208–228. doi:10.1137/19M1274067

    doi:10.1137/19m1274067 2021

  12. [12]

    T. G. Grossmann, U. J. Komorowska, J. Latz, C.-B. Schönlieb, Can physics-informed neural net- works beat the finite element method?, IMA Journal of Applied Mathematics 89 (1) (2024) 143–174. doi:10.1093/imamat/hxae011

    doi:10.1093/imamat/hxae011 2024

  13. [13]

    J. D. Toscano, V. Oommen, A. J. Varghese, Z. Zou, N. Ahmadi Daryakenari, C. Wu, G. E. Karniadakis, From pinns to pikans: recent advances in physics-informed machine learning, Machine Learning for Computational Science and Engineering 1 (1) (Mar. 2025). doi:10.1007/s44379-025-00015-1

    doi:10.1007/s44379-025-00015-1 2025

  14. [14]

    X. Guo, W. Li, F. Iorio, Convolutional neural networks for steady flow approximation, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, ACM, 2016, pp. 481–490. doi:10.1145/2939672.2939738

    arXiv 2016

  15. [15]

    Bhatnagar, Y

    S. Bhatnagar, Y. Afshar, S. Pan, K. Duraisamy, S. Kaushik, Prediction of aerodynamic flow fields using convolutional neural networks, Computational Mechanics 64 (2) (2019) 525–545. doi:10.1007/s00466- 019-01740-0. 29

    doi:10.1007/s00466- 2019

  16. [16]

    Eichinger, A

    M. Eichinger, A. Heinlein, A. Klawonn, Stationary Flow Predictions Using Convolutional Neural Net- works, in: F. J. Vermolen, C. Vuik (Eds.), Numerical Mathematics and Advanced Applications ENU- MATH 2019, Lecture Notes in Computational Science and Engineering, Springer International Pub- lishing, 2021, pp. 541–549. doi:10.1007/978-3-030-55874-1_53

    work page doi:10.1007/978-3-030-55874-1_53 2019

  17. [17]

    Eichinger, A

    M. Eichinger, A. Heinlein, A. Klawonn, Surrogate convolutional neural network models for steady computational fluid dynamics simulations, Electronic Transactions on Numerical Analysis 56 (2022) 235–255

    2022

  18. [18]

    L. Lu, P. Jin, G. Pang, Z. Zhang, G. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence 3 (3) (2021) 218–229. doi:10.1038/s42256-021-00302-5

    doi:10.1038/s42256-021-00302-5 2021

  19. [19]

    Kovachki, Z

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, A. Anandkumar, Neural operator: Learning maps between function spaces with applications to pdes, Journal of Machine Learn- ing Research 24 (89) (2023) 1–97. URLhttp://jmlr.org/papers/v24/21-1524.html

    2023

  20. [20]

    Nayak, S

    D. Nayak, S. Goswami, Ti-deeponet: Learnable time integration for stable long-term extrapolation (May 2025). arXiv:2505.17341, doi:10.48550/ARXIV.2505.17341

    doi:10.48550/arxiv.2505.17341 2025

  21. [21]

    W. Diab, M. Al Kobaisi, Temporal neural operator for modeling time-dependent physical phenomena, Scientific Reports 15 (1) (Sep. 2025). doi:10.1038/s41598-025-16922-5

    doi:10.1038/s41598-025-16922-5 2025

  22. [22]

    I. Luz, M. Galun, H. Maron, R. Basri, I. Yavneh, Learning Algebraic Multigrid Using Graph Neural Networks, in: Proceedings of the 37th International Conference on Machine Learning, PMLR, 2020, pp. 6489–6499. URLhttps://proceedings.mlr.press/v119/luz20a.html

    2020

  23. [23]

    Markidis, The old and the new: Can physics-informed deep-learning replace traditional linear solvers?, Frontiers in Big Data 4 (Nov

    S. Markidis, The old and the new: Can physics-informed deep-learning replace traditional linear solvers?, Frontiers in Big Data 4 (Nov. 2021). doi:10.3389/fdata.2021.669097

    arXiv 2021

  24. [24]

    Huang, R

    R. Huang, R. Li, Y. Xi, Learning Optimal Multigrid Smoothers via Neural Networks, SIAM J. Sci. Comput. 45 (3) (2023) S199–S225. doi:10.1137/21M1430030

    doi:10.1137/21m1430030 2023

  25. [25]

    Accelerating multigrid solver with generative super-resolution

    F. Holguin, G. Sidharth, G. Portwood, Accelerating multigrid solver with generative super-resolution (Mar. 2024). arXiv:2403.07936, doi:10.48550/ARXIV.2403.07936

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2403.07936 2024

  26. [26]

    Sofos, D

    F. Sofos, D. Drikakis, A review of deep learning for super-resolution in fluid flows, Physics of Fluids 37 (4) (Apr. 2025). doi:10.1063/5.0265738

    work page doi:10.1063/5.0265738 2025

  27. [27]

    Y. Xie, E. Franz, M. Chu, N. Thuerey, Tempogan: A temporally coherent, volumetric gan for super- resolution fluid flow, ACM Trans. Graph. 37 (4) (Jul. 2018). doi:10.1145/3197517.3201304. URLhttps://doi.org/10.1145/3197517.3201304

    arXiv 2018

  28. [28]

    Schmidt, L

    J. Schmidt, L. Schmidt, F. M. Strnad, N. Ludwig, P. Hennig, A generative framework for probabilistic, spatiotemporally coherent downscaling of climate simulation, npj Climate and Atmospheric Science 8 (1) (Jul. 2025). doi:10.1038/s41612-025-01157-y

    work page doi:10.1038/s41612-025-01157-y 2025

  29. [29]

    Kochkov, J

    D. Kochkov, J. Yuval, I. Langmore, P. Norgaard, J. Smith, G. Mooers, M. Klöwer, J. Lottes, S. Rasp, P. Düben, S. Hatfield, P. Battaglia, A. Sanchez-Gonzalez, M. Willson, M. P. Brenner, S. Hoyer, Neural general circulation models for weather and climate, Nature 632 (8027) (2024) 1060–1066. doi:10.1038/s41586-024-07744-y

    doi:10.1038/s41586-024-07744-y 2024

  30. [30]

    Gregory, M

    W. Gregory, M. Bushuk, Y.-F. Zhang, A. Adcroft, L. Zanna, C. McHugh, L. Jia, Advancing global sea ice prediction capabilities using a fully-coupled climate model with integrated machine learning (2025). doi:10.48550/ARXIV.2505.18328. 30

    work page doi:10.48550/arxiv.2505.18328 2025

  31. [31]

    Witte, F

    M. Witte, F. R. Lapolli, P. Freese, S. Götschel, D. Ruprecht, P. Korn, C. Kadow, Dynamic deep learning based super-resolution for the shallow water equations, Machine Learning: Science and Technology 6 (1) (2025) 015060. doi:10.1088/2632-2153/ada19f

    work page doi:10.1088/2632-2153/ada19f 2025

  32. [32]

    Fabra, J

    A. Fabra, J. Baiges, R. Codina, Finite element approximation of wave problems with correcting terms basedontrainingartificialneuralnetworkswithfinesolutions, ComputerMethodsinAppliedMechanics and Engineering 399 (2022) 115280. doi:10.1016/j.cma.2022.115280

    arXiv 2022

  33. [33]

    Manrique de Lara, E

    F. Manrique de Lara, E. Ferrer, Accelerating high order discontinuous Galerkin solvers using neural networks: 1D Burgers’ equation, Computers & Fluids 235 (2022) 105274. doi:10.1016/j.compfluid.2021.105274

    arXiv 2022

  34. [34]

    Manrique de Lara, E

    F. Manrique de Lara, E. Ferrer, Accelerating high order discontinuous Galerkin solvers using neural networks: 3D compressible Navier-Stokes equations, Journal of Computational Physics 489 (2023) 112253. doi:10.1016/j.jcp.2023.112253

    arXiv 2023

  35. [35]

    Brandstetter, D

    J. Brandstetter, D. Worrall, M. Welling, Message passing neural pde solvers (Feb. 2022). arXiv:2202.03376, doi:10.48550/ARXIV.2202.03376

    Pith/arXiv arXiv doi:10.48550/arxiv.2202.03376 2022

  36. [36]

    K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, Q. Tian, Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast (2022). arXiv:2211.02556, doi:10.48550/arXiv.2211.02556

    Pith/arXiv arXiv doi:10.48550/arxiv.2211.02556 2022

  37. [37]

    R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, A. Pritzel, S. Ravuri, T. Ewalds, F. Alet, Z. Eaton-Rosen, W. Hu, A. Merose, S. Hoyer, G. Holland, J. Stott, O. Vinyals, S. Mohamed, P. Battaglia, GraphCast: Learning skillful medium-range global weather forecasting (2022). arXiv:2212.12794, doi:10.48550/arXiv.2212.12794

    Pith/arXiv arXiv doi:10.48550/arxiv.2212.12794 2022

  38. [38]

    S. Lang, M. Alexe, M. Chantry, J. Dramsch, F. Pinault, B. Raoult, M. C. A. Clare, C. Lessig, M. Maier-Gerber, L. Magnusson, Z. B. Bouallègue, A. P. Nemesio, P. D. Dueben, A. Brown, F. Pap- penberger, F. Rabier, Aifs – ecmwf’s data-driven forecasting system (Jun. 2024). arXiv:2406.01465, doi:10.48550/ARXIV.2406.01465

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.01465 2024

  39. [39]

    N. D. Brenowitz, C. S. Bretherton, Prognostic validation of a neural network unified physics parame- terization, Geophysical Research Letters 45 (12) (2018) 6289–6298. doi:10.1029/2018gl078510

    doi:10.1029/2018gl078510 2018

  40. [40]

    J. Chen, M. Zhang, T. Zhang, W. Lin, W. Xue, Stable simulation of the community atmosphere model using machine-learning physical parameterization trained with experience replay, Journal of Advances in Modeling Earth Systems 17 (6) (Jun. 2025). doi:10.1029/2024ms004722

    work page doi:10.1029/2024ms004722 2025

  41. [41]

    K. Chen, T. Han, F. Ling, J. Gong, L. Bai, X. Wang, J.-J. Luo, B. Fei, W. Zhang, X. Chen, L. Ma, T. Zhang, R. Su, Y. Ci, B. Li, X. Yang, W. Ouyang, The operational medium-range deterministic weather forecasting can be extended beyond a 10-day lead time, Communications Earth & Environment 6 (1) (Jul. 2025). doi:10.1038/s43247-025-02502-y

    doi:10.1038/s43247-025-02502-y 2025

  42. [42]

    S. Rasp, Coupled online learning as a way to tackle instabilities and biases in neural network parame- terizations: general algorithms and lorenz 96 case study (v1.0), Geoscientific Model Development 13 (5) (2020) 2185–2196. doi:10.5194/gmd-13-2185-2020

    doi:10.5194/gmd-13-2185-2020 2020

  43. [43]

    Gregory, M

    W. Gregory, M. Bushuk, Y. Zhang, A. Adcroft, L. Zanna, Machine learning for online sea ice bias correction within global ice-ocean simulations, Geophysical Research Letters 51 (3) (Jan. 2024). doi:10.1029/2023gl106776

    work page doi:10.1029/2023gl106776 2024

  44. [44]

    Pedersen, L

    C. Pedersen, L. Zanna, J. Bruna, Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos (Mar. 2025). arXiv:2503.18731, doi:10.48550/ARXIV.2503.18731. 31

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.18731 2025

  45. [45]

    Margenberg, T

    N. Margenberg, T. Richter, Parallel time-stepping for fluid-structure interactions, Math- ematical Modelling of Natural Phenomena 16 (2021) 20, https://arxiv.org/abs/1907.01252. doi:10.1051/mmnp/2021005

    Pith/arXiv arXiv 2021

  46. [46]

    Margenberg, C

    N. Margenberg, C. Lessig, T. Richter, Structure preservation for the Deep Neural Network Multigrid Solver, ETNA - Electronic Transactions on Numerical Analysis 56 (2021) 86–101. doi:10.1553/etna_vol56s86

    work page doi:10.1553/etna_vol56s86 2021

  47. [47]

    Margenberg, R

    N. Margenberg, R. Jendersie, C. Lessig, T. Richter, Dnn-mg: A hybrid neural network/finite element method with applications to 3d simulations of the navier–stokes equations 420 116692. doi:10.1016/j.cma.2023.116692

    arXiv 2023

  48. [48]

    Kapustsin, U

    U. Kapustsin, U. Kaya, T. Richter, A hybrid finite element/neural network solver and its application to the Poisson problem 23 (3) (2023) e202300135. doi:10.1002/pamm.202300135

    doi:10.1002/pamm.202300135 2023

  49. [50]

    Becker, M

    R. Becker, M. Braack, D. Meidner, T. Richter, B. Vexler, The finite element toolkitGascoigne, http://www.gascoigne.de

  50. [51]

    Richter, Fluid-Structure Interactions: Models, Analysis and Finite Elements, Vol

    T. Richter, Fluid-Structure Interactions: Models, Analysis and Finite Elements, Vol. 118 of Lec- ture Notes in Computational Science and Engineering, Springer International Publishing, 2017. doi:10.1007/978-3-319-63970-3

    work page doi:10.1007/978-3-319-63970-3 2017

  51. [52]

    Becker, M

    R. Becker, M. Braack, A finite element pressure gradient stabilization for the Stokes equations based on local projections, Calcolo 38 (4) (2001) 173–199. doi:10.1007/s10092-001-8180-4

    doi:10.1007/s10092-001-8180-4 2001

  52. [53]

    C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations, SIAM, Philadelphia, 1995

    1995

  53. [54]

    Becker, M

    R. Becker, M. Braack, Multigrid techniques for finite elements on locally refined meshes, Numeri- cal Linear Algebra with Applications 7 (2000) 363–379. doi:10.1002/1099-1506(200009)7:6<363::AID- NLA202>3.0.CO;2-V

    work page doi:10.1002/1099-1506(200009)7:6 2000

  54. [55]

    Failer, T

    L. Failer, T. Richter, A Newton multigrid framework for optimal control of fluid-structure interactions, Optimization and Engineering 22 (4) (2020). doi:10.1007/s11081-020-09498-8

    work page doi:10.1007/s11081-020-09498-8 2020

  55. [56]

    Kapustsin, U

    U. Kapustsin, U. Kaya, T. Richter, Error analysis for hybrid finite element/neural network discretiza- tions (2023). doi:10.48550/ARXIV.2310.11271

    work page doi:10.48550/arxiv.2310.11271 2023

  56. [57]

    Golestaneh, M

    P. Golestaneh, M. Taheri, J. Lederer, How many samples are needed to train a deep neural network?, in: The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=q6zrZbth1F

    2025

  57. [58]

    Deep neural networks for geometric multigrid methods

    N. Margenberg, R. Jendersie, T. Richter, C. Lessig, Deep neural networks for geometric multigrid methods (2021). arXiv:2106.07687, doi:10.48550/arXiv.2106.07687

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2106.07687 2021

  58. [59]

    J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization (Jul. 2016). arXiv:1607.06450, doi:10.48550/ARXIV.1607.06450

    Pith/arXiv arXiv doi:10.48550/arxiv.1607.06450 2016

  59. [60]

    Huang, J

    L. Huang, J. Qin, Y. Zhou, F. Zhu, L. Liu, L. Shao, Normalization techniques in training DNNs: Methodology, analysis and application 45 (8) (2023) 10173–10196. doi:10.1109/tpami.2023.3250241

    arXiv 2023

  60. [61]

    Ramachandran, B

    P. Ramachandran, B. Zoph, Q. V. Le, Searching for activation functions (Oct. 2017). arXiv:1710.05941, doi:10.48550/ARXIV.1710.05941

    Pith/arXiv arXiv doi:10.48550/arxiv.1710.05941 2017

  61. [62]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Decoupled Weight Decay Regularization. URLhttps://openreview.net/forum?id=Bkg6RiCqY7 32

  62. [63]

    D. P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. URLhttp://arxiv.org/abs/1412.6980

    Pith/arXiv arXiv 2015

  63. [64]

    Margenberg, D

    N. Margenberg, D. Hartmann, C. Lessig, T. Richter, A neural network multigrid solver for the Navier- Stokes equations, Journal of Computational Physics 460 (2022) 110983. doi:10.1016/j.jcp.2022.110983

    arXiv 2022

  64. [65]

    K. Cho, B. van Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk, Y. Bengio, Learning phrase repre- sentations using RNN encoder-decoder for statistical machine translation, in: Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), 2014

    2014

  65. [66]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. u. Kaiser, I. Polosukhin, Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, Vol. 30, Curran Associates, Inc., 2017. URLhttps://proceedings.neurips.cc/pap...

    2017

  66. [67]

    Palanisamy, V

    B. Palanisamy, V. Hassija, A. Chatterjee, A. Mandal, D. Chakraborty, A. Pandey, G. S. S. Chalapathi, D. Kumar, Transformers for vision: A survey on innovative methods for computer vision, IEEE Access 13 (2025) 95496–95523. doi:10.1109/access.2025.3571735

    arXiv 2025

  67. [68]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Assoc...

    doi:10.18653/v1/n19-1423 2019

  68. [69]

    Lessig, I

    C. Lessig, I. Luise, B. Gong, M. Langguth, S. Stadtler, M. Schultz, Atmorep: A stochastic model of atmosphere dynamics using large scale representation learning (Aug. 2023). arXiv:2308.13280, doi:10.48550/ARXIV.2308.13280

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.13280 2023

  69. [70]

    D. Nix, A. Weigend, Estimating the mean and variance of the target probability distribution, in: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), IEEE, 1994, pp. 55–60 vol.1. doi:10.1109/icnn.1994.374138

    arXiv 1994

  70. [71]

    Heywood, R

    J. Heywood, R. Rannacher, S. Turek, Artificial boundaries and flux and pressure conditions for the incompressible Navier-Stokes equations 22 (1992) 325–352

    1992

  71. [72]

    Schäfer, S

    M. Schäfer, S. Turek, Benchmark computations of laminar flow around a cylinder. (With support by F. Durst, E. Krause and R. Rannacher), in: E. Hirschel (Ed.), Flow Simulation with High-Performance Computers II. DFG priority research program results 1993-1995, no. 52 in Notes Numer. Fluid Mech., Vieweg, Wiesbaden, 1996, pp. 547–566

    1993

  72. [73]

    Bradbury, R

    J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, Q. Zhang, JAX: composable transformations of Python+NumPy programs (2018). URLhttp://github.com/google/jax

    2018

  73. [74]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, PyTorch: An Imperative Style, High-Performance Deep Learning Library, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d’...

    2019

  74. [75]

    Moses, V

    W. Moses, V. Churavy, Instead of rewriting foreign code for machine learning, automatically synthesize fast gradients, in: H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems, Vol. 33, Curran Associates, Inc., 2020, pp. 12472–12485. URLhttps://proceedings.neurips.cc/paper/2020/file/9332c513...

    2020

  75. [76]

    W. S. Moses, S. H. K. Narayanan, L. Paehler, V. Churavy, M. Schanen, J. Hückelheim, J. Doerfert, P. Hovland, Scalable automatic differentiation of multiple parallel paradigms through compiler augmen- tation, in: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC ’22, IEEE Press, 2022

    2022

  76. [77]

    Jendersie, C

    R. Jendersie, C. Lessig, T. Richter, A gpu parallelization of the nextsim-dg dynamical core (v0.3.1), Geoscientific Model Development 18 (10) (2025) 3017–3040. doi:10.5194/gmd-18-3017-2025

    work page doi:10.5194/gmd-18-3017-2025 2025

  77. [78]

    M. Rath, A. P. Condurache, Boosting deep neural networks with geometrical prior knowledge: a survey, Artificial Intelligence Review 57 (4) (Mar. 2024). doi:10.1007/s10462-024-10722-5

    work page doi:10.1007/s10462-024-10722-5 2024

  78. [79]

    URLhttps://github.com//NVIDIA/TransformerEngine

    NVIDIA, Transformer Engine, [accessed 12-September-2025] (2025). URLhttps://github.com//NVIDIA/TransformerEngine

    2025

  79. [80]

    S. S. Sengar, A. B. Hasan, S. Kumar, F. Carroll, Generative artificial intelligence: a systematic review and applications, Multimedia Tools and Applications 84 (21) (2024) 23661–23700. doi:10.1007/s11042- 024-20016-1

    doi:10.1007/s11042- 2024

  80. [81]

    Braack, P

    M. Braack, P. Mucha, Directional do-nothing condition for the navier-stokes equations, Journal of Computational Mathematics 32 (5) (2014) 507–521. doi:10.4208/jcm.1405-m4347

    work page doi:10.4208/jcm.1405-m4347 2014

Showing first 80 references.