Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Some Best Practices in Operator Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper establishes four robust defaults for operator learning: GELU activations, no dropout, stochastic weight averaging at the original learning rate, and no learning-rate finder.

desk verdict Useful activation/dropout tables, but the SWA and learning-rate-finder recommendations are contradicted or unsupported by the paper's own data. read the letter →

arxiv 2412.06686 v1 pith:I7U5UK52 submitted 2024-12-09 cs.LG physics.comp-ph

Dustin Enyeart , Guang Lin This is my paper · ORCID
classification cs.LGphysics.comp-ph
keywords operatorlearningbestpracticeshyperparametersactivationfunctiondropoutstochasticweightaveragingratefinderneuraloperators
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to turn hyperparameter choice in operator learning from an expensive search into a small set of defaults. Training DeepONets, Fourier neural operators, and Koopman autoencoders on five differential equations, it compares activation functions, dropout, stochastic weight averaging, and learning-rate finders one at a time. The consistent result is that GELU always beat tanh, ReLU, and ELU; dropout always made accuracy worse; stochastic weight averaging helped when its learning rate was the original or one tenth of it; and a learning-rate finder did not find good rates. If these trends hold beyond the six tested cases, practitioners can adopt these defaults and skip much of the hyperparameter search.

What carries the argument

The machinery is a controlled hyperparameter sweep: each architecture is trained on two benchmark equations with a base learning rate of 0.001, one choice is varied at a time, and the mean squared error is compared. The test beds are the pendulum, Lorenz system, and fluid-attractor ODEs plus Burger's and KdV PDEs. For Koopman autoencoders, the loss includes accuracy, encoder–decoder consistency, and a unitarity penalty on the Koopman operator, which defines the training regime in which the comparisons happen.

What would settle it

Train the same three architectures on a new operator-learning task outside the tested set, such as a two-dimensional PDE with noisy initial conditions, comparing GELU against tanh/ReLU, dropout against none, and SWA at one-tenth the base rate; any result where GELU is not best, dropout helps, or lower-rate SWA hurts would falsify the recommendations.

Watch

Extended reading notes

Core claim

The central claim is a set of four recommendations for operator learning: use GELU as the activation function; do not use dropout; use stochastic weight averaging with a learning rate at most the original learning rate; and do not use a learning-rate finder. The evidence is empirical: six experiments, one per architecture–equation pair, with errors reported in tables. The paper also notes that tanh, despite being common in practice, was worst in three of the six activation experiments, while ReLU was worst in the other three. The claims are presented as robust trends rather than as a theoretical guarantee.

Load-bearing premise

The recommendations assume the six tested benchmark combinations—three architectures matched with two equations each—are representative enough to support defaults for all operator learning tasks.

Editorial extensions

If this is right

  • A full hyperparameter search over activation functions and dropout can be skipped, saving a large share of the compute.
  • GELU can replace tanh as the default smooth activation, even though tanh is the more common choice in practice.
  • Stochastic weight averaging at the original or one-tenth learning rate becomes a cheap accuracy improvement with no extra search.
  • Learning-rate finders should be avoided for these architectures; a fixed standard rate with SWA is the safer route.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test noisy data or small training sets, so its no-dropout recommendation may not extend to regimes where dropout's regularization is most useful.
  • All benchmark equations are low-dimensional, so whether GELU remains best on high-dimensional PDEs is an open question the paper leaves unanswered.
  • The learning-rate finder result is tied to the specific finder implementation used, and other finder schemes might behave differently.
  • If the SWA trend reflects flat-minima selection, a natural extension would be to test the same default in transfer learning or fine-tuning settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This manuscript reports a series of hyperparameter experiments for three neural-operator architectures (DeepONet, Fourier neural operator, and Koopman autoencoder) on five differential equations. It examines four choices: activation function, dropout, stochastic weight averaging, and learning-rate finder. The paper concludes with four recommended defaults: use gelu, do not use dropout, use stochastic weight averaging with a learning rate no larger than the base learning rate, and do not use a learning-rate finder. The reported evidence consists mainly of error tables for activations, dropout, and SWA, plus a qualitative account of learning-rate-finder experiments.

Significance. If the claims held, these would be practical and inexpensive defaults for practitioners, and the public code repository and the explicit non-comparative framing across architectures are strengths. The activation and dropout findings are consistently supported by all six reported benchmark pairs. The self-citation to Ref. [43] is not circular: that reference supplies only the Koopman training loss, while the activation, dropout, SWA, and learning-rate-finder conclusions are independent empirical findings. However, the SWA recommendation is contradicted by two entries in the paper's own tables, and the learning-rate-finder recommendation is presented without any quantitative evidence. As stated, the Section 8 recommendations therefore overstate what the reported data can support.

major comments (3)
  1. [6, Tables 7 and 9] The sentence in Section 6, 'In each experiment, stochastic weight averaging improved the accuracy of the model when the learning rate was the same or one tenth of the original learning rate,' is contradicted by the paper's own tables. For the DeepONet KdV experiment, the no-SWA error is 1.261e-2, while SWA with LR=1e-4 gives 4.700e-2, a degradation of roughly 3.7x. For the Koopman pendulum experiment, the no-SWA error is 4.370e-4, while SWA with LR=1e-3 gives 4.919e-4. Both LR values lie inside the range 'at most the original learning rate' that Section 8 recommends. Counting all within-range rows (LR=1e-3 and LR=1e-4), 2 of 12 rows are worse than no-SWA, so the claim 'in each experiment' is false and the Section 8 recommendation is stronger than the data allow.
  2. [7] Section 7 recommends not using a learning-rate finder, but it reports no measurements: there is no table, no error value, no experiment count, no description of the swept learning-rate range, no definition of the optimization protocol, and no comparison with a tuned baseline. A recommendation with zero reported quantitative evidence cannot be evaluated or replicated. Since this is one of the four headline conclusions in Section 8, the empirical basis for it is missing.
  3. [4 and 5] All activation and dropout tables report a single error value per configuration, with no number of random seeds, no standard deviation, and no statement about whether the reported comparisons were repeated. This matters for the strength of the claimed defaults: for example, in Table 3 the pendulum elu error (1.360e-4) is lower than the gelu error (1.752e-4), and in Table 2 the KdV gelu error (3.731e-3) is only slightly below the elu error (3.909e-3). Without run-to-run variability, the universal statement that gelu is best in operator learning is not yet supported with statistical assurance, even though the six-table trend is suggestive.
minor comments (4)
  1. [Sections 4 and 6] There are several typos: 'preformed' should be 'performed', 'is is recommend' should be 'it is recommended', and 'The dropout ;is' should be 'The dropout is'.
  2. [Section 3] The differential equation notation 'd2t', 'd2x', and 'd3x' should be rendered as d²t, d²x, and d³x for readability.
  3. [Section 2.3] In the Koopman loss description, 'where E is the encoder, R is the encoder' should read 'where E is the encoder and R is the decoder'.
  4. [Table 8] The left caption says 'for the equation for burger's equation'; it should say 'for Burger's equation'. Also, 'Burger' should be capitalized consistently throughout the tables.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the best-practice recommendations are empirical summaries of reported tables, and the sole self-citation is a non-load-bearing training-loss choice.

full rationale

The paper's four recommendations are inductive summaries of measured errors: gelu from Tables 1-3, no dropout from Tables 4-6, stochastic weight averaging at a learning rate at most the base rate from Tables 7-9, and no learning-rate finder from Section 7. None of these quantities is defined in terms of the recommendation; each is a raw error value compared across configurations, so there is no self-definitional or fit-renamed-as-prediction step. The only self-citation is Ref. [43], used to justify the Koopman autoencoder loss: 'it is better to use additional loss terms during training [43].' That loss is an experimental input for one architecture, not a derived consequence of the recommended hyperparameters, and it does not by construction force the activation, dropout, SWA, or LR-finder outcomes. The tridiagonal mask and unitarity penalty are likewise stated assumptions, not smuggled ansatze. The skeptic's observation that Table 7 contains within-range SWA rows worse than baseline is a correctness and evidence-strength concern about the phrase 'in each experiment', not a circularity: the recommendation is overclaimed, not true by definition. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. Verdict: one minor, non-load-bearing self-citation; the empirical derivation chain is otherwise self-contained.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new theoretical entities are introduced. The paper's conclusions depend on a small set of hand-chosen experimental hyperparameters and on the assumption that the benchmark set is representative. The self-cited Koopman loss is a domain assumption that shapes the Koopman experiments.

free parameters (3)
  • base_learning_rate = 0.001
    Set once for all experiments; every SWA comparison is relative to this value, and no sweep or sensitivity analysis is reported.
  • swa_learning_rates = 1e-4, 1e-3, 1e-2, 1e-1
    Chosen by hand for the SWA experiments; the recommendation 'at most the original learning rate' depends on this grid.
  • dropout_rates = 0.05, 0.1, 0.15
    Chosen by hand; conclusion that dropout hurts is based only on these three rates plus zero.
assumptions (3)
  • domain assumption The six benchmark equation-architecture pairs are representative of operator learning tasks.
    Used to generalize experimental results into unqualified best practices in the conclusion; no evidence is given that the sample spans the diversity of operator-learning problems.
  • domain assumption Held-out mean squared error is the only meaningful measure of model quality.
    All comparisons and recommendations are based on a single scalar error per configuration; no other metrics, such as robustness or physical constraint satisfaction, are considered.
  • domain assumption The Koopman autoencoder training loss from the authors' own concurrent paper (Ref [43]) is appropriate for all Koopman experiments.
    Adopted without independent validation, though the hyperparameter conclusions for Koopman autoencoders depend on this loss choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Some Best Practices in Operator Learning." pith.science (2026). https://pith.science/paper/I7U5UK52

@misc{pith2026241206686,
  author       = {Pith},
  title        = {Pith review of: Some Best Practices in Operator Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I7U5UK52}},
  note         = {Machine review of arXiv:2412.06686}
}
read the original abstract

Hyperparameters searches are computationally expensive. This paper studies some general choices of hyperparameters and training methods specifically for operator learning. It considers the architectures DeepONets, Fourier neural operators and Koopman autoencoders for several differential equations to find robust trends. Some options considered are activation functions, dropout and stochastic weight averaging.

Figures

Figures reproduced from arXiv: 2412.06686 by the authors.

Figure 1
Figure 1. The DeepONet architecture: The input u is the input function, and the input x is the point where the output function is evaluated. Their encodings are denoted by Eu and Ex, respectively. The output is denoted by y. 2.2 Fourier Operator Networks Fourier neural operators, which are abbreviated as FNO, are a neural operator architecture [9–16]. They are based on spectral convolution layers. 2 [PITH_FULL_IMAGE:figures/… view at source ↗
Figure 2
Figure 2. A Fourier neural operator: The input is denoted by [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A spectral convolution layer: The input is denoted by [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Discretization of the Koopman formulation into a numerical scheme: [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Activation functions: The upper left is the hyperbolic tangent. The [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Progress for different activation functions: Both plots show the tra [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Validation loss using dropout: A DeepONet is used for the Lorenz [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Stochastic weight averaging: The optimizer dictates the learning rate [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Learning rate finder: The dot represents the optimal learning rate [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adversarial Autoencoders in Operator Learning

    cs.LG 2024-12 conditional novelty 4.0 of 10

    Adding adversarial autoencoder training improves DeepONet and Koopman autoencoder accuracy by 4% to 27% on five small-data benchmarks.

Reference graph

Works this paper leans on

63 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [43]

    Dustin Enyeart and Guang Lin.Loss Terms and Operator Forms of Koopman Autoencoders

  2. [1]

    Operator learning: Algorithms and analysis

    Nikola B Kovachki, Samuel Lanthaler, and Andrew M Stuart. “Operator learning: Algorithms and analysis”. In:arXiv preprint arXiv:2402.15715(2024)

    arXiv 2024

  3. [2]

    A mathematical guide to operator learning

    Nicolas Boullé and Alex Townsend. “A mathematical guide to operator learning”. In:arXiv preprint arXiv:2312.14688 (2023)

    arXiv 2023

  4. [3]

    NeuralNetworkApproximationstoSolutionOperatorsforPartialDifferential Equations

    NickWinovich.“NeuralNetworkApproximationstoSolutionOperatorsforPartialDifferential Equations”. PhD thesis. Purdue University, 2021. 17

    work page 2021

  5. [4]

    LearningnonlinearoperatorsviaDeepONetbasedontheuniversalapproximation theorem of operators

    LuLuetal.“LearningnonlinearoperatorsviaDeepONetbasedontheuniversalapproximation theorem of operators”. In:Nature Machine Intelligence 3.3 (Mar. 2021), pp. 218–229.issn: 2522-5839. doi: 10.1038/s42256-021-00302-5. url: http://dx.doi.org/10.1038/s42256- 021-00302-5

    doi:10.1038/s42256-021-00302-5 2021

  6. [5]

    Error estimates for deep- onets: A deep learning framework in infinite dimensions

    Samuel Lanthaler, Siddhartha Mishra, and George E Karniadakis. “Error estimates for deep- onets: A deep learning framework in infinite dimensions”. In:Transactions of Mathematics and Its Applications6.1 (2022), tnac001

    2022

  7. [6]

    A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials

    Somdatta Goswami et al. “A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials”. In:Computer Methods in Applied Mechanics and Engineering391 (2022), p. 114587

    2022

  8. [7]

    Novel DeepONet architecture to predict stresses in elastoplastic structures with variable complex geometries and loads

    Junyan He et al. “Novel DeepONet architecture to predict stresses in elastoplastic structures with variable complex geometries and loads”. In:Computer Methods in Applied Mechanics and Engineering 415 (2023), p. 116277

    2023

Show all 63 references
  1. [8]

    Learning time-dependent PDE via graph neural networks and deep operator network for robust accuracy on irregular grids

    Sung Woong Cho, Jae Yong Lee, and Hyung Ju Hwang. “Learning time-dependent PDE via graph neural networks and deep operator network for robust accuracy on irregular grids”. In: arXiv preprint arXiv:2402.08187(2024)

    2024 arXiv

  2. [9]

    A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data

    Lu Lu et al. “A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data”. In:Computer Methods in Applied Mechanics and Engineering 393 (2022), p. 114778

    2022

  3. [10]

    Fourier neural operator for parametric partial differential equations

    Zongyi Li et al. “Fourier neural operator for parametric partial differential equations”. In: arXiv preprint arXiv:2010.08895(2020)

    2020 arXiv

  4. [11]

    Toward a Better Understanding of Fourier Neural Operators: Analysis and Improvement from a Spectral Perspective

    Shaoxiang Qin et al. “Toward a Better Understanding of Fourier Neural Operators: Analysis and Improvement from a Spectral Perspective”. In:arXiv preprint arXiv:2404.07200(2024)

    2024 arXiv

  5. [12]

    Fourier neural operator with learned deformations for pdes on general ge- ometries

    Zongyi Li et al. “Fourier neural operator with learned deformations for pdes on general ge- ometries”. In:Journal of Machine Learning Research24.388 (2023), pp. 1–26

    2023

  6. [13]

    Lie point symmetry data aug- mentation for neural PDE solvers

    Johannes Brandstetter, Max Welling, and Daniel E Worrall. “Lie point symmetry data aug- mentation for neural PDE solvers”. In:International Conference on Machine Learning. PMLR. 2022, pp. 2241–2256. 18

    2022

  7. [14]

    On universal approximation and error bounds for Fourier neural operators

    Nikola Kovachki, Samuel Lanthaler, and Siddhartha Mishra. “On universal approximation and error bounds for Fourier neural operators”. In:Journal of Machine Learning Research22.290 (2021), pp. 1–76

    2021

  8. [15]

    Factorized fourier neural operators

    Alasdair Tran et al. “Factorized fourier neural operators”. In:arXiv preprint arXiv:2111.13802 (2021)

    2021 arXiv

  9. [16]

    Fourier features let networks learn high frequency functions in low di- mensionaldomains

    Matthew Tancik et al. “Fourier features let networks learn high frequency functions in low di- mensionaldomains”.In: Advances in neural information processing systems33(2020),pp.7537– 7547

    2020

  10. [17]

    Elias M Stein and Rami Shakarchi.Fourier analysis: an introduction. Vol. 1. Princeton Uni- versity Press, 2011

    2011

  11. [18]

    Springer Science & Business Media, 2011

    Kamisetty Ramamohan Rao, Do Nyeon Kim, and Jae Jeong Hwang.Fast Fourier transform- algorithms and applications. Springer Science & Business Media, 2011

    2011

  12. [19]

    Deep learning for universal linear embeddings of nonlinear dynamics

    Bethany Lusch, J Nathan Kutz, and Steven L Brunton. “Deep learning for universal linear embeddings of nonlinear dynamics”. In:Nature communications9.1 (2018), p. 4950

    2018

  13. [20]

    A survey on the methods and results of data-driven koopman analysis in the visualization of dynamical systems

    Nishaal Parmar, Hazem H Refai, and Thordur Runolfsson. “A survey on the methods and results of data-driven koopman analysis in the visualization of dynamical systems”. In:IEEE Transactions on Big Data8.3 (2020), pp. 723–738

    2020

  14. [21]

    Learning data-driven stable Koopman operators

    Giorgos Mamakoukas, Ian Abraham, and Todd D Murphey. “Learning data-driven stable Koopman operators”. In:Free radical biology & medicine.(2020)

    2020

  15. [22]

    Data-driven nonlinear stabilization using koop- man operator

    Bowen Huang, Xu Ma, and Umesh Vaidya. “Data-driven nonlinear stabilization using koop- man operator”. In:The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications(2020), pp. 313–334

    2020

  16. [23]

    Data-driven approximation of the Koopman generator: Model reduc- tion, system identification, and control

    Stefan Klus et al. “Data-driven approximation of the Koopman generator: Model reduc- tion, system identification, and control”. In:Physica D: Nonlinear Phenomena406 (2020), p. 132416

    2020

  17. [24]

    Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control

    Steven L Brunton et al. “Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control”. In:PloS one 11.2 (2016), e0150171

    2016

  18. [25]

    Applied koopmanism

    Marko Budišić, Ryan Mohr, and Igor Mezić. “Applied koopmanism”. In:Chaos: An Interdis- ciplinary Journal of Nonlinear Science22.4 (2012). 19

    2012

  19. [26]

    Multiresolution dynamic mode decomposi- tion

    J Nathan Kutz, Xing Fu, and Steven L Brunton. “Multiresolution dynamic mode decomposi- tion”. In:SIAM Journal on Applied Dynamical Systems15.2 (2016), pp. 713–735

    2016

  20. [27]

    Learning Koopman invariant sub- spaces for dynamic mode decomposition

    Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. “Learning Koopman invariant sub- spaces for dynamic mode decomposition”. In:Advances in neural information processing sys- tems 30 (2017)

    2017

  21. [28]

    Koopman-mode decomposition of the cylinder wake

    Shervin Bagheri. “Koopman-mode decomposition of the cylinder wake”. In:Journal of Fluid Mechanics 726 (2013), pp. 596–623

    2013

  22. [29]

    M Budišic, R Mohr, and I Mezic.The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications. 2020

    2020

  23. [30]

    Learning compositional koopman operators for model-based control

    Yunzhu Li et al. “Learning compositional koopman operators for model-based control”. In: arXiv preprint arXiv:1910.08264(2019)

    2019 arXiv

  24. [31]

    Data-driven approximations of dy- namical systems operators for control

    Eurika Kaiser, J Nathan Kutz, and Steven L Brunton. “Data-driven approximations of dy- namical systems operators for control”. In:The Koopman Operator in Systems and Control: Concepts, Methodologies, and Applications(2020), pp. 197–234

    2020

  25. [32]

    Deep learning of Koopman representation for control

    Yiqiang Han, Wenjian Hao, and Umesh Vaidya. “Deep learning of Koopman representation for control”. In:2020 59th IEEE Conference on Decision and Control (CDC). IEEE. 2020, pp. 1890–1895

    2020

  26. [33]

    Koopman-based control of a soft continuum manipulator under variable loading conditions

    Daniel Bruder et al. “Koopman-based control of a soft continuum manipulator under variable loading conditions”. In:IEEE robotics and automation letters6.4 (2021), pp. 6852–6859

    2021

  27. [34]

    Modeling and control of soft robots using the koopman operator and model predictive control

    Daniel Bruder et al. “Modeling and control of soft robots using the koopman operator and model predictive control”. In:arXiv preprint arXiv:1902.02827(2019)

    2019 arXiv

  28. [35]

    A data-driven koopman model predictive control framework for nonlinear partial differential equations

    Hassan Arbabi, Milan Korda, and Igor Mezić. “A data-driven koopman model predictive control framework for nonlinear partial differential equations”. In:2018 IEEE Conference on Decision and Control (CDC). IEEE. 2018, pp. 6409–6414

    2018

  29. [36]

    Model-based control using Koop- man operators

    Ian Abraham, Gerardo De La Torre, and Todd D Murphey. “Model-based control using Koop- man operators”. In:arXiv preprint arXiv:1709.01568(2017)

    2017 arXiv

  30. [37]

    Hamiltonian systems and transformation in Hilbert space

    Bernard O Koopman. “Hamiltonian systems and transformation in Hilbert space”. In:Pro- ceedings of the National Academy of Sciences17.5 (1931), pp. 315–318

    1931

  31. [38]

    Modern Koopman theory for dynamical systems

    Steven L Brunton et al. “Modern Koopman theory for dynamical systems”. In:arXiv preprint arXiv:2102.12086 (2021). 20

    2021 arXiv

  32. [39]

    What is the Koopman operator? a simplified treatment for discrete-time systems

    Adam L Bruce, Vera M Zeidan, and Dennis S Bernstein. “What is the Koopman operator? a simplified treatment for discrete-time systems”. In:2019 American Control Conference (ACC). IEEE. 2019, pp. 1912–1917

    2019

  33. [40]

    Understanding quantum physics: A user’s manual

    Michael A Morrison. Understanding quantum physics: A user’s manual. 1990

    1990

  34. [41]

    Hans C. Ohanian. Principles of Quantum Mechanics. 1989

    1989

  35. [42]

    Griffiths

    David J. Griffiths. Introduction to Quantum Mechanics. 2018

    2018

  36. [44]

    Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems

    Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007

    2007

  37. [45]

    A hierarchy of low-dimensional models for the transient and post- transient cylinder wake

    Bernd R Noack et al. “A hierarchy of low-dimensional models for the transient and post- transient cylinder wake”. In:Journal of Fluid Mechanics497 (2003), pp. 335–363

    2003

  38. [46]

    Randall J LeVeque and Randall J Leveque.Numerical methods for conservation laws. Vol. 214. Springer, 1992

    1992

  39. [47]

    Interaction of “solitons

    Norman J Zabusky and Martin D Kruskal. “Interaction of “solitons" in a collisionless plasma and the recurrence of initial states”. In:Physical review letters15.6 (1965), p. 240

    1965

  40. [48]

    Improving neural networks by preventing co-adaptation of feature detectors

    GE Hinton. “Improving neural networks by preventing co-adaptation of feature detectors”. In: arXiv preprint arXiv:1207.0580(2012)

    2012 arXiv

  41. [49]

    Dropout: a simple way to prevent neural networks from overfitting

    Nitish Srivastava et al. “Dropout: a simple way to prevent neural networks from overfitting”. In: The journal of machine learning research15.1 (2014), pp. 1929–1958

    2014

  42. [50]

    Averaging weights leads to wider optima and better generalization

    Pavel Izmailov et al. “Averaging weights leads to wider optima and better generalization”. In: arXiv preprint arXiv:1803.05407(2018)

    2018 arXiv

  43. [51]

    There are many consistent explanations of unlabeled data: Why you should average

    Ben Athiwaratkun et al. “There are many consistent explanations of unlabeled data: Why you should average”. In:arXiv preprint arXiv:1806.05594(2018)

    2018 arXiv

  44. [52]

    Improving stability in deep reinforcement learning with weight av- eraging

    Evgenii Nikishin et al. “Improving stability in deep reinforcement learning with weight av- eraging”. In: Uncertainty in artificial intelligence workshop on uncertainty in Deep learning. 2018. 21

    2018

  45. [53]

    Accessed: October 2024

    Pavel Izmailov and Andew Gordon Wilson.Stochastic Weight Averaging in PyTorch. Accessed: October 2024. 2019.url: https://pytorch.org/blog/stochastic-weight-averaging-in- pytorch/

    2024

  46. [54]

    Cyclical learning rates for training neural networks

    Leslie N Smith. “Cyclical learning rates for training neural networks”. In:2017 IEEE winter conference on applications of computer vision (WACV). IEEE. 2017, pp. 464–472

    2017

  47. [55]

    On the variance of the adaptive learning rate and beyond

    Liyuan Liu et al. “On the variance of the adaptive learning rate and beyond”. In:arXiv preprint arXiv:1908.03265 (2019)

    2019 arXiv

  48. [56]

    A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation

    Akhilesh Gotmare et al. “A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation”. In:arXiv preprint arXiv:1810.13243(2018)

    2018 arXiv

  49. [57]

    Automatic differentiation in PyTorch

    Adam Paszke et al. “Automatic differentiation in PyTorch”. In:NIPS-W. 2017

    2017

  50. [58]

    Programming pytorch for deep learning: Creating and deploying deep learning applications

    Ian Pointer. Programming pytorch for deep learning: Creating and deploying deep learning applications. O’Reilly Media, 2019

    2019

  51. [59]

    UvA Deep Learning Tutorials

    Phillip Lippe. UvA Deep Learning Tutorials. https://uvadlc-notebooks.readthedocs.io/ en/latest/. 2024

    2024

  52. [60]

    url: https://github.com/Lightning-AI/pytorch-lightning

    Torch Lightning. url: https://github.com/Lightning-AI/pytorch-lightning

  53. [61]

    Hydra - A framework for elegantly configuring complex applications

    Omry Yadan. Hydra - A framework for elegantly configuring complex applications. Github

  54. [2019]

    url: https://github.com/facebookresearch/hydra. 22

  55. [2024]

    url: https://arxiv.org/abs/2412.04578

    arXiv: 2412.04578 [cs.LG]. url: https://arxiv.org/abs/2412.04578

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.