pith. sign in

arxiv: 2503.14568 · v1 · submitted 2025-03-18 · ❄️ cond-mat.mtrl-sci · cs.AI· cs.CE· cs.LG· physics.comp-ph

Teaching Artificial Intelligence to Perform Rapid, Resolution-Invariant Grain Growth Modeling via Fourier Neural Operator

Iman Peivaste , Ahmed Makradi , Salim Belouettar This is my paper

Pith reviewed 2026-05-22 23:49 UTC · model grok-4.3

classification ❄️ cond-mat.mtrl-sci cs.AIcs.CEcs.LGphysics.comp-ph
keywords Fourier Neural Operatorgrain growthphase-field modelingmicrostructure evolutionresolution invariancesurrogate modelingmaterials simulation
0
0 comments X

The pith

Fourier Neural Operator learns a resolution-invariant surrogate for phase-field grain growth from time-shifted microstructure sequences.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper trains a Fourier Neural Operator on sequences of microstructures generated by the Fan-Chen phase-field model so that the network maps current grain configurations to future states. Because the operator acts in Fourier space, the learned mapping is claimed to remain accurate when the input grid is finer than any training data and when the grain arrangement is entirely new. This matters for materials modeling because conventional phase-field runs become prohibitively slow at large system sizes or high spatial resolution, while most machine-learning surrogates lose accuracy the moment resolution changes. The central demonstration is that a single trained network produces long-term evolution predictions that match the underlying phase-field solver across these unseen cases.

Core claim

By operating in Fourier space the FNO learns an operator that maps phase-field data between function spaces of arbitrary resolution; after training on time-shifted pairs from the Fan-Chen model the same network accurately forecasts grain evolution for both novel grain topologies and spatial grids finer than those used in training.

What carries the argument

The Fourier Neural Operator, which learns integral kernels in Fourier space to map between function spaces and thereby decouples the learned evolution rule from any fixed grid size.

If this is right

  • A single trained network can replace repeated phase-field runs for long-time microstructure prediction at any chosen resolution.
  • Computational cost drops from hours or days per simulation to seconds per forward pass once the operator is learned.
  • The same model can be deployed on experimental image sequences whose pixel count differs from the training data.
  • Time-stepping can be performed at variable effective rates without retraining or interpolation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because no explicit conservation laws are enforced, long-time integration may accumulate small violations of volume or topology that only appear after many operator applications.
  • The method could be tested on three-dimensional grain-growth data or on other evolution phenomena such as recrystallization without changing the network architecture.
  • Pairing the operator with uncertainty quantification would indicate when the prediction is drifting outside the training distribution.
  • The approach opens the possibility of coupling the learned operator directly to continuum mechanics solvers that operate on different meshes.

Load-bearing premise

The operator learned from the training resolutions and grain configurations will continue to produce physically plausible evolution when applied to arbitrary higher resolutions and to grain arrangements never seen during training.

What would settle it

Generate a new phase-field simulation at twice the finest training resolution with a grain configuration outside the training distribution, then compare the FNO-predicted microstructure at a fixed later time against the full phase-field result; statistically significant deviation in grain-size distribution or topology falsifies the invariance claim.

read the original abstract

Microstructural evolution, particularly grain growth, plays a critical role in shaping the physical, optical, and electronic properties of materials. Traditional phase-field modeling accurately simulates these phenomena but is computationally intensive, especially for large systems and fine spatial resolutions. While machine learning approaches have been employed to accelerate simulations, they often struggle with resolution dependence and generalization across different grain scales. This study introduces a novel approach utilizing Fourier Neural Operator (FNO) to achieve resolution-invariant modeling of microstructure evolution in multi-grain systems. FNO operates in the Fourier space and can inherently handle varying resolutions by learning mappings between function spaces. By integrating FNO with the phase field method, we developed a surrogate model that significantly reduces computational costs while maintaining high accuracy across different spatial scales. We generated a comprehensive dataset from phase-field simulations using the Fan Chen model, capturing grain evolution over time. Data preparation involved creating input-output pairs with a time shift, allowing the model to predict future microstructures based on current and past states. The FNO-based neural network was trained using sequences of microstructures and demonstrated remarkable accuracy in predicting long-term evolution, even for unseen configurations and higher-resolution grids not encountered during training.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces a Fourier Neural Operator (FNO) surrogate model trained on sequences of microstructures generated by phase-field simulations of the Fan-Chen model. It claims that the learned operator achieves resolution-invariant predictions of long-term grain growth evolution, including accurate rollouts on unseen grain configurations and on spatial grids finer than those used in training, while substantially reducing computational cost relative to direct phase-field integration.

Significance. If the resolution-extrapolation and long-horizon stability claims are quantitatively substantiated, the work would provide a practical, resolution-agnostic surrogate for grain-growth modeling that could accelerate large-scale materials simulations without requiring retraining at every target resolution.

major comments (2)
  1. [Abstract and Results] Abstract and Results section: the assertion of 'remarkable accuracy' on higher-resolution grids and unseen configurations is unsupported by any reported quantitative metrics (e.g., L2 or H1 errors, grain-size distribution statistics, or structural similarity indices) that isolate resolution-extrapolation error from in-distribution single-step performance.
  2. [Methods and Results] Methods and Results: training is performed on simple time-shifted pairs with no explicit penalty or post-hoc verification for conservation of the order-parameter integral or grain-boundary energy; therefore the multi-step rollout stability asserted for long-term evolution rests solely on the empirical behavior of the FNO without demonstrated physical consistency checks.
minor comments (1)
  1. [Abstract] Abstract: 'Fan Chen model' should be written consistently as 'Fan-Chen model' and a brief citation to the original phase-field formulation should be supplied.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful review and constructive feedback. We address each major comment below and will revise the manuscript accordingly to strengthen the quantitative support and physical consistency checks.

read point-by-point responses

  1. Referee: [Abstract and Results] Abstract and Results section: the assertion of 'remarkable accuracy' on higher-resolution grids and unseen configurations is unsupported by any reported quantitative metrics (e.g., L2 or H1 errors, grain-size distribution statistics, or structural similarity indices) that isolate resolution-extrapolation error from in-distribution single-step performance.

    Authors: We agree that the abstract and results as written rely primarily on qualitative demonstrations and do not report quantitative metrics that specifically isolate resolution-extrapolation performance. The full manuscript contains visual comparisons of rollouts on finer grids and unseen configurations, but these do not include the requested error measures. In the revised version we will add a dedicated quantitative analysis subsection reporting L2 and H1 errors, grain-size distribution statistics (mean and variance), and structural similarity indices, computed separately for in-distribution single-step predictions, multi-step rollouts, and resolution-extrapolated cases. revision: yes

  2. Referee: [Methods and Results] Methods and Results: training is performed on simple time-shifted pairs with no explicit penalty or post-hoc verification for conservation of the order-parameter integral or grain-boundary energy; therefore the multi-step rollout stability asserted for long-term evolution rests solely on the empirical behavior of the FNO without demonstrated physical consistency checks.

    Authors: Training indeed uses time-shifted pairs drawn from the conservative Fan-Chen phase-field model without an explicit conservation term in the loss. While the data-driven operator inherits approximate conservation from the training distribution, we acknowledge that this does not constitute a demonstrated physical consistency check. In the revision we will add post-hoc verification: we will compute and tabulate the relative drift in the integrated order parameter and in the grain-boundary energy over long rollouts (both in-distribution and at extrapolated resolutions) and report these quantities alongside the visual results. If the observed drift is non-negligible we will also test a soft conservation penalty during retraining. revision: yes

Circularity Check

0 steps flagged

No significant circularity; training and evaluation use independently generated external simulation data

full rationale

The paper generates a dataset from separate phase-field simulations (Fan-Chen model), forms time-shifted input-output pairs, and trains an FNO surrogate to map current states to future states. Reported accuracy on unseen grain configurations and higher-resolution grids is measured on held-out simulation trajectories, not by reducing the prediction to a fitted parameter or self-citation that is itself defined by the target result. No equations, uniqueness theorems, or ansatzes are invoked that would make the claimed generalization equivalent to the training inputs by construction. This is a standard supervised operator-learning setup with external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Approach rests on the mathematical properties of Fourier Neural Operators and the assumption that phase-field data sufficiently represents the target physics.

axioms (2)
  • standard math Fourier space operations enable resolution-independent function mappings
    Core architectural property of FNO invoked to justify invariance claim
  • domain assumption Fan Chen phase-field model produces representative grain growth trajectories
    Used to generate all training and test data

pith-pipeline@v0.9.0 · 5756 in / 1155 out tokens · 30923 ms · 2026-05-22T23:49:29.758235+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

66 extracted references · 66 canonical work pages · 4 internal anchors

  1. [1]

    Biner, S. B. An overview of the phase-field method and its formalisms. Programming Phase-Field Modeling 1–7 (2017)

    work page 2017

  2. [3]

    & Francis, J

    Flint, T., Sun, Y., Xiong, Q., Smith, M. & Francis, J. Phase-field simulation of grain boundary evolution in microstructures containing second-phase particles with heterogeneous thermal properties. Scientific Reports 9, 1–11 (2019)

    work page 2019

  3. [4]

    & Yan, W

    Yang, M., Wang, L. & Yan, W. Phase-field modeling of grain evolutions in additive manufacturing from nucleation, growth, to coarsening. Npj Computational Materials 7, 1–12 (2021)

    work page 2021

  4. [5]

    Comprehensive materials processing (Newnes, 2014)

    Hashmi, S. Comprehensive materials processing (Newnes, 2014)

    work page 2014

  5. [6]

    Grain growth

    Z¨ ollner, D. Grain growth. InReference Module in Materials Science and Materials Engineering (Elsevier, 2016). URL https://www.sciencedirect.com/science/article/pii/ B9780128035818031581

    work page 2016

  6. [7]

    J., Wang, A., Zhang, T

    Jiang, C., Moniz, S. J., Wang, A., Zhang, T. & Tang, J. Photoelectrochemical devices for solar water splitting–materials and challenges. Chemical Society Reviews 46, 4645–4660 (2017)

    work page 2017

  7. [8]

    Tan, J. et al. Fullerene as a photoelectron transfer promoter enabling sta- ble tio2-protected sb2se3 photocathodes for photo-electrochemical water splitting. Advanced Energy Materials 9, 1900179 (2019)

    work page 2019

  8. [9]

    W., Ahn, S

    Yang, J. W., Ahn, S. H. & Jang, H. W. Crucial role of het- erostructures in highly advanced water splitting photoelectrodes. Current Opinion in Green and Sustainable Chemistry 29, 100454 (2021)

    work page 2021

  9. [10]

    J., Phillpot, S

    Haslam, A. J., Phillpot, S. R., Wolf, D., Moldovan, D. & Gleiter, H. Mechanisms of grain growth in nanocrystalline fcc metals by molecular-dynamics simulation. Materials Science and Engineering A-structural Materials Properties Microstructure and Processing 318, 293–312 (2001)

    work page 2001

  10. [11]

    & Mirzadeh, H

    Najafkhani, F., Kheiri, S., Pourbahari, B. & Mirzadeh, H. Recent advances in the kinetics of normal/abnormal grain growth: a review. Archives of Civil and Mechanical Engineering 21, 1–20 (2021)

    work page 2021

  11. [12]

    Gao, Y. J. et al. Modes of grain growth and mechanism of disloca- tion reaction under applied biaxial strain: Atomistic and continuum modeling. Journal of Materials Science & Technology 49, 236–250 (2020)

    work page 2020

  12. [13]

    On the theory of normal grain growth

    Louat, N. On the theory of normal grain growth. Acta metallurgica 22, 721–724 (1974). 21

    work page 1974

  13. [14]

    & Mishin, Y

    Trautt, Z., Adland, A., Karma, A. & Mishin, Y. Coupled motion of asymmetri- cal tilt grain boundaries: molecular dynamics and phase field crystal simulations. arXiv: Materials Science (2012)

    work page 2012

  14. [15]

    & Mishin, Y

    Trautt, Z. & Mishin, Y. Grain boundary migration and grain rotation studied by molecular dynamics. Acta Materialia 60, 2407–2424 (2012)

    work page 2012

  15. [16]

    Kang, S.-J. L. Sintering: densification, grain growth and microstructure (Elsevier, 2004)

    work page 2004

  16. [17]

    & DebRoy, T

    Wei, H., Elmer, J. & DebRoy, T. Three-dimensional modeling of grain structure evolution during welding of an aluminum alloy. Acta Materialia 126, 413–425 (2017)

    work page 2017

  17. [18]

    Okita, S. et al. Grain growth kinetics in submicrometer-scale molecular dynamics simulation. Acta Materialia 153, 108–116 (2018)

    work page 2018

  18. [19]

    & Mumtaz, K

    Lopez-Botello, O., Martinez-Hernandez, U., Ram´ ırez, J., Pinna, C. & Mumtaz, K. Two-dimensional simulation of grain structure growth within selective laser melted aa-2024. Materials & Design 113, 369–376 (2017)

    work page 2024

  19. [20]

    & Moelans, N

    Ravash, H., Vanherpe, L., Vleugels, J. & Moelans, N. Three-dimensional phase-field study of grain coarsening and grain shape accommodation in the final stage of liquid- phase sintering. Journal of the European Ceramic Society 37, 2265–2275 (2017)

    work page 2017

  20. [21]

    Korbuly, B. et al. Orientation-field models for polycrystalline solidification: grain coarsening and complex growth forms. Journal of Crystal Growth 457, 32–37 (2017)

    work page 2017

  21. [22]

    Liu, Y. et al. Microstructure evolution in 439 stainless steels under tensile: phase field simulation and experiment. Modelling and Simulation in Materials Science and Engineering (2024)

    work page 2024

  22. [23]

    & Chen, M

    Shang, Y. & Chen, M. Phase-field modelling of microstructure evolution in solid oxide cells. Meeting abstracts MA2023-02, 2253–2253 (2023)

    work page 2023

  23. [24]

    Cahn, J. W. & Hilliard, J. E. Free energy of a nonuniform system. i. interfacial free energy. J. Comp. Phys. 28, 258–267 (1958)

    work page 1958

  24. [25]

    Cahn, J. W. On spinodal decomposition. Acta metallurgica 9, 795–801 (1961)

    work page 1961

  25. [26]

    Allen, S. M. & Cahn, J. W. Ground state structures in ordered binary alloys with second neighbor interactions. Acta Metallurgica 20, 423–433 (1972)

    work page 1972

  26. [27]

    Allen, S. M. & Cahn, J. W. A correction to the ground state of fcc binary ordered alloys with first and second neighbor pairwise interactions. Scripta Metallurgica 7, 1261–1264 (1973)

    work page 1973

  27. [28]

    Montes de Oca Zapiain, D., Stewart, J. A. & Dingreville, R. Accelerating phase-field- based microstructure evolution predictions via surrogate models trained by machine learning methods. npj Computational Materials 7, 1–11 (2021)

    work page 2021

  28. [29]

    & Garikipati, K

    Zhang, X. & Garikipati, K. Machine learning materials physics: Multi-resolution neural networks learn the free energy and nonlinear elastic response of evolv- ing microstructures. Computer Methods in Applied Mechanics and Engineering 372, 22 113362 (2020). URL https://www.sciencedirect.com/science/article/pii/ S0045782520305478

    work page 2020

  29. [31]

    Peivaste, I. et al. Machine-learning-based surrogate modeling of microstructure evo- lution using phase-field. Computational Materials Science 214, 111750 (2022)

    work page 2022

  30. [32]

    & Karniadakis, G

    Oommen, V., Shukla, K., Goswami, S., Dingreville, R. & Karniadakis, G. E. Learning two-phase microstructure evolution using neural operators and autoencoder architec- tures. npj Computational Materials 8, 190 (2022)

    work page 2022

  31. [33]

    Bonneville, C. et al. Accelerating phase field simulations through a hybrid adaptive fourier neural operator with u-net backbone. arXiv preprint arXiv:2406.17119 (2024)

    work page arXiv 2024

  32. [34]

    & Aoki, T

    Sakane, S., Takaki, T., Ohno, M., Shimokawabe, T. & Aoki, T. Gpu-accelerated 3d phase-field simulations of dendrite competitive growth during directional solidification of binary alloy (2015)

    work page 2015

  33. [35]

    Miyoshi, E. et al. Ultra-large-scale phase-field simulation study of ideal grain growth. NPJ Computational Materials 3, 1–6 (2017)

    work page 2017

  34. [36]

    & Tang, T

    Cheng, Y., Kurganov, A., Qu, Z. & Tang, T. Fast and stable explicit oper- ator splitting methods for phase-field models. Journal of Computational Physics 303, 45–65 (2015). URL https://www.sciencedirect.com/science/article/pii/ S0021999115005859

    work page 2015

  35. [37]

    Mianroodi, J. R., H. Siboni, N. & Raabe, D. Teaching solid mechanics to artificial intelligence—a fast solver for heterogeneous materials. Npj Computational Materials 7, 99 (2021)

    work page 2021

  36. [38]

    Nyshadham, C. et al. Machine-learned multi-system surrogate models for materials prediction. npj Computational Materials 5, 51 (2019)

    work page 2019

  37. [39]

    & Sun, W

    Wang, K. & Sun, W. A multiscale multi-permeability poroplas- ticity model linked by recursive homogenizations and deep learning. Computer Methods in Applied Mechanics and Engineering 334, 337–380 (2018)

    work page 2018

  38. [40]

    & Garikipati, K

    Zhang, X. & Garikipati, K. Machine learning materials physics: Multi-resolution neural networks learn the free energy and nonlinear elastic response of evolv- ing microstructures. Computer Methods in Applied Mechanics and Engineering 372, 113362 (2020)

    work page 2020

  39. [41]

    Y., Xue, T., Liao, S

    Choi, J. Y., Xue, T., Liao, S. & Cao, J. Accelerating phase-field simulation of three-dimensional microstructure evolution in laser powder bed fusion with compos- able machine learning predictions. Additive Manufacturing 79, 103938 (2024). URL https://www.sciencedirect.com/science/article/pii/S2214860423005511

    work page 2024

  40. [42]

    Yang, K. et al. Self-supervised learning and prediction of microstructure evolution with convolutional recurrent neural networks. Patterns 2 (2021). 23

    work page 2021

  41. [43]

    & Gautier, B

    Alhada-Lahbabi, K., Deleruyelle, D. & Gautier, B. Transfer learning for accelerat- ing phase-field modeling of ferroelectric domain formation in large-scale 3d systems. Computer Methods in Applied Mechanics and Engineering 429, 117167 (2024). URL https://www.sciencedirect.com/science/article/pii/S0045782524004237

    work page 2024

  42. [44]

    Li, Z. et al. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  43. [45]

    DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators

    Lu, L., Jin, P. & Karniadakis, G. E. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  44. [46]

    Miyoshi, E. et al. Large-scale phase-field study of anisotropic grain growth: Effects of misorientation-dependent grain boundary energy and mobility. Computational Materials Science 186, 109992 (2021)

    work page 2021

  45. [47]

    & Takaki, T

    Miyoshi, E. & Takaki, T. Extended higher-order multi-phase-field model for three- dimensional anisotropic-grain-growth simulations. Computational Materials Science 120, 77–83 (2016)

    work page 2016

  46. [48]

    & Takaki, T

    Miyoshi, E. & Takaki, T. Validation of a novel higher-order multi-phase-field model for grain-growth simulations using anisotropic grain-boundary properties. Computational Materials Science 112, 44–51 (2016)

    work page 2016

  47. [49]

    & Ohno, M

    Miyoshi, E., Takaki, T., Shibuta, Y. & Ohno, M. Bridging molecular dynamics and phase-field methods for grain growth prediction. Computational Materials Science (2018)

    work page 2018

  48. [50]

    Miyoshi, E. et al. Large-scale phase-field simulation of three- dimensional isotropic grain growth in polycrystalline thin films. Modelling and Simulation in Materials Science and Engineering (2019)

    work page 2019

  49. [51]

    S., Clarke, A

    Tourret, D., Song, Y. S., Clarke, A. J. & Karma, A. Grain growth competition during thin-sample directional solidification of dendritic microstructures: A phase- field study. Acta Materialia 122, 220–235 (2017)

    work page 2017

  50. [52]

    & Shibuta, Y

    Takaki, T., Sakane, S., Ohno, M. & Shibuta, Y. Competitive growth during directional solidification of a binary alloy with natural convection: two-dimensional phase-field study. Modelling and Simulation in Materials Science and Engineering (2019)

    work page 2019

  51. [53]

    Guo, C. et al. Competitive growth of diverging columnar grains during directional so- lidification: A three-dimensional phase-field study. Computational Materials Science (2021)

    work page 2021

  52. [54]

    Guo, C. et al. Overgrowth behavior at converging grain boundaries during competitive grain growth: A two-dimensional phase-field study. International Journal of Heat and Mass Transfer 160, 120196 (2020)

    work page 2020

  53. [55]

    & Chen, L.-Q

    Krill III, C. & Chen, L.-Q. Computer simulation of 3-d grain growth using a phase-field model. Acta Materialia 50, 3059–3075 (2002). URL https://www. sciencedirect.com/science/article/pii/S1359645402000848. 24

    work page 2002

  54. [56]

    Kazaryan, A., Wang, Y., Dregia, S. A. & Patton, B. R. Generalized phase-field model for computer simulation of grain growth in anisotropic systems. Phys. Rev. B 61, 14275–14278 (2000). URL https://link.aps.org/doi/10.1103/PhysRevB.61. 14275

    work page doi:10.1103/physrevb.61 2000

  55. [57]

    & Chen, L.-Q

    Fan, D. & Chen, L.-Q. Computer simulation of grain growth using a continuum field model. Acta Materialia 45, 611–622 (1997)

    work page 1997

  56. [58]

    Krill, C. E. & Chen, L.-Q. Computer simulation of 3-d grain growth using a phase- field model. Acta Materialia 50, 3059–3075 (2002)

    work page 2002

  57. [59]

    & Wang, L.-L.Spectral methods: algorithms, analysis and applications, vol

    Shen, J., Tang, T. & Wang, L.-L.Spectral methods: algorithms, analysis and applications, vol. 41 (Springer Science & Business Media, 2011)

    work page 2011

  58. [60]

    Chen, L. Q. & Shen, J. Applications of semi-implicit fourier-spectral method to phase field equations. Computer Physics Communications 108, 147–158 (1998)

    work page 1998

  59. [61]

    Harris, C. R. et al. Array programming with NumPy. Nature 585, 357–362 (2020). URL https://doi.org/10.1038/s41586-020-2649-2

    work page doi:10.1038/s41586-020-2649-2 2020

  60. [62]

    Virtanen, P. et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17, 261–272 (2020)

    work page 2020

  61. [63]

    Abadi, M. et al. TensorFlow: Large-scale machine learning on heterogeneous sys- tems (2015). URL https://www.tensorflow.org/. Software available from tensor- flow.org

    work page 2015

  62. [64]

    Peivaste, I. et al. Accelerating phase-field-based simulation via machine learning. arXiv preprint arXiv:2205.02121 (2022)

    work page arXiv 2022

  63. [65]

    & Tiamiyu, A

    Peivaste, I., Jossou, E. & Tiamiyu, A. A. Data-driven analysis and prediction of stable phases for high-entropy alloy design. Scientific Reports 13, 22556 (2023)

    work page 2023

  64. [66]

    Kovachki, N. B. et al. Neural operator: Learning maps between function spaces. ArXiv abs/2108.08481 (2021)

    work page arXiv 2021

  65. [67]

    Gaussian Error Linear Units (GELUs)

    Hendrycks, D. & Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016)

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  66. [68]

    Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). 25

    work page internal anchor Pith review Pith/arXiv arXiv 2014