pith. sign in

arxiv: 2605.31027 · v1 · pith:GHZ22KYPnew · submitted 2026-05-29 · 💻 cs.LG

Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs

Qihong Yang , Qiaolin He This is my paper

Pith reviewed 2026-06-28 23:54 UTC · model grok-4.3

classification 💻 cs.LG
keywords high-frequency PDEsseparable neural networksFourier featuresrandom weightsleast squaresphysics-informed neural networksmulti-scale approximation
0
0 comments X

The pith

MS-SFNN solves high-frequency PDEs to high accuracy using fixed random weights in separable subnetworks with tunable scaling and cosine activations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Multi-Scale Separable Fourier Neural Networks to approximate solutions of linear and nonlinear high-frequency PDEs. It builds basis functions by multiplying outputs from independent one-dimensional subnetworks, each with fixed random weights initialized from a unit-variance uniform distribution and cosine activations whose frequencies are adjusted by a single tunable scale per subnetwork. Coefficients of the linear combination are found by least squares, with derivatives computed analytically and large systems handled by a batched QR algorithm. Experiments claim this fixed-basis approach yields lower errors than Physics-Informed Neural Networks and Separated-Variable Spectral Neural Networks across tested problems while avoiding parameter training and reducing memory use in three dimensions or at high frequencies.

Core claim

The central claim is that a separable architecture of d independent subnetworks, each producing Fourier features via fixed random weights and cosine activation modulated by a tunable scale, generates an expressive basis whose linear combination, obtained via least squares, accurately solves arbitrary high-frequency PDEs; analytical derivatives and batched QR further make the method practical for large or three-dimensional cases without any weight training after the single random initialization.

What carries the argument

Separable Fourier basis formed by element-wise multiplication of outputs from d coordinate-wise subnetworks, each using fixed random weights, cosine activation, and one tunable scaling factor to control frequency content.

If this is right

  • Accuracy exceeds that of PINNs and SV-SNN on the reported high-frequency linear and nonlinear test problems.
  • Memory cost stays manageable in three dimensions and at high frequencies because automatic differentiation is replaced by analytic derivatives and large least-squares problems are solved with batched QR.
  • Only the scaling factors and the least-squares coefficients need adjustment; all network weights remain frozen after random initialization.
  • The same separable construction applies uniformly to both linear and nonlinear PDEs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The fixed random Fourier basis could be reused across multiple related PDEs, amortizing the one-time initialization cost.
  • The separable product structure may extend naturally to other high-dimensional approximation tasks where full tensor-product bases become prohibitive.
  • Because only scales are tuned, the method might integrate into optimization loops that adjust scales adaptively during time-stepping of time-dependent PDEs.
  • The explicit Fourier embedding suggests possible theoretical links to classical spectral methods that could be explored for convergence rates.

Load-bearing premise

Randomly initialized fixed weights drawn once from a uniform distribution with unit variance, together with per-subnetwork tunable scaling factors, generate a sufficiently expressive basis set for arbitrary high-frequency linear and nonlinear PDEs without any parameter training.

What would settle it

A high-frequency PDE test case in which the reported error of MS-SFNN remains larger than that of a tuned PINN after optimal choice of the scaling factors.

Figures

Figures reproduced from arXiv: 2605.31027 by Qiaolin He, Qihong Yang.

Figure 1
Figure 1. Figure 1: Multi-Scale Separable Fourier Neural Networks (MS-SFNN) architecture dia [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Heat maps illustrate the heat conduction equation (21) with [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Heat maps illustrate the heat conduction equation (21) with [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Heat maps illustrate the heat conduction equation (21) with [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Heat maps illustrate the two-dimensional Helmholtz equation (22) with [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Heat maps illustrate the two-dimensional Helmholtz equation (22) with [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Heat maps illustrate the two-dimensional complex geometry Helmholtz equation [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Heat maps illustrate the two-dimensional complex geometry Helmholtz equation [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Heat maps illustrate the complex geometry Poisson equation (23). Left: the [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Heat maps illustrate the complex source term Poisson equation (24). Left: the [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Heat maps illustrate the three-dimensional Helmholtz equation (25) with [PITH_FULL_IMAGE:figures/full_fig_p025_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Heat maps illustrate the three-dimensional Helmholtz equation (25). Left: the [PITH_FULL_IMAGE:figures/full_fig_p026_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Heat maps illustrate the two-dimensional flower-shaped problems with Dirichlet [PITH_FULL_IMAGE:figures/full_fig_p027_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Heat maps illustrate the two-dimensional flower-shaped problems with Dirichlet [PITH_FULL_IMAGE:figures/full_fig_p028_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Heat maps illustrate the two-dimensional flower-shaped problems with mixed [PITH_FULL_IMAGE:figures/full_fig_p029_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Heat maps illustrate the two-dimensional flower-shaped problems with mixed [PITH_FULL_IMAGE:figures/full_fig_p029_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Heat maps illustrate the nonlinear elliptic equation (28). Left: the exact [PITH_FULL_IMAGE:figures/full_fig_p031_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Heat maps illustrate the u component of the nonlinear elliptic equation (28). Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y True 0.98 0.70 0.42 0.14 0.14 0.42 0.70 0.98 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y Pred 0.98 0.70 0.42 0.14 0.14 0.42 0.70 0.98 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y Error 0.000 0.345 0.691 1.036 1.38… view at source ↗
Figure 19
Figure 19. Figure 19: Heat maps illustrate the v component of the nonlinear elliptic equation (28). Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 34 [PITH_FULL_IMAGE:figures/full_fig_p034_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Heat maps illustrate the pressure p of the nonlinear elliptic equation (28). Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. problem is defined within circular domain Ω = (x, y) : x 2 + y 2 ≤ 3.0 2\(Ω1 ∪ Ω2, with two cylindrical obstacles centered at (−1.0, 0.5) with radius r1 = 0.3 and (1.0, −0.5) with radius r2 = 0.3. Governing equations are … view at source ↗
Figure 21
Figure 21. Figure 21: Heat maps illustrate the u component of the double-cylinder steady Navier￾Stokes equations (32) with p is chosen as case 1. Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y True 1.380 0.986 0.591 0.197 0.197 0.591 0.986 1.380 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y Pred 1.380 0.986 0.591 0.197 0.197 0.591 0.986 1.380 3 2 … view at source ↗
Figure 22
Figure 22. Figure 22: Heat maps illustrate the v component of the double-cylinder steady Navier￾Stokes equations (32) with p is chosen as case 1. Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 37 [PITH_FULL_IMAGE:figures/full_fig_p037_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Heat maps illustrate the u component of the double-cylinder steady Navier￾Stokes equations (32) with p is chosen as case 2. Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 4.12. Analysis of Effects of scale factors To evaluate the effects of scale factors ρ, we solve the two-dimensional Helmholtz equations (22) with MS-SFNN, performing a grid s… view at source ↗
Figure 24
Figure 24. Figure 24: Heat maps illustrate the v component of the double-cylinder steady Navier￾Stokes equations (32) with p is chosen as case 2. Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y True 2.000 1.428 0.857 0.286 0.286 0.857 1.428 2.000 3 2 1 0 1 2 3 x 3 2 1 0 1 2 3 y Pred 2.000 1.428 0.857 0.286 0.286 0.857 1.428 2.000 3 2 … view at source ↗
Figure 25
Figure 25. Figure 25: Heat maps illustrate the p component of the double-cylinder steady Navier￾Stokes equations (32) with p is chosen as case 2. Left: the exact solution; Middle: the prediction solution of MS-SFNN; Right: the absolute error between them. 39 [PITH_FULL_IMAGE:figures/full_fig_p039_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Left: the exact solution is selected as case [PITH_FULL_IMAGE:figures/full_fig_p041_26.png] view at source ↗
read the original abstract

We propose a novel neural network architecture, termed Multi-Scale Separable Fourier Neural Networks (MS-SFNN), for the accurate and efficient solution of linear and nonlinear high-frequency partial differential equations (PDEs). MS-SFNN exploits a separable representation: given a $d$-dimensional input, it employs $d$ independent subnetworks -- each acting on a single coordinate -- and constructs basis functions via element-wise multiplication of their outputs. The PDE solution is approximated as a linear combination of these basis functions, with coefficients determined by least squares. Critically, all network weights and biases are randomly initialized once, from a uniform distribution with unit variance, and remain fixed thereafter. To enhance expressivity, a tunable scaling factor is introduced in each subnetwork to modulate the frequency content of the resulting basis functions. Fourier features are explicitly embedded through cosine activations, endowing the method with strong spectral approximation capabilities. To mitigate the memory bottleneck associated with dense collocation in high-frequency or three-dimensional problems, we replace automatic differentiation with analytically derived basis function derivatives and develop a memory-efficient batched QR decomposition algorithm for solving large-scale least-squares systems. Numerical experiments demonstrate that MS-SFNN achieves unprecedented accuracy across a range of challenging PDEs, significantly outperforming state-of-the-art methods such as Physics-Informed Neural Networks (PINN) and Separated-Variable Spectral Neural Networks (SV-SNN).

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes Multi-Scale Separable Fourier Neural Networks (MS-SFNN) for linear and nonlinear high-frequency PDEs. The architecture uses d independent subnetworks (one per coordinate) whose outputs are multiplied elementwise to form separable basis functions; the PDE solution is a linear combination of these bases with coefficients obtained by least squares. All network weights and biases are drawn once from a unit-variance uniform distribution and then frozen; only per-subnetwork scaling factors are tuned. Cosine activations embed Fourier features. Analytical derivatives replace autodiff and a batched QR solver addresses memory limits. Numerical experiments are reported to show unprecedented accuracy that significantly outperforms PINNs and SV-SNNs.

Significance. If the reported accuracy holds under the fixed-random-basis regime, the method would supply a training-light, spectrally capable alternative for high-frequency PDEs that avoids the optimization difficulties of PINNs while retaining a neural-network interface. The separable construction and memory-efficient QR implementation are practical strengths for high-dimensional or high-frequency regimes.

major comments (2)
  1. [Abstract] Abstract: the headline claim that MS-SFNN achieves 'unprecedented accuracy' on 'a range of challenging PDEs' and 'significantly outperforming' PINNs and SV-SNNs rests on unspecified numerical experiments. No error norms, collocation-point counts, random-seed statistics, or exclusion criteria are supplied, so the data-to-claim link cannot be verified.
  2. [Abstract] Abstract (core procedure): the method fixes all weights after a single draw from Uniform[-1,1] (unit variance) and solves only for linear coefficients plus a handful of scaling factors. No analysis or theorem shows that this particular random separable Fourier basis is guaranteed to be sufficiently rich for arbitrary high-frequency linear and nonlinear PDEs; failure of expressivity would render the reported outperformance an artifact of the chosen test problems or the specific draw rather than a general property.
minor comments (2)
  1. [Abstract] The abstract states that 'Fourier features are explicitly embedded through cosine activations' but does not clarify whether the frequencies themselves are also scaled by the tunable factors or remain fixed at the random draw.
  2. [Abstract] The memory-efficient batched QR algorithm is mentioned but no complexity or stability analysis is referenced in the provided text.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. The comments correctly identify areas where the abstract could be strengthened with quantitative details and where the empirical character of the method should be stated more explicitly. We respond to each point below and indicate planned revisions.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the headline claim that MS-SFNN achieves 'unprecedented accuracy' on 'a range of challenging PDEs' and 'significantly outperforming' PINNs and SV-SNNs rests on unspecified numerical experiments. No error norms, collocation-point counts, random-seed statistics, or exclusion criteria are supplied, so the data-to-claim link cannot be verified.

    Authors: We agree that the abstract would be improved by including concrete quantitative support for the performance claims. In the revised version we will add a concise summary of representative L2 error norms, collocation-point counts, and comparison metrics against PINNs and SV-SNNs, while preserving the abstract's brevity. These numbers are already reported in the numerical-experiments section and will now be referenced directly in the abstract. revision: yes

  2. Referee: [Abstract] Abstract (core procedure): the method fixes all weights after a single draw from Uniform[-1,1] (unit variance) and solves only for linear coefficients plus a handful of scaling factors. No analysis or theorem shows that this particular random separable Fourier basis is guaranteed to be sufficiently rich for arbitrary high-frequency linear and nonlinear PDEs; failure of expressivity would render the reported outperformance an artifact of the chosen test problems or the specific draw rather than a general property.

    Authors: The MS-SFNN is presented as an empirical method that exploits the known spectral properties of random Fourier features together with a separable construction and tunable scaling. We do not provide a new theorem guaranteeing universal expressivity for every possible high-frequency PDE; the approach relies on the richness of the random basis (modulated by scaling) and is validated through extensive numerical tests. We will insert a clarifying paragraph in the methodology section that explicitly notes the empirical nature of the claims, references prior theoretical results on random Fourier features, and cautions that performance may vary with problem class and random draw. revision: partial

Circularity Check

0 steps flagged

No significant circularity; method is a fixed-basis least-squares fit validated empirically

full rationale

The paper proposes MS-SFNN with randomly initialized fixed weights, per-subnetwork scaling factors, cosine activations for Fourier features, and least-squares solution for linear coefficients (plus analytic derivatives and batched QR). The central claims rest on numerical experiments showing accuracy on test PDEs, not on any derivation that reduces the reported performance back to a fitted hyperparameter or self-citation chain of the same result. No self-definitional step, no fitted-input-called-prediction, and no load-bearing uniqueness theorem from prior author work appears in the provided text. The random-basis expressivity is an assumption whose validity is tested externally rather than enforced by construction.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The method rests on the domain assumption that fixed random weights plus tunable scaling suffice for expressivity; the scaling factor itself functions as a free parameter chosen per subnetwork.

free parameters (1)
  • tunable scaling factor per subnetwork
    Introduced explicitly to modulate frequency content of the basis functions; value chosen to enhance expressivity.
axioms (1)
  • domain assumption Random initialization from uniform distribution with unit variance produces usable basis functions when combined with cosine activations and scaling
    Stated as critical to the method and kept fixed thereafter.

pith-pipeline@v0.9.1-grok · 5778 in / 1234 out tokens · 24993 ms · 2026-06-28T23:54:47.322393+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 8 canonical work pages · 1 internal anchor

  1. [1]

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics- informed neural networks: A deep learning framework for solving for- ward and inverse problems involving nonlinear partial differential equa- tions.Journal of Computational physics, 378:686–707, 2019

    2019

  2. [2]

    Convergence of the finite element method applied to the eigenvalue problemδu+λu= 0.Publications of the Research Institute for Mathematical Sciences, 13(1):47–60, 1977

    Kazuo Ishihara. Convergence of the finite element method applied to the eigenvalue problemδu+λu= 0.Publications of the Research Institute for Mathematical Sciences, 13(1):47–60, 1977

    1977

  3. [3]

    A mixed finite element method for the biharmonic eigenvalue problems of plate bending.Publications of the Research In- stitute for Mathematical Sciences, 14(2):399–414, 1978

    Kazuo Ishihara. A mixed finite element method for the biharmonic eigenvalue problems of plate bending.Publications of the Research In- stitute for Mathematical Sciences, 14(2):399–414, 1978

    1978

  4. [4]

    Finite difference boundary value method for solv- ing one-dimensional eigenvalue equations.Journal of Computational Physics, 10(1):123–132, 1972

    Donald G Truhlar. Finite difference boundary value method for solv- ing one-dimensional eigenvalue equations.Journal of Computational Physics, 10(1):123–132, 1972

    1972

  5. [5]

    A finite-difference method for the numerical solution of the Schrödinger equation.Journal of Computational and Applied Mathematics, 79(2):189–205, 1997

    TE Simos and PS Williams. A finite-difference method for the numerical solution of the Schrödinger equation.Journal of Computational and Applied Mathematics, 79(2):189–205, 1997

    1997

  6. [6]

    Adam: A Method for Stochastic Optimization

    Diederik P Kingma. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  7. [7]

    On the limited memory BFGS method for large scale optimization.Mathematical programming, 45(1):503–528, 1989

    Dong C Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization.Mathematical programming, 45(1):503–528, 1989

    1989

  8. [8]

    Deep- XDE: A deep learning library for solving differential equations.SIAM review, 63(1):208–228, 2021

    Lu Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. Deep- XDE: A deep learning library for solving differential equations.SIAM review, 63(1):208–228, 2021

    2021

  9. [9]

    fPINNs: Fractional physics-informed neural networks.SIAM Journal on Scientific Comput- ing, 41(4):A2603–A2626, 2019

    Guofei Pang, Lu Lu, and George Em Karniadakis. fPINNs: Fractional physics-informed neural networks.SIAM Journal on Scientific Comput- ing, 41(4):A2603–A2626, 2019

    2019

  10. [10]

    Quan- tifying total uncertainty in physics-informed neural networks for solv- ing forward and inverse stochastic problems.Journal of Computational Physics, 397:108850, 2019

    Dongkun Zhang, Lu Lu, Ling Guo, and George Em Karniadakis. Quan- tifying total uncertainty in physics-informed neural networks for solv- ing forward and inverse stochastic problems.Journal of Computational Physics, 397:108850, 2019. 44

    2019

  11. [11]

    Learning in modal space: Solving time-dependent stochastic PDEs using physics- informed neural networks.SIAM Journal on Scientific Computing, 42(2):A639–A665, 2020

    Dongkun Zhang, Ling Guo, and George Em Karniadakis. Learning in modal space: Solving time-dependent stochastic PDEs using physics- informed neural networks.SIAM Journal on Scientific Computing, 42(2):A639–A665, 2020

    2020

  12. [12]

    Physics-informed neural networks for inverse problems in nano-optics and metamaterials.Optics express, 28(8):11618–11633, 2020

    Yuyao Chen, Lu Lu, George Em Karniadakis, and Luca Dal Negro. Physics-informed neural networks for inverse problems in nano-optics and metamaterials.Optics express, 28(8):11618–11633, 2020

    2020

  13. [13]

    Physics-informed neural networks with hard constraints for inverse design.SIAM Journal on Scientific Com- puting, 43(6):B1105–B1132, 2021

    Lu Lu, Raphael Pestourie, Wenjie Yao, Zhicheng Wang, Francesc Ver- dugo, and Steven G Johnson. Physics-informed neural networks with hard constraints for inverse design.SIAM Journal on Scientific Com- puting, 43(6):B1105–B1132, 2021

    2021

  14. [14]

    Hidden fluid mechanics: Learning velocity and pressure fields from flow visual- izations.Science, 367(6481):1026–1030, 2020

    Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visual- izations.Science, 367(6481):1026–1030, 2020

    2020

  15. [15]

    Systems biology informed deep learning for inferring parameters and hidden dynamics.PLoS computational biology, 16(11):e1007575, 2020

    Alireza Yazdani, Lu Lu, Maziar Raissi, and George Em Karniadakis. Systems biology informed deep learning for inferring parameters and hidden dynamics.PLoS computational biology, 16(11):e1007575, 2020

    2020

  16. [16]

    Physics-informed neural networks for cardiac activation mapping.Frontiers in Physics, 8:42, 2020

    Francisco Sahli Costabal, Yibo Yang, Paris Perdikaris, Daniel E Hur- tado, and Ellen Kuhl. Physics-informed neural networks for cardiac activation mapping.Frontiers in Physics, 8:42, 2020

    2020

  17. [17]

    Han Gao, Luning Sun, and Jian-Xun Wang. PhyGeoNet: Physics- informed geometry-adaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain.Journal of Com- putational Physics, 428:110079, 2021

    2021

  18. [18]

    A high-efficient hybrid physics-informed neural networks based on convolutional neural network.IEEE Transactions on Neural Networks and Learning Systems, 33(10):5514–5526, 2021

    Zhiwei Fang. A high-efficient hybrid physics-informed neural networks based on convolutional neural network.IEEE Transactions on Neural Networks and Learning Systems, 33(10):5514–5526, 2021

    2021

  19. [19]

    Spline-pinn: Approaching pdes without data using fast, physics- informed hermite-spline cnns

    Nils Wandel, Michael Weinmann, Michael Neidlin, and Reinhard Klein. Spline-pinn: Approaching pdes without data using fast, physics- informed hermite-spline cnns. InProceedings of the AAAI conference on artificial intelligence, volume 36, pages 8529–8538, 2022. 45

    2022

  20. [20]

    PhyCRNet: Physics-informed convolutional-recurrent network for solv- ing spatiotemporal PDEs.Computer Methods in Applied Mechanics and Engineering, 389:114399, 2022

    Pu Ren, Chengping Rao, Yang Liu, Jian-Xun Wang, and Hao Sun. PhyCRNet: Physics-informed convolutional-recurrent network for solv- ing spatiotemporal PDEs.Computer Methods in Applied Mechanics and Engineering, 389:114399, 2022

    2022

  21. [21]

    Arda Mavi, Ali Can Bekar, Ehsan Haghighat, and Erdogan Madenci. An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic dif- ferential operator.Computer Methods in Applied Mechanics and Engi- neering, 407:115944, 2023

    2023

  22. [22]

    Physics- informed generative adversarial networks for stochastic differential equa- tions.SIAM Journal on Scientific Computing, 42(1):A292–A317, 2020

    Liu Yang, Dongkun Zhang, and George Em Karniadakis. Physics- informed generative adversarial networks for stochastic differential equa- tions.SIAM Journal on Scientific Computing, 42(1):A292–A317, 2020

    2020

  23. [23]

    Wasserstein generative adversarial un- certainty quantification in physics-informed neural networks.Journal of Computational Physics, 463:111270, 2022

    Yihang Gao and Michael K Ng. Wasserstein generative adversarial un- certainty quantification in physics-informed neural networks.Journal of Computational Physics, 463:111270, 2022

    2022

  24. [24]

    Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Moham- mad Sadegh Eshaghi, Xiaoying Zhuang, Timon Rabczuk, and Yinghua Liu. Kolmogorov–Arnold-Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov–Arnold Networks.Computer Methods in Applied Me- chanics and Engineering, 433:117...

    2025

  25. [25]

    Physics-informed transformer networks

    Fabricio Dos Santos, Tara Akhound-Sadegh, and Siamak Ravanbakhsh. Physics-informed transformer networks. InThe Symbiosis of Deep Learning and Differential Equations III, 2023

    2023

  26. [26]

    Pinnsformer: A transformer-based framework for physics-informed neural networks

    Zhiyuan Zhao, Xueying Ding, and B Aditya Prakash. Pinnsformer: A transformer-based framework for physics-informed neural networks. arXiv preprint arXiv:2307.11833, 2023

    work page arXiv 2023

  27. [27]

    Efficient training of physics-informed neural networks via impor- tance sampling.Computer-Aided Civil and Infrastructure Engineering, 36(8):962–977, 2021

    Mohammad Amin Nabian, Rini Jasmine Gladstone, and Hadi Mei- dani. Efficient training of physics-informed neural networks via impor- tance sampling.Computer-Aided Civil and Infrastructure Engineering, 36(8):962–977, 2021

    2021

  28. [28]

    Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A com- prehensive study of non-adaptive and residual-based adaptive sampling 46 for physics-informed neural networks.Computer Methods in Applied Mechanics and Engineering, 403:115671, 2023

    2023

  29. [29]

    DAS-PINNs: A deep adaptive sampling method for solving high-dimensional partial differen- tial equations.Journal of Computational Physics, 476:111868, 2023

    Kejun Tang, Xiaoliang Wan, and Chao Yang. DAS-PINNs: A deep adaptive sampling method for solving high-dimensional partial differen- tial equations.Journal of Computational Physics, 476:111868, 2023

    2023

  30. [30]

    Understanding and mit- igating gradient flow pathologies in physics-informed neural networks

    Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mit- igating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021

    2021

  31. [31]

    When and why PINNs fail to train: A neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022

    Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why PINNs fail to train: A neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022

    2022

  32. [32]

    Self-adaptive loss bal- anced physics-informed neural networks.Neurocomputing, 496:11–34, 2022

    Zixue Xiang, Wei Peng, Xu Liu, and Wen Yao. Self-adaptive loss bal- anced physics-informed neural networks.Neurocomputing, 496:11–34, 2022

    2022

  33. [33]

    Xi’an Li, Jiaxin Deng, Jinran Wu, Shaotong Zhang, Weide Li, and You-Gan Wang. Physical informed neural networks with soft and hard boundary constraints for solving advection-diffusion equations us- ing fourier expansions.Computers & Mathematics with Applications, 159:60–75, 2024

    2024

  34. [34]

    Solving a class of multi-scale elliptic PDEs by Fourier-based mixed physics in- formed neural networks.Journal of Computational Physics, 508:113012, 2024

    Xi’an Li, Jinran Wu, Xin Tai, Jianhua Xu, and You-Gan Wang. Solving a class of multi-scale elliptic PDEs by Fourier-based mixed physics in- formed neural networks.Journal of Computational Physics, 508:113012, 2024

    2024

  35. [35]

    A dimension-augmented physics-informed neural network (DaP- INN) with high level accuracy and efficiency.Journal of Computational Physics, 491:112360, 2023

    Weilong Guan, Kaihan Yang, Yinsheng Chen, Shaolin Liao, and Zhong Guan. A dimension-augmented physics-informed neural network (DaP- INN) with high level accuracy and efficiency.Journal of Computational Physics, 491:112360, 2023

    2023

  36. [36]

    Multi-stage neural networks: Function approximator of machine precision.Journal of Computational Physics, page 112865, 2024

    Yongji Wang and Ching-Yao Lai. Multi-stage neural networks: Function approximator of machine precision.Journal of Computational Physics, page 112865, 2024

    2024

  37. [37]

    Multi-level neural networks for accurate solutions of boundary-value 47 problems.Computer Methods in Applied Mechanics and Engineering, 419:116666, 2024

    Ziad Aldirany, Régis Cottereau, Marc Laforest, and Serge Prudhomme. Multi-level neural networks for accurate solutions of boundary-value 47 problems.Computer Methods in Applied Mechanics and Engineering, 419:116666, 2024

    2024

  38. [38]

    On computing the hyperparameter of extreme learning machines: Algorithm and application to computational PDEs, and comparison with classical and high-order finite elements

    Suchuan Dong and Jielin Yang. On computing the hyperparameter of extreme learning machines: Algorithm and application to computational PDEs, and comparison with classical and high-order finite elements. Journal of Computational Physics, 463:111290, 2022

    2022

  39. [39]

    Suchuan Dong and Zongwei Li. Local extreme learning machines and do- main decomposition for solving linear and nonlinear partial differential equations.Computer Methods in Applied Mechanics and Engineering, 387:114129, 2021

    2021

  40. [40]

    Optimization of random fea- ture method in the high-precision regime.Communications on Applied Mathematics and Computation, 6(2):1490–1517, 2024

    Jingrun Chen, Weinan E, and Yifei Sun. Optimization of random fea- ture method in the high-precision regime.Communications on Applied Mathematics and Computation, 6(2):1490–1517, 2024

    2024

  41. [41]

    The random feature method for time-depensent problems.https://arxiv.org/pdf/2304.06913v1, 2023

    Jingrun Chen, Yixin Luo, et al. The random feature method for time- dependent problems.arXiv preprint arXiv:2304.06913, 2023

    work page arXiv 2023

  42. [42]

    Bridging traditional and machine learning-based algorithms for solving PDEs: The random feature method.Journal of Machine Learning, 1(3):268–298, 2022

    Jingrun Chen, Xurong Chi, and Zhouwang Yang. Bridging traditional and machine learning-based algorithms for solving PDEs: The random feature method.Journal of Machine Learning, 1(3):268–298, 2022

    2022

  43. [43]

    Yong Shang, Fei Wang, and Jingbo Sun. Randomized neural network with Petrov–Galerkin methods for solving linear and nonlinear partial differential equations.Communications in Nonlinear Science and Nu- merical Simulation, 127:107518, 2023

    2023

  44. [44]

    Randomized neural networks with Petrov– Galerkin methods for solving linear elasticity and navier–stokes equa- tions.Journal of Engineering Mechanics, 150(4):04024010, 2024

    Yong Shang and Fei Wang. Randomized neural networks with Petrov– Galerkin methods for solving linear elasticity and navier–stokes equa- tions.Journal of Engineering Mechanics, 150(4):04024010, 2024

    2024

  45. [45]

    Randomized neural networks with petrov- galerkin methods for solving linear elasticity problems.arXiv preprint arXiv:2308.03088, 2023

    Yong Shang and Fei Wang. Randomized neural networks with Petrov- Galerkin methods for solving linear elasticity problems.arXiv preprint arXiv:2308.03088, 2023

    work page arXiv 2023

  46. [46]

    Randomized neural network methods for solving obstacle problems.Banach Center Publications, 127:261–276, 2024

    Fei Wang and Haoning Dang. Randomized neural network methods for solving obstacle problems.Banach Center Publications, 127:261–276, 2024. 48

    2024

  47. [47]

    Numerical computation of partial differ- entialequationsbyhidden-layerconcatenatedextremelearningmachine

    Naxian Ni and Suchuan Dong. Numerical computation of partial differ- entialequationsbyhidden-layerconcatenatedextremelearningmachine. Journal of Scientific Computing, 95(2):35, 2023

    2023

  48. [49]

    Separated-variable spectral neural networks: a physics-informed learn- ing approach for high-frequency pdes.arXiv preprint arXiv:2508.00628, 2025

    Xiong Xiong, Zhuo Zhang, Rongchun Hu, Chen Gao, and Zichen Deng. Separated-variable spectral neural networks: a physics-informed learn- ing approach for high-frequency pdes.arXiv preprint arXiv:2508.00628, 2025

    work page arXiv 2025

  49. [50]

    FG- PINNs: frequency-guided physics-informed neural networks for solving pdes with high frequency components.arXiv preprint arXiv:2511.12055, 2025

    Jiachun Zheng, Yunqing Huang, Nianyu Yi, and Yunlei Yang. FG- PINNs: frequency-guided physics-informed neural networks for solving pdes with high frequency components.arXiv preprint arXiv:2511.12055, 2025

    work page arXiv 2025

  50. [51]

    Physics-informed neural networks for high-frequency and multi- scale problems using transfer learning.Applied Sciences, 14(8), 2024

    Abdul Hannan Mustajab, Hao Lyu, Zarghaam Rizvi, and Frank Wut- tke. Physics-informed neural networks for high-frequency and multi- scale problems using transfer learning.Applied Sciences, 14(8), 2024

    2024

  51. [52]

    High-frequency flow field super- resolutionviaphysics-informedhierarchicaladaptiveFourierfeaturenet- works.Physics of Fluids, 37(9):097111, 09 2025

    Xiong Xiong, Kang Lu, Zhuo Zhang, Zheng Zeng, Sheng Zhou, Rongchun Hu, and Zichen Deng. High-frequency flow field super- resolutionviaphysics-informedhierarchicaladaptiveFourierfeaturenet- works.Physics of Fluids, 37(9):097111, 09 2025

    2025

  52. [53]

    Solving high frequency and multi-scale PDEs with Gaussian processes

    Shikai Fang, Madison Cooley, Da Long, Shibo Li, Mike Kirby, and Shan- dian Zhe. Solving high frequency and multi-scale PDEs with Gaussian processes. InThe Twelfth International Conference on Learning Repre- sentations, 2024

    2024

  53. [54]

    A novel Fourier feature network for solving partial differential equations.Communica- tions in Nonlinear Science and Numerical Simulation, 152:109274, 2026

    Qihong Yang, Zhijie Su, Yangtao Deng, and Qiaolin He. A novel Fourier feature network for solving partial differential equations.Communica- tions in Nonlinear Science and Numerical Simulation, 152:109274, 2026

    2026

  54. [55]

    pyRFM: A high-performance python im- plementation and applications of random feature methods.Journal On Numerical Methods and Computer Applications, pages 1–21

    Jingrun Chen and Yifei Sun. pyRFM: A high-performance python im- plementation and applications of random feature methods.Journal On Numerical Methods and Computer Applications, pages 1–21. 49

  55. [56]

    Random features for large-scale kernel machines.Advances in neural information processing systems, 20, 2007

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines.Advances in neural information processing systems, 20, 2007

    2007

  56. [57]

    Uniform approximation of functions with random bases

    Ali Rahimi and Benjamin Recht. Uniform approximation of functions with random bases. In2008 46th annual allerton conference on commu- nication, control, and computing, pages 555–561. IEEE, 2008

    2008

  57. [58]

    Spectral connvergece of random feature method in one dimension.arXiv preprint arXiv:2507.07371, 2025

    Pingbing Ming and Hao Yu. Spectral connvergece of random feature method in one dimension.arXiv preprint arXiv:2507.07371, 2025

    work page arXiv 2025

  58. [59]

    A learning based numerical method for Helmholtz equations with high frequency.Journal of Computational Physics, 520:113478, 2025

    Yu Chen, Jin Cheng, Tingyue Li, and Yun Miao. A learning based numerical method for Helmholtz equations with high frequency.Journal of Computational Physics, 520:113478, 2025

    2025

  59. [60]

    Learning based numerical methods for acoustic frequency-domain simulation with high frequency.Engineering Analysis with Boundary Elements, 163:200–210, 2024

    Tingyue Li, Yu Chen, Yun Miao, and Dingjiong Ma. Learning based numerical methods for acoustic frequency-domain simulation with high frequency.Engineering Analysis with Boundary Elements, 163:200–210, 2024

    2024

  60. [61]

    Sparse-regularized high-frequency enhanced neural network for solving high-frequency problems.Journal of Computational Physics, 523:113676, 2025

    Qilin Huang, Mingjin Fang, Dongsheng Cheng, Chunyuan Lu, and Tais- han Zeng. Sparse-regularized high-frequency enhanced neural network for solving high-frequency problems.Journal of Computational Physics, 523:113676, 2025

    2025

  61. [62]

    Understanding the difficulty of train- ing deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the difficulty of train- ing deep feedforward neural networks. InProceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010

    2010

  62. [63]

    Ameya D Jagtap and George Em Karniadakis. Extended physics- informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial dif- ferential equations.Communications in Computational Physics, 28(5), 2020

    2020

  63. [64]

    Ben Moseley, Andrew Markham, and Tarje Nissen-Meyer. Finite basis physics-informed neural networks (FBPINNs): a scalable domain de- composition approach for solving differential equations.Advances in Computational Mathematics, 49(4):62, 2023. 50

    2023

  64. [65]

    Fourier PINNs: from strong boundary conditions to adaptive Fourier bases.arXiv preprint arXiv:2410.03496, 2024

    Madison Cooley, Varun Shankar, Robert M Kirby, and Shandian Zhe. Fourier PINNs: from strong boundary conditions to adaptive Fourier bases.arXiv preprint arXiv:2410.03496, 2024

    work page arXiv 2024

  65. [66]

    Binary structured physics- informed neural networks for solving equations with rapidly changing solutions.Journal of Computational Physics, 518:113341, 2024

    Yanzhi Liu, Ruifan Wu, and Ying Jiang. Binary structured physics- informed neural networks for solving equations with rapidly changing solutions.Journal of Computational Physics, 518:113341, 2024. 51

    2024