REVIEW 18 cited by
PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks
read the original abstract
Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions to partial differential equations (PDEs). However, conventional PINNs, relying on multilayer perceptrons (MLP), neglect the crucial temporal dependencies inherent in practical physics systems and thus fail to propagate the initial condition constraints globally and accurately capture the true solutions under various scenarios. In this paper, we introduce a novel Transformer-based framework, termed PINNsFormer, designed to address this limitation. PINNsFormer can accurately approximate PDE solutions by utilizing multi-head attention mechanisms to capture temporal dependencies. PINNsFormer transforms point-wise inputs into pseudo sequences and replaces point-wise PINNs loss with a sequential loss. Additionally, it incorporates a novel activation function, Wavelet, which anticipates Fourier decomposition through deep neural networks. Empirical results demonstrate that PINNsFormer achieves superior generalization ability and accuracy across various scenarios, including PINNs failure modes and high-dimensional PDEs. Moreover, PINNsFormer offers flexibility in integrating existing learning schemes for PINNs, further enhancing its performance.
Forward citations
Cited by 18 Pith papers
-
Curvature-aware dynamic precision approach for physics-informed neural networks
The curvature-aware precision controller adapts between FP32 and FP64 during PINN training to match double-precision accuracy at reduced computational cost.
-
Cosmo-SPINN: Fuzzy Dark Matter Simulations with Physics-Informed Generative Networks
Physics-informed generative U-Nets evolve and super-resolve fuzzy dark matter fields under Schrödinger–Poisson constraints with far less supervised data than pure data-driven baselines.
-
Harnessing Machine Learning for Hybrid Constitutive Modelling of Viscoelastic Fluid Flows in Computational Rheology
A four-term tensor-basis neural network trained only on oscillatory shear data, embedded in a finite-volume CFD solver, qualitatively reproduces viscoelastic contraction and cross-slot flows, including the onset of el...
-
A Physics-Informed Fourier-Wavelet Transformer for Multiscale Computational Fluid Dynamics Surrogate Modeling
A physics-informed Fourier-wavelet transformer model reports the lowest normalized mean-squared error on cylinder-wake and fluid-structure interaction velocity-field benchmarks compared with spectral, transformer, ope...
-
Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs
MS-SFNN builds PDE solutions from element-wise products of outputs from d independent fixed-random-weight subnetworks with tunable scaling and cosine activations, then solves coefficients by least squares, claiming su...
-
Physics-Informed Neural Networks with Attention Feature Expansion for Monge-Amp\`ere Equations
PINN-AFE uses multi-head attention and input convex networks to solve Monge-Ampère equations with claimed accuracy, efficiency, and extensions to image enhancement and medical registration.
-
Deep Wave Network for Modeling Multi-Scale Physical Dynamics
DW-Net improves the accuracy versus computational cost Pareto front over standard U-Nets for 2D and 3D multi-scale flow benchmarks by stacking multiple waves while keeping training settings identical.
-
Can Transformers predict system collapse in dynamical systems?
Transformers fail to predict catastrophic collapse in unseen parameter regimes of nonlinear dynamical systems, while reservoir computing reliably succeeds.
-
When PINNs Go Wrong: Pseudo-Time Stepping Against Spurious Solutions
PINNs fail on spurious solutions admitted by the residual loss; adaptive pseudo-time stepping with Jacobian-based step selection improves accuracy and robustness on PDE benchmarks.
-
LLT: Local Linear Transformer for PDE Operator Learning
Local Linear Transformer learns PDE operators by combining linear global attention with local spatial mixing, achieving competitive accuracy and lower training-step cost than prior transformers.
-
Physics-Informed Neural Network with Squeeze-Excitation-like Attention
SEA-PINN adds squeeze-excitation attention to PINNs, yielding stable low-variance initialization and competitive accuracy on 17 of 20 benchmarks without Fourier features or periodic activations.
-
Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs
MS-SFNN encodes multi-scale Fourier features in a separable product of fixed-weight cosine subnetworks and solves for linear coefficients by least squares, claiming better accuracy than PINN and SV-SNN on high-frequency PDEs.
-
Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs
MS-SFNN encodes multi-scale Fourier features in a separable product of 1D cosine subnetworks and solves high-frequency PDEs via least-squares basis coefficients, claiming better accuracy than PINN and SV-SNN.
-
Integrating Fourier Neural Operator with Diffusion Model for Autoregressive Predictions of Three-dimensional Turbulence
DiAFNO, an implicit adaptive Fourier neural operator used as the denoiser inside an EDM diffusion model, gives more accurate autoregressive predictions of 3D turbulence than EDM or dynamic Smagorinsky LES.
-
Neural Multiscale Decomposition for Solving The Nonlinear Klein-Gordon Equation with Time Oscillation
NeuralMD solves the oscillatory NKGE by training one network on the slow NLSW envelope and another on the remainder, but its model-selection step requires the exact solution as ground truth.
-
Split Complex-Valued Physics-Informed Neural Networks for Forward and Inverse Nonlinear PDEs
Split complex-valued PINNs achieve lower benchmark errors than real-valued PINNs, but the comparison is confounded by doubled parameters and unresolved internal error inconsistencies.
-
A new strategy for physics-informed neural networks based on hierarchical collocation point refinement
Training a single PINN on progressively finer collocation point sets, carrying over network parameters, cuts training time by an order of magnitude on tested Poisson, convection-diffusion-reaction, and Helmholtz bench...
-
A Simple but Efficient Transformer-Based Physics-Informed Neural Network for Incompressible Navier--Stokes Equations
PhysicsFormer applies a lightweight Transformer PINN with pseudo-sequential representations to convection, Burgers, lid-driven cavity, and inverse Navier-Stokes problems, reporting near-zero error in parameter identif...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.