REVIEW 31 cited by
Transformer for Partial Differential Equations' Operator Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Transformer for Partial Differential Equations' Operator Learning
read the original abstract
Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.
Forward citations
Cited by 31 Pith papers
-
GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems
GAIA introduces a geometry-adaptive integral autoencoder that unifies forward, boundary-value, and inverse PDE operator learning on arbitrary domains via geometry tokens and cross-attention.
-
HS-FNO: History-Space Fourier Neural Operator for Non-Markovian Partial Differential Equations
HS-FNO lifts the state to include history and decomposes updates into a learned future-slice predictor plus an exact shift-append transport, yielding lower rollout errors than standard or lag-stack FNO baselines on fi...
-
Temperature Field Reconstruction of Tungsten Monoblock Divertor on EAST using Physics-aware Neural Operator Transformer
PNOT combines graph attention on boundary heat flux with a physics-aware neural operator and gradient-constrained loss to reconstruct divertor temperature fields for real-time fusion control.
-
HAMNO: A Hierarchical Adaptive Multi-scale Neural Operator with Physics-Informed Learning for Dynamical Systems
HAMNO introduces adaptive gating between local and global operators in a hierarchical setup, with PI-HAMNO adding PDE residual constraints, demonstrating better performance on Allen-Cahn, Cahn-Hilliard, and Swift-Hohe...
-
MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers
MeshTok uses AMR-inspired adaptive multiscale tokenization to improve the efficiency-accuracy trade-off of Transformer models for PDEs over uniform-grid baselines.
-
HS-FNO: History-Space Fourier Neural Operator for Non-Markovian Partial Differential Equations
HS-FNO learns non-Markovian PDE dynamics on history-space states by predicting only the new future slice and using exact shift-append transport, cutting aggregate rollout error from 0.185-0.241 to 0.094 across benchma...
-
CATO: Charted Attention for Neural PDE Operators
CATO learns a continuous latent chart for efficient axial attention on PDE meshes and adds derivative-aware supervision to improve accuracy and reduce oversmoothing on general geometries.
-
QuadNorm: Resolution-Robust Normalization for Neural Operators
QuadNorm uses quadrature-based moments instead of uniform averaging in normalization layers, achieving O(h²) consistency across resolutions and better cross-resolution transfer in neural operators.
-
Learning Neural Operator Surrogates for the Black Hole Accretion Code
Physics-informed Fourier neural operators recover plasmoid formation in sparse SRRMHD vortex data where data-only models fail, and transformer operators approximate AMR jet evolution, marking first reported uses in th...
-
One Scale at a Time: Scale-Autoregressive Modeling for Fluid Flow Distributions
Scale-autoregressive modeling (SAR) samples fluid flow distributions hierarchically from coarse to fine resolutions on meshes, achieving lower distributional error and 2-7x faster runtime than diffusion or flow-matchi...
-
Latent Generative Solvers for Generalizable Long-Term Physics Simulation
LGS pretrained on 2.5M trajectories across 16 systems matches deterministic baselines at one step and halves 20-step error while using far less compute and adapting to held-out higher-resolution flows.
-
Deep Gaussian Processes for Functional Maps
DGPFM stacks GP-based linear and nonlinear transformations in function space via kernel integrals and inducing-point variational learning for function-on-function regression.
-
Physics Transformer: Tailoring Transformer for General PDE Prediction
Patch-wise function projection plus factorized space/state attention yields a Transformer PDE solver that reports state-of-the-art accuracy from 2D dynamics to million-scale 3D CFD.
-
Adaptive Mamba Neural Operators
AMO builds adaptive Takenaka-Malmquist bases inside a Mamba state-space model for PDE operator learning, but the claimed equivalence to adaptive Fourier decomposition is not supported by the implemented recurrence.
-
Evaluation of State-of-the-Art Deep Learning Architectures for Aerodynamical Predictions
Benchmarking four neural operators for airfoil and NASA CRM pressure prediction: Transolver best on 2D, BSMS-GNN best on 3D; UPT and GAOT lag.
-
Attention mechanism for scalable mesh-based neural surrogates of free-surface fluids
Self-attention mechanisms are used to build mesh-preserving neural surrogates that approximate PFEM dynamics for free-surface flows, delivering accurate transient predictions and improved scalability on 2D and 3D benchmarks.
-
EqGINO: Equivariant Geometry-Informed Fourier Neural Operators for 3D PDEs
EqGINO adds a spectral isotropy prior to FNOs to guarantee discrete equivariance and enable generalization to continuous SE(3) transformations on 3D PDEs with limited training data.
-
IV-Net: A neural network for elliptic PDEs with random and highly varying coefficients
IV-Net is a multigrid-inspired convolutional neural operator that approximates solutions to linear elliptic PDEs with high-contrast coefficients and shows better accuracy than POD and other neural operators on heterog...
-
AOT-POT: Adaptive Operator Transformation for Large-Scale PDE Pre-training
AOT-POT adaptively reshapes complex PDE solution operators via input-dependent transformations and parallel stream mixing to enable effective large-scale pre-training, yielding SOTA results on 12 benchmarks with minim...
-
Pretrained Video Models as Differentiable Physics Simulators for Urban Wind Flows
WinDiNet repurposes a 2B-parameter video diffusion model as a differentiable surrogate that generates 112-frame urban wind flow rollouts in under one second and enables direct gradient optimization of building positions.
-
Pretrained Video Models as Differentiable Physics Simulators for Urban Wind Flows
A fine-tuned video diffusion model becomes a fast, differentiable CFD surrogate for urban wind, enabling gradient-based building-layout optimization confirmed by ground-truth simulations.
-
ArGEnT: Arbitrary Geometry-encoded Transformer for Operator Learning
ArGEnT adds self-, cross-, and hybrid-attention transformers to DeepONet to learn geometry-dependent operators from point-cloud inputs, yielding higher accuracy than standard DeepONet on fluid, solid, and electrochemi...
-
Differentiable Autoencoding Neural Operator for Interpretable and Integrable Latent Space Modeling
DIANO builds coarse-grid latent spaces for fluid dynamics data via neural operator encoding and decoding while integrating a differentiable PDE solver directly in the latent space for end-to-end physics-constrained training.
-
Flow marching for a generative PDE foundation model
Flow Marching jointly samples noise and physical time to learn a velocity field for generative PDE modeling, paired with a latent autoencoder and efficient transformer for large-scale pretraining on 2.5M trajectories.
-
TAEN: A Model-Constrained Tikhonov Autoencoder Network for Forward and Inverse Problems
TAE combines Tikhonov regularization with autoencoders and a data randomization strategy to learn forward and inverse surrogates from one sample, with linear error bounds and tests on heat inversion and Navier-Stokes ...
-
PI-DOSnet: A Physics-Informed Deep Operator-Splitting Network for Evolution Partial Differential Equations
PI-DOSnet is a physics-informed operator learning framework using operator splitting that enables data-free long-time inference of evolution PDE solutions, with energy stability shown for the Allen-Cahn equation at la...
-
Integrating Fourier Neural Operator with Diffusion Model for Autoregressive Predictions of Three-dimensional Turbulence
DiAFNO, an implicit adaptive Fourier neural operator used as the denoiser inside an EDM diffusion model, gives more accurate autoregressive predictions of 3D turbulence than EDM or dynamic Smagorinsky LES.
-
FEDONet : Fourier-Embedded DeepONet for Spectrally Accurate Operator Learning
FEDONet augments DeepONet with Fourier-embedded trunk networks using random Fourier features, yielding lower L2 reconstruction errors than standard DeepONet on Burgers', 2D Poisson, Eikonal, Allen-Cahn, and Kuramoto-S...
-
On the training of physics-informed neural operators for solving parametric partial differential equations
Empirical benchmarks indicate CViT-based PINOs with adapted PINN mitigations can match or exceed data-driven neural operators on parametric PDE tasks.
-
High-Fidelity Industrial Crash Dynamics Prediction via Geometry-Aware Operator Learning with Memory-Efficient Low-Rank Attention
GeoTransolver applies geometry-aware operator learning and low-rank attention to predict high-fidelity crash dynamics on bumper and full-vehicle datasets, with one-shot temporal prediction achieving state-of-the-art a...
-
RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamics
RETO achieves relative L2 errors of 0.063 on ShapeNet and 0.089/0.097 on DrivAerML surface pressure/velocity, outperforming Transolver and other baselines.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.