GAIA introduces a geometry-adaptive integral autoencoder that unifies forward, boundary-value, and inverse PDE operator learning on arbitrary domains via geometry tokens and cross-attention.
hub Canonical reference
Transolver: A Fast Transformer Solver for PDEs on General Geometries
Canonical reference. 83% of citing Pith papers cite this work as background.
36
Pith papers citing it
10
external citations · Pith
Background
83% of classified citations
abstract
Transformers have empowered many milestones across various fields and have recently been applied to solve partial differential equations (PDEs). However, since PDEs are typically discretized into large-scale meshes with complex geometries, it is challenging for Transformers to capture intricate physical correlations directly from massive individual points. Going beyond superficial and unwieldy meshes, we present Transolver based on a more foundational idea, which is learning intrinsic physical states hidden behind discretized geometries. Specifically, we propose a new Physics-Attention to adaptively split the discretized domain into a series of learnable slices of flexible shapes, where mesh points under similar physical states will be ascribed to the same slice. By calculating attention to physics-aware tokens encoded from slices, Transovler can effectively capture intricate physical correlations under complex geometrics, which also empowers the solver with endogenetic geometry-general modeling capacity and can be efficiently computed in linear complexity. Transolver achieves consistent state-of-the-art with 22% relative gain across six standard benchmarks and also excels in large-scale industrial simulations, including car and airfoil designs. Code is available at https://github.com/thuml/Transolver.
hub tools
citation-role summary
background 5
method 1
citation-polarity summary
representative citing papers
MEEC equips point clouds with a discrete exterior calculus that satisfies exact conservation and is differentiable in point positions, allowing a single trained kernel to produce compatible physics on unseen geometries and parameters.
PNOT combines graph attention on boundary heat flux with a physics-aware neural operator and gradient-constrained loss to reconstruct divertor temperature fields for real-time fusion control.
Learning only the Stokes boundary correction on an exact free-space core yields a 5–16× more data-efficient operator than black-box DeepONet, with geometric generalization controlled by descriptor invariance and training coverage.
Functional Attention replaces pairwise softmax attention with structured linear operators inspired by geometric functional maps to produce compact, resolution-invariant representations for operator learning.
NEO is a mass-aware neural operator that learns the invariant low-frequency eigenspace of the LBO on point clouds for fast spectral geometry.
Releases first open high-fidelity CFD dataset of 1800 samples from 180 variants of NASA high-lift CRM at 10 angles of attack using GPU-accelerated wall-modeled LES.
Fine-tuning neural PDE operators to regime endpoints reveals a physical direction in weight space that CCM uses to compose accurate merged models for new or extrapolated regimes from metadata or short prefixes.
CATO learns a continuous latent chart for efficient axial attention on PDE meshes and adds derivative-aware supervision to improve accuracy and reduce oversmoothing on general geometries.
Physics-informed Fourier neural operators recover plasmoid formation in sparse SRRMHD vortex data where data-only models fail, and transformer operators approximate AMR jet evolution, marking first reported uses in these relativistic MHD settings.
A graph-based neural operator trained on expert-validated race-car CFD data reaches accuracy levels usable for early-stage interactive aerodynamic design exploration.
A kernel operator learning framework constructs property-preserving bases so that predicted incompressible velocity fields satisfy divergence-free and periodicity conditions exactly, delivering up to six orders lower error and five orders faster training than neural operators.
LSR-Net learns long-short-range operators for pattern dynamics on manifolds via Fourier multipliers and Gaussian gridding, showing lower RMSE than SFNO on Allen-Cahn and similar systems.
Self-attention mechanisms are used to build mesh-preserving neural surrogates that approximate PFEM dynamics for free-surface flows, delivering accurate transient predictions and improved scalability on 2D and 3D benchmarks.
EqGINO adds a spectral isotropy prior to FNOs to guarantee discrete equivariance and enable generalization to continuous SE(3) transformations on 3D PDEs with limited training data.
A transformer-based dual-attention neural operator framework predicts 3D wind fields over complex terrain, outperforming baselines by 10% with further gains from sparse data.
Courant is a state-adaptive Perceiver encoder-processor-decoder surrogate trained with L2 loss that yields interpretable, multiscale, locally supported latent features acting as time-evolving spatial basis functions.
IKNO replaces first-order kernel integrals in neural operators with infinite-order versions that have efficient closed-form approximations and reports SOTA accuracy on time-dependent and time-independent benchmarks.
Mask-Morph Graph U-Net morphs coarse graph hierarchies with barycentric parameterization and applies masked supervised pretraining to improve generalizability of hierarchical GNN surrogates for crashworthiness prediction on variable meshes.
U-HNO uses adaptive per-point routing in a U-shaped hybrid architecture to achieve state-of-the-art accuracy on PDE benchmarks with sharp localized features.
ShardTensor is a domain-parallelism system for SciML that enables flexible scaling of extreme-resolution spatial datasets by removing the constraint of batch size one per device.
ABLE learns a spatially adaptive Parseval frame from data via an ancillary density to replace fixed bases in spectral neural operators for PDEs.
CarCrashNet supplies a large multi-modal crash simulation benchmark and CrashSolver neural model for data-driven full-vehicle crash prediction, validated against experiments and commercial solvers.
AeroJEPA applies joint-embedding predictive learning to produce scalable, semantically organized latent representations for 3D aerodynamic fields that support both field reconstruction and downstream design tasks.
citing papers explorer
-
GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems
GAIA introduces a geometry-adaptive integral autoencoder that unifies forward, boundary-value, and inverse PDE operator learning on arbitrary domains via geometry tokens and cross-attention.
-
A meshfree exterior calculus for generalizable and data-efficient learning of physics from point clouds
MEEC equips point clouds with a discrete exterior calculus that satisfies exact conservation and is differentiable in point positions, allowing a single trained kernel to produce compatible physics on unseen geometries and parameters.
-
Temperature Field Reconstruction of Tungsten Monoblock Divertor on EAST using Physics-aware Neural Operator Transformer
PNOT combines graph attention on boundary heat flux with a physics-aware neural operator and gradient-constrained loss to reconstruct divertor temperature fields for real-time fusion control.
-
Solver Exactness, Learned Flexibility: Equivariant Boundary-Correction Operators for Stokes Flow
Learning only the Stokes boundary correction on an exact free-space core yields a 5–16× more data-efficient operator than black-box DeepONet, with geometric generalization controlled by descriptor invariance and training coverage.
-
Functional Attention: From Pairwise Affinities to Functional Correspondences
Functional Attention replaces pairwise softmax attention with structured linear operators inspired by geometric functional maps to produce compact, resolution-invariant representations for operator learning.
-
Learning Laplacian Eigenspace with Mass-Aware Neural Operators on Point Clouds
NEO is a mass-aware neural operator that learns the invariant low-frequency eigenspace of the LBO on point clouds for fast spectral geometry.
-
HiLiftAeroML: High-Fidelity Computational Fluid Dynamics Dataset for High-Lift Aircraft Aerodynamics
Releases first open high-fidelity CFD dataset of 1800 samples from 180 variants of NASA high-lift CRM at 10 angles of attack using GPU-accelerated wall-modeled LES.
-
Discovering Physical Directions in Weight Space: Composing Neural PDE Experts
Fine-tuning neural PDE operators to regime endpoints reveals a physical direction in weight space that CCM uses to compose accurate merged models for new or extrapolated regimes from metadata or short prefixes.
-
CATO: Charted Attention for Neural PDE Operators
CATO learns a continuous latent chart for efficient axial attention on PDE meshes and adds derivative-aware supervision to improve accuracy and reduce oversmoothing on general geometries.
-
Learning Neural Operator Surrogates for the Black Hole Accretion Code
Physics-informed Fourier neural operators recover plasmoid formation in sparse SRRMHD vortex data where data-only models fail, and transformer operators approximate AMR jet evolution, marking first reported uses in these relativistic MHD settings.
-
Faster by Design: Interactive Aerodynamics via Neural Surrogates Trained on Expert-Validated CFD
A graph-based neural operator trained on expert-validated race-car CFD data reaches accuracy levels usable for early-stage interactive aerodynamic design exploration.
-
Fluids You Can Trust: Property-Preserving Operator Learning for Incompressible Flows
A kernel operator learning framework constructs property-preserving bases so that predicted incompressible velocity fields satisfy divergence-free and periodicity conditions exactly, delivering up to six orders lower error and five orders faster training than neural operators.
-
LSR-Net: Long-Short-Range Operator Learning for Pattern Dynamics on Manifolds
LSR-Net learns long-short-range operators for pattern dynamics on manifolds via Fourier multipliers and Gaussian gridding, showing lower RMSE than SFNO on Allen-Cahn and similar systems.
-
Attention mechanism for scalable mesh-based neural surrogates of free-surface fluids
Self-attention mechanisms are used to build mesh-preserving neural surrogates that approximate PFEM dynamics for free-surface flows, delivering accurate transient predictions and improved scalability on 2D and 3D benchmarks.
-
EqGINO: Equivariant Geometry-Informed Fourier Neural Operators for 3D PDEs
EqGINO adds a spectral isotropy prior to FNOs to guarantee discrete equivariance and enable generalization to continuous SE(3) transformations on 3D PDEs with limited training data.
-
Transformer-based Neural Operators for 3D Wind Field Prediction over Complex Mountainous Terrain
A transformer-based dual-attention neural operator framework predicts 3D wind fields over complex terrain, outperforming baselines by 10% with further gains from sparse data.
-
Courant: a State-Adaptive Perceiver-Based Neural Surrogate with Local Support and Interpretable Field Decomposition
Courant is a state-adaptive Perceiver encoder-processor-decoder surrogate trained with L2 loss that yields interpretable, multiscale, locally supported latent features acting as time-evolving spatial basis functions.
-
IKNO: Infinite-order Kernel Neural Operators
IKNO replaces first-order kernel integrals in neural operators with infinite-order versions that have efficient closed-form approximations and reports SOTA accuracy on time-dependent and time-independent benchmarks.
-
Mask-Morph Graph U-Net: A Generalisable Mesh-Based Surrogate for Crashworthiness Field Prediction under Large Geometric Variation
Mask-Morph Graph U-Net morphs coarse graph hierarchies with barycentric parameterization and applies masked supervised pretraining to improve generalizability of hierarchical GNN surrogates for crashworthiness prediction on variable meshes.
-
U-HNO: A U-shaped Hybrid Neural Operator with Sparse-Point Adaptive Routing for Non-stationary PDE Dynamics
U-HNO uses adaptive per-point routing in a U-shaped hybrid architecture to achieve state-of-the-art accuracy on PDE benchmarks with sharp localized features.
-
ShardTensor: Domain Parallelism for Scientific Machine Learning
ShardTensor is a domain-parallelism system for SciML that enables flexible scaling of extreme-resolution spatial datasets by removing the constraint of batch size one per device.
-
Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEs
ABLE learns a spatially adaptive Parseval frame from data via an ancillary density to replace fixed bases in spectral neural operators for PDEs.
-
CarCrashNet: A Large-Scale Dataset and Hierarchical Neural Solver for Data-Driven Structural Crash Simulation
CarCrashNet supplies a large multi-modal crash simulation benchmark and CrashSolver neural model for data-driven full-vehicle crash prediction, validated against experiments and commercial solvers.
-
AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modeling
AeroJEPA applies joint-embedding predictive learning to produce scalable, semantically organized latent representations for 3D aerodynamic fields that support both field reconstruction and downstream design tasks.
-
PerFlow: Physics-Embedded Rectified Flow for Efficient Reconstruction and Uncertainty Quantification of Spatiotemporal Dynamics
PerFlow decouples observation conditioning from physics enforcement in rectified flows using constraint-preserving projections and invariance guarantees for fast, physics-consistent reconstruction of spatiotemporal dynamics.
-
A Hybridizable Neural Time Integrator for Stable Autoregressive Forecasting
A hybrid transformer-FEM integrator provides provable discrete energy preservation and gradient bounds for stable autoregressive forecasting of chaotic systems, with 65x fewer parameters and 9000x speedup in a fusion surrogate trained on 12 simulations.
-
FLARE: A Data-Efficient Surrogate for Predicting Displacement Fields in Directed Energy Deposition
FLARE predicts post-cooling displacement fields in directed energy deposition by encoding simulations as implicit neural fields whose weights are regularized to follow an affine structure in parameter space, enabling data-efficient prediction via weight mixing.
-
A Structure-Preserving Graph Neural Solver for Parametric Hyperbolic Conservation Laws
A structure-preserving GNN solver for parametric hyperbolic conservation laws achieves superior long-horizon stability and orders-of-magnitude speedups over high-resolution simulations on supersonic flow benchmarks.
-
SuperWing: a comprehensive transonic wing dataset for data-driven aerodynamic design
SuperWing supplies 4,239 diverse wing shapes and 28,856 flow-field solutions that let Transformer models predict surface aerodynamics to 2.5 drag-count error and generalize zero-shot to DLR-F6 and NASA CRM wings.
-
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Di-BiLPS combines a variational autoencoder, latent diffusion, and contrastive learning to achieve state-of-the-art accuracy on PDE problems with as little as 3% observations while supporting zero-shot super-resolution and lower computational cost.
-
An Exploratory Study into using Machine-Learning for Fast Step-by-step Emulation of Numerical Mechanical Thrombectomy Simulations for Ischemic Stroke
ML surrogates accurately emulate single steps of simplified thrombectomy simulations with speedups but lack stability over long times with complex geometries.
-
High-Fidelity Industrial Crash Dynamics Prediction via Geometry-Aware Operator Learning with Memory-Efficient Low-Rank Attention
GeoTransolver applies geometry-aware operator learning and low-rank attention to predict high-fidelity crash dynamics on bumper and full-vehicle datasets, with one-shot temporal prediction achieving state-of-the-art accuracy and reduced overhead.
-
PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting
PnP-Corrector decouples pre-trained physics engines from a correction agent to mitigate reciprocal error amplification in coupled spatiotemporal forecasting, cutting error by 28% on a 300-day ocean-atmosphere task.
-
Replay-Based Continual Learning for Physics-Informed Neural Operators
A replay-based continual learning strategy for physics-informed neural operators mitigates catastrophic forgetting on prior physical problems while enabling efficient adaptation to new data using only physical constraints.
-
A Simple but Efficient Transformer-Based Physics-Informed Neural Network for Incompressible Navier--Stokes Equations
PhysicsFormer applies a lightweight Transformer PINN with pseudo-sequential representations to convection, Burgers, lid-driven cavity, and inverse Navier-Stokes problems, reporting near-zero error in parameter identification and flow reconstruction from sparse noisy data.
- RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamics