REVIEW 16 cited by
DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training
read the original abstract
Pre-training has been investigated to improve the efficiency and performance of training neural operators in data-scarce settings. However, it is largely in its infancy due to the inherent complexity and diversity, such as long trajectories, multiple scales and varying dimensions of partial differential equations (PDEs) data. In this paper, we present a new auto-regressive denoising pre-training strategy, which allows for more stable and efficient pre-training on PDE data and generalizes to various downstream tasks. Moreover, by designing a flexible and scalable model architecture based on Fourier attention, we can easily scale up the model for large-scale pre-training. We train our PDE foundation model with up to 0.5B parameters on 10+ PDE datasets with more than 100k trajectories. Extensive experiments show that we achieve SOTA on these benchmarks and validate the strong generalizability of our model to significantly enhance performance on diverse downstream PDE tasks like 3D data. Code is available at \url{https://github.com/thu-ml/DPOT}.
Forward citations
Cited by 16 Pith papers
-
Temperature Field Reconstruction of Tungsten Monoblock Divertor on EAST using Physics-aware Neural Operator Transformer
PNOT combines graph attention on boundary heat flux with a physics-aware neural operator and gradient-constrained loss to reconstruct divertor temperature fields for real-time fusion control.
-
Therm-FM: Foundation Model is ALL YOU NEED for 3D-ICs Thermal Simulation
Therm-FM adapts a pretrained PDE foundation model using thermal-equivalent multi-fidelity training to achieve up to 10.6x lower error in 3D-IC thermal simulation with under 20% of typical training data and strong cros...
-
Latent Generative Solvers for Generalizable Long-Term Physics Simulation
LGS pretrained on 2.5M trajectories across 16 systems matches deterministic baselines at one step and halves 20-step error while using far less compute and adapting to held-out higher-resolution flows.
-
Physics-Informed Neural Quantum Control for Rovibrational Photoassociation in a Morse Molecular System
PINQC optimizes laser pulses via neural networks and differentiable quantum dynamics to achieve continuum-to-bound rovibrational photoassociation in extended Morse models with larger rotational spaces.
-
Harness In-Context Operator Learning with Chain of Operators
CHOP reduces relative inference error on OOD operator tasks for scalar conservation laws and mean-field control by composing frozen ICON with explicit closed-form elementary operators that remain interpretable.
-
AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategies
AutoPDE maintains an explicit solver strategy through PDE analysis, numerical method selection, and adaptive tuning, achieving 54.5% pass rate on PDE Agent Bench, 14.2 points above the strongest baseline.
-
Small Models, Strong Priors: Architectural Inductive Bias for Parameter-Efficient Neural PDE Solvers
WaveLiT combines wavelet tokenization, linear attention, and multiscale pyramids to produce parameter-efficient neural PDE solvers that match much larger models on TheWell benchmarks.
-
Therm-FM: Foundation Model is ALL YOU NEED for 3D-ICs Thermal Simulation
Therm-FM adapts pretrained diffusion PDE foundation models to 3D-IC thermal simulation with multi-fidelity adaptation, reporting up to 10.6x mean error reduction and strong cross-design performance using under 20% of ...
-
ARC-STAR: Auditable Post-Hoc Correction for PDE Foundation Models
ARC-STAR reduces velocity rollout error by at least 36x over raw Poseidon across all tested regime cells via auditable global and local correction stages on five flow benchmarks.
-
ARC-STAR: Auditable Post-Hoc Correction for PDE Foundation Models
ARC-STAR is a frozen, auditable post-hoc correction method that reduces velocity rollout error by at least 36x over raw Poseidon across five flow benchmarks using global and local stages with budget-aware triage.
-
ARC-STAR: Auditable Post-Hoc Correction for PDE Foundation Models
ARC-STAR is an auditable, budget-aware post-hoc correction method that reduces velocity rollout error by at least 36x over raw Poseidon across five flow benchmarks.
-
Flow marching for a generative PDE foundation model
Flow Marching jointly samples noise and physical time to learn a velocity field for generative PDE modeling, paired with a latent autoencoder and efficient transformer for large-scale pretraining on 2.5M trajectories.
-
Neuro-Symbolic AI for Analytical Solutions of Differential Equations
SIGS is a neuro-symbolic framework that discovers analytical solutions to PDEs by generating grammar-constrained expressions, embedding them in a topology-regularised latent manifold, and refining structure and coeffi...
-
Physics-Informed Neural Quantum Control for Rovibrational Photoassociation in a Morse Molecular System
PINQC optimizes neural-network laser fields via differentiable Schrödinger propagation to photoassociate a Morse molecule, stably reaching l_max=6 versus prior TBQCP limits near l_max=4.
-
Physics-Informed Neural Quantum Control for Rovibrational Photoassociation in a Morse Molecular System
Physics-informed neural control optimizes laser pulses that transfer continuum Gaussian wave packets into the vibrational ground state of a Morse molecule up to l_max=6 with stable high fidelity.
-
Replay-Based Continual Learning for Physics-Informed Neural Operators
A replay-based continual learning strategy for physics-informed neural operators mitigates catastrophic forgetting on prior physical problems while enabling efficient adaptation to new data using only physical constraints.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.