REVIEW 11 cited by
CViT: Continuous Vision Transformer for Operator Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
CViT: Continuous Vision Transformer for Operator Learning
read the original abstract
Operator learning, which aims to approximate maps between infinite-dimensional function spaces, is an important area in scientific machine learning with applications across various physical domains. Here we introduce the Continuous Vision Transformer (CViT), a novel neural operator architecture that leverages advances in computer vision to address challenges in learning complex physical systems. CViT combines a vision transformer encoder, a novel grid-based coordinate embedding, and a query-wise cross-attention mechanism to effectively capture multi-scale dependencies. This design allows for flexible output representations and consistent evaluation at arbitrary resolutions. We demonstrate CViT's effectiveness across a diverse range of partial differential equation (PDE) systems, including fluid dynamics, climate modeling, and reaction-diffusion processes. Our comprehensive experiments show that CViT achieves state-of-the-art performance on multiple benchmarks, often surpassing larger foundation models, even without extensive pretraining and roll-out fine-tuning. Taken together, CViT exhibits robust handling of discontinuous solutions, multi-scale features, and intricate spatio-temporal dynamics. Our contributions can be viewed as a significant step towards adapting advanced computer vision architectures for building more flexible and accurate machine learning models in the physical sciences.
Forward citations
Cited by 11 Pith papers
-
CATO: Charted Attention for Neural PDE Operators
CATO learns a continuous latent chart for efficient axial attention on PDE meshes and adds derivative-aware supervision to improve accuracy and reduce oversmoothing on general geometries.
-
Physics-Informed Neural PDE Solvers via Spatio-Temporal MeanFlow
Spatio-Temporal MeanFlow adapts MeanFlow to PDEs by replacing the generative velocity field with the physical operator and extending the integral constraint to the spatio-temporal domain, yielding a unified solver for...
-
AI models of unstable flow exhibit hallucination
AI models of viscous fingering exhibit hallucinations from spectral bias; DeepFingers combines FNO and DeepONet with time-contrast conditioning to predict accurate finger dynamics while preserving mixing metrics.
-
Small Models, Strong Priors: Architectural Inductive Bias for Parameter-Efficient Neural PDE Solvers
WaveLiT combines wavelet tokenization, linear attention, and multiscale pyramids to produce parameter-efficient neural PDE solvers that match much larger models on TheWell benchmarks.
-
Do Neural Operators Forget Geometry? The Forgetting Hypothesis in Deep Operator Learning
Neural operators progressively forget domain geometry with depth due to Markovian layers and global mixing; a geometry memory injection mechanism mitigates this forgetting.
-
Conditional Neural Field based Reduced Order Model for Dynamic Ditching Load Prediction
Conditional neural fields combined with LSTM networks predict aircraft ditching loads accurately across heterogeneous spatial discretizations using fewer parameters than convolutional autoencoders.
-
Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems
APT, a mesh-agnostic neural operator fusing graph-based local features with global attention, is claimed to be the first architecture trained directly on adaptive-mesh-refinement simulations and outperforms state-of-t...
-
Structure-Preserving Learning Improves Geometry Generalization in Neural PDEs
A geometry-conditioned Whitney-form neural network that solves a learned discrete conservation law improves out-of-distribution geometry generalization for steady-state PDEs compared with regression-based neural operators.
-
On the training of physics-informed neural operators for solving parametric partial differential equations
Empirical benchmarks indicate CViT-based PINOs with adapted PINN mitigations can match or exceed data-driven neural operators on parametric PDE tasks.
-
Towards a Foundation Model for the Martian Atmosphere
The paper reviews data sources, physical models, downstream applications, and AI techniques to outline considerations for building a foundation model for the Martian atmosphere.
-
A Practitioner's Guide to Kolmogorov-Arnold Networks
A systematic review of Kolmogorov-Arnold Networks that maps their relation to Kolmogorov superposition theory, MLPs, and kernels, examines basis-function design choices, summarizes performance advances, and supplies a...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.