KAN-SAE applies nonlinear per-feature B-spline activations in sparse autoencoders to discover 72% more alive climate features and interpretable patterns such as European heatwaves and Pacific typhoons in deep learning weather models.
hub
Climax: A foundation model for weather and climate
26 Pith papers cite this work. Polarity classification is still indexing.
26
Pith papers citing it
abstract
Most state-of-the-art approaches for weather and climate modeling are based on physics-informed numerical models of the atmosphere. These approaches aim to model the non-linear dynamics and complex interactions between multiple variables, which are challenging to approximate. Additionally, many such numerical models are computationally intensive, especially when modeling the atmospheric phenomenon at a fine-grained spatial and temporal resolution. Recent data-driven approaches based on machine learning instead aim to directly solve a downstream forecasting or projection task by learning a data-driven functional mapping using deep neural networks. However, these networks are trained using curated and homogeneous climate datasets for specific spatiotemporal tasks, and thus lack the generality of numerical models. We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings. ClimaX extends the Transformer architecture with novel encoding and aggregation blocks that allow effective use of available compute while maintaining general utility. ClimaX is pre-trained with a self-supervised learning objective on climate datasets derived from CMIP6. The pre-trained ClimaX can then be fine-tuned to address a breadth of climate and weather tasks, including those that involve atmospheric variables and spatio-temporal scales unseen during pretraining. Compared to existing data-driven baselines, we show that this generality in ClimaX results in superior performance on benchmarks for weather forecasting and climate projections, even when pretrained at lower resolutions and compute budgets. The source code is available at https://github.com/microsoft/ClimaX.
hub tools
citation-role summary
background 4
citation-polarity summary
fields
cs.LG 16 cs.CV 2 physics.ao-ph 2 astro-ph.EP 1 astro-ph.IM 1 cs.AI 1 cs.CL 1 cs.MM 1 physics.flu-dyn 1roles
background 4polarities
background 4representative citing papers
STaT is a Symbolic-Temporal-Textual Alignment model that integrates three modalities to reduce shape distortion in non-stationary time series forecasting, reporting up to 8.9% gains in magnitude metrics and 8.5% less distortion on eight benchmarks.
ML climate emulators degrade under seasonal distribution shifts that proxy long-term climate change, but physically motivated compositional decompositions improve out-of-distribution performance with modest in-distribution trade-offs.
SAFE-SVD introduces a sensitivity-aware fidelity-enforcing SVD framework for compressing physics foundation models that maintains higher accuracy than standard methods at greater compression ratios.
Introduces WILDFIRE-FM and a fixed-contract evaluation framework demonstrating that wildfire model transfer conclusions depend strongly on evaluation design and task formulation.
Q-SRDRN multi-quantile network with pinball loss and per-quantile heads detects extreme precipitation events up to 18 times more effectively than deterministic baselines while preserving augmentation benefits for the median.
MixINN combines mixed models and deep learning to predict genotype-environment interactions in corn trials, yielding 5.8-7.2% higher average yields when selecting top-performing genotypes compared to standard methods.
DW-Net improves the accuracy versus computational cost Pareto front over standard U-Nets for 2D and 3D multi-scale flow benchmarks by stacking multiple waves while keeping training settings identical.
A multimodal SwinV2-UNet vision transformer conditioned on data modality and time predicts spatiotemporal fluid flows and reconstructs unobserved fields from limited views using CFD data of argon jet injection.
GSNO uses position-dependent spherical Green's functions to create flexible neural operators that adapt to non-equivariant systems on spheres while keeping spectral efficiency and grid invariance.
GAIR introduces a geo-aligned implicit representation module inside a multi-encoder contrastive SSL framework that produces location-aware embeddings and outperforms prior geo-foundation models on 22 geospatial datasets across 9 tasks.
Diffusion models recover known ENSO variability structure from synthetic LIM data when given enough samples, but require pre-training on CMIP6 plus fine-tuning to match observations with the ~700 samples available in ERSSTv5.
Otter Weather is a spatiotemporal model that outperforms NWP baselines by 9.6% at 24h lead with under 3.5 A100-days training and extends efficiency gains to probabilistic forecasting via CRPS.
Standardized pretraining and evaluation of geospatial multimodal foundation models on GEOBench reveals design trade-offs in flexibility, modality alignment, and task performance.
FREUD applies rectified flow transformers with frame-wise encoding and a unified decoder to achieve state-of-the-art probabilistic precipitation nowcasting on the SEVIR benchmark.
Hybrid quantum-classical corrective diffusion model improves MAE and CRPS on 2020 validation wind data but exhibits a generalization gap on 2021 out-of-distribution tests.
PINN-Cast combines continuous-depth Neural ODEs inside transformer blocks with a two-branch attention module and physics-informed loss to produce short-term weather forecasts that respect governing physical principles.
Scaling laws for weather models exhibit strong cross-channel and cross-horizon heterogeneity, where globally pooled metrics appear favorable while many individual channels degrade at longer leads.
LangRetrieval is a conditional flow matching framework with semantic warm-up and GRPO-based self-evolving optimization using CSI rewards to improve satellite-to-radar precipitation retrieval.
A survey organizing AI-powered research automation into five workflow stages, defining AutoResearch and Vibe Research, and proposing five evaluation dimensions while noting domain-conditioned limits on autonomy.
Remote-sensing foundation models need domain-specific design and evaluation around measurement physics and decision constraints; benchmark accuracy alone is insufficient for trustworthy EO deployment.
The paper reviews data sources, physical models, downstream applications, and AI techniques to outline considerations for building a foundation model for the Martian atmosphere.
A review of Earth science foundation models covering capability evolution from perception to discovery, applications across atmosphere/hydrosphere/lithosphere/biosphere/anthroposphere/cryosphere, over 200 datasets, and key challenges.
AI methods can strengthen cross-domain interactions and support more coherent multi-component representations in Earth system models.
citing papers explorer
-
Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
KAN-SAE applies nonlinear per-feature B-spline activations in sparse autoencoders to discover 72% more alive climate features and interpretable patterns such as European heatwaves and Pacific typhoons in deep learning weather models.
-
STaT: Resolving Shape Distortion in Non-Stationary Time Series via Tri-Modal Synergy
STaT is a Symbolic-Temporal-Textual Alignment model that integrates three modalities to reduce shape distortion in non-stationary time series forecasting, reporting up to 8.9% gains in magnitude metrics and 8.5% less distortion on eight benchmarks.
-
No Epoch Like the Present: Robust Climate Emulation Requires Out-of-Distribution Generalisation
ML climate emulators degrade under seasonal distribution shifts that proxy long-term climate change, but physically motivated compositional decompositions improve out-of-distribution performance with modest in-distribution trade-offs.
-
SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
SAFE-SVD introduces a sensitivity-aware fidelity-enforcing SVD framework for compressing physics foundation models that maintains higher accuracy than standard methods at greater compression ratios.
-
Does Your Wildfire Prediction Model Actually Work, or Just Score Well?
Introduces WILDFIRE-FM and a fixed-contract evaluation framework demonstrating that wildfire model transfer conclusions depend strongly on evaluation design and task formulation.
-
Multi-Quantile Regression for Extreme Precipitation Downscaling
Q-SRDRN multi-quantile network with pinball loss and per-quantile heads detects extreme precipitation events up to 18 times more effectively than deterministic baselines while preserving augmentation benefits for the median.
-
MixINN: Accelerating Plant Breeding by Combining Mixed Models and Deep Learning for Interaction Prediction
MixINN combines mixed models and deep learning to predict genotype-environment interactions in corn trials, yielding 5.8-7.2% higher average yields when selecting top-performing genotypes compared to standard methods.
-
Deep Wave Network for Modeling Multi-Scale Physical Dynamics
DW-Net improves the accuracy versus computational cost Pareto front over standard U-Nets for 2D and 3D multi-scale flow benchmarks by stacking multiple waves while keeping training settings identical.
-
A Multimodal Vision Transformer-based Modeling Framework for Prediction of Fluid Flows in Energy Systems
A multimodal SwinV2-UNet vision transformer conditioned on data modality and time predicts spatiotemporal fluid flows and reconstructs unobserved fields from limited views using CFD data of argon jet injection.
-
Generalized Spherical Neural Operators: Green's Function Formulation
GSNO uses position-dependent spherical Green's functions to create flexible neural operators that adapt to non-equivariant systems on spheres while keeping spectral efficiency and grid invariance.
-
GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations
GAIR introduces a geo-aligned implicit representation module inside a multi-encoder contrastive SSL framework that produces location-aware embeddings and outperforms prior geo-foundation models on 22 geospatial datasets across 9 tasks.
-
Learning Climate Variability from Scarce Data with Diffusion Models: A Test Case for ENSO
Diffusion models recover known ENSO variability structure from synthetic LIM data when given enough samples, but require pre-training on CMIP6 plus fine-tuning to match observations with the ~700 samples available in ERSSTv5.
-
Otter Weather: Skillful and Computationally Efficient Medium-Range Weather Forecasting
Otter Weather is a spatiotemporal model that outperforms NWP baselines by 9.6% at 24h lead with under 3.5 A100-days training and extends efficiency gains to probabilistic forecasting via CRPS.
-
Emerging Flexible Designs for Geospatial Multimodal Foundation Models
Standardized pretraining and evaluation of geospatial multimodal foundation models on GEOBench reveals design trade-offs in flexibility, modality alignment, and task performance.
-
Probabilistic Precipitation Nowcasting with Rectified Flow Transformers
FREUD applies rectified flow transformers with frame-wise encoding and a unified decoder to achieve state-of-the-art probabilistic precipitation nowcasting on the SEVIR benchmark.
-
Hybrid Quantum-Classical Corrective Diffusion Modeling for Meteorological Downscaling
Hybrid quantum-classical corrective diffusion model improves MAE and CRPS on 2020 validation wind data but exhibits a generalization gap on 2021 out-of-distribution tests.
-
PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting
PINN-Cast combines continuous-depth Neural ODEs inside transformer blocks with a two-branch attention module and physics-informed loss to produce short-term weather forecasts that respect governing physical principles.
-
Towards Scaling Law Analysis For Spatiotemporal Weather Data
Scaling laws for weather models exhibit strong cross-channel and cross-horizon heterogeneity, where globally pooled metrics appear favorable while many individual channels degrade at longer leads.
-
LangRetrieval: Language-Guided Self-Evolving Satellite-to-Radar Retrieval via CSI-Driven Reward
LangRetrieval is a conditional flow matching framework with semantic warm-up and GRPO-based self-evolving optimization using CSI rewards to improve satellite-to-radar precipitation retrieval.
-
AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery
A survey organizing AI-powered research automation into five workflow stages, defining AutoResearch and Vibe Research, and proposing five evaluation dimensions while noting domain-conditioned limits on autonomy.
-
Scalable and Trustworthy Earth Observation Foundation Models
Remote-sensing foundation models need domain-specific design and evaluation around measurement physics and decision constraints; benchmark accuracy alone is insufficient for trustworthy EO deployment.
-
Towards a Foundation Model for the Martian Atmosphere
The paper reviews data sources, physical models, downstream applications, and AI techniques to outline considerations for building a foundation model for the Martian atmosphere.
-
Earth Science Foundation Models: From Perception to Reasoning and Discovery
A review of Earth science foundation models covering capability evolution from perception to discovery, applications across atmosphere/hydrosphere/lithosphere/biosphere/anthroposphere/cryosphere, over 200 datasets, and key challenges.
-
Toward Artificial Intelligence Enabled Earth System Coupling
AI methods can strengthen cross-domain interactions and support more coherent multi-component representations in Earth system models.
-
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning
Position paper claims multimodal LLMs can significantly advance scientific reasoning and proposes a four-stage roadmap plus challenges and suggestions.
- A Self-Supervised Framework for Space Object Behaviour Characterisation