Pith. sign in

REVIEW 5 major objections 5 minor 42 references

Paraformer: Parameterization of Sub-grid Scale Processes Using Transformers

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Paraformer, an encoder-only Transformer with a five-step temporal context window, claims to outperform classical deep-learning baselines on the ClimSim climate parameterization benchmark.

desk verdict A legitimate but narrow empirical result: Paraformer's 5-step context Transformer beats a single-step MLP on ClimSim, yet the paper's attention claim is confounded by the context window and the imported baselines. read the letter →

arxiv 2412.16763 v1 pith:CMBCMY4Q submitted 2024-12-21 cs.LG physics.ao-ph

Shuochen Wang , Nishant Yadav , Auroop R. Ganguly This is my paper · ORCID
classification cs.LGphysics.ao-ph
keywords sub-gridscaleparameterizationclimateemulationTransformerattentionmechanismClimSimdatasetdeeplearningglobalmodelsmemory-awarecontextwindow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Paraformer, an encoder-only Transformer for emulating sub-grid scale physics—clouds, radiation, precipitation—that global climate models cannot resolve explicitly. Trained on the low-resolution, real-geography ClimSim dataset, the model sequences grid-scale inputs into non-overlapping five-step windows (about 12 hours of memory) and predicts sub-grid tendencies. The authors report that Paraformer achieves lower mean absolute error than all six classical deep-learning baselines on every tested quantity in the smaller variable set, and lower error than a reimplemented MLP on every variable in the larger set. If correct, this introduces attention-based temporal memory into a field that has relied on MLPs, CNNs, and similar architectures, and suggests that roughly half a day of prior state carries useful information for parameterization.

What carries the argument

The load-bearing object is an encoder-only Transformer followed by a fully-connected layer, with a context window of five non-overlapping time steps acting as the model's memory. At the 140-minute subsampled resolution, five steps equal roughly 12 hours of simulated climate, and self-attention assigns importance scores across those steps so earlier grid-scale states can influence the current sub-grid prediction. The paper calls the design memory-aware because the window size controls how much history the model holds, and reports that non-overlapping windows matched sliding windows in accuracy while avoiding data duplication. The best configuration found by grid search uses six encoder layers, four attention heads, and an embedding dimension of 256 (v1) or 512 (v2).

What would settle it

Retrain the CNN, encoder-decoder, HSR, RPN, and cVAE baselines under the paper's exact 140-minute subsampling and train/validation/test splits; if any of them matches or beats Paraformer's v1 MAE (for example, a CNN with dT/dt MAE below 2.332 W/m²), the claimed general superiority over classical deep-learning architectures would not hold.

Watch

Extended reading notes

Core claim

The paper's central claim is that attention over time, not just current-state mapping, is what improves sub-grid parameterization. Paraformer feeds an encoder-only Transformer five consecutive 140-minute grid-scale states and lets self-attention weight those states when predicting the current sub-grid response. In the v1 experiment this configuration achieved the lowest mean absolute error among the seven compared models on all ten target quantities and the best reported R² on all but one, with the exception coming from the precipitation variable PRECC; in v2 it beat the reimplemented MLP on every variable. The authors read the pattern of gains—concentrated in the lower atmosphere and on uncommon far-from-zero values—as evidence that the attention mechanism captures inter-state and intra-state temporal dependencies that instantaneous baselines miss.

Load-bearing premise

The broad claim that Paraformer beats classical deep-learning architectures assumes that the published baseline numbers from the original ClimSim study transfer to this paper's different temporal subsampling, training split, and custom test set; only the MLP baseline was retrained under the identical protocol.

Editorial extensions

If this is right

  • If Paraformer is right, an attention-based emulator can replace or supplement MLP parameterizations inside a GCM by consuming the last roughly 12 hours of grid-scale state rather than a single snapshot.
  • Five non-overlapping steps being optimal suggests that the sub-grid parameterization problem has exploitable short-term memory, and that longer context windows add cost without accuracy gains in these experiments.
  • The v2 results show that adding physically relevant variables improves both models but shrinks Paraformer's relative advantage, pointing to diminishing returns from input expansion alone.
  • The smoother R² profiles across atmospheric levels imply attention is more consistent than the MLP at representing vertical structure for temperature and moisture tendencies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the authors leave implicit is that the relevant temporal memory for column-wise parameterization may be short (about 12 hours); this could be tested by comparing regimes with slow radiative or stratiform processes, where longer context might matter.
  • A direct extension of the reported scatter plots is to evaluate extreme-event metrics explicitly, since the biggest visible gains are on far-from-zero outliers that MAE and R² underweight.
  • Because only the MLP was reimplemented under the new 140-minute and 7-year protocol, the general superiority claim over the other listed architectures becomes fully established only if those baselines are retrained on identical splits; until then the controlled result is Paraformer versus MLP.
  • The spatial dimension is deliberately ignored; extending attention to neighboring columns through graph- or mesh-based tokenization is a natural next step that the paper flags as future work rather than tests.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes Paraformer, an encoder-only Transformer with a five-time-step context window, for learning sub-grid scale parameterization from the ClimSim dataset. The authors evaluate on two variable sets (v1 and v2) and report that Paraformer achieves lower MAE and generally higher R2 than several baseline models. However, the comparison to most baselines relies on numbers taken from the original ClimSim paper [23] rather than models retrained under the same data protocol, and the only reimplemented baseline (MLP) does not have access to the same temporal context window. The paper attributes the improvement to the attention mechanism, but no ablation isolates the effect of the context window from the architecture.

Significance. If the central claim were substantiated, the paper would make a useful contribution by introducing attention to full-physics ClimSim and showing that temporal memory can improve parameterization. The extensive visualizations of vertical and spatial error patterns are informative. However, the current evidence does not support the headline claim of outperforming classical deep-learning architectures, because the comparison protocol is inconsistent and the mechanism attribution is confounded. The paper also provides no code or reproducibility package, so the empirical claims cannot be independently verified from the manuscript alone.

major comments (5)
  1. [Section 2.1 and Table 2] The comparison to CNN, ED, HSR, RPN, and cVAE uses baseline numbers taken directly from [23], while the present paper uses a different data protocol: training on 7 years of data subsampled to 140-minute steps, a validation year, and a v1 test set at 120-minute resolution. The original baselines were trained on the original ClimSim splits. Without retraining these baselines under the same protocol, the claim that Paraformer 'outperforms classical deep-learning architectures' is not established; the only valid comparison is the reimplemented MLP.
  2. [Section 2.2 and Table 2] The reported improvements cannot be attributed to the attention mechanism because the MLP baseline is reimplemented based on [23] with instantaneous grid-scale input, while Paraformer uses a five-time-step context window (about 12 hours). No experiment supplies the same five-step context to a non-attention model or evaluates Paraformer with a context length of 1. Therefore the MAE/R2 advantages may result solely from the additional temporal input, not from the attention mechanism.
  3. [Table A3 and Section 3] The statement 'Paraformer demonstrates lower errors on all variables in v2 compared to the MLP baseline' is contradicted by the RMSE values for du/dt and dv/dt: Paraformer reports 9.84E-05 vs. 9.83E-05 for du/dt and 9.47E-05 vs. 9.42E-05 for dv/dt. The claim should be restricted to MAE or the RMSE discrepancy should be acknowledged.
  4. [Table 2 and Section 3] The phrase 'outperforms all other structures on all but one variable' is misleading for PRECC, where Paraformer's R2 is -1.764 while the CNN baseline reports 0.077. If the claim is about MAE, it should say so explicitly; if it is about both metrics, it is inaccurate for R2.
  5. [Section 2.2] The choice of window size 5 is an empirical hyperparameter, but no ablation results are shown to justify it, and the paper does not provide a physical rationale. This is acceptable as hyperparameter tuning, but it increases the need for a context-matched baseline or an ablation that varies the window for both Paraformer and a non-attention model to separate the effect of memory from that of attention.
minor comments (5)
  1. [Abstract] The phrase 'attenuation mechanism' should be 'attention mechanism'.
  2. [Figure 1 caption] The variables B, B_new, num_seq, and seq_len are not defined clearly; please provide explicit definitions in the caption.
  3. [Section 2.1] Clarify whether the stated dataset size (10,099,200 data points) refers to the raw data or the subsampled data used here.
  4. [Table 2 caption] The caption states that large negative R2 values are not shown for dq/dt and PRECSC, but PRECC also contains negative R2 values; please clarify the criterion for omitting versus showing negative R2.
  5. [Section 3] The hyperparameters for the MLP baseline are not reported; please include them so the reader can judge the fairness of the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the paper is an empirical benchmark with held-out test evaluation, and the main concerns are experimental-control issues, not circular reasoning.

full rationale

This is an empirical benchmark paper, not a derivation. The central claim is grounded in held-out test-set evaluation (v1 and v2 test splits) after a standard training/validation hyperparameter search; no parameter is fitted to the test targets and then renamed as a prediction. The context window size was selected by validation performance, which is a model-selection step rather than a circular definition of the result; the paper does not fit the window and then 'predict' the window's importance on the same data. The imported baseline numbers from ClimSim [23] and the absence of a context-matched non-attention baseline raise external-validity and attribution concerns, but these are experimental-control issues rather than circular reductions: the paper's equations and conclusions do not use the target result as an input. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation; the only external benchmark [23] is not authored by the present authors. Honest non-finding.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper does not introduce new physical entities or derive new equations; its free parameters are hyperparameters and data-processing choices selected by grid search or hand. The central claim rests on standard Transformer machinery plus the domain assumption that ClimSim data and the chosen splits adequately represent the parameterization problem.

free parameters (4)
  • Context window size = 5
    Chosen by testing window sizes 5 to 40; non-overlapping windows with size 5 gave the best results (Section 2.2). This parameter is fit to validation performance and affects the temporal memory framing.
  • Transformer hyperparameters (layers, embedding dim, heads, batch size, optimizer, learning rate) = 6 layers, 256/512 embedding, 4 heads, batch 512, AdamW, lr 1e-4
    Selected by grid search per variable set (Section 2.3); these are fitted to validation performance and could inflate performance if the search budget differs from the baselines.
  • MLP hyperparameters = Unspecified; based on [23] but differing between v1 and v2
    The paper states MLP hyperparameters differ between v1 and v2 reflecting independent search processes, but exact values are not reported, making it hard to assess tuning equality.
  • Temporal subsampling factor = 20 minutes to 140 minutes
    The original 20-minute data are subsampled to 140 minutes to address memory limits; this is a data-processing choice that changes the temporal correlations seen by the model and is not justified beyond computational constraints (Section 2.1).
assumptions (4)
  • standard math Standard Transformer attention and encoder architecture as defined in Vaswani et al. [33] are taken as given.
    The paper builds on the attention mechanism and encoder structure without deriving them.
  • domain assumption The ClimSim low-resolution real-geography dataset is a valid proxy for the sub-grid scale parameterization problem in E3SM-MMF, and the 7/1/2 year temporal split is an appropriate evaluation protocol.
    Used as the training and evaluation bed without independent validation of its representativeness (Sections 2.1 and 3).
  • domain assumption The v1 test set, which uses 120-minute temporal resolution over the same period as the validation set, is a meaningful generalization test for a model trained on 140-minute data.
    The test protocol is inherited from [23] and used to claim performance under a different model timestep (Section 2.1).
  • ad hoc to paper A context window of five time steps (about 12 hours) is sufficient temporal memory for the parameterization task; this window size was chosen empirically, not from physical reasoning.
    Window size 5 was selected because it gave the best validation performance among sizes 5 to 40 (Section 2.2), so the 'memory-aware' framing is tied to a fitted choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Paraformer: Parameterization of Sub-grid Scale Processes Using Transformers." pith.science (2026). https://pith.science/paper/CMBCMY4Q

@misc{pith2026241216763,
  author       = {Pith},
  title        = {Pith review of: Paraformer: Parameterization of Sub-grid Scale Processes Using Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CMBCMY4Q}},
  note         = {Machine review of arXiv:2412.16763}
}
read the original abstract

One of the major sources of uncertainty in the current generation of Global Climate Models (GCMs) is the representation of sub-grid scale physical processes. Over the years, a series of deep-learning-based parameterization schemes have been developed and tested on both idealized and real-geography GCMs. However, datasets on which previous deep-learning models were trained either contain limited variables or have low spatial-temporal coverage, which can not fully simulate the parameterization process. Additionally, these schemes rely on classical architectures while the latest attention mechanism used in Transformer models remains unexplored in this field. In this paper, we propose Paraformer, a "memory-aware" Transformer-based model on ClimSim, the largest dataset ever created for climate parameterization. Our results demonstrate that the proposed model successfully captures the complex non-linear dependencies in the sub-grid scale variables and outperforms classical deep-learning architectures. This work highlights the applicability of the attenuation mechanism in this field and provides valuable insights for developing future deep-learning-based climate parameterization schemes.

Figures

Figures reproduced from arXiv: 2412.16763 by the authors.

Figure 1
Figure 1. The architecture of Paraformer and the data processing workflow. The shape of data [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. MAE, RMSE and R2 of dT /dt and dq/dt using MLP and Paraformer on variable set v1. Each index on the x-axis represents a vertical level in the atmosphere starting from the top (i.e. level index 0 represents the top of the atmosphere). Units of non-energy flux variables are converted to a common energy unit, W/m2 [23]. Negative R2 values are not shown [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. R2 of daily-mean, zonal-mean dT /dt and dq/dt for MLP and Paraformer at different pressure levels in variable set v1. Yellow contours cover regions of > 0.9R2 , orange contours cover regions of > 0.7R2 . 5 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Spatial distribution of R2 using MLP (left column) and Paraformer (right column) of 8 scalar target variables in v1. The names of the variables are labeled on the left. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 29 canonical work pages

  1. [23]

    Climsim: A large multi-scale dataset for hybrid physics-ml climate emulation

    Sungduk Yu, Walter Hannah, Liran Peng, Jerry Lin, Mohamed Aziz Bhouri, Ritwik Gupta, Björn Lütjens, Justus C Will, Gunnar Behrens, Julius Busecke, et al. Climsim: A large multi-scale dataset for hybrid physics-ml climate emulation. Advances in Neural Information Processing Systems, 36, 2024

    2024

  2. [1]

    Carbon dioxide and climate

    CA Senior and JFB Mitchell. Carbon dioxide and climate. The impact of cloud parameterization. Journal of Climate, 6(3):393–418, 1993

    work page 1993

  3. [2]

    A GCM parameterization for the shortwave radiative properties of water clouds

    A Slingo. A GCM parameterization for the shortwave radiative properties of water clouds. Journal of Atmospheric Sciences, 46(10):1419–1427, 1989

    work page 1989

  4. [3]

    Climate goals and computing the future of clouds

    Tapio Schneider, João Teixeira, Christopher S Bretherton, Florent Brient, Kyle G Pressel, Christoph Schär, and A Pier Siebesma. Climate goals and computing the future of clouds. Nature Climate Change, 7(1):3–5, 2017

    2017

  5. [4]

    Causes of higher climate sensitivity in CMIP6 models

    Mark D Zelinka, Timothy A Myers, Daniel T McCoy, Stephen Po-Chedley, Peter M Caldwell, Paulo Ceppi, Stephen A Klein, and Karl E Taylor. Causes of higher climate sensitivity in CMIP6 models. Geophysical Research Letters, 47(1):e2019GL085782, 2020

    work page 2020

  6. [5]

    An evaluation of proposed representations of subgrid hydrologic processes in climate models

    G Thomas and A Henderson-Sellers. An evaluation of proposed representations of subgrid hydrologic processes in climate models. Journal of climate, 4(9):898–910, 1991

    work page 1991

  7. [6]

    Improving a subgrid runoff parameterization scheme for climate models by the use of high resolution data derived from satellite observations.Climate Dynamics, 21:349–359, 2003

    Stefan Hagemann and L Dümenil Gates. Improving a subgrid runoff parameterization scheme for climate models by the use of high resolution data derived from satellite observations.Climate Dynamics, 21:349–359, 2003

    work page 2003

  8. [7]

    Subgrid-scale physical parameterization in atmospheric modeling: How can we make it consistent? Journal of Physics A: Mathematical and Theoretical, 49(28):284001, 2016

    Jun-Ichi Yano. Subgrid-scale physical parameterization in atmospheric modeling: How can we make it consistent? Journal of Physics A: Mathematical and Theoretical, 49(28):284001, 2016

    work page 2016

Show all 42 references
  1. [8]

    Robustness of neural network emulations of radiative transfer parameterizations in a state-of-the-art general circulation model

    Alexei Belochitski and Vladimir Krasnopolsky. Robustness of neural network emulations of radiative transfer parameterizations in a state-of-the-art general circulation model. Geoscientific Model Development, 14(12):7425–7437, 2021

    2021

  2. [9]

    A physics-incorporated deep learning framework for parameterization of atmospheric radiative transfer

    Yichen Yao, Xiaohui Zhong, Yongjun Zheng, and Zhibin Wang. A physics-incorporated deep learning framework for parameterization of atmospheric radiative transfer. Journal of Advances in Modeling Earth Systems, 15(5):e2022MS003445, 2023

    2023

  3. [10]

    Validation of a high-resolution regional climate model for the Alpine region and effects of a subgrid-scale topography and land use representation

    Eun Soon Im, E Coppola, F Giorgi, and X Bi. Validation of a high-resolution regional climate model for the Alpine region and effects of a subgrid-scale topography and land use representation. Journal of Climate, 23(7):1854–1873, 2010

    2010

  4. [11]

    Ensemble data assimilation with the NCEP global forecast system

    Jeffrey S Whitaker, Thomas M Hamill, Xue Wei, Yucheng Song, and Zoltan Toth. Ensemble data assimilation with the NCEP global forecast system. Monthly Weather Review, 136(2): 463–482, 2008

    2008

  5. [12]

    A generalized approach to parameterizing convection combining ensemble and data assimilation techniques

    Georg A Grell and Dezs ˝o Dévényi. A generalized approach to parameterizing convection combining ensemble and data assimilation techniques. Geophysical Research Letters, 29(14): 38–1, 2002

    2002

  6. [13]

    A prognostic cloud water parameterization for global climate models

    Anthony D Del Genio, Mao-Sung Yao, William Kovari, and Kenneth KW Lo. A prognostic cloud water parameterization for global climate models. Journal of Climate, 9(2):270–304, 1996

    1996

  7. [14]

    An improved strategy for the evaluation of cloud parameterizations in GCMs

    Christian Jakob. An improved strategy for the evaluation of cloud parameterizations in GCMs. Bulletin of the American Meteorological Society, 84(10):1387–1402, 2003

    2003

  8. [15]

    Ensemble data assimilation in the whole atmosphere community climate model

    NM Pedatella, K Raeder, JL Anderson, and H-L Liu. Ensemble data assimilation in the whole atmosphere community climate model. Journal of Geophysical Research: Atmospheres, 119 (16):9793–9809, 2014

    2014

  9. [16]

    Bias and data assimilation

    Dick P Dee. Bias and data assimilation. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 131 (613):3323–3343, 2005

    2005

  10. [17]

    A machine learning augmented data assimilation method for high-resolution observations

    Lucas J Howard, Aneesh Subramanian, and Ibrahim Hoteit. A machine learning augmented data assimilation method for high-resolution observations. Journal of Advances in Modeling Earth Systems, 16(1):e2023MS003774, 2024. 9

    2024

  11. [18]

    Deep learning to represent subgrid processes in climate models

    Stephan Rasp, Michael S Pritchard, and Pierre Gentine. Deep learning to represent subgrid processes in climate models. Proceedings of the national academy of sciences, 115(39):9684– 9689, 2018

    2018

  12. [19]

    Could machine learning break the convection parameterization deadlock? Geophysical Research Letters, 45(11):5742–5751, 2018

    Pierre Gentine, Mike Pritchard, Stephan Rasp, Gael Reinaudi, and Galen Yacalis. Could machine learning break the convection parameterization deadlock? Geophysical Research Letters, 45(11):5742–5751, 2018

    2018

  13. [20]

    Effects of stochastic parametrizations in the Lorenz’96 system

    Daniel S Wilks. Effects of stochastic parametrizations in the Lorenz’96 system. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 131(606):389–407, 2005

    2005

  14. [21]

    Machine learning for stochastic parameterization: Generative adversarial networks in the Lorenz’96 model

    David John Gagne, Hannah M Christensen, Aneesh C Subramanian, and Adam H Monahan. Machine learning for stochastic parameterization: Generative adversarial networks in the Lorenz’96 model. Journal of Advances in Modeling Earth Systems, 12(3):e2019MS001896, 2020

    2020

  15. [22]

    Stochastic parametrizations and model uncertainty in the lorenz’96 system

    HM Arnold, IM Moroz, and TN Palmer. Stochastic parametrizations and model uncertainty in the lorenz’96 system. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1991):20110479, 2013

    1991

  16. [24]

    Using machine learning to parameterize moist convection: Potential for modeling of climate, climate change, and extreme events

    Paul A O’Gorman and John G Dwyer. Using machine learning to parameterize moist convection: Potential for modeling of climate, climate change, and extreme events. Journal of Advances in Modeling Earth Systems, 10(10):2548–2563, 2018

    2018

  17. [25]

    Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions

    Janni Yuval and Paul A O’Gorman. Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions. Nature communications, 11(1):3295, 2020

    2020

  18. [26]

    Stable machine-learning parameter- ization of subgrid processes with real geography and full-physics emulation

    Zeyuan Hu, Akshay Subramaniam, Zhiming Kuang, Jerry Lin, Sungduk Yu, Walter M Hannah, Noah D Brenowitz, Josh Romero, and Michael S Pritchard. Stable machine-learning parameter- ization of subgrid processes with real geography and full-physics emulation. arXiv preprint arXiv:24...

    2024 arXiv

  19. [27]

    Applications of deep learning to ocean data inference and subgrid parameterization

    Thomas Bolton and Laure Zanna. Applications of deep learning to ocean data inference and subgrid parameterization. Journal of Advances in Modeling Earth Systems , 11(1):376–399, 2019

    2019

  20. [28]

    A data-driven ap- proach to precipitation parameterizations using convolutional encoder-decoder neural networks

    Pablo Rozas Larraondo, Luigi J Renzullo, Inaki Inza, and Jose A Lozano. A data-driven ap- proach to precipitation parameterizations using convolutional encoder-decoder neural networks. arXiv preprint arXiv:1903.10274, 2019

    1903 arXiv

  21. [29]

    Improved weather forecasting using neural network emulation for radiation parameterization

    Hwan-Jin Song and Soonyoung Roh. Improved weather forecasting using neural network emulation for radiation parameterization. Journal of Advances in Modeling Earth Systems, 13 (10):e2021MS002609, 2021

    2021

  22. [30]

    Vladimir M Krasnopolsky, Michael S Fox-Rabinovitz, and Alexei A Belochitski. Using ensemble of neural networks to learn stochastic convection parameterizations for climate and numerical weather prediction models from data simulated by a cloud resolving model.Advances in Artifi...

    2013

  23. [31]

    Stochastic parameterization of column physics using generative adversarial networks

    Balasubramanya T Nadiga, Xiaoming Sun, and Cody Nash. Stochastic parameterization of column physics using generative adversarial networks. Environmental Data Science, 1:e22, 2022

    2022

  24. [32]

    Generative data-driven ap- proaches for stochastic subgrid parameterizations in an idealized ocean model

    Pavel Perezhogin, Laure Zanna, and Carlos Fernandez-Granda. Generative data-driven ap- proaches for stochastic subgrid parameterizations in an idealized ocean model. Journal of Advances in Modeling Earth Systems, 15(10):e2023MS003681, 2023. 10

    2023

  25. [33]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  26. [34]

    Learning deep transformer models for machine translation

    Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F Wong, and Lidia S Chao. Learning deep transformer models for machine translation. arXiv preprint arXiv:1906.01787, 2019

    1906 arXiv

  27. [35]

    Pre-trained language models for text generation: A survey

    Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Pre-trained language models for text generation: A survey. ACM Computing Surveys, 56(9):1–39, 2024

    2024

  28. [36]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021

    2021

  29. [37]

    A transformer-based framework for multivariate time series representation learning

    George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2114...

    2021

  30. [38]

    Transformers in time series: A survey

    Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125, 2022

    2022 arXiv

  31. [39]

    Efficient attention: Attention with linear complexities

    Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Efficient attention: Attention with linear complexities. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3531–3539, 2021

    2021

  32. [40]

    Reformer: The efficient transformer

    Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451, 2020

    2001 arXiv

  33. [41]

    Earthformer: Exploring space-time transformers for earth system forecasting

    Zhihan Gao, Xingjian Shi, Hao Wang, Yi Zhu, Yuyang Bernie Wang, Mu Li, and Dit-Yan Yeung. Earthformer: Exploring space-time transformers for earth system forecasting. Advances in Neural Information Processing Systems, 35:25390–25403, 2022

    2022

  34. [42]

    Physics-informed machine learning: case studies for weather and climate modelling

    Karthik Kashinath, M Mustafa, Adrian Albert, JL Wu, C Jiang, Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, R Wang, Ashesh Chattopadhyay, A Singh, et al. Physics-informed machine learning: case studies for weather and climate modelling. Philosophical Transactions of the Royal So...

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.