REVIEW 3 major objections 5 minor 16 references
Self-attentive Transformer for Fast and Accurate Postprocessing of Temperature and Wind Speed Forecasts
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A self-attentive Transformer corrects temperature and wind-speed ensemble forecasts for twenty lead times simultaneously, cutting the continuous ranked probability score by up to 16.5% while training roughly six times faster than a…
desk verdict Useful gridded wind benchmark entry with joint-lead-time Transformer; solid relative comparison, but absolute wind skill rests on ERA5 as ground truth. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the self-attentive ensemble Transformer, an adaptation of Finn's ensemble Transformer. The input is a tensor $Z \in \mathbb{R}^{b\times k\times t\times h\times w\times c}$ of $k$ ensemble members, $t$ lead times, a spatial grid of size $h\times w$, and $c$ predictors; a linear layer projects the $c$ predictors to $\tilde{c}$ features, then $n=4$ transformer blocks apply multi-headed self-attention with 8 heads, computing the softmax across the ensemble-member axis while the query, key, and value weights are shared across space, time, and features. This sharing is what lets information propagate implicitly across grid points and lead times, analogous to an ensemble Kalman filter, so one model can postprocess all 20 lead times at once. The output is projected back to a single variable, and training minimizes the CRPS, analytically for temperature and via a regularized kernel CRPS for wind speed.
What would settle it
Compare the Transformer's wind-speed corrections against independent offshore and coastal observations such as buoys, masts, or lidar instead of ERA5; if the CRPS advantage over raw forecasts shrinks or reverses at coastal grid points where ERA5 is known to underestimate strong winds, part of the claimed wind-speed gain is an artifact of the reference field.
Extended reading notes
Core claim
The central claim is that a self-attentive ensemble Transformer, using multi-headed self-attention across ensemble members with weights shared over spatial, temporal, and predictor dimensions, can correct all twenty lead times of gridded ECMWF ensemble forecasts simultaneously and more accurately than a classical member-by-member method. The authors report CRPS reductions over raw forecasts of 16.5% for two-meter temperature, 10% for ten-meter wind speed, and 9% for one-hundred-meter wind speed, with the Transformer ahead of classical MBM at every lead time and over nearly the whole EUPPBench domain. They further report that training takes about one-sixth of the time for temperature and one-fifth for wind speed, and that the Transformer attains spread-error ratios close to one, indicating markedly better calibrated ensembles than the raw forecasts. The paper presents this as the first postprocessing of gridded ten- and one-hundred-meter wind speed in EUPPBench.
Load-bearing premise
The evaluation treats the ERA5 reanalysis, a computer-generated weather reconstruction, as the true observation for all three variables; because ERA5 is known to underestimate strong winds near coasts and offshore, part of the measured wind-speed improvement could be correction toward a biased reference rather than genuine forecast skill.
Editorial extensions
If this is right
- A single Transformer can replace per-lead-time postprocessing models for the three target variables, cutting training time by roughly a factor of five to six while improving CRPS at every lead time.
- The method supplies the first gridded wind-speed baselines in EUPPBench, giving future postprocessing studies a direct comparison point for 10 m and 100 m wind speed.
- Adding predictors is cheap: increasing from three to ten predictors raises training time by about 7%, so operational systems can include richer input sets without retraining per lead time.
- The Transformer's spread-error ratios near one and more uniform rank histograms imply a substantially better calibrated ensemble than raw forecasts, which matters for probabilistic decision-making.
Reading between the lines
- An untested next step is whether the same shared-attention design transfers to non-smooth variables such as precipitation or gusts; the paper's evidence covers only temperature and wind speed.
- The attention maps could be used diagnostically to locate where operational corrections are most needed, a use the paper sketches but does not develop into a method.
- Because ERA5 is also the training target, the reported wind-speed gains should be re-evaluated against in-situ coastal observations before being used to justify offshore renewable-energy applications.
- The speed comparison is hardware-dependent (GPU versus CPU), so an end-to-end operational test at national-forecasting scale is needed to confirm the wall-clock advantage.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a self-attentive Transformer for gridded postprocessing of ensemble weather forecasts, applied to two-meter temperature and ten- and one-hundred-meter wind speed over the EUPPBench domain. Unlike many previous postprocessing models, the Transformer processes 20 lead times simultaneously and exchanges information across ensemble members, variables, spatial locations, and lead times via multi-headed self-attention. The model is trained on EUPPBench forecasts with ERA5 as the reference and is compared against a classical member-by-member (MBM) baseline. The authors report CRPS improvements over the raw ensemble of 16.5% for temperature, 10% for ten-meter wind speed, and 9% for one-hundred-meter wind speed, with the Transformer outperforming classical MBM on average CRPS and spread-error ratio (SER) while requiring substantially less training time.
Significance. If the reported results hold, the paper makes a useful practical contribution: a single postprocessing model covering multiple variables and all lead times, with better average CRPS than raw forecasts and a classical MBM baseline, and with substantially lower training cost. The use of a held-out test year, the gridded multi-lead-time setting, and the first application to one-hundred-meter wind speed in EUPPBench are concrete advances. The evaluation is generally coherent, and the authors are transparent about several limitations, including ERA5's known weaknesses for coastal and offshore winds. The main risks are the lack of uncertainty quantification for the headline scores, the ambiguous selection procedure for the MBM baseline variant, and the dependence of the wind-speed claims on ERA5 as ground truth in exactly the coastal regimes where ERA5 is least reliable.
major comments (3)
- [Section 3, Table 3] All headline results are point estimates over a single test year, with no confidence intervals or significance tests. The Transformer-versus-MBM differences for wind speed are small (0.467 vs. 0.477 m/s for w10 and 0.732 vs. 0.747 m/s for w100), and the verification scores are strongly spatially and temporally correlated, so it is unclear whether these differences are sampling noise. Please add block-bootstrap confidence intervals on CRPS/SER differences or paired significance tests, and restrict the word 'significantly' to statistically supported claims.
- [Section 2.4] The sentence 'For each target variable, we report results for the most competitive MBM variant' does not specify how the variant was selected. If the choice between the single-predictor and multi-predictor versions of classical MBM was made using the 2017 test data, the benchmark is optimistically biased and the comparison is not a fair baseline. Please state explicitly that variant selection used the 2016 validation split, or report both variants on the test set.
- [Section 4 and Figures A.2-A.3] The wind-speed evaluation is anchored entirely to ERA5 as ground truth, and the paper itself cites evidence that ERA5 underestimates strong winds offshore and struggles with coastal variability (Gandoin and Garza, 2024; Alkhalidi et al., 2025). The Transformer's negative bias in the northwest Netherlands and its increasingly negative bias with lead time appear in the regime where ERA5 is least reliable. Part of the measured CRPS gain for wind speed could therefore be convergence toward a biased reference rather than genuine forecast skill. Please add a robustness check against independent observations (e.g., coastal stations for w10 and tall mast or lidar data for w100, where available), or explicitly restrict the wind-speed claims to skill relative to ERA5 and soften the offshore renewable-energy interpretation.
minor comments (5)
- [Section 2.4 and Appendix A] The description of the attention-map computation is inconsistent: Section 2.4 mentions averaging the element-wise product of the key and value matrix, while Appendix A and the surrounding text define the map via Q ⊙ K. Please harmonize the wording.
- [Eq. (2)] The scaling factor sc is defined just before the attention equation but does not appear in Eq. (2); either include it in the softmax argument or remove the definition.
- [Eq. (4)] In the definition of ϵm = V^m − \bar{V}, the symbol \bar{V} is not defined; presumably it is the ensemble mean of the target variable, while \bar{V}_i denotes the ensemble mean of predictor i. Please clarify the notation.
- [Table 1] The units for geopotential height at 500 hPa are given as 'm 2s−2'; this should be m² s⁻² or geopotential metres, and the entry should be made consistent.
- [Section 2.4 and Abstract] The 'up to six times faster' claim is based on indicative wall-clock times on different hardware (GPU for the Transformer, CPU for MBM), as acknowledged in Section 2.4; please carry this caveat into the abstract or phrase the claim as 'up to six times faster in our setup.'
Circularity Check
No significant circularity: the claimed CRPS improvements are empirical results computed on a held-out test year; the self-citations concern architecture and interpretability, not the evidence for the result.
full rationale
Walking the derivation chain, the central claims are the CRPS improvements in Table 3 (16.5% for t2m, 10% for w10, 9% for w100) over raw forecasts and classical MBM. These are obtained by training on 1997-2015, validating on 2016, and scoring on 2017 (Section 2.3), so the reported numbers are out-of-sample evaluations rather than quantities defined by construction. The scoring rules (Eq. 6; Eqs. A.2-A.5) are standard and do not contain the model's fitted parameters; the fair ensemble CRPS used for wind verification is independent of the kernel-CRPS training loss, which weakens any concern that training and evaluation share a forced objective. The reported kernel-CRPS constants (lambda=0.0275/0.05, k=2.7/2.0) are hyperparameters of the loss, not fitted outputs renamed as predictions. The architecture is adapted from Finn (2021), with code also from Finn (2021) and Ashkboos et al. (2022), and the attention-interpretation claim cites Finn (2023); these are implementation and interpretability aids and are not what establishes the empirical skill numbers. The ERA5 ground-truth limitation in Section 4 - 'some studies report that it underestimates strong wind events offshore and struggles to capture variability at coastal regions' - is a genuine external-validity caveat for the absolute wind-speed gains, but it is acknowledged, and classical MBM is trained and scored on the same ERA5 target, so the head-to-head comparison is not a self-referential reduction. No equation in the paper defines a prediction in terms of the claimed improvement, and no load-bearing argument rests on a self-citation chain. Therefore no circular step is present.
Assumptions & free parameters
free parameters (5)
- Kernel CRPS spread penalty lambda =
0.0275 (w10), 0.05 (w100)
- Kernel CRPS spread penalty k =
2.7 (w10), 2.0 (w100)
- Learning rate =
0.001
- Batch size =
2
- Transformer depth and width =
n=4 blocks, h_n=8 heads, m_n=4 MLP factor
assumptions (5)
- domain assumption ERA5 reanalysis is an adequate ground truth for 2m temperature and 10m/100m wind speed in the EUPPBench domain.
- domain assumption A normal predictive distribution is adequate for 2m temperature.
- domain assumption The chronological split prevents leakage and the 2017 test year is representative.
- domain assumption Shared self-attention across the ensemble dimension transfers useful spatiotemporal information.
- ad hoc to paper The regularized kernel-based CRPS is a suitable training target for wind speed.
Cite this review
Pith. "Pith review of Self-attentive Transformer for Fast and Accurate Postprocessing of Temperature and Wind Speed Forecasts." pith.science (2026). https://pith.science/paper/33HMHM2P
@misc{pith2026241213957,
author = {Pith},
title = {Pith review of: Self-attentive Transformer for Fast and Accurate Postprocessing of Temperature and Wind Speed Forecasts},
year = {2026},
howpublished = {\url{https://pith.science/paper/33HMHM2P}},
note = {Machine review of arXiv:2412.13957}
}
read the original abstract
Current postprocessing techniques often require separate models for each lead time and disregard possible inter-ensemble relationships by either correcting each member separately or by employing distributional approaches. In this work, we tackle these shortcomings with an innovative, fast and accurate Transformer which postprocesses each ensemble member individually while allowing information exchange across variables, spatial dimensions and lead times by means of multi-headed self-attention. Weather forecasts are postprocessed over 20 lead times simultaneously while including up to fifteen meteorological predictors. We use the EUPPBench dataset for training which contains ensemble predictions from the European Center for Medium-range Weather Forecasts' integrated forecasting system alongside corresponding observations. The work presented here is the first to postprocess the ten and one hundred-meter wind speed forecasts within this benchmark dataset, while also correcting two-meter temperature. Our approach significantly improves the original forecasts, as measured by the CRPS, with 16.5\% for two-meter temperature, 10\% for ten-meter wind speed and 9\% for one hundred-meter wind speed, outperforming a classical member-by-member approach employed as a competitive benchmark. Furthermore, being up to six times faster, it fulfills the demand for rapid operational weather forecasts in various downstream applications, including renewable energy forecasting.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Alkhalidi, M., A. Al-Dabbous, S. Al-Dabbous, and D. Alzaid, 2025: Evaluating the accuracy of the era5 model in predicting wind speeds across coastal and offshore regions. Journal of Marine Science and Engineering , 13 (1), URL https://www.mdpi.com/2077-1312/ 13/1/149. Arroyo, ´A., ´A. Herrero, V. Tricio, and E. Corchado, 2017: Analysis of meteorological c...
work page 2025
-
[2]
ECMWF, 2024: ERA5: Data documentation. URL https://confluence.ecmwf.int/display/ CKB/ERA5%3A+data+documentation, known issues with analysed near surface winds and their diurnal cycle, Accessed: 2024-12-05. Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using monte carlo methods to forecast error statistics. Journa...
work page 2024
-
[4]
Earth System Science Data , 15 (6), 2635–2653. Dueben, P. D., M. G. Schultz, M. Chantry, D. J. Gagne, D. M. Hall, and A. McGovern, 2022: Challenges and benchmark datasets for machine learning in the atmospheric sci- ences: Definition, status, and outlook. Artificial Intelligence for the Earth Systems, 1 (3), e210
work page 2022
-
[11]
Messori, 2021: Ensemble methods for neural network-based weather forecasts
Scher, S., and G. Messori, 2021: Ensemble methods for neural network-based weather forecasts. Journal of Advances in Modeling Earth Systems , 13 (2). Schultz, M. G., C. Betancourt, B. Gong, F. Kleinert, M. Langguth, L. H. Leufen, A. Mozaf- fari, and S. Stadtler, 2021: Can deep learning beat numerical weather prediction? Philo- sophical Transactions of the...
work page 2021
-
[27]
B., and Coauthors, 2023: The rise of data-driven weather forecasting
Bouallegue, Z. B., and Coauthors, 2023: The rise of data-driven weather forecasting. URL https://arxiv.org/abs/2307.10128, 2307.10128. Bremnes, J. B., 2020: Ensemble postprocessing using quantile function regression based on neural networks and bernstein polynomials. Monthly Weather Review, 148 (1), 403–414. Burlando, M., M. Pizzo, M. Repetto, G. Solari, ...
arXiv 2023
-
[30]
Veldkamp, S., K. Whan, S. Dirksen, and M. Schmeits, 2021: Statistical postprocessing of wind speed forecasts using convolutional neural networks. Monthly Weather Review , 149 (4), 1141–1152. 22 Wessel, J. B., C. A. Ferro, and F. Kwasniok, 2024: Lead-time-continuous statistical post- processing of ensemble weather forecasts. Quarterly Journal of the Royal ...
work page 2021
-
[70]
Journal of Pharmaceutical Analysis , 101081
Jiang, J., and Coauthors, 2024: A review of transformers in drug discovery and beyond. Journal of Pharmaceutical Analysis , 101081. Jung, C., and D. Schindler, 2019: Wind speed distribution selection – a review of recent development and progress. Renewable and Sustainable Energy Reviews , 114, 109 290, https://doi.org/https://doi.org/10.1016/j.rser.2019.1...
-
[92]
Gimpel, 2016: Bridging nonlinearities and stochastic regularizers with gaussian error linear units
Hendrycks, D., and K. Gimpel, 2016: Bridging nonlinearities and stochastic regularizers with gaussian error linear units. CoRR, abs/1606.08415, URL http://arxiv.org/abs/ 1606.08415, 1606.08415. Hersbach, H., and Coauthors, 2020: The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146 (730), 1999–2049. 20 H¨ ohlein, K., B. Sc...
arXiv 2016
Show all 16 references
-
[97]
Lerch, 2022: Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison
Schulz, B., and S. Lerch, 2022: Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison. Monthly Weather Review , 150 (1), 235–257. Sørensen, J., G. Larsen, and A. Cazin-Bourguignon, 2021: Production and cost assessment of offshore...
2022
-
[107]
Naseer, M
Khan, S., M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, 2022: Transformers in vision: A survey. ACM computing surveys (CSUR) , 54 (10s), 1–41. Lakatos, M., S. Lerch, S. Hemri, and S. Baran, 2023: Comparison of multivariate post- processing methods using global ecm...
2022 arXiv
-
[162]
S., 2021: Self-attentive ensemble transformer: Representing ensemble interactions in neural networks for earth system models
Finn, T. S., 2021: Self-attentive ensemble transformer: Representing ensemble interactions in neural networks for earth system models. arXiv preprint arXiv:2106.13924 . Finn, T. S., 2023: Self-attentive ensemble transformer: Representing ensemble interactions in neural network...
2021 arXiv
-
[301]
super sapphire– phailin
Mlakar, P., J. Merˇ se, and J. Faganeli Pucer, 2024: Ensemble weather forecast post- processing with a flexible probabilistic neural network approach. Quarterly Journal of the Royal Meteorological Society, 150 (764), 4156–4177. Mohanty, U., and Coauthors, 2015: A great escape ...
2024 arXiv
-
[412]
Pinson, P., and J. W. Messner, 2018: Application of postprocessing for renewable energy. Statistical postprocessing of ensemble forecasts, Elsevier, 241–266. Rabier, F., 2024: Longer ranges. URL hhttps://www.ecmwf.int/en/newsletter/179/ editorial/longer-ranges, accessed on May 7,
2018
-
[735]
Zhang, and M
Zhang, P., J. Zhang, and M. Chen, 2017: Economic impacts of climate change on agriculture: The importance of additional climatic variables other than tempera- ture and precipitation. Journal of Environmental Economics and Management , 83, 8–31, https://doi.org/https://doi.org/...
2017 doi
-
[987]
Bi, K., L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian, 2022: Pangu-weather: A 3d high-resolution model for fast and accurate global weather forecast. arXiv preprint arXiv:2211.02556. Bouall` egue, Z. B., J. A. Weyn, M. C. Clare, J. Dramsch, P. Dueben, and M. Chantry, 2024: Imp...
2022 arXiv
-
[2024]
Lerch, 2018: Neural networks for postprocessing ensemble weather fore- casts
Rasp, S., and S. Lerch, 2018: Neural networks for postprocessing ensemble weather fore- casts. Monthly Weather Review , 146 (11), 3885–3900. Rumes, B., R. Brabant, and L. Vigin, 2022: Offshore renewable energy in the belgian part of the north sea. MEMOIRS,
2018
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.