REVIEW 3 major objections 5 minor 18 references
Using Generative Models to Produce Realistic Populations of UK Windstorms
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Diffusion-GAN best mimics UK windstorms but overshoots extremes.
desk verdict Solid first benchmark of four generative models for UK wind fields; believable ranking but the undocumented training-sample construction makes the reference distribution undefined. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the coupling of adversarial or diffusion training objectives with an extreme-event metric. The models map latent noise to $40\times 40$ fields of normalized 10-metre wind speed, using transposed-convolution generators (standard GAN and WGAN-GP), a U-net that reverses a 16-step linear noise schedule, or a timestep-dependent discriminator that sees noisy versions of real and generated samples through an exponential 32-step schedule. Success is measured not by pixel error but by four distributional comparisons: FID in an Inception feature space, SSIM between averaged storm-severity maps, and KL divergence and Earth mover's distance on the first 25 principal components (about 95% of variance). The storm severity index, computed from local 98th-percentile wind speeds, land-sea mask, and exceedances, is the device that makes the extreme tail visible and separates the models.
What would settle it
Use independent station-based gust observations or a held-out ERA5 period that none of the models saw, and compare the upper tail of generated wind speeds: if the diffusion-GAN's top SSI values still exceed the observed maximum (the Burns' Day storm, SSI 122.95 in ERA5) while its bulk distribution remains close, the claimed trade-off is confirmed; if a retrained model reproduces the held-out tail, the paper's 'overestimates extremes' conclusion would be a tuning artefact.
Extended reading notes
Core claim
On the paper's own terms, the central claim is that four off-the-shelf generative architectures, a standard GAN, a Wasserstein GAN with gradient penalty, a U-net denoising diffusion model, and a diffusion-GAN, can each learn the spatial and statistical character of UK windstorms from ERA5, and that their differences are systematic. The WGAN-GP is the most balanced across metrics but occasionally misplaces or exaggerates extreme wind regions; the standard GAN varies most across repeated sampling and diverges on PCA-based distributional metrics; the U-net diffusion model is visually best but underestimates the upper tail; the diffusion-GAN has the best KL divergence and Earth mover's distance on the first 25 principal components yet produces top SSI values, up to 181, far above anything in ERA5. The paper concludes that every model underestimates the rarest events (SSI above 60) and that an ensemble combining the models' complementary strengths could improve overall reliability.
Load-bearing premise
The load-bearing premise is that the 83 years of hourly ERA5 fields, after whatever reduction produced the training set, still represent the true population of UK windstorms, so statistical similarity to that same training distribution is a valid test of realism and of value for risk assessment.
Editorial extensions
If this is right
- The diffusion-GAN, with the best KL divergence and Earth mover's distance, is the most reliable generator for bulk wind-speed distributions, but its inflated SSI tail means it cannot be used unadjusted for extreme-event frequencies.
- All four models underproduce the rarest events (SSI above 60), so any generated catalogue will need tail correction or statistical post-processing before it can inform catastrophe models.
- Since each architecture has a complementary failure mode, an ensemble that selects or blends models per event type should improve overall fidelity relative to any single model.
- The U-net diffusion model's consistent underestimation means visual realism does not guarantee extreme-value realism; image-quality metrics alone are insufficient for hazard validation.
- The models generate independent, identically distributed, single-variable snapshots, so they can emulate the spatial population of storms but not storm lifecycles or multivariate physical consistency.
Reading between the lines
- If the diffusion-GAN's tail inflation and the U-net's tail contraction are architecture-driven rather than tuning accidents, then a conditional or mixture-of-experts generator could in principle combine U-net spatial fidelity with diffusion-GAN distributional match; the paper stops at suggesting an ensemble, not at designing one.
- Because every model was evaluated against the same ERA5 distribution it was trained on, the reported realism is an upper bound; held-out station gust observations or RCM-generated storms would be a stricter and more decision-relevant test.
- The consistent underproduction of SSI values above 60 across all four architectures suggests the limiting factor is the short observational tail of the training set, not model choice; extending training with physically simulated or statistical-surrogate storms may improve tail behaviour more than any architectural change.
- The PCA result that extreme cases scatter more widely than ERA5, with GAN-type models biased toward Scottish-coast storms, implies that generated catalogues may overstate the geographic spread of rare events; risk modellers should check regional rather than national loss aggregates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript evaluates four generative models (standard GAN, WGAN-GP, U-net diffusion model, and diffusion-GAN) for producing synthetic 10-m wind-speed fields over the UK, using ERA5 reanalysis data from 1940 to 2022 as the training reference. The authors compare the models through visual inspection of typical and extreme wind maps, SSI distributions, PCA scatter plots, point-based boxplots and return periods, and four quantitative metrics (FID, SSIM, KL divergence, EMD). They report that the diffusion-GAN performs best overall but overestimates the most extreme events, the U-net diffusion model underestimates storm intensity, and the WGAN-GP offers balanced but imperfect performance. The paper frames the work as a first comparative benchmark of generative architectures for UK windstorm emulation and discusses potential applications in risk assessment.
Significance. If the central comparison is valid, this paper would provide a useful baseline for applying modern generative models to windstorm emulation, a topic with limited prior work in meteorology. The study's strengths include multiple independent lines of evidence—spatial maps, SSI distributions, PCA diagnostics, return-period curves, and four quantitative metrics—that generally support the qualitative ranking of the models. The paper is also transparent about many limitations, explicitly conceding in Section 5c that the evaluation metrics do not fully capture physical structure and that the generated fields are IID single-variable snapshots without temporal consistency. However, the significance is currently limited by two load-bearing methodological gaps: the manuscript never documents how the hourly ERA5 record was subsampled into the training set, and all evaluation metrics are computed against the same data distribution used for training, with no independent holdout or physical validation. These gaps must be addressed before the comparative ranking and the risk-applicability claims can be considered reliable.
major comments (3)
- [Section 2 and Tables 3/4] The manuscript does not state how the hourly ERA5 record (1940–2022, 40×40 grid) was reduced to the training sample implied by the statement 'Batches per epoch 569 (one-tenth of the sample size)' in Tables 3 and 4. If the sample size is approximately 5,690 maps, this is a drastic reduction from the roughly 727,000 hourly fields in the stated period, and the subsampling rule (by season, storm status, hour, or random selection) is never given. This is load-bearing because all four models are trained on this sample, and the title and abstract claim generation of 'realistic populations of UK windstorms.' A biased or undefined subsampling procedure would corrupt the target distribution and hence the entire comparison. Please specify the exact subsampling procedure, the resulting training sample size, and whether the same subset is used for model selection and evaluation.
- [Section 4e, Table 6, and Section 5c] All evaluation metrics (FID, SSIM, KL divergence, EMD) compare generated samples to the ERA5 data, but the manuscript does not state whether the reference is the full hourly ERA5 record or the 5,690-map training subset. If the reference is the training subset, the evaluation is entirely in-sample and the reported rankings may reflect overfitting rather than generalization. The paper's own limitations section (5c) acknowledges that these metrics are statistical and do not capture physical structure, and that the outputs are IID single-variable fields. Given this, the central claim that the diffusion-GAN 'performed better than the other models in general' requires either an explicit justification for why similarity to the training distribution is the correct measure of realism, or an independent evaluation (e.g., a temporal or spatial holdout, or comparison against storm-event statistics not used in training). Please clarify the reference dataset and discuss the in-sample nature of the evaluation.
- [Section 4b, Fig 9, and Fig 14] There is an apparent inconsistency in the sample size used for SSI distributions. The text states that models were run to produce 'the same number of samples as the ERA5' (Section 4a), but Fig 14 describes each generated dataset as being of 'equal length (83 years)', which would imply roughly 727,000 hourly maps per model if every hour is used. Meanwhile, Tables 3 and 4 imply a training sample of about 5,690 maps. The manuscript never states how many generated maps are used to compute the SSI frequency distributions in Fig 9 and Fig 14, nor how the ERA5 reference curve is constructed (all hours, storm-only hours, or the training subset). Without this information, the visual comparisons of tail frequencies and the claim that 'all models underestimate the frequency of extreme events (SSI above 60)' are not quantitatively interpretable. Please report the exact number of samples used for each curve and specify the ERA5 reference sample.
minor comments (5)
- [Section 2] The text cites 'Potisomporn et al., 2023' but the reference list contains the corresponding entry under 'Ravuri, S., Lenc, K., Willson, M., ... 2023: Evaluating ERA5 reanalysis predictions...' with Ravuri listed as the first author. This misattribution should be corrected.
- [Section 3f, Eq. (2)] The FID equation is written for general feature means and covariances, but the manuscript does not explain how the Inception-v3 model, pre-trained on ImageNet, is adapted to 40×40 single-channel wind-speed fields. Please describe the feature extraction procedure (e.g., resizing, channel replication, which layer is used).
- [Figure 13] The caption for Fig 13 omits the diffusion-GAN (purple line) in the list of compared models, even though the figure and text discuss it. The caption should list all four models.
- [Section 4a, Figure 8] In the description of the U-net diffusion model, the third-highest SSI value is quoted as 50.98, but the corresponding maps and table do not provide full SSI values for all top-10 cases; adding a table with the top-10 SSI values for each model would make the comparison more concrete.
- [Appendix A5] In Table A5, several storm names are left blank (e.g., rank 4, 7, and 10). If the events do not have widely used names, this is acceptable, but the authors should state that explicitly rather than leaving the field empty.
Circularity Check
No significant circularity: the four-model comparison is an empirical fit-to-reference evaluation whose conclusions are not assumed in the inputs.
full rationale
This paper contains no derivation chain of the kind to which circularity analysis applies. It trains four generative models on ERA5 10-m wind fields (Section 2) and evaluates them with FID, SSIM, KL divergence, EMD, and SSI comparisons against the same ERA5 dataset (Section 3f, Table 6). Matching the training distribution is the explicit objective of generative modelling, so using ERA5 as the reference for these metrics is fit-for-purpose rather than a reduction of the conclusions to their inputs; the models can and do fail to match ERA5 (e.g., the U-net diffusion model underestimating SSI and the diffusion-GAN overestimating extremes), so the ranking has empirical content. There are no self-citations used as load-bearing evidence, no imported uniqueness theorems, and no fitted parameter renamed as a prediction. The paper's own limitations section (Section 5c) acknowledges that the metrics do not capture physical structure and that outputs are IID single-variable fields; this weakens the broader risk-applicability claim but does not make the comparative benchmark circular. The undocumented reduction of hourly ERA5 data to the implied ~5,690-sample training set (Tables 3–4: 'Batches per epoch 569 (one-tenth of the sample size)') is a provenance and reproducibility gap, not a circularity.
Assumptions & free parameters
free parameters (6)
- Standard GAN training hyperparameters =
LR 0.0002, batch 128, 10000 epochs, label smoothing 0.9
- WGAN-GP training hyperparameters =
LR 0.0006 (generator), 0.0004 (critic), gradient penalty weight 10, 4 critic updates per generator step
- U-net diffusion noise schedule =
16 timesteps, linear noise schedule, initial LR 0.0008, LR reduction factor 0.75
- Diffusion-GAN noise schedule =
32 timesteps, beta from 0.0001 to 0.02, exponential schedule
- PCA truncation =
25 principal components
- Training sample count and subsampling scheme =
Implied about 5,690 samples (569 batches per epoch equals one-tenth of sample size)
assumptions (6)
- domain assumption ERA5 reanalysis is an unbiased representation of UK surface windstorm risk.
- domain assumption The specific training sample is representative of the windstorm population, including extremes.
- domain assumption Distributional similarity to the same ERA5 training data is a valid measure of realism.
- ad hoc to paper IID single-variable wind-speed fields can stand in for windstorm events.
- domain assumption Klawa and Ulbrich SSI with local 98th percentile thresholds is appropriate for UK storm severity.
- standard math PCA with 25 components preserves the distributional information needed for KL and EMD.
Cite this review
Pith. "Pith review of Using Generative Models to Produce Realistic Populations of UK Windstorms." pith.science (2026). https://pith.science/paper/YYPRFFMR
@misc{pith2026250116110,
author = {Pith},
title = {Pith review of: Using Generative Models to Produce Realistic Populations of UK Windstorms},
year = {2026},
howpublished = {\url{https://pith.science/paper/YYPRFFMR}},
note = {Machine review of arXiv:2501.16110}
}
read the original abstract
This study evaluates the potential of generative models, trained on historical ERA5 reanalysis data, for simulating windstorms over the UK. Four generative models, including a standard GAN, a WGAN-GP, a U-net diffusion model, and a diffusion-GAN were assessed based on their ability to replicate spatial and statistical characteristics of windstorms. Different models have distinct strengths and limitations. The standard GAN displayed broader variability and limited alignment on the PCA dimensions. The WGAN-GP had a more balanced performance but occasionally misrepresented extreme events. The U-net diffusion model produced high-quality spatial patterns but consistently underestimated windstorm intensities. The diffusion-GAN performed better than the other models in general but overestimated extremes. An ensemble approach combining the strengths of these models could potentially improve their overall reliability. This study provides a foundation for such generative models in meteorological research and could potentially be applied in windstorm analysis and risk assessment.
Reference graph
Works this paper leans on
-
[1]
Introduction Windstorms are among the most impactful natural hazards in the United Kingdom, causing widespread disruption to society, infrastructure, and the economy. The geographical location of the country exposes it to windstorms, which frequently develop over the North Atlantic and especially affect the country during winter (Adam et al., 2016). Major...
work page 2016
-
[2]
Data description The dataset used in this study is the ERA5 reanalysis produced by the European Centre for Medium-Range Weather Forecasts (ECMWF). ERA5 provides hourly estimates of atmospheric variables, which covers the period from 1940 to the present with a spatial resolution of 0.25°×0.25° (Hersbach et al., 2020). For this study, data from 1940 to 2022...
work page 1940
-
[3]
Copyright in this work may be transferred without further notice
Methodology This work has been submitted to Artificial Intelligence for the Earth Systems. Copyright in this work may be transferred without further notice. 6 a. Model Selection and Parameter Tuning Various generative models, including variational autoencoder GANs, mean and covariance feature matching GANs, and denoising diffusion GANs, were considered an...
work page 2017
-
[4]
The spatial coverage of the ERA5 dataset used in this study and the domain for which the data samples will be generated. Prior to model training, the ERA5 data were normalized to a range of [0,1] using global minimum and maximum values across the entire domain and period. This normalization improves stability during training and generation (Glorot & Bengi...
work page 2010
-
[5]
consists of an encoder-decoder structure with skip connections (grey arrows) between them (Ibtehaz & Rahman, 2020). These connections ensure that the model retains important spatial features and recovers fine details in the outputs (Drozdzal et al., 2016). The encoder downsamples the noisy wind fields into lower-dimensional representations, while the deco...
work page 2020
-
[6]
The architecture of the discriminator in the standard GAN. LeakyReLU activations are used after convolutional layers in both networks, which return small values for negative inputs instead of zeros in the ReLU activation functions. This technique ensures small gradients passing through the networks and prevents vanishing gradients that may hinder the trai...
work page 2018
-
[7]
One-sided label smoothing is applied to the real samples, labelling them as 0.9 instead of 1 commonly found in GANs, to avoid an overconfident discriminator, and thereby vanishing gradients (Goodfellow, 2016). Generator Discriminator Optimizer Adam Learning rate 0.0002 𝛽! 0.5 𝛽" 0.999 Loss function Binary cross entropy Batch size 128 64 real samples + 64 ...
work page 2016
-
[8]
Optimization scheme and training parameters for the standard GAN model. c. Wasserstein GAN with Gradient Penalty (WGAN-GP) This work has been submitted to Artificial Intelligence for the Earth Systems. Copyright in this work may be transferred without further notice. 9 While the overall architecture of the WGAN-GP is similar to that of the standard GAN, a...
work page 2018
Show all 18 references
-
[10]
It consists of an encoder (down-sample) and a decoder (up-sample)
The architecture of the U-net used to predict the denoised data, 𝑥#$! given the input data 𝑥#. It consists of an encoder (down-sample) and a decoder (up-sample). In this study, the forward diffusion process follows a linear noise scheduling, where the magnitude of noise increa...
2019
-
[11]
Optimization scheme and training parameters for the diffusion-GAN model. f. Verification Methods The performance of the generative models was evaluated using various statistical techniques (Table 5), which focus on how well the generated outputs replicate the spatial character...
2017
-
[12]
This work has been submitted to Artificial Intelligence for the Earth Systems
Top-ranked outputs from the diffusion-GAN with improper noise scheduling. This work has been submitted to Artificial Intelligence for the Earth Systems. Copyright in this work may be transferred without further notice. 38 Rank SSI Values Time Storm/Event 1 122.95 1400 UTC 25 J...
1990
-
[13]
REFERENCES Adam, E.F., Brown, S., Nicholls, R.J
Top 10 SSI cases identified from the ERA5 dataset, ranked by their SSI values. REFERENCES Adam, E.F., Brown, S., Nicholls, R.J. and Tsimplis, M., 2016: A systematic assessment of maritime disruptions affecting UK ports, coastal areas and surrounding seas from 1950 to
2016
-
[17]
and Abbeel, P., 2020: Denoising diffusion probabilistic models
Ho, J., Jain, A. and Abbeel, P., 2020: Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, pp.6840-6851. Hu, Z. and Hong, L.J., 2013: Kullback-Leibler divergence constrained distributionally robust optimization. Available at Optimiz...
2020 arXiv
-
[27]
and Courville, A.C., 2017: Improved training of wasserstein gans
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. and Courville, A.C., 2017: Improved training of wasserstein gans. Advances in neural information processing systems,
2017
-
[30]
Natural Hazards and Earth System Sciences, 11(10), pp.2847-2857
Haylock, M.R., 2011: European extra-tropical storm damage risk from a multi-model ensemble of dynamically-downscaled global climate models. Natural Hazards and Earth System Sciences, 11(10), pp.2847-2857. Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz...
2011
-
[31]
and Yu, Y., 2018, April: Wasserstein distance guided representation learning for domain adaptation
Shen, J., Qu, Y., Zhang, W. and Yu, Y., 2018, April: Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI conference on artificial intelligence (Vol. 32, No. 1). Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z., 2...
2018 arXiv
-
[1990]
have demonstrated the potential severity of such events, resulting in billions of pounds in damage and significant loss of life (Cusack, 2023). Accurate catalogues of windstorms and their losses are essential for risk assessment, particularly in the insurance and re-insurance ...
2014
-
[2014]
Ashfaq, M., Rastogi, D., Mei, R., Kao, S.C., Gangrade, S., Naz, B.S
Natural Hazards, 83, pp.691-713. Ashfaq, M., Rastogi, D., Mei, R., Kao, S.C., Gangrade, S., Naz, B.S. and Touma, D., 2016: High‐resolution ensemble projections of near‐term regional climate over the continental United States. Journal of Geophysical Research: Atmospheres, 121(1...
2016 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.