Pith. sign in

REVIEW 6 major objections 5 minor 10 references

A Deep Learning Model of Lightning Stroke Density

T0 review · 6 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read This paper claims that U-Net convolutional networks trained on standard meteorological reanalysis fields predict lightning stroke density with about an order of magnitude less mean bias than the classic CAPE-times-precipitation product, esp

desk verdict Solid, new empirical result—CNN beats CAPE*precip on held-out years—but the paper has internal contradictions and missing details that need fixing before it is fully trustworthy. read the letter →

arxiv 2509.10399 v1 pith:LYTG2WHF submitted 2025-09-12 physics.ao-ph

classification physics.ao-ph
keywords lightningstrokedensitydeeplearningparameterizationU-NetconvolutionalneuralnetworkWWLLNCAPEFractionsSkillScoreERA5/IMERG
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a learned, nonlinear, spatially varying mapping from routine meteorological fields to lightning stroke density is more accurate than the long-standing CAPE×precipitation parameterization (called R14), particularly over oceans and in low-lightning regimes. The authors train U-Net convolutional neural networks on WWLLN lightning observations from 2010–2021 and test on held-out years 2022–2023. They report that the CNNs cut average domain mean bias by roughly an order of magnitude, raise Fractions Skill Scores across all lightning-density bins, and achieve r² values as high as 0.93 for climatological stroke density over ocean. If correct, this gives weather and Earth-system models a practical, data-driven alternative to a lightning parameterization known to overestimate tropical-ocean lightning.

What carries the argument

The central object is the U-Net convolutional neural network, an image-to-image encoder-decoder architecture whose skip connections preserve fine spatial detail while pooling layers capture large-scale context. Trained with mean squared error loss at 0.5°×0.5° and 12-hourly resolution, the network learns convolution kernels that map gridded fields—CAPE, precipitation, land-sea mask, relative humidity, wind shear, 2-meter temperature, and warm cloud depth—to stroke density. The learned mapping's nonlinearity and spatial variation is what distinguishes it from the fixed multiplication of CAPE and precipitation. Verification relies on the Fractions Skill Score (FSS) and a binned variant (MFSS)

What would settle it

Recompute the 2022–2023 evaluation using raw WWLLN counts before the detection-efficiency adjustment, or using an independent lightning dataset such as satellite-based total lightning, and check whether the CNN's advantage over R14 persists. If the margin shrinks or reverses, the central claim depends on the correction. Also check whether the main-text claim that the 0.5°/12-hourly model is statistically better than the 1°/3-hourly model survives a reproduction, since the supplement reports the opposite.

Watch

Extended reading notes

Core claim

The central discovery is that a U-Net CNN can reproduce the spatial distribution and magnitude of lightning stroke density from meteorological inputs, and that most of the improvement over R14 comes from allowing a nonlinear, spatially varying relationship rather than from adding predictors. On held-out 2022–2023 data, the best CNN (CPLRSTW) yields r² = 0.93 against WWLLN climatology over ocean, while R14 yields 0.44; over land, CPLRSTW yields 0.80 and R14 0.20. A CNN trained only on CAPE and precipitation (CP) already reaches r² = 0.90 over ocean, showing that the assumption of a linear multiplicative relationship in CAPE and precipitation is the main limitation of R14. The CNNs also captur

Load-bearing premise

The training target is WWLLN stroke counts after applying a detection-efficiency adjustment that scales observed counts to what a uniformly sensitive global network would see; if that correction is biased in space or time, the CNN learns a distorted lightning climatology and every verification score inherits the distortion.

Editorial extensions

If this is right

  • A learned lightning parameterization could replace or augment the CAPE×precipitation product in weather and Earth-system models, with the largest gains over oceans and in tropical regions.
  • Because the CAPE-plus-precipitation CNN already captures most of the skill, the key improvement is nonlinearity rather than additional predictor variables, suggesting simple nonlinear corrections to R14 may recover much of the benefit.
  • Event-scale prediction of 12-hourly lightning patterns at 0.5° resolution shows potential for use in severe-weather monitoring and short-range forecasting.
  • The land-sea mask is singled out as the variable that most reduces oceanic overestimation, indicating that explicitly separating land and ocean regimes is important for lightning parameterizations.
  • If storage is limited, the CP CNN may suffice for capturing general lightning patterns, while the full CPLRSTW CNN adds specificity at higher computational cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The CNN is only as trustworthy as the detection-efficiency-corrected WWLLN target; if that correction has regional or temporal biases, the network will learn those biases, and the reported margin over R14 may partly reflect the correction rather than true lightning physics.
  • Editorial inference: The architecture and training approach should transfer to other lightning datasets (e.g., geostationary lightning mappers) or to other convective hazards such as hail and severe wind, providing a testable route for broader operational use.
  • Editorial inference: The authors note an unresolved discrepancy in R14 performance between North America and South America; a testable next step is to examine whether the CNN is compensating for reanalysis biases in CAPE or whether the CAPE-updraft-electrification relationship actually differs by continent.
  • Editorial inference: A reader should note that the main text and the supplement appear to disagree on whether the 0.5°/12-hourly or the 1°/3-hourly configuration wins the bootstrap comparison; resolving this would clarify the paper's resolution choice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper trains U-Net CNNs on ERA5/IMERG meteorological fields to predict WWLLN lightning stroke density, with the WWLLN target corrected by the Hutchins et al. (2012) detection-efficiency adjustment. Training uses 2010–2021 and evaluation uses held-out 2022–2023. The authors compare a CAPE+precipitation CNN (CP), a seven-variable CNN (CPLRSTW), and an R14 baseline defined as a linear regression of stroke density against CAPE×precipitation, using FSS, MFSS, mean bias, and r². They report that the CNNs reduce mean bias by about an order of magnitude relative to R14, yield higher FSS in most lightning regimes and subdomains, achieve r²=0.93 over ocean, and capture two convective events. The paper concludes that nonlinear, spatially contextual image-based parameterizations are a promising alternative to multiplicative CAPE×precipitation schemes.

Significance. The strongest element is the evaluation design: 2022–2023 are true held-out years, so the CNN-versus-R14 comparison is a genuine forecast test rather than a circular fit. If the headline results survive scrutiny, the paper provides a useful demonstration that a spatially contextual, nonlinear mapping from reanalysis variables can outperform a standard CAPE×precipitation formulation, especially over tropical oceans where the latter is known to overpredict. The use of multiple verification metrics and two event case studies is a strength. The main concerns are specification gaps and internal inconsistencies in the resolution selection, baseline definition, FSS threshold, and abstract/body metric descriptions; these are fixable but should be addressed before the central claims can be fully assessed.

major comments (6)
  1. [Methods: Machine Learning Model Configuration; Fig. S2] The resolution-selection paragraph is internally inconsistent and the comparison is confounded. The main text reports mean FSS of 0.70 for 0.5°×0.5° at 12-hourly and 0.81 for 1°×1° at 3-hourly, then states that bootstrapping shows the 0.5°/12-hourly model is significantly more skillful. The supplement (Fig. S2) reports the same two configurations with means 0.694 and 0.710 and concludes that the 1°/3-hourly model is significantly better. The main text must be reconciled with the SI, and the selection of 0.5°/12-hourly—used for all subsequent results—needs to be justified. The comparison also changes spatial and temporal resolution simultaneously, so it cannot attribute the difference to either factor.
  2. [Methods: Data / Machine Learning Model Configuration] The R14 baseline is not specified sufficiently. The text says a linear regression of lightning stroke density to CAPE×precipitation 'will hereafter be referred to as R14', but it does not report the regression equation, the coefficient(s), the fitting period/domain, or whether the regression was re-estimated on the same training data used for the CNNs. Without this, the headline claim that the CNNs reduce bias by an order of magnitude relative to R14 is not reproducible, and a reader cannot assess whether the comparison is fair (e.g., original Romps et al. 2014 constant versus a domain-specific fit). Please provide the exact formula and fitting protocol.
  3. [Methods: Fractional Skill Score] The FSS is threshold-based in its standard form, but no threshold is given for the standard FSS used in Figures 2, 6a, 8a, and 10a. The text specifies only the 3×3 window. Without reporting the threshold, all quoted FSS values are ambiguous and not reproducible. Please state the threshold (or thresholds) used for each FSS calculation; the MFSS bins are defined, but the standard FSS is not.
  4. [Abstract; Results, Figure 5] The abstract's r²=0.93 is described as being between modeled and observed lightning 'climatologies', but the Results section reports that Figure 5 compares observed and modeled 12-hourly, point-by-point lightning stroke densities, with the r² values in the legends. In addition, the Figure 5 text says the comparison is at 1°×1°, despite the Methods selecting 0.5°×0.5° for the remainder of the study. Please reconcile the abstract with the actual metric and clarify the resolution of the r² calculation; this affects how the headline result is interpreted.
  5. [Methods: Data; Table S1] All verification is performed against the same Hutchins et al. (2012) detection-efficiency-corrected WWLLN target used for training. If that spatial/temporal correction is biased, the CNN—which can use the land-sea mask and spatial context—could learn the correction patterns and still score well against the same corrected target, while the R14 baseline cannot. This is not a circularity problem, because 2022–2023 are held out, but it is a correctness risk for the claim that the CNN improves on lightning physics. Table S1 compounds the concern: 91.4% of the 12-hourly target samples are in the zero bin, so aggregate metrics are strongly weighted by the ability to predict zeros. Please add an independent check (e.g., LIS/GLM climatology or uncorrected WWLLN counts) or an explicit sensitivity analysis and discuss the potential shared-target bias.
  6. [Methods: Machine Learning Model Configuration] The U-Net configuration is under-specified: the manuscript does not report number of downsampling/upsampling levels, filter counts, activation functions, optimizer, learning rate, batch size, number of epochs, data augmentation, or early stopping. 'U-Net' and '3×3 kernel' are insufficient to reproduce the model. Please provide a full architecture table or a pointer to a released implementation. The bootstrapping procedure (number of resamples, resampling unit) is also not described.
minor comments (5)
  1. [Methods: Data] The wind-shear formula is missing from the text; an equation placeholder appears before 'from ERA5 quantities'. Please insert the formula.
  2. [Introduction / References] Ayzel et al. is cited as 2010 in the introduction but the reference list gives 2020. Correct the citation year.
  3. [Table 1] Table 1 states that IMERG provides hourly precipitation rates at a 30-minute time resolution. Clarify whether the data are 30-minute fields expressed as hourly rates or hourly accumulations.
  4. [Throughout] Use R² rather than r2 for consistency with standard statistical notation.
  5. [Methods / Conclusion] The paper would benefit from a data and code availability statement, since the training configuration and evaluation routines are not otherwise fully specified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the CNN evaluation is a genuine held-out forecast comparison.

full rationale

The paper's central claim is that CNNs trained on ERA5/IMERG variables for 2010-2021 outperform the R14 (CAPE*precipitation) parameterization on WWLLN lightning observations from 2022-2023. This is a genuine out-of-sample evaluation: the target years are reserved for testing, the CNN weights are not fitted to those years, and the R14 baseline is also evaluated on the same held-out target. No equation in the paper constructs the prediction from the evaluation target, and no fitted parameter is renamed as a prediction. The only coauthor prior work entering the pipeline is the Hutchins et al. (2012) detection-efficiency adjustment, which is applied to WWLLN stroke counts before training and before all verification. That is a published, externally described correction and not a parameter fitted by this study; it affects the common target for both the CNN and the baseline, so it cannot by itself force the CNN's advantage. A possible bias in the correction is a measurement-quality concern, not a circular-derivation concern. The FSS/MFSS metrics and r2 comparisons are all evaluated against the same withheld observations but that is standard verification practice, not circularity. Accordingly, the derivation chain is self-contained for the claim of improved skill over R14, and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the detection-efficiency-corrected WWLLN target, the representativeness of the ERA5/IMERG predictors, and the specific architecture choices, none of which are independently verified here. The missing R14 fit and model hyperparameters are additional unstated inputs.

free parameters (3)
  • R14 linear regression coefficient(s) = not reported
    The baseline parameterization is defined as a linear regression of stroke density on CAPE*precipitation, fit on the training period; the slope (and intercept) are fitted constants that determine the baseline predictions, but their values are not given.
  • U-Net architecture hyperparameters (layer count, channels, optimizer, learning rate, epochs, batch size) = not reported
    The architecture is called a U-Net with 3x3 kernels and MSE loss, but no training hyperparameters or layer details are provided; these hand-chosen settings affect the predictions.
  • Model selection choices (kernel size, loss function, resolution) = 3x3 kernel, MSE loss, 0.5x0.5 deg, 12-hourly
    These configurations were chosen based on validation FSS; the choice of 0.5x0.5 12-hourly is contradicted by the supplemental bootstrapping, which found 1x1 3-hourly statistically better, so the selection is a partly data-driven choice that the paper does not resolve.
assumptions (4)
  • domain assumption The Hutchins et al. (2012) adjustment scales WWLLN stroke counts to what a uniformly sensitive global network would observe.
    Invoked in Methods/Data: 'we apply the adjustment developed previously by Hutchins et al. (2012) which scales the observed stroke counts to represent the distribution expected from a uniformly sensitive global network.' The truth of the training target depends on this correction.
  • domain assumption ERA5 and IMERG meteorological fields are sufficient predictors of lightning stroke density, and their relationship with lightning is stationary across the 2010-2023 period.
    The U-Net learns from these inputs during 2010-2021 and is tested on 2022-2023; any non-stationarity or missing predictor (e.g., aerosols) would degrade the transfer.
  • domain assumption Gridded 0.5x0.5 degree, 12-hourly lightning stroke density is a meaningful target for skill assessment, and FSS/MFSS on a 3x3 window capture the relevant skill.
    The paper uses FSS with a fixed 3x3 window and MFSS bins; these metric choices define the evaluation without sensitivity analysis.
  • standard math Convolutional U-Nets are capable of learning the desired mapping from the chosen inputs to stroke density.
    The paper relies on standard CNN universality and the U-Net architecture from Ronneberger et al. (2015); no formal guarantee is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Deep Learning Model of Lightning Stroke Density." pith.science (2026). https://pith.science/paper/LYTG2WHF

@misc{pith2026250910399,
  author       = {Pith},
  title        = {Pith review of: A Deep Learning Model of Lightning Stroke Density},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LYTG2WHF}},
  note         = {Machine review of arXiv:2509.10399}
}
read the original abstract

Lightning plays a crucial role in the Earth's climate system, yet existing parameterizations for use in forecasting and earth system models show room for improvement in capturing spatial and temporal variations in its frequency. This study develops deep learning-based parameterizations of lightning stroke density using meteorological variables from the ERA and IMERG datasets. Convolutional neural networks (CNNs) with U-Net architectures are trained using World Wide Lightning Location Network (WWLLN) data from 2010 to 2021 and evaluated on WWLLN lightning observations from 2022 and 2023. The CNNs reduce the average domain mean bias by an order of magnitude and produce significantly higher Fractions Skill Score (FSS) values across all lightning regimes compared to the multiplicative product of CAPE and precipitation. The CNNs show skill relative to previously published parameterizations over the oceans especially, with r2 values as high as 0.93 achieved between the best performing modeled and observed lightning stroke density climatologies. The CNNs are also able to accurately capture the 12-hourly evolution of lightning spatial patterns on an event-scale with high skill. These results show the potential for deep learning to improve on lightning parameterizations in weather and earth system models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 1 linked inside Pith

  1. [1]

    Thornton 1 , Chris J

    A Deep Learning Model of Lightning Stroke Density Randall Jones II 1 , Joel A. Thornton 1 , Chris J. Wright 1 , Robert A. Holzworth 2 1 University of Washington, Department of Atmospheric and Climate Sciences, 2 University of Washington, Department of Earth and Space Sciences ABSTRACT Lightning plays a crucial role in the Earth’s climate system, yet exist...

  2. [2]

    In general, as noted above, the R14 parameterization tends to have an order of magnitude larger mean bias (positive) compared to the CNN approaches, which have either positive or negative biases depending on the CNN model and region. Notably, over the northern domain, the average mean bias in the R14 parameterization is three orders of magnitude less than...

  3. [4]

    Stolz et al

    They found that the highest total lightning density occurred in areas with above average normalized CAPE, heightened concentrations of aerosols with diameters larger than 40 nanometers, and shallower warm cloud depth. Stolz et al. (2017) expanded upon this study, investigating the relative contributions of normalized CAPE, CCN concentrations, warm cloud d...

  4. [6]

    latitudexlongitude°

    We chose this region for future comparison with the domain of the Geostationary Lightning Mapper (GLM) aboard NASA’s GOES-18 satellite. Figure 1 shows the WWLLN lightning stroke density climatology for the focus region and time. Figure 1: Mean WWLLN lightning stroke density (strokes km -2 yr -1 ) for the years 2010 to 2023 at a 0.5° latitude by 0.5° longi...

  5. [8]

    A snapshot from the North American event is shown in Figure 7, with the mean lightning observed by WWLLN over the eastern continental United States on June 29 th , 2023 in the top left, and predictions from the R14 parameterization (top right), the CP CNN model (bottom left), and the CPLRSTW CNN model (bottom right) for comparison. In the observed lightni...

  6. [9]

    There are sharp gradients in the lightning stroke density over Brazil, which are not replicated by the R14 parameterization or the CNN models

    North of 20° S, the event brings high lightning stroke densities over land throughout the western coast of South America and Brazil. There are sharp gradients in the lightning stroke density over Brazil, which are not replicated by the R14 parameterization or the CNN models. There is also a high lightning feature during this event over the ocean north of ...

  7. [2011]

    However, Romps et al

    A linear regression of lightning stroke density to the multiplicative product of CAPE and precipitation will hereafter be referred to as the R14 parameterization. However, Romps et al. (2018) shows that this parameterization does not work particularly well over the ocean, overestimating lightning in the tropical oceans, pointing to the possibility of othe...

  8. [2013]

    Additionally, with the inclusion of geographic variables, such as coast and terrain, the logistic regression method accurately predicted lightning occurrence 86% of the time

    They found that an increase in lightning was linked with an increase in all environmental variables with the exception of shear. Additionally, with the inclusion of geographic variables, such as coast and terrain, the logistic regression method accurately predicted lightning occurrence 86% of the time. The goals of our study are similar to those of Stolz ...

Show all 10 references
  1. [2023]

    The CNNs reduce the average domain mean bias by an order of magnitude and produce significantly higher Fractions Skill Score (FSS) values across all lightning regimes compared to the multiplicative product of CAPE and precipitation. The CNNs show skill relative to previously p...

  2. [4289]

    https://doi.org/10.1038/s41467-022-31714-5 Phillips, V. T. J., Donner, L. J., & Garner, S. T. (2007). Nucleation processes in deep convection simulated by a cloud-system-resolving model with double-moment bulk microphysics. Journal of the Atmospheric Sciences, 64 (3), 738–761....

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.