REVIEW 4 major objections 5 minor 37 references
Advancing Heatwave Forecasting via Distribution Informed-Graph Neural Networks (DI-GNNs): Integrating Extreme Value Theory with GNNs
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Heatwave recall jumps from 13% to 85% with extreme-value GNN
desk verdict The GPD+GNN combination is plausible, but the reported metrics are arithmetically impossible under the paper's own definitions, so the central claim currently has no valid empirical support. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the per-station Generalized Pareto Distribution fit to exceedances of the 90th percentile of daily maximum temperature, summarized by shape $\xi$, scale $\sigma$, mean, variance, and 95th percentile $q_{95}$. These five numbers are used three ways: appended to each node's feature vector (Eq. 16); to reweight the Pearson-correlation adjacency matrix as $A_{ij} = \rho_{ij} w_i w_j$ with $w_i = 1 + |\xi_i| + \sigma_i / \max_j \sigma_j$ (Eq. 17); and to define a weighted F1 loss that up-weights stations with heavier tails (Eqs. 18–20). The GPD, defined by $H(y|\xi,\sigma) = 1 - (1 + \xi y/\sigma)^{-1/\xi}$, models the distribution of threshold excesses, so these descriptors encode how extreme and how variable the local temperature tail is.
What would settle it
Fit the GPD descriptors on the training years only, then append a single per-station constant equal to the historical heatwave-day frequency from those same training years to the Li et al. feature set, with no adjacency reweighting and no weighted loss. If this trivial climatology control reaches recall and AUC near DI-GNN's reported values, the GPD machinery is adding nothing beyond station priors; if the control remains near the baseline (recall ~13%, AUC ~0.68), the descriptors are carrying real distributional information.
Extended reading notes
Core claim
The central discovery is that GPD-derived descriptors — the shape parameter $\xi$, scale $\sigma$, mean, variance, and 95th percentile of daily maximum temperature — when computed per station and threaded through the feature vector, the adjacency matrix (via weights $w_i = 1 + |\xi_i| + \frac{\sigma_i}{\max_j \sigma_j}$), and a weighted F1 loss, transform the model's ability to catch rare heatwave days. On the BC dataset, DI-GNN achieves balanced accuracy 86.66%, recall 84.95%, precision 87.38%, and AUC 0.919 for the 3-day horizon (Table 2), compared with balanced accuracy 62.35%, recall 12.86%, precision 45.96%, and AUC 0.675 for the adapted Li et al. baseline (Table 1). The authors interpret this as evidence that EVT-informed descriptors give the GNN a principled way to prioritize tail events, replacing heuristic feature engineering with statistically grounded structure at every level of the model.
Load-bearing premise
The load-bearing premise is that the per-station GPD descriptors — fitted on the same Tmax record that defines heatwave labels — are legitimate predictive features rather than a backdoor that leaks the label structure; if they only encode each station's heatwave climatology, the reported forecast improvement is not genuine skill.
Editorial extensions
If this is right
- If the reported gains hold, heatwave warning systems could use DI-GNN to issue reliable alerts at 3- and 5-day lead times with recall around 85%, catching most heatwave days rather than the 13% caught by the baseline.
- The approach transfers beyond heatwaves: the authors argue that any rare-event forecasting problem with a well-defined exceedance threshold could benefit from embedding GPD descriptors into features, graph structure, and loss.
- DI-GNN's accuracy is actually lower than the baseline's (43.8% vs 89.2% for the 3-day horizon) while its balanced accuracy is much higher, showing an explicit trade-off between raw accuracy and rare-event detection that forecasters must manage.
- Because the GPD descriptors are computed per station, the framework adapts to local climate: stations with heavier tails are weighted more heavily in both the graph and the loss, which could make the model usable across diverse climates without retuning.
Reading between the lines
- An ablation the paper does not run would isolate which injection point matters: adding only the five GPD features to the Li et al. baseline, without adjacency reweighting or the weighted loss, would show whether the gain is from the extra inputs or from the graph and loss changes.
- Because the per-station GPD descriptors are constant and derive from the same Tmax record used to define heatwave labels, they may act as a station-level climatology prior rather than a time-varying forecast signal; a control feature equal to historical heatwave frequency would test this.
- DI-GNN's accuracy falls to 43.8% while the baseline achieves 89.2%, meaning the 85% recall likely comes with many extra false alarms; a cost-weighted comparison of missed heatwaves versus false alerts would assess operational value.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DI-GNN, a graph attention network for heatwave forecasting from 71 British Columbia weather stations, and claims that injecting Generalized Pareto Distribution (GPD) descriptors into the feature space, adjacency matrix, and loss function substantially improves prediction of rare heatwave days over the Li et al. baseline. The reported evaluation covers 3-day and 5-day horizons and lists balanced accuracy, recall, precision, F1, accuracy, AUC, and average precision. The central claim is that EVT-informed components yield large, stable gains over the baseline on imbalanced heatwave data.
Significance. The underlying idea is timely and potentially interesting: representing tail behavior of temperature distributions inside a GNN is a natural direction for rare-event forecasting. However, as submitted, the empirical evidence is not usable. Tables 1 and 2 violate the metric identities defined in Section 3.2, there is no independent test set or uncertainty quantification, and the GPD descriptors may encode the same threshold that defines the labels. If these issues were corrected with a sound experimental protocol, the approach could merit further study, but the present manuscript does not establish the claimed improvement. The paper is transparent about some limitations (Section 7), such as station-only predictions and the absence of atmospheric drivers, but the problems I identify are more fundamental and require redoing the evaluation.
major comments (4)
- [Tables 1 and 2; Section 3.2] The reported headline metrics are jointly inconsistent under the paper's own definitions. With positive-class proportion p = 0.1245 (Section 5.1), Eq. (7) implies Accuracy = p*Recall + (1-p)*TNR. For DI-GNN with Cout = 3, BA = 0.8666 and Recall = 0.8495 give TNR = 2*BA - Recall = 0.8837, so Accuracy must lie between 0.8495 and 0.8837; Table 2 reports 0.4377. For Cout = 5, TNR = 0.8674, which forces Accuracy ≈ 0.866, not 0.2990. In addition, Eq. (12) would give F1 ≈ 86.15% from the reported precision and recall, not the tabulated 84.36%. No confusion matrix can reproduce Table 2, so the claimed DI-GNN performance cannot be evaluated.
- [Table 1; Section 3.2] The baseline table is also internally inconsistent. For Cout = 3, BA = 0.6235 and Recall = 0.1286 imply TNR = 1.1184 > 1, which is impossible. For Cout = 5, BA = 0.3628 and Recall = 0.1200 imply TNR = 0.6056, which yields Accuracy ≈ 0.545 under the stated p = 0.1245, not the reported 0.6901. These inconsistencies make the baseline comparison meaningless as reported.
- [Sections 4.1, 4.4, 5.2; Eqs. (16) and (21)] The GPD descriptors are fitted to the same Tmax series whose 90th percentile defines the heatwave label, creating a serious risk of circularity. Section 4.4 computes exceedances above the 90th percentile of Tmax, while Section 3 labels heatwave days by Tmax exceeding the 90th percentile under the PKL criteria. If the descriptors are computed over the full record rather than strictly on training folds, the features encode information about the label-generating distribution. Even if the descriptors are computed only on training data, the reported gain over the baseline may reflect per-station climatological tail priors rather than improved temporal forecasting skill. I would want to see temporal-blocked cross-validation with descriptors recomputed per fold, plus a descriptor-only baseline (e.g., logistic regression on ξ_i, σ_i, μ_i, Var_i, q95) to separate prior information from predictive skill.
- [Sections 5.2 and 5.4] There is no independent test set and no uncertainty quantification. Section 5.2 states that the first 13 years are used for training and the last two years for validation, with early stopping based on validation loss; the results in Tables 1 and 2 therefore appear to be validation-set metrics. The claim that DI-GNN 'significantly outperforms' the baseline is unsupported without a held-out test period, repeated runs with error bars, or a significance test. The instability visible in the baseline training curves (Figures 2 and 3) further suggests run-to-run variability that should be quantified rather than described qualitatively.
minor comments (5)
- [Figures 4, 5, 8, 9] Figure numbering and captions are inconsistent: Figure 5 is labeled as Precision-Recall curves for the Li et al. model, but Section 5.4 refers to Figures 5 and 6 as though they are DI-GNN training curves, and the AP values for DI-GNN appear only in Figure 9 with no in-text reference to that figure.
- [Section 4.3, Eq. (20)] The loss function introduces a weighting parameter β, but the value of β is never reported in Section 5.2, so the training objective is not fully specified.
- [Section 4.4] The description should state explicitly whether the GPD threshold and descriptors are recomputed within each training fold or once on all data; Section 5.2 says 'derived from the training data,' but Section 4.4 does not specify any temporal split.
- [Code Availability] The statement 'Code and data is available upon request' is insufficient for reproducibility; particularly because the experimental tables contain impossible entries, the manuscript should provide public code and processed data.
- [Introduction] There is a typo ('increasing frequent') and the sentence about the 2021 heatwave breaking records 'by over 45°C' is ambiguous; presumably it means temperatures exceeded 45°C, not that records were broken by 45°C.
Circularity Check
No circular derivation: the GPD/threshold overlap is a leakage risk but not by-construction equivalence; the reported metric inconsistencies are correctness failures, not circularity.
full rationale
The only candidate for circularity is the GPD feature construction. The paper fits ξ, σ, μ, variance, and q95 to each station's Tmax exceedances above the 90th percentile (Section 4.4, Eq. 21), and the PKL heatwave label is also defined by Tmax exceeding the 90th percentile (Section 3). This overlap is a genuine leakage risk: the descriptors summarize the same tail that defines the target, so part of DI-GNN's apparent gain could come from target-derived inputs rather than genuinely predictive temporal learning. However, Section 5.2 states the GPD parameters were 'derived from the training data,' while the labels being forecast are in the validation period; the descriptors are static per-station values, not the day-by-day validation labels. Shared threshold does not make the forecast equal to the fitted descriptors by construction, so this is not a circular derivation under the paper's stated protocol. Separately, the headline empirical comparison is not reproducible. For the DI-GNN Cout=3 row, Balanced Accuracy = 0.8666 and Recall = 0.8495 imply TNR = 0.8837 via Eq. (8). Eq. (7), together with the class fraction, forces Accuracy = p·Recall + (1−p)·TNR, which must lie between 0.8495 and 0.8837, yet Table 2 reports Accuracy = 0.4377. The baseline rows are similarly impossible (row 1 of Table 1 gives TNR = 1.1184 > 1). This makes the claimed 'significantly outperforms' unsupported, but it is an arithmetic/reporting inconsistency rather than a circular step. No load-bearing self-citation chains, imported uniqueness theorems, or ansatz-by-citation patterns appear. The references to [28] are to an external baseline, not to the authors' own prior theorems. Accordingly, the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- GPD shape xi_i per station =
not reported; MLE fits
- GPD scale sigma_i per station =
not reported; MLE fits
- GPD threshold u_i (90th percentile of Tmax) per station =
not reported; 90th percentile
- 95th percentile q95_i =
not reported
- Tmax mean mu_i and variance Var_i =
not reported
- F1-weighted loss parameter beta =
not reported
assumptions (5)
- domain assumption Tmax exceedances above the 90th percentile follow a Generalized Pareto Distribution
- domain assumption The PKL heatwave definition (Tmax > T90 for 3+ consecutive days) correctly identifies heatwaves in BC
- domain assumption The 71 ECCC stations and the 2009-2024 period are homogeneous and representative for training and validation
- domain assumption Pearson correlation between station time series encodes spatial dependencies relevant to heatwaves
- domain assumption GAT message passing can learn from a sparse 71-node graph with static node features
Cite this review
Pith. "Pith review of Advancing Heatwave Forecasting via Distribution Informed-Graph Neural Networks (DI-GNNs): Integrating Extreme Value Theory with GNNs." pith.science (2026). https://pith.science/paper/UUINXQVI
@misc{pith2026241113496,
author = {Pith},
title = {Pith review of: Advancing Heatwave Forecasting via Distribution Informed-Graph Neural Networks (DI-GNNs): Integrating Extreme Value Theory with GNNs},
year = {2026},
howpublished = {\url{https://pith.science/paper/UUINXQVI}},
note = {Machine review of arXiv:2411.13496}
}
read the original abstract
Heatwaves, prolonged periods of extreme heat, have intensified in frequency and severity due to climate change, posing substantial risks to public health, ecosystems, and infrastructure. Despite advancements in Machine Learning (ML) modeling, accurate heatwave forecasting at weather scales (1--15 days) remains challenging due to the non-linear interactions between atmospheric drivers and the rarity of these extreme events. Traditional models relying on heuristic feature engineering often fail to generalize across diverse climates and capture the complexities of heatwave dynamics. This study introduces the Distribution-Informed Graph Neural Network (DI-GNN), a novel framework that integrates principles from Extreme Value Theory (EVT) into the graph neural network architecture. DI-GNN incorporates Generalized Pareto Distribution (GPD)-derived descriptors into the feature space, adjacency matrix, and loss function to enhance its sensitivity to rare heatwave occurrences. By prioritizing the tails of climatic distributions, DI-GNN addresses the limitations of existing methods, particularly in imbalanced datasets where traditional metrics like accuracy are misleading. Empirical evaluations using weather station data from British Columbia, Canada, demonstrate the superior performance of DI-GNN compared to baseline models. DI-GNN achieved significant improvements in balanced accuracy, recall, and precision, with high AUC and average precision scores, reflecting its robustness in distinguishing heatwave events.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Barriopedro, D., Garc ´ ıa-Herrera, R., Ord´ o˜ nez, C., Miralles, D. G., & Salcedo-Sanz, S. (2023). Heat Waves: Physical Understanding and Scientific Challenges. Reviews of Geophysics, 61, e2022RG000780. https://doi.org/10.1029/2022RG000780
-
[2]
IPCC. (2021). Climate Change 2021: The Physical Science Basis. Cambridge University Press. Contribution of Working Group I to the Sixth Assessment Report of the IPCC. https://doi.org/10.1017/9781009157896
-
[3]
White, R. H., Anderson, S., Booth, J. F., Braich, G., Draeger, C., Fei, C., Harley, C. D. G., Henderson, S. B., Jakob, M., Lau, C.-A., Mareshet Admasu, L., Narinesingh, V., Rodell, C., Roocroft, E., Weinberger, K. R., & West, G. (2023). The unprecedented Pacific Northwest heatwave of June 2021. Nature Communications, 14, 727. https://doi.org/10.1038/s4146...
-
[4]
Thompson, V., Kennedy-Asser, A. T., Vosper, E., Lo, Y. T. E., Huntingford, C., Andrews, O., Mitchell, D., & Hawkins, E. (2022). The 2021 western North America heat wave among the most extreme events ever recorded globally. Science Advances, 8, eabm6860. https://doi.org/10.1126/sciadv.abm6860
-
[5]
Miralles, D. G., Teuling, A. J., van Heerwaarden, C. C., & Vil` a-Guerau de Arellano, J. (2014). Mega-heatwave temperatures due to combined soil desiccation and atmospheric heat accumulation. Nature Geoscience, 7, 345–349. https://doi.org/10.1038/ngeo2141
doi:10.1038/ngeo2141 2014
-
[6]
Findell, K. L., Knutson, T. R., & Milly, P. C. D. (2017). Impact of anthropogenic land use and land cover change on climate. Nature Climate Change, 7, 762–766. https://doi.org/10.1038/nclimate3312 20/23
-
[7]
Perkins, S. E. (2015). A review on the scientific understanding of heatwaves—Their measurement, driving mechanisms, and changes at the global scale. Atmospheric Research, 164–165, 242–267. https://doi.org/10.1016/j.atmosres.2015.05.014
-
[8]
M., Zscheischler, J., Wartenburger, R., Dee, D., & Seneviratne, S
Vogel, M. M., Zscheischler, J., Wartenburger, R., Dee, D., & Seneviratne, S. I. (2020). Concurrent 2018 hot extremes across Northern Hemisphere due to human-induced climate change. Earth’s Future, 8, e2020EF001731. https://doi.org/10.1029/2020EF001731
Show all 37 references
-
[9]
R., Bessemoulin, P., Ebi, K., & Menne, B
McGregor, G. R., Bessemoulin, P., Ebi, K., & Menne, B. (2015). Heatwaves and health: Guidance on warning-system development. World Meteorological Organization and World Health Organization. https://www.who.int/publications/i/item/9789241596324
2015
-
[10]
van Straaten, C., Buehler, T., & Seneviratne, S. I. (2022). Subseasonal drivers of European high temperatures at different time leads. Journal of Advances in Modeling Earth Systems, 14, e2021MS002942. https://doi.org/10.1029/2021MS002942
2022 doi
-
[11]
A., Durran, D
Weyn, J. A., Durran, D. R., Caruana, R., & Cresswell-Clay, N. (2021). Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models. Journal of Advances in Modeling Earth Systems, 13, e2021MS002502. https://doi.org/10.1029/2021MS002502
2021 doi
-
[12]
M., Pinto, J
Lopez-Gomez, J., Sousa, P. M., Pinto, J. G., Camus, P., Garc ´ ıa-Bustamante, E., & Trigo, R. M. (2022). Skillful prediction of subseasonal heat extremes. Geophysical Research Letters, 49, e2022GL098857. https://doi.org/10.1029/2022GL098857
2022 doi
-
[13]
D., Scher, S., Weyn, J
Rasp, S., Dueben, P. D., Scher, S., Weyn, J. A., Mouatadid, S., & Thuerey, N. (2020). WeatherBench: A benchmark dataset for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12, e2020MS002203. https://doi.org/10.1029/2020MS002203
2020 doi
-
[14]
Scher, S., & Messori, G. (2021). Ensemble methods for neural network-based weather forecasts. Journal of Advances in Modeling Earth Systems, 13, e2021MS002521. https://doi.org/10.1029/2021MS002521
2021 doi
-
[15]
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566, 195–204. https://doi.org/10.1038/s41586-019-0912-1
2019 doi
- [16]
- [17]
-
[18]
Chen, L., Zhong, X., Zhang, F., Cheng, Y., Xu, Y., Qi, Y., & Li, H. (2023). FuXi: A cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 6, 1, 190. https://doi.org/10.1038/s41612-023-00372-y 21/23
2023 doi
-
[19]
Chattopadhyay, A., Nabizadeh, E., & Hassanzadeh, P. (2020). Analog forecasting of extreme-causing weather patterns using deep learning. Journal of Advances in Modeling Earth Systems, 12, 2, e2019MS001705. https://doi.org/10.1029/2019MS001705
2020 doi
-
[20]
E., Wang, Z., Li, B., Harnos, D
Miller, D. E., Wang, Z., Li, B., Harnos, D. S., & Ford, T. (2021). Skillful subseasonal prediction of extreme warm days. Journal of Climate, 34, 17, 7263–7277. https://doi.org/10.1175/JCLI-D-20-0986.1
2021 doi
-
[21]
Guigma, K., Vrac, M., Naveau, P., & Yiou, P. (2021). Enhanced subseasonal forecasts of heatwaves through machine learning. Earth Systems and Environment, 5, 4, 771–784. https://doi.org/10.1007/s41748-021-00247-6
2021 doi
-
[22]
F., & Vitart, F
Quinting, J. F., & Vitart, F. (2017). Synoptic-scale Rossby wave packets and blocking in heatwave dynamics. Geophysical Research Letters, 44, 2, 936–943. https://doi.org/10.1002/2016GL071391
2017 doi
-
[23]
I., Donat, M
Seneviratne, S. I., Donat, M. G., Pitman, A. J., Knutti, R., & Wilby, R. L. (2012). Changes in climate extremes and their impacts. In Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation(pp. 109–230). Cambridge University Press. https://doi.o...
2012 doi
-
[24]
C., Hagenbuchner, M., & Monfardini, G
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009). The Graph Neural Network Model. IEEE Transactions on Neural Networks, 20, 1, 61–80. https://doi.org/10.1109/TNN.2008.2005605
2009
- [25]
- [26]
- [27]
-
[28]
Li, P., Wang, J., Wang, X., & Lee, K. (2023). Regional Heatwave Prediction Using Graph Neural Network and Weather Station Data. Geophysical Research Letters, 50, 2, e2023GL103405. https://doi.org/10.1029/2023GL103405
2023 doi
-
[29]
Coles, S. (2001). An Introduction to Statistical Modeling of Extreme Values. Springer. https://doi.org/10.1007/978-1-4471-3675-0
2001 doi
-
[30]
(1997).Modelling Extremal Events for Insurance and Finance
Embrechts, P., Kl¨ uppelberg, C., & Mikosch, T. (1997).Modelling Extremal Events for Insurance and Finance. Springer. https://doi.org/10.1007/978-3-642-33483-2
1997 doi
-
[31]
Zscheischler, J., Westra, S., van den Hurk, B. J. J. M., Seneviratne, S. I., Pitman, A. J., AghaKouchak, A., Leonard, M., Wahl, T., & Zhang, X. (2020). Understanding Compound Weather and Climate Events. Nature Reviews Earth & Environment, 1, 333–347. https://doi.org/10.1038/s4...
2020 doi
-
[32]
Bastos, A., O’Sullivan, M., Ciais, P., Makowski, D., Sitch, S., Friedlingstein, P., et al. (2020). Combined Effects of Climate Extremes and Land Cover Changes on the Carbon Cycle. Nature Geoscience, 13, 269-274. https://doi.org/10.1038/s41561-020-0582-7
2020 doi
-
[33]
J., Shaby, B
Reich, B. J., Shaby, B. A., & Cooley, D. (2014). A hierarchical model for serially-dependent extremes: A study of heat waves in the western US. Journal of Agricultural, Biological, and Environmental Statistics, 19, 2, 119–135. https://doi.org/10.1007/s13253-014-0160-4
2014 doi
-
[34]
McGregor, G. (2024). Heatwaves as Extreme Events. In Heatwaves: Causes, Consequences and Responses(pp. 81–133). Springer International Publishing. https://doi.org/10.1007/978-3-031-13012-6_4
2024 doi
-
[35]
Hu, Y., Jia, G., Gao, H., Li, Y., Hou, M., Li, J., & Miao, C. (2023). Spatial characterization of global heat waves using satellite-based land surface temperature. International Journal of Applied Earth Observation and Geoinformation, 125, 103604. https://doi.org/10.1016/j.jag...
2023
-
[36]
Zeppetello, L. R. V., Zhang, L. N., Battisti, D. S., & Lagu¨ e, M. M. (2024). How Much Does Land–Atmosphere Coupling Influence Summertime Temperature Variability in the Western United States?. Journal of Climate, 37, 13, 3457–3478. https://doi.org/10.1175/JCLI-D-23-0052.1
2024 doi
-
[37]
Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55, 10, 78–87. https://doi.org/10.1145/2347736.2347755 23/23
2012
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.