REVIEW 2 major objections 6 minor 31 references
HourGlass reconstructs the hours between 6-hourly data-driven weather forecasts as skillful, temporally coherent probabilistic trajectories with realistic small-scale structure.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-14 05:27 UTC pith:2BFBSEEA
load-bearing objection Solid operational methods paper: probabilistic CRPS temporal downscaling trained on continuous NWP trajectories, verified on independent SYNOP, with code released. the 2 major comments →
HourGlass: A probabilistic data-driven temporal downscaler for global and regional weather forecasting
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
A probabilistic temporal downscaler trained with almost-fair CRPS plus window-wise min/max/mean and difference terms on continuous NWP forecast segments can fill the hours between 6-hourly data-driven forecasts while preserving upstream skill, realistic small-scale spatial variability, and coherent physical evolution, rather than producing the temporally inconsistent smoothing of deterministic MSE-trained interpolators.
What carries the argument
Almost-fair CRPS on each output hour, augmented by the same score applied to min, max, mean and successive differences over the six-hour window (plus a regional spectral term), with shared latent noise so one forward pass yields a single coherent trajectory; residual connections force the final hour of prognostic fields to match the parent forecast.
Load-bearing premise
Continuous stretches of numerical weather prediction forecast trajectories teach the true physical hour-to-hour evolution without imprinting the parent model’s biases and spin-up so strongly that skill collapses when the downscaler is later applied to analysis-trained AI forecasts.
What would settle it
Hourly verification against surface stations would show larger errors or broken temporal-increment spectra inside each 6-hour window relative to cubic-spline interpolation of the same parent forecast, or storm case studies would exhibit unphysical jumps or loss of frontal structure at mid-window.
If this is right
- Existing 6-hourly data-driven forecast systems can supply operational hourly products without being retrained for hourly steps.
- Skill gains that come from analysis-trained parent models (for example better 2 m temperature) can be carried into the intermediate hours even though the downscaler itself never saw analyses as targets.
- Deterministic or MSE-trained temporal interpolators become inferior defaults whenever small-scale spatial realism and coherent storm timing matter.
- Each ensemble member can be downscaled independently, preserving a probabilistic hourly product at modest extra cost.
- Improvements to the 6-hourly parent forecasts, especially of intensity extremes, are expected to translate directly into better hourly fields.
Where Pith is reading between the lines
- The same boundary-anchored residual design should generalise to other fixed temporal gaps (for example 3-hourly to hourly) provided the parent anchors remain skilful.
- Diagnostic precipitation’s over-confidence near the window edges suggests multi-scale or boundary-aware probabilistic losses will be needed before hourly rain products match operational needs.
- Training only on post-spin-up forecast segments may systematically under-represent the first hours after analysis, limiting direct use for nowcasting-style applications.
- Once upstream AI models reduce dry bias and raise extreme rain rates, HourGlass should inherit those gains at hourly resolution with little additional training cost.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces HourGlass, a probabilistic GNN-based temporal downscaler that reconstructs intermediate hourly states between two 6-hourly forecast anchors. Models are trained with almost-fair CRPS plus min/max/mean/difference aggregate terms (and a regional spectral term for Bris) on continuous NWP forecast segments (IFS and MEPS) rather than reanalysis, to avoid assimilation-window jumps. Two applications are presented: global AIFS-HourGlass (applied to AIFS-Single and AIFS-ENS) and regional stretched-grid Bris-HourGlass. Verification against independent 2025 SYNOP stations, spectral diagnostics of fields and temporal increments, loss ablations, cubic-spline and NWP baselines, and case studies of Storm Amy and US Southern Plains convection support the claim that skill of the upstream 6-hourly systems is retained while producing temporally coherent hourly trajectories with improved small-scale spatial variability. Precipitation extremes remain underestimated, which the authors acknowledge as a shared data-driven limitation.
Significance. If the results hold, HourGlass supplies a practical, operationally relevant bridge from the dominant 6-hourly data-driven forecast paradigm to the hourly products required by many applications, without the error accumulation of direct hourly forecasting or the smoothing of deterministic temporal interpolators. Strengths include independent SYNOP verification for 2025, spectral analysis of increments (Fig. 10), explicit loss ablations (Figs. 6–8), residual connections that leave 6-hour anchors unchanged, training on forecast trajectories to avoid ERA5 jumpiness (Figs. 4–5), dual global/regional demonstration, and open code in the Anemoi framework. These elements make the contribution concrete and reusable rather than purely methodological.
major comments (2)
- §3.2 and Fig. 14: When Bris-HourGlass is driven by Bris (or by MEPS analysis), mean hourly precipitation exhibits clear mid-window relaxation toward the training distribution and a dry bias inherited from the upstream model near the 6-hour anchors. The paper correctly notes that 6-hourly accumulated precipitation CRPS improves, but the residual boundary dependence and spin-up imprint remain load-bearing for the claim of seamless hourly products. A short quantitative summary of how much of the mid-window skill gain is pure spread versus genuine error reduction (already hinted at by Fig. 13) would strengthen the operational interpretation.
- §2.2–2.3 and Fig. 5: The decision to train exclusively on IFS/MEPS forecast segments (lead times 12–29 h / 6–23 h) is well motivated by ERA5 jumpiness, yet the manuscript does not fully quantify how much parent-NWP bias or spin-up structure is transferred into the downscaler when it is later applied to analysis-trained AI forecasts. Fig. 5 shows comparable RMSE for ERA5-only vs IFS-only training on a few variables, but a parallel diagnostic for precipitation intensity distributions (analogous to Fig. 8) and for the regional model would make the weakest assumption more transparent.
minor comments (6)
- §2.1: The switch from a pure transformer processor (prior AIFS) to a graph transformer is stated but not motivated; a sentence on why this choice was made for multi-time-step decoding would help.
- Table 1: Asterisks and daggers for diagnostic vs prognostic variables and for global-only fields are dense; a short legend or footnote clarifying which variables are residual-connected would improve readability.
- Fig. 2 caption: “Hours 4-6 share the same random noise” is useful; stating explicitly that the same noise seed is used across the ablation would make the visual comparison clearer.
- §3.1: The small RMSE jumps attributed to diurnal observation density are plausible; a brief note on the number of stations per hour (or a supplementary plot) would remove residual ambiguity.
- Code availability: The GitHub link points to a feature branch; a commit hash or release tag would improve long-term reproducibility.
- Typographical: “availabilty” in the Code availability heading; “Code availabilty” should be corrected.
Circularity Check
No significant circularity; skill-preservation and coherence claims rest on external SYNOP/MRMS verification, with residual anchors left unchanged by design and openly discounted.
full rationale
HourGlass is trained on continuous NWP forecast segments (IFS lead times 12–29 h; MEPS 6–23 h) with almost-fair CRPS plus min/max/mean/difference aggregate terms, then applied to independent AIFS/Bris 6-hourly states and scored against SYNOP stations (2025) and MRMS QPE. The residual skip connection (Sec. 2.1) forces prognostic fields at the t6 anchor to equal the upstream forecast by construction, so 6-hourly skill is preserved tautologically; the authors explicitly state that “the scores at every 6th hour are not pertinent” and evaluate only the intermediate hours, increment spectra (Fig. 10), 24 h extrema (Fig. 15), and case-study evolution. Self-citations supply the upstream models (AIFS, Bris) and the CRPS recipe; they are not uniqueness theorems that force the downscaler result. No parameter fitted to one quantity is re-labelled a prediction of a closely related quantity, and precipitation biases inherited from the training distribution are documented rather than hidden (Fig. 14, Sec. 3.2). The central claims therefore stand on external benchmarks and are not circular.
Axiom & Free-Parameter Ledger
free parameters (4)
- almost-fair CRPS epsilon =
0.025
- spectral loss weight lambda_f (Bris) =
0.15
- learning rates and iteration counts =
various (see §2.4)
- variable tendency weights w_i = 1/sigma(Delta x_i) =
data-derived
axioms (3)
- domain assumption Continuous segments of operational NWP forecasts (IFS, MEPS) provide temporally consistent and sufficiently realistic hourly targets for learning physical evolution.
- ad hoc to paper Almost-fair CRPS plus min/max/mean/difference aggregates is an adequate surrogate for temporal coherence and small-scale spatial variability.
- domain assumption A residual connection that forces the final (t+6) prognostic state to equal the input forecaster state is valid and does not harm intermediate-hour skill.
invented entities (1)
-
HourGlass temporal downscaler (AIFS-HourGlass / Bris-HourGlass)
no independent evidence
read the original abstract
Many forecast applications require high frequency temporal resolution, yet most state-of-the-art data-driven weather forecasting systems operate at 6-hourly resolution. Although direct hourly forecasting is possible, it suffers from error accumulation and temporal inconsistency. We introduce HourGlass, a probabilistic data-driven temporal downscaling method that reconstructs the evolution between forecast states. HourGlass is trained using variants of the continuous ranked probability score (CRPS) preserving small-scale spatial variability while encouraging temporal consistency. Unlike existing deterministic temporal downscaling approaches, which tend to produce overly smooth fields, HourGlass generates realistic probabilistic forecasts. Training on forecast trajectories rather than reanalysis or analysis data also avoids the temporal inconsistencies present in datasets used by previous methods. We evaluate HourGlass in two settings: AIFS-HourGlass, applied globally to ECMWF's AIFS-Single and AIFS-ENS forecast systems, and Bris-HourGlass, applied regionally to MET Norway's high-resolution stretched-grid ensemble model, Bris. Verification against observations shows that both models retain the skill of their underlying forecasting systems while producing temporally coherent hourly forecasts with realistic small-scale variability. Case studies demonstrate physically consistent evolution during rapidly developing weather events, including extratropical cyclones and organised convection. Hourly precipitation remains challenging: HourGlass improves the spatial realism of precipitation fields but still underestimates the most intense extremes, a common limitation of data-driven weather forecasting models. These results demonstrate that HourGlass effectively bridges the gap between 6-hourly data-driven forecasts and the hourly products required for operational regional and global forecasting.
Figures
Reference graph
Works this paper leans on
-
[1]
Machine learning for precipitation nowcasting from radar im- ages, 2019
Shreya Agrawal, Luke Barrington, Carla Bromberg, John Burge, Cenk Gazen, and Jason Hickey. Machine learning for precipitation nowcasting from radar im- ages, 2019. URL https://arxiv.org/abs/1912 .12132
2019
-
[2]
The rise of data-driven weather forecasting: A first statisti- cal assessment of machine learning–based weather forecasts in an operational-like context
Zied Ben Bouallègue, Mariana CA Clare, Linus Mag- nusson, Estibaliz Gascon, Michael Maier-Gerber, Martin Janoušek, Mark Rodwell, Florian Pinault, Jes- per S Dramsch, Simon TK Lang, et al. The rise of data-driven weather forecasting: A first statisti- cal assessment of machine learning–based weather forecasts in an operational-like context. Bulletin of the...
2024
-
[3]
Pangu-weather: A 3d high- resolution model for fast and accurate global weather forecast, 2022
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Pangu-weather: A 3d high- resolution model for fast and accurate global weather forecast, 2022. URL http://arxiv.org/abs/22 11.02556
2022
-
[4]
FuXi: a cascade machine learning forecasting system for 15-day global weather forecast
Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. FuXi: a cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science, 6(1):190, 2023. ISSN 2397-
2023
-
[5]
URL http s://www.nature.com/articles/s41612-023 -00512-1
doi:10.1038/s41612-023-00512-1. URL http s://www.nature.com/articles/s41612-023 -00512-1
-
[6]
URL https://www.ecmwf.int/ en/about/media-centre/news/2024/anemoi -new-framework-weather-forecasting-bas ed-machine-learning
ECMWF, 2024. URL https://www.ecmwf.int/ en/about/media-centre/news/2024/anemoi -new-framework-weather-forecasting-bas ed-machine-learning
2024
-
[7]
IFS Documentation CY49R1 - Part III: Dynamics and Numerical Procedures, chapter 3
ECMWF. IFS Documentation CY49R1 - Part III: Dynamics and Numerical Procedures, chapter 3. ECMWF, 2024. doi:10.21957/D04FB7A27E. URL https://www.ecmwf.int/bibcite/referenc e/81625
-
[8]
IFS Documentation CY49R1 - Part IV: Physical Processes, chapter 4
ECMWF. IFS Documentation CY49R1 - Part IV: Physical Processes, chapter 4. ECMWF, 2024. doi:10.21957/C731EE1102. URL https://www. ecmwf.int/bibcite/reference/81626
-
[9]
The operational forecast process at metcoop
Reima Eresmaa, Ulf Andrae, Lars Berggren, John Bjørnar Bremnes, Carl Fortelius, Inger-Lise Frogner, Erik Gregow, Rafael Grote, Martin Ridal, Ole Vignes, Ivar Ansper, Roohollah Azad, Jevg¯enijs Begens, Lisbeth Bergholt, Jostein Blyverket, Laila Daniel, Bjørg Jenny Engdahl, Helene Erlandsen, Mehdi Eshagh, Mikael Hasu, Mariken Homleid, Petter Ideström, Karl-...
-
[10]
HarmonEPS—The HARMONIE En- semble Prediction System
Inger-Lise Frogner, Ulf Andrae, Jelena Bojarova, Al- fons Callado, Pau Escribà, Henrik Feddersen, Alan Hally, Janne Kauhanen, Roger Randriamampianina, Andrew Singleton, Geert Smet, Sibbo van der Veen, and Ole Vignes. HarmonEPS—The HARMONIE En- semble Prediction System. Weather and Forecasting, 34(6):1909–1937, December 2019. ISSN 1520-0434, 0882-8156. doi...
-
[11]
Inger-Lise Frogner, Andrew T. Singleton, Morten Ø. Køltzow, and Ulf Andrae. Convection-permitting ensembles: Challenges related to their design and use. Quarterly Journal of the Royal Meteorological Society, 145(S1):90–106, September 2019. ISSN 0035-9009, 1477-870X. doi:10.1002/qj.3525. URL https://rmets.onlinelibrary.wiley.com/ doi/10.1002/qj.3525
-
[12]
Joseph G. Galway. The lifted index as a predic- tor of latent instability. Bulletin of the American Meteorological Society, 37(10):528–529, 1956. doi:10.1175/1520-0477-37.10.528
-
[13]
The ERA5 global reanaly- sis
Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Di- nand Schepers, et al. The ERA5 global reanaly- sis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999–2049, 2020
1999
-
[14]
Forecasting Global Weather with Graph Neural Networks, February 2022
Ryan Keisler. Forecasting Global Weather with Graph Neural Networks, February 2022. URL h t tp : / / a r x i v . o r g / a b s / 2 2 0 2 . 0 7 5 75. arXiv:2202.07575 [physics]
Pith/arXiv arXiv 2022
-
[15]
Graphcast: Learning skillful medium- range global weather forecasting, 2023
Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Will- son, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton- Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Pe- ter Battaglia. Graphcast: Learning skillful medium- range global weather f...
Pith/arXiv arXiv 2023
-
[16]
Simon Lang, Mihai Alexe, Matthew Chantry, Jesper Dramsch, Florian Pinault, Baudouin Raoult, Mariana C. A. Clare, Christian Lessig, Michael Maier-Gerber, Linus Magnusson, Zied Ben Bouallègue, Ana Pri- eto Nemesio, Peter D. Dueben, Andrew Brown, Florian Pappenberger, and Florence Rabier. AIFS - ECMWF’s data-driven forecasting system, 2024. URLhttp://arxiv.o...
Pith/arXiv arXiv 2024
-
[17]
Simon Lang, Martin Leutbecher, and Pedro Maciel. A multi-scale loss formulation for learning a proba- bilistic model with proper score optimisation, 2025. URLhttp://arxiv.org/abs/2506.10868
Pith/arXiv arXiv 2025
-
[18]
Aifs-crps: en- semble forecasting using a model trained with a loss function based on the continuous ranked probability score
Simon Lang, Mihai Alexe, Mariana CA Clare, Christopher Roberts, Rilwan Adewoyin, Zied Ben Bouallègue, Matthew Chantry, Jesper Dramsch, Peter D Dueben, Sara Hahner, et al. Aifs-crps: en- semble forecasting using a model trained with a loss function based on the continuous ranked probability score. npj Artificial Intelligence, 2(1):18, 2026
2026
-
[19]
Modulated adaptive fourier neural oper- ators for temporal interpolation of weather forecasts,
Jussi Leinonen, Boris Bonev, Thorsten Kurth, and Yair Cohen. Modulated adaptive fourier neural oper- ators for temporal interpolation of weather forecasts,
-
[20]
URL https://arxiv.org/abs/2410.189
-
[21]
arXiv:2410.18904 [physics.ao-ph]
-
[22]
An update to ECMWF’s machine-learned weather forecast model AIFS, 2025
Gabriel Moldovan, Ewan Pinnington, Ana Prieto Nemesio, Simon Lang, Zied Ben Bouallègue, Jes- per Dramsch, Mihai Alexe, Mario Santa Cruz, Sara Hahner, Harrison Cook, Helen Theissen, Mariana Clare, Cathal O’Brien, Jan Polster, Linus Magnus- son, Gert Mertes, Florian Pinault, Baudouin Raoult, Patricia de Rosnay, Richard Forbes, and Matthew Chantry. An update...
arXiv 2025
-
[23]
Re- gional data-driven weather modeling with a global stretched-grid
Thomas Nils Nipen, Håvard Homleid Haugen, Magnus Sikora Ingstad, Even Marius Nordha- gen, Aram Farhad Shafiq Salihi, Paulina Tedesco, Ivar Ambjørn Seierstad, Jørn Kristiansen, Simon Lang, Mihai Alexe, Jesper Dramsch, Baudouin Raoult, Gert Mertes, and Matthew Chantry. Re- gional data-driven weather modeling with a global stretched-grid. Artificial Intellig...
2025
-
[24]
URL https: //journals.ametsoc.org/view/journals/a ies/aop/AIES-D-25-0001.1/AIES-D-25-000 1.1.xml
doi:10.1175/AIES-D-25-0001.1. URL https: //journals.ametsoc.org/view/journals/a ies/aop/AIES-D-25-0001.1/AIES-D-25-000 1.1.xml. Publisher: American Meteorological So- ciety Section: Artificial Intelligence for the Earth Systems
-
[25]
High-resolution probabilistic data- driven weather modeling with a stretched-grid
Even Marius Nordhagen, Håvard Homleid Haugen, Aram Farhad Shafiq Salihi, Magnus Sikora Ingstad, Thomas Nils Nipen, Ivar Ambjørn Seierstad, Inger- Lise Frogner, Mariana Clare, Simon Lang, Matthew Chantry, et al. High-resolution probabilistic data- driven weather modeling with a stretched-grid. arXiv preprint arXiv:2511.23043, 2025
arXiv 2025
-
[26]
Graph-based neural weather prediction for lim- ited area modeling, 2023
Joel Oskarsson, Tomas Landelius, and Fredrik Lind- sten. Graph-based neural weather prediction for lim- ited area modeling, 2023. URL http://arxiv.or g/abs/2309.17370
Pith/arXiv arXiv 2023
-
[27]
Jaideep Pathak, Shashank Subramanian, Peter Har- rington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassan- zadeh, Karthik Kashinath, and Animashree Anand- kumar. FourCastNet: A global data-driven high- resolution weather model using adaptive fourier neu- 19 HourGlass: A prob...
Pith/arXiv arXiv 2022
-
[28]
Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Mas- ters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. GenCast: Diffusion-based ensemble forecasting for medium-range weather, 2024. URL http://arxiv. org/abs/2312.15796
Pith/arXiv arXiv 2024
-
[29]
Piet Termonia, Claude Fischer, Eric Bazile, François Bouyssel, Radmila Brožková, Pierre Bénard, Bogdan Bochenek, Daan Degrauwe, Mariá Derková, Ryad El Khatib, Rafiq Hamdi, Ján Mašek, Patricia Pottier, Neva Pristov, Yann Seity, Petra Smolíková, Oldˇrich Španiel, Martina Tudor, Yong Wang, Christoph Wittmann, and Alain Joly. The ALADIN System and its canonic...
-
[30]
Jian Zhang, Kenneth Howard, Carrie Langston, Brian Kaney, Youcun Qi, Lin Tang, Heather Grams, Yadong Wang, Stephen Cocks, Steven Martinaitis, Ami Arthur, Karen Cooper, Jeff Brogden, and David Kitzmiller. Multi-radar multi-sensor (mrms) quanti- tative precipitation estimation: Initial operating ca- pabilities. Bulletin of the American Meteorological Societ...
doi:10.1175/bams- 2016
-
[31]
FuXi-2.0: Advancing machine learning weather forecasting model for practical ap- plications, 2024
Xiaohui Zhong, Lei Chen, Xu Fan, Wenxu Qian, Jun Liu, and Hao Li. FuXi-2.0: Advancing machine learning weather forecasting model for practical ap- plications, 2024. URL http://arxiv.org/abs/ 2409.07188. 20
Pith/arXiv arXiv 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.