Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Probabilistic measures afford fair comparisons of AIWP and NWP model output

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces the potential continuous ranked probability score (PC), a loss-function-free measure built by post-processing deterministic forecasts with isotonic distributional regression, and reports that under PC the data-driven…

desk verdict A clear, honest proposal for a potential-score benchmark, but the headline GraphCast result needs uncertainty bounds and the optimism-parity assumption should be tested. read the letter →

arxiv 2506.03744 v1 pith:HOD4CGN4 submitted 2025-06-04 stat.AP physics.ao-phstat.ML

Tilmann Gneiting , Tobias Biegert , Kristof Kraus , Eva-Maria Walz , Alexander I. Jordan , Sebastian Lerch This is my paper · ORCID
classification stat.APphysics.ao-phstat.ML MSC 62M2062G0562G0862P12 PACS 92.60.Ry
keywords potentialCRPSisotonicdistributionalregressionEasyUQforecastevaluationAIWPNWPWeatherBench2continuousrankedprobabilityscore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces the potential continuous ranked probability score ($\mathrm{PC}$), a measure meant to make single-valued forecasts from AI-based and physics-based weather models comparable without choosing a loss function. The trick is to post-process every deterministic forecast in the evaluation set with the same automated method, isotonic distributional regression, and take the mean continuous ranked probability score of the resulting probabilistic forecasts. The paper argues this yields a fair, loss-independent yardstick: $\mathrm{PC}$ is invariant under strictly increasing transformations of model output, is measured in the units of the weather variable, and is zero exactly when the outcome is a fixed non-decreasing function of the forecast. On WeatherBench 2 data, the paper finds that GraphCast dominates Pangu-Weather and ECMWF's high-resolution model under $\mathrm{PC}$ for all three studied variables and all lead times, and that HRES's $\mathrm{PC}$ closely tracks the operational ECMWF ensemble's mean CRPS. If the measure is accepted, it changes how AIWP and NWP backbones are benchmarked, removing the objection that AI models are tuned to the metric being reported.

What carries the argument

The load-bearing object is the EasyUQ solution, a special case of isotonic distributional regression: given paired model output and outcomes, it finds the unique predictive distributions that minimize the average CRPS subject to the quantile functions being nondecreasing in model output. Pool-adjacent-violators algorithms compute this solution in about $O(n \log n)$ operations, and the $\mathrm{PC}$ score is then the mean CRPS of these fitted distributions. This machinery gives $\mathrm{PC}$ its invariance under strictly increasing transformations, its zero-in-the-perfect-monotone-predictor property, and its interpretation as a slightly optimistic proxy for a real-time postprocessed probabilistic forecast.

What would settle it

Fit PC on data where the outcome is a strictly decreasing function of the model output. The score will be positive even though the forecast is perfect, because isotonicity forbids decreasing relationships; that outcome would refute any claim that PC fairly compares arbitrary model encodings.

Watch

Extended reading notes

Core claim

The paper's central claim is that the $\mathrm{PC}$ measure, defined as the in-sample mean CRPS of the EasyUQ/IDR postprocessed predictive distribution, gives a principled and fair way to compare deterministic forecast output across models trained under different objectives. Under this measure, the paper reports, GraphCast outperforms both Pangu-Weather and ECMWF HRES for mean sea level pressure, 2 m temperature, and 10 m wind speed at lead times from one to ten days in WeatherBench 2 when all models are evaluated against the IFS analysis; Pangu-Weather generally beats HRES for pressure and wind, while HRES beats Pangu-Weather for temperature. The paper also claims that HRES's $\mathrm{PC}$ is a close proxy for the mean CRPS of the operational ECMWF ensemble, with slight optimism at one day and pessimism at longer leads. The measure achieves these properties by construction: $\mathrm{PC}$ is invariant under strictly increasing transformations of model output, equals zero if and only if the outcome is a non-decreasing function of the forecast, and is bounded above by the in-sample climatological CRPS.

Load-bearing premise

The whole comparison depends on the assumption that all useful predictive information in a deterministic forecast can be extracted by a monotone transformation; if a model encodes information non-monotonically, PC understates its potential and the comparison is no longer fair.

Editorial extensions

If this is right

  • Benchmarks of deterministic weather forecasts can be run without pre-specifying a loss function, because PC treats every model's output through the same post-processing lens.
  • AI models lose the advantage of having been trained on the exact RMSE or other metric used for evaluation, since PC is defined by the evaluation data alone.
  • PC values for an operational deterministic backbone can serve as a check on the expected skill of the operational ensemble, flagging lead times where the ensemble adds little.
  • Because PCS is a normalized skill score with a meaningful climatology baseline, forecasters can compare model families across variables and lead times on a common 0-to-1 scale.
  • The invariance property means ranking by PC is stable under monotone rescaling of forecasts, so unit changes or calibrations of model output do not distort comparisons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the isotonicity constraint means PC actually measures monotone information content; a non-monotone model would be undervalued, so PC is best read as a lower bound on potential skill under monotone post-processing.
  • Inference: the same construction transfers to other single-valued forecast settings where competitors train under different loss functions, such as energy load, finance, or epidemiology, whenever an evaluation set of a few hundred to a few thousand paired forecasts and outcomes is available.
  • Inference: replacing CRPS in the defining criterion with threshold- or quantile-weighted CRPS would yield a tail-focused version of PC aimed at extreme weather, a direction the paper sketches but does not implement.
  • Inference: a directed stress test would invert a skilful model's output, making the forecast-outcome relation strictly decreasing; a fully fair skill measure should still credit it, while PC would not unless the ordering is restored.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces the potential continuous ranked probability score (PC), defined as the mean CRPS of the EasyUQ/IDR post-hoc probabilistic forecast fitted on the evaluation data itself, together with a skill score version PCS. The authors establish basic properties (nonnegativity, zero characterization, invariance under strictly increasing transformations of the model output, upper bound given by the climatological CRPS), illustrate them in a simulation, and apply the measures to WeatherBench 1 and WeatherBench 2 data. They report that in WeatherBench 2, GraphCast dominates Pangu-Weather and ECMWF HRES for all considered variables and lead times when evaluated with the IFS analysis as ground truth, and that PC for HRES aligns closely with the mean CRPS of the operational ECMWF ensemble. The paper argues that PC provides a fair, loss-function-independent comparison of deterministic AIWP and NWP backbones.

Significance. The proposal is attractive and original in its simplicity: applying the same CRPS-optimal isotonic postprocessing to every deterministic backbone removes the need to pre-specify a scoring function and addresses the unequal footing that arises when AI models are trained on the evaluation metric. The mathematical properties follow cleanly from established isotonic distributional regression theory, the computational cost is modest, and the authors provide code and replication material. The external check against the operational ensemble CRPS is a useful validation. The main risk is that PC is an in-sample fit of the evaluation objective, so its fairness across models depends on the optimism being comparable across models, which is neither shown nor tested. If this issue is resolved, the measure could be a valuable benchmark tool for the community.

major comments (4)
  1. [Section 2.1, Eq. (4) and Section 3.2, Figure 2] The fairness claim rests on the assumption that the in-sample optimism of PC is comparable across models, but this is not demonstrated. PC is defined as the minimized in-sample mean CRPS of the EasyUQ fit, which is an optimistically biased estimate of the expected out-of-sample CRPS. The bias depends on n, the distribution and tie structure of the model output, and the signal-to-noise relationship; GraphCast, Pangu-Weather, and HRES outputs differ in these respects. The paper acknowledges that PC is "a slightly optimistic proxy" but provides no quantification or cross-model evidence of equal optimism. Because Figure 2 is entirely in-sample and lacks uncertainty quantification, the headline ranking could be partly an artifact of differential in-sample adaptability. A split-sample or cross-validated variant of PC (fit on training data, evaluate on holdout), or a bound/analysis of the optimism differences, is needed to support the "fair comparisons" claim. Alternatively, the claims should be weakened to comparisons of in-sample potential under the fitted isotonic model, rather than fair assessment of expected predictive ability.
  2. [Section 2.2, property (b) and the definition of EasyUQ in Section 2] The isotonicity constraint is a load-bearing assumption for the interpretation of PC as a measure of potential predictive ability. Property (b) shows that PC = 0 only when the outcome is a nondecreasing function of the model output, and the EasyUQ fit itself is constrained to be stochastically increasing in the model output. If a model encodes information non-monotonically (e.g., a change in sign or a U-shaped relationship), PC will understate that model's potential, and the comparison is no longer fair in the sense claimed in the abstract and Section 1. Since the paper explicitly targets "across application domains," the scope of the fairness claim should be stated as applying only when isotonicity holds, or the paper should provide evidence that the variables and lead times considered satisfy this condition. This is not an optional caveat; it defines what PC measures.
  3. [Section 3.2, Figure 2 and Figure 4] The central empirical result that GraphCast dominates both Pangu-Weather and HRES for all considered variables and lead times is presented without any uncertainty quantification. The permutation test introduced in Section 3.2 and displayed in Figure 4 is applied to the comparisons in Figure 3 (ERA5 vs IFS ground truth) but not to the ranking in Figure 2. Since the differences in PC between models may be small at many grid points, and since PC itself is in-sample, the reader cannot assess whether the all-dominance pattern is statistically meaningful. I suggest adding block-bootstrap confidence intervals for the latitude-weighted PC differences or a permutation test for the Figure 2 comparisons, following the paper's own method.
  4. [Section 3.3, Figure 7] The proxy claim for the operational ensemble CRPS is not fully supported by the evidence presented. The paper does not state how the operational CRPS values in Figure 7 were obtained (which dataset, what ground truth or observations, what period), nor does it explain the grid-point matching. The heuristic that optimistic and pessimistic effects "balance each other" is plausible but is only tested for HRES, not for AIWP models. Please specify the data source and computation of the operational CRPS, and consider showing similar scatterplots for GraphCast and Pangu-Weather if such data are available; otherwise, soften the claim to a case study for HRES.
minor comments (4)
  1. [Section 3.2] There are several typos: "ground thruth" should be "ground truth," and "PC-ERA5" in the sentence comparing HRES, GC-ERA5, and PC-ERA5 should presumably be "PW-ERA5." Also, Section 2.1 has "addresss" for "addresses."
  2. [Footnote 2 of Section 3.2] The block permutation procedure is described very tersely; please clarify how p-values are computed at the grid-point level, especially with respect to the two daily runs and the handling of the block length of 2k.
  3. [Section 2.1] The paragraph that calls PC a "slightly optimistic proxy" and then a "pessimistic proxy" is confusing; the two directions of bias refer to different baselines (in-sample fitting versus operational product sophistication) and should be stated more explicitly and consistently.
  4. [Section 3.2, Figure 2] The latitude-weighted PC values are spatially and temporally correlated, yet the figure shows no measure of uncertainty; even a brief statement about the dependence structure and its implications would be helpful.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: PC is an explicitly in-sample potential measure; its operational-proxy claim is checked against independent ECMWF ensemble CRPS data.

full rationale

The paper defines PC directly as the in-sample mean CRPS of the EasyUQ/IDR solution fitted on the evaluation data (Section 2.1, eq. (4)), and it explicitly acknowledges that PC is both the target criterion of the constrained minimization and 'a slightly optimistic proxy' for real-time CRPS. The measure is therefore not a hidden fitted quantity masquerading as an out-of-sample forecast; it is presented and interpreted as a measure of potential predictive ability. The headline comparison in Section 3.2 applies the identical postprocessing protocol to every model and reports the resulting PC values, so the 'fair comparison' claim rests on equal treatment rather than on a concealed prediction. The proxy claim in Section 3.3 is validated against the operational ECMWF ensemble's CRPS in Figure 7, which is independent external evidence and does not reduce to the fitted EasyUQ values. Citations to Henzi et al. (2021) and Walz et al. (2024a) are to peer-reviewed prior work whose stated assumptions do not include the present target result, so self-citation is not load-bearing. The concern that in-sample optimism may differ across models is a legitimate statistical bias or correctness issue, but it is not circularity, because the paper does not claim that PC is an unbiased out-of-sample score. Overall, the derivation is self-contained with respect to its stated definition and the external proxy check, and no circular step can be exhibited from the text.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free scalar parameters are fitted; the only fitted objects are the nonparametric isotonic CDFs produced by EasyUQ, which are the output of an optimization algorithm rather than hand-tuned constants. The main axioms are the isotonicity constraint on the relationship between model output and outcome, the representativeness of the 2020 evaluation period, standard CRPS theory, and the block-permutation testing assumption.

assumptions (4)
  • domain assumption Isotonicity: the relationship between each model's deterministic output and the weather outcome is monotone, so that the IDR/EasyUQ constraint is appropriate.
    Invoked in Section 2 when defining EasyUQ as the constrained minimizer of mean CRPS under isotonicity. If violated, PC understates potential performance for that model.
  • domain assumption Representativeness: the 2020 WeatherBench 2 evaluation period is representative of the models' operational behavior, with no significant distribution shift.
    The empirical rankings and proxy validation in Sections 3.2 and 3.3 are computed on this single year.
  • standard math CRPS representation and scoring-rule theory as in Gneiting and Raftery (2007).
    Used in eqs. (1) and (2) and in the derivation of PC(0) as half the Gini mean difference.
  • domain assumption Validity of the block permutation test: temporal dependence in score differences is captured by blocks of length 2k, and the test statistic distribution under the null is exchangeable under block sign flips.
    Section 3.2, footnote 2. If dependence extends beyond 2k, the p-values could be anti-conservative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Probabilistic measures afford fair comparisons of AIWP and NWP model output." pith.science (2026). https://pith.science/paper/HOD4CGN4

@misc{pith2026250603744,
  author       = {Pith},
  title        = {Pith review of: Probabilistic measures afford fair comparisons of AIWP and NWP model output},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HOD4CGN4}},
  note         = {Machine review of arXiv:2506.03744}
}
read the original abstract

We introduce a new measure for fair and meaningful comparisons of single-valued output from artificial intelligence based weather prediction (AIWP) and numerical weather prediction (NWP) models, called potential continuous ranked probability score (PC). In a nutshell, we subject the deterministic backbone of physics-based and data-driven models post hoc to the same statistical postprocessing technique, namely, isotonic distributional regression (IDR). Then we find PC as the mean continuous ranked probability score (CRPS) of the postprocessed probabilistic forecasts. The nonnegative PC measure quantifies potential predictive performance and is invariant under strictly increasing transformations of the model output. PC attains its most desirable value of zero if, and only if, the weather outcome Y is a fixed, non-decreasing function of the model output X. The PC measure is recorded in the unit of the outcome, has an upper bound of one half times the mean absolute difference between outcomes, and serves as a proxy for the mean CRPS of real-time, operational probabilistic products. When applied to WeatherBench 2 data, our approach demonstrates that the data-driven GraphCast model outperforms the leading, physics-based European Centre for Medium Range Weather Forecasts (ECMWF) high-resolution (HRES) model. Furthermore, the PC measure for the HRES model aligns exceptionally well with the mean CRPS of the operational ECMWF ensemble. Across application domains, our approach affords comparisons of single-valued forecasts in settings where the pre-specification of a loss function -- which is the usual, and principally superior, procedure in forecast contests, administrative, and benchmarks settings -- places competitors on unequal footings.

Figures

Figures reproduced from arXiv: 2506.03744 by the authors.

Figure 1
Figure 1. Predictive ability of WeatherBench 1 (Rasp et al., 2020) models three days ahead for 850 hPa temperature in 2017 and 2018 as a function of latitude in terms of RMSE, PC, ACC, PCS, and CPA, respectively. The acronyms HRES, T63, and T42 represent physics-based NWP models run at decreasing grid resolution, which are compared to purely data-driven, neural network based (CNN), linear regression based (LR), and persistenc… view at source ↗
Figure 2
Figure 2. Potential predictive ability of HRES, GC-IFS, and PW-IFS model output from WeatherBench 2 (Rasp et al., 2024) for mean sea level pressure (MSLP), 2 m temperature (T2M), and 10 m wind speed (WS10) at lead times of one, three, five, seven, and ten days in terms of latitude-weighted PC, with the IFS analysis serving as ground truth. and evaluated against, the ERA5 analysis (Hersbach et al., 2020) as ground truth, where… view at source ↗
Figure 3
Figure 3. Potential predictive ability of HRES, GC-ERA5, and PW-ERA5 model output from Weath￾erBench 2 (Rasp et al., 2024) for mean sea level pressure (MSLP), 2 m temperature (T2M), and 10 m wind speed (WS10) at lead times of one, three, five, seven, and ten days in terms of latitude-weighted PC, with either ERA5 (top panel) or the IFS analysis (bottom panel) serving as ground truth. For reference, we also show the latitude-w… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Statistical significance of results from [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Scatterplot of latitude-weighted PCS of (left column) GC-ERA5 versus GC-IFS and (right column) PW-ERA5 versus PW-IFS with (upper row) the IFS analysis and (lower row) the ERA5 analysis serving as ground truth. For each variable, the points represent lead times of one, …
Figure 6
Figure 6. Figure 6: Grid point level potential predictive ability of GC-ERA5 for mean sea level pressure (MSLP), 2 m temperature (T2M), and 10 m wind speed (WS10) at lead times of one, three, five, seven, and ten days in terms of PC skill relative to the ERA5 climatology forecast from Wea…
Figure 7
Figure 7. Figure 7: Scatterplots of the mean CRPS for the operational ECMWF ensemble versus PC for IFS HRES model output, for mean sea level pressure (MSLP), 2 m temperature (T2M), and 10 m wind speed (WS10) at lead times of one, three, five, seven, and ten days, with the IFS analysis ser…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. "What is a realistic forecast?" Assessing data-driven weather forecasts, a journey from verification to falsification

    physics.ao-ph 2026-01 conditional novelty 5.0 of 10

    Forecast realism is decomposed into functional, structural, and physical kinds, with falsification against scientific knowledge proposed as a third evaluation step for AI weather models.

  2. Evaluating Extreme Precipitation Forecasts: A Threshold-Weighted, Spatial Verification Approach for Comparing an AI Weather Prediction Model Against a High-Resolution NWP Model

    physics.ao-ph 2025-10 conditional novelty 5.0 of 10

    Combining HiRA neighborhood verification with threshold-weighted CRPS shows that AI-vs-NWP rankings for extreme precipitation depend strongly on neighborhood size.

  3. Assessing Monotone Dependence: Area Under the Curve Meets Rank Correlation

    stat.ME 2025-10 conditional novelty 5.0 of 10

    A new asymmetric rank-based measure, CMA, unifies AUC and Spearman's rho for all linearly ordered outcomes, with CLTs and a DeLong-type test.

Reference graph

Works this paper leans on

41 extracted references · 39 canonical work pages · cited by 3 Pith papers

  1. [1]

    and Ziegel, J

    Arnold, S. and Ziegel, J. (2025). Isotonic conditional laws. Bernoulli , 31:1140--1159

    work page 2025

  2. [2]

    C., Magnusson, L., Gascon, E., Maier-Gerber, M., Janou s ek, M., Rodwell, M., Pinault, F., Dramsch, J

    Ben Bouall\` e gue, Z., Clare, M. C., Magnusson, L., Gascon, E., Maier-Gerber, M., Janou s ek, M., Rodwell, M., Pinault, F., Dramsch, J. S., Lang, S. T., Raoult, B., Rabier, F., Chevallier, M., Sandu, I., Dueben, P., Chantry, M., and Pappenberger, F. (2024). The rise of data-driven weather forecasting: A first statistical assessment of machine learning-ba...

    work page 2024

  3. [3]

    Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., and Tian, Q. (2023). Accurate medium-range global weather forecasting with 3d neural networks. Nature , 619:533--538

    work page 2023

  4. [4]

    P., Lucic, A., Stanley, M., Vaughan, A., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J

    Bodnar, C., Bruinsma, W. P., Lucic, A., Stanley, M., Vaughan, A., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J. A., Dong, H., Gupta, J. K., Thambiratnam, K., Archibald, A. T., Wu, C.-C., Heider, E., Welling, M., Turner, R. E., and Perdikaris, P. (2025). A foundation model for the E arth system. Nature , 641:1180--1187

    work page 2025

  5. [5]

    Bonavita, M. (2024). On some limitations of current machine learning weather prediction models. Geophysical Research Letters , 51:e2023GL107377

    work page 2024

  6. [6]

    D., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D

    Brenowitz, N. D., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D. R., Harrington, P., and Pritchard, M. S. (2025). A practical probabilistic benchmark for AI weather models. Geophysical Research Letters , 52:e2024GL113656

    work page 2025

  7. [7]

    B \"u lte, C., Horat, N., Quinting, J., and Lerch, S. (2025). Uncertainty quantification for data-driven weather models. Artificial Intelligence for the Earth Systems , https://doi.org/10.1175/AIES-D-24-0049.1

    doi:10.1175/aies-d-24-0049.1 2025

  8. [8]

    S., and Gneiting, T

    Feldmann, K., Richardson, D. S., and Gneiting, T. (2019). Grid-versus station-based postprocessing of ensemble temperature forecasts. Geophysical Research Letters , 46:7744--7751

    work page 2019

Show all 41 references
  1. [9]

    Gerds, T. A. and Schumacher, M. (2006). Consistent estimation of the expected B rier score in general survival models with right-censored event times. Biometrical Journal , 48:1029--1040

    2006

  2. [10]

    Glasser, G. J. (1962). Variance formulas for the mean difference and coefficient of concentration. Journal of the American Statistical Association , 57:648--654

    1962

  3. [11]

    Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association , 106:746--762

    2011

  4. [12]

    Gneiting, T., Balabdaoui, F., and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology , 69:243--268

    2007

  5. [13]

    and Raftery, A

    Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association , 102:359--378

    2007

  6. [14]

    and Ranjan, R

    Gneiting, T. and Ranjan, R. (2011). Comparing density forecasts using threshold-and quantile-weighted scoring rules. Journal of Business & Economic Statistics , 29:411--422

    2011

  7. [15]

    and Walz, E.-M

    Gneiting, T. and Walz, E.-M. (2022). Receiver operating characteristic ( ROC ) movies, universal ROC ( UROC ) curves, and coefficient of predictive ability ( CPA ). Machine Learning , 111:2769--2797

    2022

  8. [16]

    Hamill, T. M. and Juras, J. (2006). Measuring forecast skill: I s it real skill or is it the varying climatology? Quarterly Journal of the Royal Meteorological Society , 132:2905--2923

    2006

  9. [17]

    o sching, A., and D \

    Henzi, A., M \"o sching, A., and D \"u mbgen, L. (2022). Accelerating the pool-adjacent-violators algorithm for isotonic distributional regression. Methodology and Computing in Applied Probability , 24:2633--2645

    2022

  10. [18]

    F., and Gneiting, T

    Henzi, A., Ziegel, J. F., and Gneiting, T. (2021). Isotonic distributional regression. Journal of the Royal Statistical Society Series B: Statistical Methodology , 83:963--993

    2021

  11. [19]

    D., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R

    Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Hor \'a nyi, A., Mu \ n oz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., Chiara, G. D., Dahl...

    2020

  12. [20]

    I., M \"u hlemann, A., and Ziegel, J

    Jordan, A. I., M \"u hlemann, A., and Ziegel, J. F. (2022). Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals. Annals of the Institute of Statistical Mathematics , 74:489--514

    2022

  13. [21]

    J., Parker, D

    Keane, R. J., Parker, D. J., Dunn-Sigouin, E., Kolstad, E. W., and Marsham, J. H. (2025). Mid-latitude versus tropical scales of predictability and their implications for forecasting. Meteorological Applications . Preprint available at https://eprints.whiterose.ac.uk/id/eprint/226102/

    2025

  14. [22]

    o wer, M., Lottes, J., Rasp, S., D \

    Kochkov, D., Yuval, J., Langmore, I., Norgaard, P., Smith, J., Mooers, G., Kl \"o wer, M., Lottes, J., Rasp, S., D \"u ben, P., Hatfield, S., Battaglia, P., Sanchez-Gonzalez, A., Willson, M., Brenner, M. P., and Hoyer, S. (2024). Neural general circulation models for weather a...

    2024

  15. [23]

    Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Alet, F., Ravuri, S., Ewalds, T., Eaton-Rosen, Z., Hu, W., Merose, A., Hoyer, S., Holland, G., Vinyals, O., Stott, J., Pritzel, A., Mohamed, S., and Battaglia, P. (2023). Learning skillful medium-range...

    2023

  16. [24]

    C., Roberts, C., Adewoyin, R., Bouall \`e gue, Z

    Lang, S., Alexe, M., Clare, M. C., Roberts, C., Adewoyin, R., Bouall \`e gue, Z. B., Chantry, M., Dramsch, J., Dueben, P. D., Hahner, S., Maciel, P., Prieto-Nemesio, A., O'Brien, C., Pinault, F., Polster, J., Raoult, B., Tietsche, S., and Leutbecher, M. (2024). AIFS-CRPS: E ns...

    2024 arXiv

  17. [25]

    L., Ravazzolo, F., and Gneiting, T

    Lerch, S., Thorarinsdottir, T. L., Ravazzolo, F., and Gneiting, T. (2017). Forecaster's dilemma: E xtreme events and forecast evaluation. Statistical Science , 32:106--127

    2017

  18. [26]

    Lipiecki, A., Uniejewski, B., and Weron, R. (2024). Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: T he benefits of using isotonic distributional regression. Energy Economics , 139:107934

    2024

  19. [27]

    N., and Petroliagis, T

    Molteni, F., Buizza, R., Palmer, T. N., and Petroliagis, T. (1996). The ECMWF ensemble prediction system: M ethodology and validation. Quarterly Journal of the Royal Meteorological Society , 122:73--119

    1996

  20. [28]

    o sching, A. and D \

    M \"o sching, A. and D \"u mbgen, L. (2020). Monotone least squares and isotonic quantiles. Electronic Journal of Statistics , 14:24--49

    2020

  21. [29]

    Murphy, A. H. and Daan, H. (1985). Forecast evaluation. In Murphy, A. H. and Katz, R., editors, Probability, Statistics, and Decision Making in the Atmospheric Sciences , pages 379--437. Westview Press, Boulder, Colorado

    1985

  22. [30]

    and Messori, G

    Olivetti, L. and Messori, G. (2024). Do data-driven models beat numerical models in forecasting weather extremes? A comparison of IFS HRES , P angu- W eather, and G raph C ast. Geoscientific Model Development , 17:7915--7962

    2024

  23. [31]

    Python language reference

    Python Software Foundation (2025). Python language reference. Available at http://www.python.org

    2025

  24. [32]

    R : A language and environment for statistical computing

    R Core Team (2025). R : A language and environment for statistical computing. Available at http://www.r-project.org

    2025

  25. [33]

    T., Ebert-Uphoff, I., and Stewart, J

    Radford, J. T., Ebert-Uphoff, I., and Stewart, J. Q. (2025a). A comparison of AI weather prediction and numerical weather prediction models for 1--7-day precipitation forecasts. Weather and Forecasting , 40:561--575

    2025

  26. [34]

    T., Ebert-Uphoff, I., Stewart, J

    Radford, J. T., Ebert-Uphoff, I., Stewart, J. Q., Musgrave, K. D., DeMaria, R., Tourville, N., and Hilburn, K. (2025b). Accelerating community-wide evaluation of AI models for global weather prediction by facilitating access to model output. Bulletin of the American Meteorolog...

    2025

  27. [35]

    D., Scher, S., Weyn, J

    Rasp, S., Dueben, P. D., Scher, S., Weyn, J. A., Mouatadid, S., and Thuerey, N. (2020). Weather B ench: A benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems , 12:e2020MS002203

    2020

  28. [36]

    Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russell, T., Sanchez-Gonzalez, A., Yang, V., Carver, R., Agrawal, S., Chantry, M., Ben Bouall\` e gue, Z., Dueben, P., Bromberg, C., Sisk, J., Barrington, L., Bell, A., and Sha, F. (2024). Weather B ench 2: A benchm...

    2024

  29. [37]

    C., Markou, S., Turner, R., and Vaughan, A

    Selz, T., Bruinsma, W., Craig, G. C., Markou, S., Turner, R., and Vaughan, A. (2025). On the effective resolution of AI weather prediction models. Preprint available at http://dx.doi.org/10.22541/essoar.174139239.94807670/v1

    2025

  30. [38]

    B., Demaeyer, J., Evans, G

    Vannitsem, S., Bremnes, J. B., Demaeyer, J., Evans, G. R., Flowerdew, J., Hemri, S., Lerch, S., Roberts, N., Theis, S., Atencia, A., Ben Bouall\` e gue, Z., Bhend, J., Dabernig, M., De Cruz, L., Hieta, L., Mestre, O., Moret, L., Plenkovi\' e , I. O., Schmeits, M., Taillardat, ...

    2021

  31. [39]

    Walz, E.-M., Henzi, A., Ziegel, J., and Gneiting, T. (2024a). E asy U ncertainty Q uantification ( E asy UQ ): G enerating predictive distributions from single-valued model output. SIAM Review , 66:91--122

    2024

  32. [40]

    H., K \"o hler, G., and Gneiting, T

    Walz, E.-M., Knippertz, P., Fink, A. H., K \"o hler, G., and Gneiting, T. (2024b). Physics-based vs data-driven 24-hour probabilistic forecasts of precipitation for northern tropical A frica. Monthly Weather Review , 152:2011--2031

    2024

  33. [41]

    Yitzhaki, S. (2003). Gini’s mean difference: A superior measure of variability for non-normal distributions. Metron , 61:285--316

    2003

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.