Pith. sign in

REVIEW 2 major objections 5 minor 15 references

Learning low-dimensional representations of ensemble forecast fields using autoencoder-based methods

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A permutation-invariant autoencoder compresses 50-member weather forecast fields into a low-dimensional Gaussian whose decoded samples preserve the ensemble spread that PCA and AE two-step baselines underestimate.

desk verdict Useful paper with a real but confounded central claim: the iVAE advantage partly comes from training on the same metric used for evaluation. read the letter →

arxiv 2502.04409 v1 pith:EUQO5C4A submitted 2025-02-06 cs.LG physics.ao-ph

Jieyu Chen , Kevin Höhlein , Sebastian Lerch This is my paper · ORCID
classification cs.LGphysics.ao-ph MSC 62H2568T07
keywords dimensionalityreductionensembleweatherforecastsvariationalautoencoderpermutationinvarianceprobabilisticlatentrepresentationenergydistanceSinkhornnumericalprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to establish that an ensemble of weather forecast fields, 50 gridded simulations per day each sampling the same unknown forecast distribution, can be compressed into a low-dimensional latent probability distribution that still behaves like the original ensemble when decoded back into fields. The authors propose two frameworks: a two-step approach that encodes each member separately with PCA or an autoencoder and then fits a Gaussian to the per-member latent codes, and an invariant variational autoencoder (iVAE) that encodes all members jointly through a network invariant to member ordering and outputs a Gaussian in latent space directly. The central empirical claim, tested on ten years of temperature and wind speed forecasts over Europe, is that the iVAE reproduces the ensemble's member-to-member variability at the correct magnitude, while both two-step methods produce reconstructed ensembles that are too smooth. If this holds, downstream consumers of forecasts, including post-processing, analog forecasting, and data storage, could work with a compact probabilistic code instead of storing full high-dimensional ensemble fields.

What carries the argument

The central object is the distributional latent representation $D = \mathcal{N}(\mu, \Sigma)$ on a space $\mathbb{R}^{d_{\text{latent}}}$ that stands in for the entire ensemble. In the two-step approaches, $D$ is assembled after the fact: a deterministic encoder maps each member field to a latent code, and a Gaussian is fitted to the 50 codes, with $\mu$ the code mean and $\Sigma$ the code covariance. In the iVAE, $D$ is learned end-to-end: a shared encoder maps each member to an intermediate vector, an average-pooling layer makes the whole encoding invariant to the ordering of members, a second encoder outputs the Gaussian parameters $(\mu, \log\sigma)$, and the reparameterization trick allows gradients to flow through the random sampling. The decoder maps sampled $z_n$ back to fields, and the reconstruction loss is itself distributional, a weighted sum of the energy distance and the Sinkhorn distance between the raw and reconstructed field ensembles plus a small KL regularization term, instead of a field-wise error, which would be ill-defined because input and output members are not paired.

What would settle it

Train the iVAE with the same architecture and loss on daily precipitation accumulations over the same European grid, and at grid points where the raw ensemble has a substantial frequency of exactly-zero values, count the frequency in the generated members. If the decoded members are almost never zero, or if the pixel-wise standard-deviation difference grows far beyond the near-zero values achieved for temperature and wind, the Gaussian latent representation, the load-bearing assumption, fails, and the reconstructed ensembles are not statistically indistinguishable from the raw ensemble for that variable.

Watch

Extended reading notes

Core claim

The paper argues that a distributional latent representation of an ensemble forecast is learnable end-to-end: encode all 50 members into a Gaussian $D = \mathcal{N}(\mu, \Sigma)$ on a low-dimensional latent space (for the iVAE, $q_\phi(z|X) = \mathcal{N}(z; \mu, \text{diag}(\sigma^2))$), then decode any number of samples from $D$ into new fields intended to be statistically indistinguishable from members of the raw ensemble. The central comparison appears in Section 4.2: at every latent dimension from 2 to 32, the iVAE's decoded ensembles preserve the raw ensemble's member-to-member standard deviation (mean absolute difference about 0.11 to 0.25 K for temperature, and within about 0.06 m/s of zero for wind speed), whereas the PCA- and AE-based two-step methods systematically underestimate it (about 0.29 to 0.30 K for temperature). The iVAE also achieves positive energy-distance skill scores against the PCA baseline at all latent dimensions in both pixel-wise and whole-field comparisons, while the AE-based method beats PCA only in some settings and mainly at low dimensions.

Load-bearing premise

All three methods represent each forecast day's ensemble as a multivariate Gaussian in latent space, so if the true forecast distribution is not Gaussian, such as bimodal, skewed, or with a point mass at zero as in precipitation, the compressed representation cannot faithfully capture it, a limitation the authors state explicitly in the discussion.

Editorial extensions

If this is right

  • The iVAE's decoded members preserve the raw ensemble's spread far better than the two-step methods (standard-deviation differences of about 0.11 to 0.25 K for temperature versus 0.29 to 0.51 K for PCA and AE across latent dimensions 2 to 32), so the compressed code carries genuine uncertainty information rather than a smoothed mean field.
  • Because any number of samples can be drawn from the learned distribution and decoded, the representation doubles as a generative model for building larger or resampled ensembles without rerunning the numerical simulation.
  • The neural-network methods (AE and iVAE) outperform PCA at low latent dimensions, while PCA becomes competitive or better at high dimensions, so the choice of method depends on the target compression rate.
  • The ablation of the iVAE loss shows a trade-off between energy-distance and Sinkhorn-distance reconstruction quality, meaning the training objective can be weighted toward whichever distributional property a downstream task needs.
  • Two-dimensional latent representations separate months of the year and, for geopotential height, distinct weather regimes, indicating the compressed representations retain meteorologically meaningful structure beyond raw reconstruction accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A plausible reason the iVAE keeps ensemble spread while the two-step methods do not is its permutation invariance: the encoder is forced to summarize 50 exchangeable inputs into one distribution before any member-specific detail can pass through, so it cannot encode a single smoothed mean field. A testable extension would regularize a two-step autoencoder's per-member codes toward genuine exchange
  • The Gaussian latent assumption sets a ceiling: for a variable with a point mass at zero, such as precipitation, a single Gaussian cannot represent the mixture, as the authors note. The natural stress test is a zero-inflated decoding head or a mixture- or flow-based latent prior applied to the same architecture, checking whether the spread-matching result transfers.
  • The same recipe, a permutation-invariant shared encoder plus distributional reconstruction losses (energy and Sinkhorn distances), applies to any ensemble simulation pipeline, such as climate-model initial-condition ensembles or perturbed-physics runs, not just weather forecasts; the paper's skill-score evaluation protocol is portable as well.
  • For wind speed the iVAE's standard-deviation difference reaches slightly negative values (about minus 0.01 m/s at dimension 32), hinting that the model can marginally over-disperse; if that persists when generating large numbers of members, rank-histogram or variogram checks would reveal whether the decoded ensembles are reliable probabilistic forecasts or only moment-matching reconstructions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper addresses the problem of learning low-dimensional distributional representations of ensemble forecast fields, motivated by numerical weather prediction. It proposes two frameworks: a two-step approach that applies a deterministic dimensionality reduction (PCA or autoencoder) member-by-member and then fits a multivariate Gaussian to the latent codes, and an invariant variational autoencoder (iVAE) that encodes all ensemble members jointly through a permutation-invariant mean-pooling encoder and outputs a Gaussian latent posterior. Both frameworks are evaluated on 10 years of ECMWF ensemble forecasts over Europe for temperature, wind components, and geopotential height, across latent dimensions 2 to 32. The main evaluation compares reconstructed ensembles to the raw ensembles via pixel-wise mean and standard deviation differences, univariate and multivariate energy distances, and univariate Wasserstein / multivariate Sinkhorn distances. The paper's central empirical claim is that the iVAE preserves ensemble variability substantially better than the two-step methods, while the two-step methods and iVAE are comparable on mean-field reconstruction.

Significance. If the central claim were robust, the iVAE would be a valuable tool for probabilistic dimensionality reduction of ensemble forecasts, with clear applications in data compression and downstream post-processing. The paper is well placed within the literature and has several strengths: a clear problem formulation, a publicly available implementation, a non-trivial case study with multiple variables and latent dimensions, and an ablation study of the loss weighting. The limitation of the Gaussian latent assumption is explicitly acknowledged in Section 5. However, the central claim is currently supported primarily by metrics that are also part of the iVAE training objective; the independent Sinkhorn-based evaluation shows the iVAE performing worst. This confound, detailed below, limits the significance of the contribution as stated and requires additional investigation before the manuscript can be accepted.

major comments (2)
  1. [3.3, 4.1, 4.2, Figs. 6-7, S2-S3, S6, S9] The iVAE loss in Eq. (1) includes the multivariate energy distance D(X, X-tilde) with weight omega1=0.5 and a factor of 2, while the Sinkhorn term is divided by 50 for temperature and by 500 for wind. With omega2=0.5, the effective Sinkhorn contribution is 0.5/50 = 0.01 (temperature) or 0.5/500 = 0.001 (wind), so the loss is dominated by the energy distance, which is exactly the main evaluation metric in Figures 6-7. The reported iVAE superiority on energy distance is therefore partly a restatement of its training objective. The independent Sinkhorn evaluation in the Supplement (Figures S2-S3, S6, S9) shows the iVAE performing worst, with negative skill scores down to -0.57 for z500, despite Sinkhorn being nominally present in the loss. Since the paper's central claim is that the iVAE preserves ensemble variability across the board, the authors should either provide support from a metric not used in the training loss (e.g., a spectral or variogram-based score), or explicitly restrict the claim to energy-type metrics and discuss the Sinkhorn failure.
  2. [3.3, 4.3, Fig. 8] The rescaling of the loss components (energy multiplied by 2, Sinkhorn divided by 50 or 500, KL divided by 10) is chosen heuristically by comparing loss values over the first 20 epochs. The ablation in Section 4.3 shows that varying the relative weight omega2 produces a clear trade-off between energy distance and Sinkhorn distance, so the reported 'best' behavior of the iVAE is contingent on these hand-picked scaling constants. The paper should either justify the rescaling with a principled criterion (e.g., tuning on a validation target that includes the evaluation metrics of interest) or demonstrate that the variability-preservation result is stable across a range of rescaling choices. As written, the conclusion that the iVAE 'performs best at preserving the variability information of the input ensemble forecasts' is not robust evidence against the alternative explanation that the heuristic loss weighting, rather than the permutation-invariant architecture, drives the observed advantage.
minor comments (5)
  1. [Section 2] Typo: 'forcasts' should be 'forecasts' in the second sentence.
  2. [Section 4.1] Typo: 'idential' in 'the ensemble members follow an idential distribution' should be 'identical'.
  3. [Supplement, Fig. S1] Typo: 'pricipal' in 'First n pricipal components' should be 'principal'.
  4. [Section 3.1 and Eq. (1)] The symbol D is used both for the latent distribution in Section 3.1 (D = E(X)) and for the energy distance in Eq. (1) and Section 4.1. Please introduce a distinct notation (e.g., E_D for energy distance) to avoid confusion.
  5. [Section 5] The acknowledged Gaussian assumption is a genuine scope limitation, especially for variables with point masses such as precipitation; the manuscript would be clearer if the implications for the two-step methods (where the Gaussian is fitted after training) versus the iVAE (where it is the posterior family) were stated separately.

Circularity Check

1 steps flagged · score 6.0 of 10

The iVAE's headline edge over PCA/AE is partly an artifact of training on the same energy distance used for evaluation; the claimed general superiority is contradicted by the supplement's Sinkhorn results.

  1. fitted input called prediction [Section 3.3, Eq. (1); Section 4.1-4.2, Figs. 6-7; Supplement Figs. S2-S3, S6, S9]
    "ℓ(X, ˜X) = ω1 · 2D(X, ˜X) + ω2 · 1 50 SD(X, ˜X) + ω3 · 1 10 DKL(qϕ(z|X)∥pθ(z)), (1) where D(·) represents the energy distance and SD(·) denotes the Sinkhorn distance."

    Eq. (1) makes the multivariate energy distance D the dominant term of the iVAE objective: with ω1=ω2=0.5, ω3=0.01, and the stated rescaling (Sinkhorn ÷50 for t2m, ÷500 for u10, KL ÷10), the effective weights are 1.0 for D, 0.01 or 0.001 for Sinkhorn, and 0.001 for KL. The paper's main evidence for iVAE superiority (Figs. 6-7) is the skill score computed from this same D and its per-pixel version, while the PCA and AE baselines are fit to variance and MAE respectively. The positive energy-distance skill is therefore largely a restatement of the training objective, not an independent finding. The supplement confirms this metric-specificity: iVAE attains the worst Sinkhorn skill scores (down to -0.57 for z500) despite Sinkhorn nominally appearing in Eq. (1).

full rationale

The paper is a method paper with a clear, self-contained experimental setup: it proposes PCA/AE two-step and iVAE frameworks, evaluates them on held-out test data, provides code, and cites prior work mainly for motivation. I found no self-citation chain or imported uniqueness theorem; the Gaussian-latent assumption is a stated modeling choice, not a derived prediction. The significant circularity concern is confined to the headline quantitative claim. In Eq. (1) the iVAE is trained to minimize a weighted sum whose dominant term is the multivariate energy distance D(X, X̃), with Sinkhorn down-weighted to effective weights 0.01 (temperature) or 0.001 (wind) and KL to 0.001. The main evidence for iVAE superiority in Section 4.2 (Figs. 6-7) is the skill score computed from this same energy distance, while the PCA and AE baselines are trained on variance and MAE respectively. The paper acknowledges this connection ('potentially benefit from the inclusion of the multivariate energy distance as part of the loss function'), and the supplement shows iVAE performs worst on Sinkhorn skill scores, so the energy-distance advantage is a metric-specific consequence of the training objective rather than a general representation-quality finding. The pixel-wise standard-deviation diagnostic (Figs. 4-5) provides some independent support for better variability preservation, which keeps the paper from being fully circular; however, it is closely related to univariate energy distance and does not restore the fairness of the headline comparison. Overall score 6: a central 'prediction' reduces by construction, while the method, ablations, and code remain transparent and partially independent.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central result rests on the Gaussian latent assumption and, for the iVAE, on the sufficiency of mean pooling of per-member encodings. The loss weights and rescaling factors are fitted constants derived from training data. No new physical entities are introduced.

free parameters (2)
  • iVAE loss weights omega1, omega2, omega3 = 0.5, 0.5, 0.01
    Chosen heuristically: omega3 small to avoid posterior collapse, omega1=omega2 equal. Affects the reconstruction-variability trade-off; the ablation only varies omega2.
  • iVAE loss rescaling factors = KL/10, energy*2, Sinkhorn/50 (t2m), Sinkhorn/500 (u10)
    Determined by comparing mean loss components over the first 20 training epochs; these are data-dependent constants that set the relative contribution of each term in Eq. (1).
assumptions (4)
  • domain assumption Ensemble members are independent and identically distributed samples from an unknown multivariate distribution P (X_m ~ P).
    Used to justify the exchangeability requirement and the iVAE's permutation invariance (Section 3.1). In practice ECMWF ensemble members are conditionally dependent and may have distinct perturbation structures.
  • domain assumption The latent representation is a multivariate Gaussian: D = N(mu, Sigma), or q(z|X)=N(mu, diag(sigma^2)).
    Core to both proposed frameworks; acknowledged in Section 5 as limiting for variables such as precipitation. Not derived from the data distribution.
  • ad hoc to paper Mean pooling of per-member intermediate encodings (y-bar = (1/50) sum y_m) preserves the distributional information needed to represent the ensemble.
    Design choice in the iVAE encoder (Section 3.3, Figure 2). Higher-order moments of the per-member features are discarded before mapping to the latent Gaussian; if the decoder cannot recover the ensemble shape from this summary, the method fails.
  • domain assumption Energy distance and Sinkhorn distance between input and reconstructed ensembles are appropriate training objectives for representation learning.
    Adopted in Section 3.3 because member-wise MSE is inappropriate; this choice makes the evaluation metric partly coincide with the training objective.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning low-dimensional representations of ensemble forecast fields using autoencoder-based methods." pith.science (2026). https://pith.science/paper/EUQO5C4A

@misc{pith2026250204409,
  author       = {Pith},
  title        = {Pith review of: Learning low-dimensional representations of ensemble forecast fields using autoencoder-based methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUQO5C4A}},
  note         = {Machine review of arXiv:2502.04409}
}
read the original abstract

Large-scale numerical simulations often produce high-dimensional gridded data that is challenging to process for downstream applications. A prime example is numerical weather prediction, where atmospheric processes are modeled using discrete gridded representations of the physical variables and dynamics. Uncertainties are assessed by running the simulations multiple times, yielding ensembles of simulated fields as a high-dimensional stochastic representation of the forecast distribution. The high-dimensionality and large volume of ensemble datasets poses major computing challenges for subsequent forecasting stages. Data-driven dimensionality reduction techniques could help to reduce the data volume before further processing by learning meaningful and compact representations. However, existing dimensionality reduction methods are typically designed for deterministic and single-valued inputs, and thus cannot handle ensemble data from multiple randomized simulations. In this study, we propose novel dimensionality reduction approaches specifically tailored to the format of ensemble forecast fields. We present two alternative frameworks, which yield low-dimensional representations of ensemble forecasts while respecting their probabilistic character. The first approach derives a distribution-based representation of an input ensemble by applying standard dimensionality reduction techniques in a member-by-member fashion and merging the member representations into a joint parametric distribution model. The second approach achieves a similar representation by encoding all members jointly using a tailored variational autoencoder. We evaluate and compare both approaches in a case study using 10 years of temperature and wind speed forecasts over Europe. The approaches preserve key spatial and statistical characteristics of the ensemble and enable probabilistic reconstructions of the forecast fields.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages

  1. [1]

    R., Buchanan, P

    Allen, S., Evans, G. R., Buchanan, P. and Kwasniok, F. (2021). Incorporating the north atlantic oscillation into the post-processing of mogreps-g wind speed forecasts. Quarterly Journal of the Royal Meteorological Society, 147, 1403–1418. 22 An, J. and Cho, S. (2015). Variational autoencoder based anomaly detection using reconstruction probability. Specia...

    work page 2021

  2. [6]

    Horat, N

    Morgan-Kaufmann. Horat, N. and Lerch, S. (2024). Deep learning for postprocessing global probabilistic forecasts on subseasonal time scales. Monthly Weather Review , 152, 667–687. H¨ ohlein, K., Schulz, B., Westermann, R. and Lerch, S. (2024). Postprocessing of ensemble weather forecasts using permutation-invariant neural networks. Artificial Intelligence...

    arXiv 2024

  3. [11]

    Mockert, F., Grams, C

    MIT Press. Mockert, F., Grams, C. M., Lerch, S., Osman, M. and Quinting, J. (2024). Multivariate post- processing of probabilistic sub-seasonal weather regime forecasts. Quarterly Journal of the Royal Meteorological Society, 150, 4771–4787. Patrini, G., van den Berg, R., Forr´ e, P., Carioni, M., Bhargav, S., Welling, M., Genewein, T. and Nielsen, F. (202...

    arXiv 2024

  4. [26]

    Dorninger, M., Gilleland, E., Casati, B., Mittermaier, M

    Curran Associates, Inc. Dorninger, M., Gilleland, E., Casati, B., Mittermaier, M. P., Ebert, E. E., Brown, B. G. and Wil- son, L. J. (2018). The setup of the mesovict project. Bulletin of the American Meteorological Society, 99, 1887 –

    work page 2018

  5. [29]

    Curran Associates, Inc. Rasp, S. and Lerch, S. (2018). Neural networks for postprocessing ensemble weather forecasts. Monthly Weather Review , 146, 3885–3900. Rizzo, M. L. and Sz´ ekely, G. J. (2016). Energy distance. WIREs Computational Statistics , 8, 27–38. Roberts, N. M. and Lean, H. W. (2008). Scale-selective verification of rainfall accumulations fr...

    work page 2018

  6. [30]

    Low-dimensional representation of ensemble forecast fields using autoencoder-based methods

    Curran Associates, Inc. Zhong, X., Chen, L., Li, H., Liu, J., Fan, X., Feng, J., Dai, K., Luo, J.-J., Wu, J. and Lu, B. (2024). Fuxi-ens: A machine learning model for medium-range ensemble weather forecasting. 2405.05925, URL https://arxiv.org/abs/2405.05925. 27 Zhou, C. and Paffenroth, R. C. (2017). Anomaly detection with robust deep autoencoders. In Pro...

    arXiv 2024

  7. [32]

    Notably, the iV AE method performs the worst in terms of Sinkhorn distance, despite incorporating the Sinkhorn distance component in its loss function. Conversely, both neural network-based approaches demonstrate significantly better performance for wind speed data, consistent with our findings on energy distances in the main paper. Further evaluation res...

    work page 2017

  8. [97]

    J., Richardson, D

    Rodwell, M. J., Richardson, D. S., Parsons, D. B. and Wernli, H. (2018). Flow-dependent reliability: A path to more skillful ensemble forecasts. Bulletin of the American Meteorological Society, 99, 1015 –

    work page 2018

Show all 15 references
  1. [234]

    and Lerch, S

    Chen, J., Janke, T., Steinke, F. and Lerch, S. (2024). Generative machine learning methods for multivariate ensemble postprocessing. The Annals of Applied Statistics , 18, 159–183. Courty, N., Flamary, R., Tuia, D. and Rakotomamonjy, A. (2016). Optimal transport for domain ada...

    2024

  2. [257]

    Sz´ ekely, G. J. and Rizzo, M. L. (2013). Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference , 143, 1249–1272. Tenenbaum, J. B., de Silva, V. and Langford, J. C. (2000). A global geometric framework for nonlinear dimensio...

    2013 arXiv

  3. [689]

    and Cremers, D

    Eisenberger, M., Toker, A., Leal-Taix´ e, L., Bernard, F. and Cremers, D. (2022). A unified framework for implicit sinkhorn differentiation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 509–518. Feik, M., Lerch, S. and St¨ uhmer...

    2022 arXiv

  4. [1026]

    Roweis, S. T. and Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326. Sakurada, M. and Yairi, T. (2014). Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MLSDA 2014 2nd ...

    2000 arXiv

  5. [1376]

    Gondara, L. (2016). Medical image denoising using convolutional denoising autoencoders. In 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW) . 241–246. Grams, C. M., Beerli, R., Pfenninger, S., Staffell, I. and Wernli, H. (2017). Balancing europe’s wind-...

    2016

  6. [1509]

    B¨ ulte, C., Horat, N., Quinting, J

    00519, URL https://arxiv.org/abs/1509.00519. B¨ ulte, C., Horat, N., Quinting, J. and Lerch, S. (2024). Uncertainty quantification for data- driven weather models. 2403.13458, URL https://arxiv.org/abs/2403.13458. Chapman, W. E., Monache, L. D., Alessandrini, S., Subramanian, ...

    2024 arXiv

  7. [1906]

    and Houlsby, N

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J. and Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. 2010.11929, URL https: ...

    2021 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.