REVIEW 3 major objections 4 minor 43 references
On a known non-stationary Gaussian random field, all four deep generative models recover the mean surface, but covariance recovery varies by family: DDPM and score-SDE do best, flow matching attenuates non-stationarity, and the VAE fails to
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
On a known non-stationary Gaussian random field, DDPM and score-SDE reproduce the covariance structure best, flow matching mildly attenuates it, and the VAE largely fails.
T0 review reviewed 2026-08-01 challenge →
load-bearing objection Useful controlled benchmark, but the headline FM non-stationarity attenuation claim doesn't survive the paper's own multi-seed table; worth refereeing after fixes. the 3 major comments →
Can Deep Generative Models Reproduce Non-Stationary Gaussian Random Fields?
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
Under the paper's controlled benchmark, the ground-truth non-stationary covariance is defined by a Paciorek–Schervish Matérn construction with a spatially varying local range parameter β(s). The paper's central discovery is a consistent family-level ordering: DDPM and score-SDE, which reverse a many-step noising process, recover both the degree and the location of non-stationarity almost as well as fresh draws from the true process; flow matching, which transports noise along a learned near-straight path, lands close on global correlation measures but shows attenuated NS-grad and corr(β̂, β) values and a small variance deficit; and the VAE, trained on an evidence lower bound with a 128-dimen
What carries the argument
The load-bearing device is a non-stationary Matérn covariance built from a spatially varying local range parameter β(s), plus a local-stationarity estimation protocol: the domain is cut into nine 3×3 windows, a stationary Matérn model is fit inside each window from pooled empirical variograms, and the resulting local range estimates feed three diagnostics — NS-grad (the ratio of estimated ranges at the two corners where β is largest and smallest), corr(β̂, β) (whether the spatial pattern of variation is correctly placed), and tot-var/Buresk summary metrics that compare the generated ensemble covariance directly to the ground truth. Coordinate-aware network inputs break translation equivarian
Load-bearing premise
The paper's ranking of models rests on nine local stationary Matérn fits in non-overlapping 3×3 windows; if estimator smoothing from these small local fits compresses NS-grad and corr(β̂, β) unequally across model families, the apparent attenuation of flow matching (or the 'strong recovery' of the diffusion models) could be an artifact of the estimator rather than a property of the generators.
What would settle it
Run the same evaluation with the windows replaced by much finer or overlapping local estimates (say 6×6 or 8×8 windows) and with covariance recovery measured directly in the spectral domain, avoiding the variogram fit; if FM's NS-grad and corr(β̂, β) rise to the oracle's level, the reported flow-matching attenuation is an estimation artifact, not a generative failure.
If this is right
- If the benchmark ranking is right, model family choice matters more than training budget for reproducing spatial dependence: diffusion-style models are the safer default when the covariance structure is the object of interest.
- Because CRPS does not separate the models, observation-only validation is insufficient; targeted covariance diagnostics are needed to certify a DGM for spatial uncertainty quantification.
- FM's mild under-dispersion is a scale error, not a structural one, so calibration rescaling may fix it; VAE's under-dispersion is structural (missing nugget-scale noise) and cannot be fixed by rescaling.
- The ERA5 adaptation shows the same toolkit works when ground truth is unavailable, using held-out years as an empirical reference; this gives practitioners a way to audit DGMs on real climate fields.
Where Pith is reading between the lines
- A direct test of the paper's mechanism: increase the window count (or use overlapping windows) to see whether FM's attenuated NS-grad persists; if it weakens toward the oracle, part of the reported attenuation is estimator smoothing rather than model behavior.
- The same evaluation logic could be extended to non-Gaussian processes (e.g., precipitation with zeros and heavy tails) where the covariance alone is not sufficient; the Bures/whitened-geometry approach would need a different discrepancy measure.
- The finding that VAE collapses the nugget suggests latent-variable models need per-pixel stochastic decoding for spatial noise; a hierarchical VAE with spatial latent noise is an immediate, testable fix.
- For climate applications, the paper implies that ensemble spread from a VAE (or any smoothed generator) is not trustworthy for fine-scale risk, even if the mean forecast is good.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a controlled benchmark for evaluating whether deep generative models (FM, DDPM, score-SDE, VAE) can reproduce a known non-stationary Gaussian random field. The DGP has a spatially varying mean and a non-stationary Matérn covariance with a spatially varying local range parameter. The authors introduce several targeted metrics — mean MSE, Bures distance on whitened dominant eigenspaces, local-window estimates of the range surface summarized by NS-grad and corr(β̂, β), total-variance ratio, and fitted variance components — together with an oracle and a stationary control. They report that all four models recover the mean surface, while covariance recovery differs by family: DDPM and score-SDE recover the non-stationary covariance well, FM shows mildly attenuated non-stationarity and slight under-dispersion, and VAE struggles with covariance recovery. An ERA5 temperature-anomaly experiment is presented as a real-data extension. The central claim is therefore that model family choices materially affect the reproduction of spatial dependence and variance structure.
Significance. If the central claim were fully supported, the paper would make a useful contribution: it constructs a known-DGP benchmark with oracle and stationary references, proposes targeted diagnostics that go beyond aggregate scores like CRPS, and ships a reproducible, seeded pipeline with public reference implementations. The observation that CRPS does not separate models while targeted covariance metrics do is a valuable cautionary result. The multi-seed robustness study is a welcome feature. However, the headline family-level ordering depends on metrics that are shown within the paper's own appendix to be unstable across training seeds and to be computed from a very small number of local windows. Because the paper does not currently resolve this internal inconsistency, the required confidence in the central ordering is not yet established.
major comments (3)
- [§5, Table 1 vs. §B.5, Table A6] The abstract and Section 5 claim that FM exhibits 'attenuated non-stationarity' relative to DDPM and score-SDE, but the paper's own multi-seed robustness table does not reproduce this ordering. In Table 1, FM has corr(β̂, β)=0.623 and NS-grad=1.98, while DDPM and score-SDE have larger correlation and gradient values. In Table A6, across five training seeds, FM has corr(β̂, β)=0.711±0.051, DDPM 0.707±0.020, and score-SDE 0.716±0.051 — statistically indistinguishable. For NS-grad, FM averages 2.58±0.39, above the oracle value of 2.51 and far above the Table 1 value of 1.98. The appendix's statement that 'the main conclusions persist across retrainings' is not supported for these two load-bearing metrics. Since the FM-attenuation claim rests on NS-grad and corr(β̂, β), this is an internal inconsistency that must be addressed, either by showing the single-checkpoint results are representativ
- [§3.4, Table 1] Bures_k is presented as a headline metric for covariance-structure recovery, yet the paper reports it without any uncertainty interval (the table caption explicitly excludes Bures_k and CRPS from the bootstrap intervals). The near-oracle ordering among FM, DDPM, and score-SDE (Bures_k = 0.538, 0.582, 0.555 vs. oracle 0.532) needs an uncertainty quantification before one can conclude that these models 'recover the covariance structure reasonably well.' In addition, the truncation parameter k used in Bures_k is never stated in the manuscript or tables; readers cannot reproduce the metric without it. Please report k and provide intervals or a sensitivity analysis over k and over the evaluation draws.
- [§3.4, Appendix B.5] The local-stationarity estimator uses only nine non-overlapping 3×3 windows, each containing about 55 points, to produce β̂(s), NS-grad, and corr(β̂, β). This is a high-variance procedure, and the paper's own across-seed results show that NS-grad shifts by ±0.3–0.5, which is comparable to the FM-vs-diffusion differences in Table 1. The oracle floor does not control for estimator smoothing or window size, because both the oracle and the DGMs are passed through the same local-fitting step. To support the claim that FM attenuates while DDPM/score-SDE recover non-stationarity, the paper should quantify the resolution limit of the estimator (e.g., by bootstrap over windows, by increasing the number of windows, or by reporting the estimator's behavior on known true realizations) and show that the model differences exceed that limit.
minor comments (4)
- [§3.4] Typo: 'metrices' should be 'metrics'.
- [Table A3] The row for σ²=1 appears to have been copied from the non-stationary dependence table (it shows Bures_k, NS-grad, and corr values) rather than the variance-component columns (tot-var, σ̂², τ̂²). Please correct this transcription error.
- [Table A6] The caption says B=300 bootstrap samples per seed, while Table 1 says B=2000. Clarify whether different bootstrap sizes are intentional and whether the main-table intervals would change with B=300.
- [§5 / §4.2] The spectral-bias explanation for the FM/VAE differences is speculative and not directly tested. It is fine as a hypothesis, but the conclusion section repeats it as if established; consider softening the language.
Circularity Check
No significant circularity: the benchmark compares generated samples against a pre-specified GRF ground truth and an oracle floor.
full rationale
The paper makes no derivation claim whose conclusion is built into its inputs. Its central assertion is an empirical comparison: four DGMs are trained on realizations from a known non-stationary GRF, and their generated samples are evaluated against the true mean, true covariance, true local-range field, and variance components. The evaluation metrics (Bures_k, NS-grad, corr(beta-hat, beta), tot-var, sigma-hat^2, tau-hat^2) are computed from generated samples or from local Matérn fits to those samples, and are then compared with ground-truth values or with oracle draws; nothing is fitted to the quantities being predicted. The local-stationarity estimator could be noisy or biased, and the reported cross-seed variation in Table A6 does not fully reproduce the single-checkpoint ranking, but that is an internal statistical-robustness concern, not circularity. There are no load-bearing self-citations, no imported uniqueness theorems, and no fitted parameters relabeled as predictions. The oracle and stationary-control references provide independent anchors for the comparisons. Accordingly, no circular step is present.
Axiom & Free-Parameter Ledger
free parameters (7)
- Mean-trend coefficients c and a =
c=(0.5,1.5,−2.0,1.2,−1.5,2.0), a=0.8
- Covariance parameters σ², τ², ν =
σ²=2, τ²=0.3, ν=1.5 (main), with sweep values
- Local range surface β(s) =
β(s)=0.12+0.38(0.5s1+0.5s2)^1.5
- Stationary-control range φ =
0.25
- Bures_k truncation k =
not stated
- VAE latent dimension =
128
- Local-window configuration =
3×3 windows, 500 subsampled points
axioms (8)
- domain assumption Paciorek–Schervish non-stationary Matérn covariance is a meaningful target for spatial non-stationarity.
- domain assumption Local-stationarity approximation within 3×3 windows is accurate enough for local range/variance estimation.
- domain assumption Bures_k on the leading k eigenspace captures the dominant correlation structure.
- domain assumption Equal optimizer steps, matched sampling budgets, and standard hyperparameters yield a fair model-family comparison.
- domain assumption Coordinate-aware input channels suffice for models to represent location-dependent covariance.
- domain assumption scikit-gstat's Matérn parameterization mapping (r=4√ν φ, c0=σ², s=ν, b=τ²) is correct.
- domain assumption Held-out ERA5 years (2015–2022) are a valid empirical reference despite possible temporal distribution shift.
- standard math Gaussian initial noise and the ODE/SDE samplers behave as specified.
Cite this review
Pith. "Pith review of Can Deep Generative Models Reproduce Non-Stationary Gaussian Random Fields?." pith.science (2026). https://pith.science/paper/AQLXZXPJ
@misc{pith2026260725929,
author = {Pith},
title = {Pith review of: Can Deep Generative Models Reproduce Non-Stationary Gaussian Random Fields?},
year = {2026},
howpublished = {\url{https://pith.science/paper/AQLXZXPJ}},
note = {Machine review of arXiv:2607.25929}
}
read the original abstract
Deep generative models (DGMs) are widely used for complex high-dimensional data and increasingly applied to spatial and spatio-temporal modeling. Their generated samples implicitly represent the learned data distribution and associated uncertainty. However, for real-world data, assessing whether DGMs have learned the underlying process is difficult because the ground truth is unknown and evaluation often relies on observations alone. We evaluate representative DGMs, flow matching (FM), DDPM, score-SDE, and VAE, on a known non-stationary Gaussian random field. This paper provides comprehensive metrics to assess recovery of the ground-truth mean and covariance structures, with oracle samples and a stationary control as references. All four models recover the mean surface, while their covariance recovery differs across model families: DDPM and score-SDE recover the covariance structure reasonably well, FM exhibits mildly attenuated non-stationarity and slight variance under-dispersion, and VAE has difficulty recovering the covariance structure. An experiment on ERA5 temperature anomalies further demonstrates how the framework can support the validation and development of DGMs for complex real-world spatio-temporal data.
Figures
Reference graph
Works this paper leans on
-
[1]
Bhambra, P.; Joachimi, B.; Lahav, O.; and Piras, D. 2025. Psi- GAN : a power-spectrum-informed generative adversarial network for the emulation of large-scale structure maps across cosmologies and redshifts. Monthly Notices of the Royal Astronomical Society, 536(3): 3138--3157
2025
-
[2]
K.; Macke, J
Bischoff, S.; Darcher, A.; Deistler, M.; Gao, R.; Gerken, F.; Gloeckler, M.; Haxel, L.; Kapoor, J.; Lappalainen, J. K.; Macke, J. H.; Moss, G.; Pals, M.; Pei, F.; Rapp, R.; Sagtekin, A. E.; Schr\"oder, C.; Schulz, A.; Stefanidi, Z.; Toyota, S.; Ulmer, L.; and Vetter, J. 2024. A practical guide to sample-based statistical distances for evaluating generativ...
2024
-
[3]
B\"ulte, C.; Horat, N.; Quinting, J.; and Lerch, S. 2025. Uncertainty quantification for data-driven weather models. Artificial Intelligence for the Earth Systems
2025
-
[4]
Cardoso, G. V.; and Pereira, M. 2025. Predictive posterior sampling from non-stationary Gaussian process priors via diffusion models with application to climate data. arXiv:2505.24556
Pith/arXiv arXiv 2025
-
[5]
Cressie, N. 1985. Fitting variogram models by weighted least squares. Journal of the International Association for Mathematical Geology, 17(5): 563--586
1985
-
[6]
Cressie, N.; and Wikle, C. K. 2011. Statistics for Spatio-Temporal Data. Hoboken, NJ: Wiley
2011
-
[7]
J.; and Ribeiro Jr., P
Diggle, P. J.; and Ribeiro Jr., P. J. 2007. Model-based Geostatistics. New York: Springer
2007
-
[8]
Ferro, C. A. T. 2014. Fair scores for ensemble forecasts. Quarterly Journal of the Royal Meteorological Society, 140(683): 1917--1923
2014
-
[9]
W.; Rezende, D.; and Eslami, S
Garnelo, M.; Rosenbaum, D.; Maddison, C.; Ramalho, T.; Saxton, D.; Shanahan, M.; Teh, Y. W.; Rezende, D.; and Eslami, S. M. A. 2018
2018
-
[10]
Gneiting, T.; and Raftery, A. E. 2007. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477): 359--378
2007
-
[11]
In International Conference on Learning Representations
Convolutional conditional neural processes. In International Conference on Learning Representations
-
[12]
Hamill, T. M. 2001. Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review, 129(3): 550--560
2001
-
[13]
Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems
2020
-
[14]
Karras, T.; Aittala, M.; Aila, T.; and Laine, S. 2022. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems
2022
-
[15]
P.; and Welling, M
Kingma, D. P.; and Welling, M. 2014. Auto-encoding variational Bayes . In International Conference on Learning Representations
2014
-
[16]
Li, L.; Carver, R.; Lopez-Gomez, I.; Sha, F.; and Anderson, J. 2024. Generative emulation of weather forecast ensembles with diffusion models. Science Advances, 10(13): eadk4489
2024
-
[17]
Lipman, Y.; Chen, R. T. Q.; Ben-Hamu, H.; Nickel, M.; and Le, M. 2023. Flow matching for generative modeling. In International Conference on Learning Representations
2023
-
[18]
Lipman, Y.; Havasi, M.; Holderrieth, P.; Shaul, N.; Le, M.; Karrer, B.; Chen, R. T. Q.; Lopez-Paz, D.; Ben-Hamu, H.; and Gat, I. 2024. Flow matching guide and code. arXiv:2412.06264
Pith/arXiv arXiv 2024
-
[19]
M\"alicke, M. 2022. SciKit-GStat 1.0: a SciPy -flavored geostatistical variogram estimation toolbox written in Python . Geoscientific Model Development, 15(6): 2505--2532
2022
-
[20]
A.; Ge, T.; Subramaniam, A.; Kashinath, K.; Kautz, J.; and Pritchard, M
Mardani, M.; Brenowitz, N.; Cohen, Y.; Pathak, J.; Chen, C.-Y.; Liu, C.-C.; Vahdat, A.; Nabian, M. A.; Ge, T.; Subramaniam, A.; Kashinath, K.; Kautz, J.; and Pritchard, M. 2025. Residual corrective diffusion modeling for km-scale atmospheric downscaling. Communications Earth & Environment, 6: 124
2025
-
[21]
Matheron, G. 1963. Principles of geostatistics. Economic Geology, 58(8): 1246--1266
1963
-
[22]
Nichol, A.; and Dhariwal, P. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning
2021
-
[23]
J.; and Schervish, M
Paciorek, C. J.; and Schervish, M. J. 2003. Nonstationary covariance functions for Gaussian process regression. In Advances in Neural Information Processing Systems
2003
-
[24]
J.; and Schervish, M
Paciorek, C. J.; and Schervish, M. J. 2006. Spatial modelling using a new class of nonstationary covariance functions. Environmetrics, 17(5): 483--506
2006
-
[25]
R.; El-Kadi, A.; Masters, D.; Ewalds, T.; Stott, J.; Mohamed, S.; Battaglia, P.; Lam, R.; and Willson, M
Price, I.; Sanchez-Gonzalez, A.; Alet, F.; Andersson, T. R.; El-Kadi, A.; Masters, D.; Ewalds, T.; Stott, J.; Mohamed, S.; Battaglia, P.; Lam, R.; and Willson, M. 2025. Probabilistic weather forecasting with machine learning. Nature, 637: 84--90
2025
-
[26]
B.; Dueben, P.; Bromberg, C.; Sisk, J.; Barrington, L.; Bell, A.; and Sha, F
Rasp, S.; Hoyer, S.; Merose, A.; Langmore, I.; Battaglia, P.; Russell, T.; Sanchez-Gonzalez, A.; Yang, V.; Carver, R.; Agrawal, S.; Chantry, M.; Bouall\`egue, Z. B.; Dueben, P.; Bromberg, C.; Sisk, J.; Barrington, L.; Bell, A.; and Sha, F. 2024. WeatherBench 2: A benchmark for the next generation of data-driven global weather models. Journal of Advances i...
2024
-
[27]
Risser, M. D. 2016. Review: nonstationary spatial modeling, with emphasis on process convolution and covariate-driven approaches. arXiv:1610.02447
Pith/arXiv arXiv 2016
-
[28]
Rozet, F.; and Louppe, G. 2023. Score-based data assimilation. In Advances in Neural Information Processing Systems
2023
-
[29]
Salimans, T.; and Ho, J. 2022. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations
2022
-
[30]
Scheuerer, M.; and Hamill, T. M. 2015. Variogram-based proper scoring rules for probabilistic forecasts of multivariate quantities. Monthly Weather Review, 143(4): 1321--1334
2015
-
[31]
Song, J.; Meng, C.; and Ermon, S. 2021. Denoising diffusion implicit models. In International Conference on Learning Representations
2021
-
[32]
P.; Kumar, A.; Ermon, S.; and Poole, B
Song, Y.; Sohl-Dickstein, J.; Kingma, D. P.; Kumar, A.; Ermon, S.; and Poole, B. 2021. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations
2021
-
[33]
Stein, M. L. 1999. Interpolation of Spatial Data: Some Theory for Kriging. New York: Springer
1999
-
[34]
Subramanian, A. K. 2020. PyTorch-VAE : A collection of variational autoencoder implementations in PyTorch . GitHub repository. https://github.com/AntixK/PyTorch-VAE
2020
-
[35]
L.; Scheuerer, M.; and Heinz, C
Thorarinsdottir, T. L.; Scheuerer, M.; and Heinz, C. 2016. Assessing the calibration of high-dimensional ensemble forecasts using rank histograms. Journal of Computational and Graphical Statistics, 25(1): 105--122
2016
-
[36]
Tong, A.; Fatras, K.; Malkin, N.; Huguet, G.; Zhang, Y.; Rector-Brooks, J.; Wolf, G.; and Bengio, Y. 2024. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research
2024
-
[37]
Walchessen, J.; Zammit-Mangion, A.; Huser, R.; and Kuusela, M. 2025. Neural conditional simulation for complex spatial processes. arXiv:2508.20067
Pith/arXiv arXiv 2025
-
[38]
Wang, B.; and Pehlevan, C. 2025. An analytical theory of spectral bias in the learning dynamics of diffusion models. In Advances in Neural Information Processing Systems
2025
-
[39]
Wang, K.; Abdulah, S.; Sun, Y.; and Genton, M. G. 2023. Which parameterization of the Mat\'ern covariance function? Spatial Statistics, 58: 100787
2023
-
[40]
Wang, P. 2020. denoising-diffusion-pytorch: Implementation of denoising diffusion probabilistic models in PyTorch . GitHub repository. https://github.com/lucidrains/denoising-diffusion-pytorch
2020
-
[41]
Warner, J. E.; Shah, T. A.; Leser, P. E.; Bomarito, G. F.; Pribe, J. D.; and Stanley, M. C. 2025. Latent generative modeling of random fields from limited training data. arXiv:2505.13007
Pith/arXiv arXiv 2025
-
[42]
C.; and Landau, B
Dowson, D. C.; and Landau, B. V. 1982. The Fr\'echet distance between multivariate normal distributions. Journal of Multivariate Analysis, 12(3): 450--455
1982
-
[43]
Bhatia, R.; Jain, T.; and Lim, Y. 2019. On the Bures--Wasserstein distance between positive definite matrices. Expositiones Mathematicae, 37(2): 165--191
2019
This paper was first reviewed by deepseek-v4-flash on August 1, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.