pith. sign in

arxiv: 2509.12884 · v2 · pith:WOYCJV24new · submitted 2025-09-16 · 📊 stat.ME · stat.ML

Modeling nonstationary spatial processes with normalizing flows

Pratik Nag , Andrew Zammit-Mangion , Ying Sun This is my paper

Pith reviewed 2026-05-21 23:08 UTC · model grok-4.3

classification 📊 stat.ME stat.ML
keywords nonstationary spatial processesnormalizing flowsneural autoregressive flowsspatial warpinggeostatisticsanisotropic processesArgo floats
0
0 comments X

The pith

Neural autoregressive flows create flexible high-dimensional warpings to model nonstationary spatial processes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Nonstationary spatial processes appear stationary after an appropriate warping of the input space. Earlier warping methods required manual selection of functions and stayed mostly in two dimensions because suitable transformations proved hard to find. The paper replaces those functions with neural autoregressive flows, which are invertible mappings that can produce complex transformations in any dimension. Simulation experiments show the resulting model captures more intricate nonstationary and anisotropic structure than standard alternatives. The approach is then used on three-dimensional ocean temperature and salinity measurements to illustrate practical value.

Core claim

The authors introduce neural autoregressive flows as a warping mechanism that turns nonstationary anisotropic spatial processes into stationary processes on the transformed domain, thereby allowing the same stationary model to be used in three or more dimensions without hand-crafted warping functions.

What carries the argument

Neural autoregressive flows, a class of invertible neural mappings that compose autoregressive transformations to generate complex, high-dimensional spatial warpings.

If this is right

  • Spatial modeling becomes feasible in three or higher dimensions without manual choice of warping functions.
  • The same stationary covariance can be applied after the learned warping to produce more accurate predictions for anisotropic data.
  • Simulation results indicate higher representational capacity than commonly used spatial process models for complex nonstationarity.
  • The framework can be applied directly to real three-dimensional datasets such as ocean float measurements.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The learned warping functions could be inspected to reveal physically meaningful coordinate transformations in the data.
  • The method may be combined with existing geostatistical software by treating the flow output as a transformed coordinate input.
  • Extension to spatio-temporal settings would require adding a time dimension to the flow input while preserving invertibility.

Load-bearing premise

Neural autoregressive flows can learn warpings that correctly represent the nonstationarity present in a given spatial application.

What would settle it

A controlled simulation in which the NAF-based model fails to recover known nonstationary structure better than a stationary Gaussian process or a low-dimensional warping baseline.

Figures

Figures reproduced from arXiv: 2509.12884 by Andrew Zammit-Mangion, Pratik Nag, Ying Sun.

Figure 1
Figure 1. Figure 1: Simulated spatial fields Y1(·) (left panel) and Y2(·) (right panel). we generated data from a Gaussian process under the following “spiral” warping function, f(s1, s2) = r [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Predicted spatial fields (top row) and their corresponding standard errors (bottom [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Same as Figure [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Map of the Argo profiling network consisting of 604 locations in the Atlantic Ocean, [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Predictions and corresponding prediction intervals ( [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Top row: Spatial predictions of ocean temperature from Argo data at three pressure [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Test data (red points) at (29.993◦W, 61.802◦N) and observations (blue points) from the five nearest depth profiles. References Banerjee, S., Carlin, B. P., and Gelfand, A. E. (2003). Hierarchical Modeling and Analysis for Spatial Data. Chapman and Hall/CRC, Boca Raton, FL. Castro Morales, F. E., Gamerman, D., and Paez, M. S. (2013). State space models with spatial deformation. Environmental and Ecological … view at source ↗
read the original abstract

Nonstationary spatial processes can often be represented as stationary processes on a warped spatial domain. Selecting an appropriate spatial warping function for a given application is often difficult and, as a result of this, warping methods have largely been limited to two-dimensional spatial domains. In this paper, we introduce a novel approach to modeling nonstationary, anisotropic spatial processes using neural autoregressive flows (NAFs), a class of invertible mappings capable of generating complex, high-dimensional warpings. Through simulation studies we demonstrate that a NAF-based model has greater representational capacity than other commonly used spatial process models. We apply our proposed modeling framework to a subset of the 3D Argo Floats dataset, highlighting the utility of our framework in real-world applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces neural autoregressive flows (NAFs) as a flexible class of invertible mappings to construct complex, high-dimensional spatial warpings for representing nonstationary and anisotropic spatial processes as stationary processes on the transformed domain. This addresses limitations of prior warping methods, which have been restricted mainly to 2D domains due to difficulties in selecting suitable warping functions. The central claims are that simulation studies demonstrate greater representational capacity for the NAF-based model relative to other commonly used spatial process models, and that the framework is useful for real-world applications as shown on a subset of the 3D Argo Floats dataset.

Significance. If the simulation results are robust, the approach could meaningfully extend warping-based nonstationary modeling to higher dimensions and more intricate covariance structures by automating the learning of warpings via NAFs. This would build on classical ideas in spatial statistics while leveraging modern normalizing-flow techniques, potentially offering improved flexibility over fixed-kernel or manually specified nonstationary models.

major comments (2)
  1. [Simulation studies] The abstract states that simulations demonstrate greater representational capacity, but the manuscript must provide concrete quantitative support for this claim. In the simulation studies section, specify the exact performance metrics (e.g., predictive log-likelihood, MSE on held-out points, or coverage of credible intervals), the baseline models (stationary GPs, other nonstationary constructions such as kernel convolutions or treed GPs), and include tables or figures with numerical results, standard errors, and direct comparisons. Without these details the central claim remains difficult to evaluate.
  2. [Real-data application] For the real-data application, clarify how the NAF warping is fitted and validated on the 3D Argo Floats subset. Report model diagnostics, computational scaling with dimension, and any comparison to alternative nonstationary models on the same data; this is needed to substantiate the utility claim beyond the simulation results.
minor comments (2)
  1. Define all acronyms at first use (e.g., NAF, GP) and ensure consistent notation for the warping function and the induced covariance throughout the text.
  2. Provide more explicit details on the NAF architecture (number of layers, hidden dimensions, activation functions) and training procedure to support reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful review and constructive suggestions. We have carefully considered each comment and revised the manuscript accordingly to provide greater clarity and quantitative support for our claims.

read point-by-point responses

  1. Referee: [Simulation studies] The abstract states that simulations demonstrate greater representational capacity, but the manuscript must provide concrete quantitative support for this claim. In the simulation studies section, specify the exact performance metrics (e.g., predictive log-likelihood, MSE on held-out points, or coverage of credible intervals), the baseline models (stationary GPs, other nonstationary constructions such as kernel convolutions or treed GPs), and include tables or figures with numerical results, standard errors, and direct comparisons. Without these details the central claim remains difficult to evaluate.

    Authors: We agree that the simulation studies section would benefit from more explicit quantitative details to substantiate the claim. In the revised version, we have added Table 2 which presents the average predictive log-likelihood and MSE on held-out points, along with standard errors computed over 10 simulation replicates. The baselines include a stationary Gaussian process and a nonstationary model using kernel convolutions. The NAF-based model outperforms both, with higher log-likelihoods and lower MSE values, confirming greater representational capacity. We have also referenced these results in the abstract. revision: yes

  2. Referee: [Real-data application] For the real-data application, clarify how the NAF warping is fitted and validated on the 3D Argo Floats subset. Report model diagnostics, computational scaling with dimension, and any comparison to alternative nonstationary models on the same data; this is needed to substantiate the utility claim beyond the simulation results.

    Authors: Thank you for this recommendation. We have expanded the real-data application section to describe the fitting process: the NAF is trained by maximizing the log-likelihood using gradient-based optimization. Validation is done through 5-fold cross-validation on the Argo Floats data, reporting predictive performance metrics. We include model diagnostics such as residual plots and coverage probabilities for predictive intervals. For computational scaling, we provide timing results showing that the approach scales to 3D without prohibitive cost. Although we did not run comparisons with other nonstationary models like treed GPs on this dataset (due to implementation challenges in 3D), the simulation studies already demonstrate advantages over standard models, and we have added a discussion of this in the revised text. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper introduces NAF-based warpings for nonstationary spatial processes and supports its claims of greater representational capacity through simulation studies and a real-world Argo Floats application. No load-bearing step reduces by construction to a fitted input, self-definition, or self-citation chain; the method extends classical warping ideas using the known flexibility of neural autoregressive flows without tautological reduction. The derivation remains self-contained against external benchmarks and empirical validation.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central approach rests on the domain assumption that nonstationary processes can be represented via warping to stationarity, with NAF parameters learned from data; no invented entities or additional free parameters beyond standard neural network fitting are explicitly introduced in the abstract.

free parameters (1)
  • NAF network parameters
    Parameters of the neural autoregressive flow are fitted during model training to define the warping function.
axioms (1)
  • domain assumption Nonstationary spatial processes can often be represented as stationary processes on a warped spatial domain
    This representation is presented as a common starting point in the abstract.

pith-pipeline@v0.9.0 · 5650 in / 1247 out tokens · 46471 ms · 2026-05-21T23:08:10.443423+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 2 internal anchors

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence :=...

    work page

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

    work page

  3. [3]

    P., and Gelfand, A

    Banerjee, S., Carlin, B. P., and Gelfand, A. E. (2003). Hierarchical Modeling and Analysis for Spatial Data . Chapman and Hall/CRC, Boca Raton, FL

    work page 2003

  4. [4]

    E., Gamerman, D., and Paez, M

    Castro Morales, F. E., Gamerman, D., and Paez, M. S. (2013). State space models with spatial deformation. Environmental and Ecological Statistics , 20:191--214

    work page 2013

  5. [5]

    J., and Sun, Y

    Chen, W., Li, Y., Reich, B. J., and Sun, Y. (2024). Deepkriging: Spatially dependent deep neural networks for spatial prediction. Statistica Sinica , 34(1):291--311

    work page 2024

  6. [6]

    Cressie, N. (1993). Statistics for Spatial Data . John Wiley & Sons, Hoboken, NJ, revised edition

    work page 1993

  7. [7]

    and Johannesson, G

    Cressie, N. and Johannesson, G. (2008). Fixed rank kriging for very large spatial data sets. Journal of the Royal Statistical Society B , 70(1):209--226

    work page 2008

  8. [8]

    Dinh, L., Krueger, D., and Bengio, Y. (2014). Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  9. [9]

    Germain, M., Gregor, K., Murray, I., and Larochelle, H. (2015). MADE : Masked autoencoder for distribution estimation. In International Conference on Machine Learning , pages 881--889. PMLR

    work page 2015

  10. [10]

    A., Martin, M

    Good, S. A., Martin, M. J., and Rayner, N. A. (2013). EN4 : Quality controlled ocean temperature and salinity profiles and monthly objective analyses with uncertainty estimates. Journal of Geophysical Research: Oceans , 118(12):6704--6716

    work page 2013

  11. [11]

    Higdon, D., Swall, J., and Kern, J. (1999). Non-stationary spatial modeling. In Bernardo, J. M., Berger, J. O., Dawid, A. P., and Smith, A. F. M., editors, Bayesian Statistics 6: Proceedings of the Sixth Valencia International Meeting June 6-10, 1998 , pages 761--768. Oxford University Press, Oxford, UK

    work page 1999

  12. [12]

    Hosoda, S. et al. (2008). New global monthly objective analysis using A rgo data. Journal of Oceanography , 64(4):333--340

    work page 2008

  13. [13]

    Huang, C.-W., Krueger, D., Lacoste, A., and Courville, A. (2018). Neural autoregressive flows. In International Conference on Machine Learning , pages 2078--2087. PMLR

    work page 2018

  14. [14]

    E., Harbin, J., and Richards, C

    Kelley, D. E., Harbin, J., and Richards, C. (2021). A rgofloats: An R package for analyzing A rgo data. Frontiers in Marine Science , 8:635922

    work page 2021

  15. [15]

    P., Bitz, C., Bryan, F., Collins, W., Dennis, J., Hearn, N., Kinter, J

    Kirtman, B. P., Bitz, C., Bryan, F., Collins, W., Dennis, J., Hearn, N., Kinter, J. L., Loft, R., Rousset, C., Siqueira, L., Stan, C., Tomas, R., and Vertenstein, M. (2012). Impact of ocean model resolution on CCSM climate simulations. Climate Dynamics , 39(6):1303--1328

    work page 2012

  16. [16]

    J., and Brubaker, M

    Kobyzev, I., Prince, S. J., and Brubaker, M. A. (2020). Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43(11):3964--3979

    work page 2020

  17. [17]

    and Sun, Y

    Li, Y. and Sun, Y. (2019). Efficient estimation of nonstationary spatial covariance functions with application to high-resolution climate model emulation. Statistica Sinica , 29(3):1209--1231

    work page 2019

  18. [18]

    Lindgren, F., Rue, H., and Lindstr \"o m, J. (2011). An explicit link between G aussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. Journal of the Royal Statistical Society B , 73(4):423--498

    work page 2011

  19. [19]

    Lorenc, A. C. (1986). Analysis methods for numerical weather prediction. Quarterly Journal of the Royal Meteorological Society , 112(474):1177--1194

    work page 1986

  20. [20]

    A., Genton, M

    Nag, P., Hong, Y., Abdulah, S., Qadir, G. A., Genton, M. G., and Sun, Y. (2025). Efficient large-scale nonstationary spatial covariance function estimation using convolutional neural networks. Journal of Computational and Graphical Statistics , 34(2):683--696

    work page 2025

  21. [21]

    Nag, P., Sun, Y., and Reich, B. J. (2023). Spatio-temporal deepkriging for interpolation and probabilistic forecasting. Spatial Statistics , 57:100773

    work page 2023

  22. [22]

    Paciorek, C. J. and Schervish, M. J. (2006). Spatial modelling using a new class of nonstationary covariance functions. Environmetrics , 17(5):483--506

    work page 2006

  23. [23]

    Papamakarios, G., Pavlakou, T., and Murray, I. (2017). Masked autoregressive flow for density estimation. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 30, pages 2339--2348, Red Hook, NY. Curran Associates, Inc

    work page 2017

  24. [24]

    and Monestiez, P

    Perrin, O. and Monestiez, P. (1999). Modelling of non-stationary spatial structure using parametric radial basis deformations. In G \'o mez-Hern \'a ndez, J., Soares, A. O., and Froidevaux, R., editors, geoENV II—Geostatistics for Environmental Applications: Proceedings of the Second European Conference on Geostatistics for Environmental Applications , pa...

    work page 1999

  25. [25]

    A., Sun, Y., and Kurtek, S

    Qadir, G. A., Sun, Y., and Kurtek, S. (2021). Estimation of spatial deformation for nonstationary processes via variogram alignment. Technometrics , 63(4):548--561

    work page 2021

  26. [26]

    Risser, M. D. (2016). Review: Nonstationary spatial modeling, with emphasis on process convolution and covariate-driven approaches. arXiv:1610.02447

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  27. [27]

    C., Riser, S., Davis, R., Gilson, J., Owens, W

    Roemmich, D., Johnson, G. C., Riser, S., Davis, R., Gilson, J., Owens, W. B., Garzoli, S. L., Schmid, C., and Ignaszewski, M. (2009). The A rgo program: Observing the global ocean with profiling floats. Oceanography , 22(2):34--43

    work page 2009

  28. [28]

    Sampson, P. D. and Guttorp, P. (1992). Nonparametric estimation of nonstationary spatial covariance structure. Journal of the American Statistical Association , 87(417):108--119

    work page 1992

  29. [29]

    Schmidt, A. M. and O'Hagan, A. (2003). Bayesian inference for non-stationary spatial covariance structure via spatial deformations. Journal of the Royal Statistical Society B , 65(3):743--758

    work page 2003

  30. [30]

    Schmitt, R. W. (2008). Salinity and the global water cycle. Oceanography , 21(1):12--19

    work page 2008

  31. [31]

    Smith, R. L. (1996). Estimating nonstationary spatial correlations. Technical report. Available from https://rls.sites.oasis.unc.edu/postscript/rs/nonstationary.pdf

    work page 1996

  32. [32]

    and Chassignet, E

    Stammer, D. and Chassignet, E. (2000). Ocean state estimation and prediction in support of oceanographic research. Oceanography , 13(2):51--56

    work page 2000

  33. [33]

    o hler, J., and K \

    Stammer, D., Martins, M. S., K \"o hler, J., and K \"o hl, A. (2021). How well do we know ocean salinity and its changes? Progress in Oceanography , 190:102478

    work page 2021

  34. [34]

    Stein, M. L. (2005). Space--time covariance functions. Journal of the American Statistical Association , 100(469):310--321

    work page 2005

  35. [35]

    Vu, Q., Zammit-Mangion, A., and Cressie, N. (2022). Modeling nonstationary and asymmetric multivariate spatial covariances via deformations. Statistica Sinica , 32(4):2071--2093

    work page 2022

  36. [36]

    A., and Lund, R

    Wang, Q., Parker, P. A., and Lund, R. (2025). Spatial deep convolutional neural networks. Spatial Statistics , 66:100883

    work page 2025

  37. [37]

    Zammit-Mangion, A., Ng, T. L. J., Vu, Q., and Filippone, M. (2022). Deep compositional spatial models. Journal of the American Statistical Association , 117(540):1787--1808

    work page 2022