Variational Approximated Restricted Maximum Likelihood Estimation for Spatial Data

Debjoy Thakur

arxiv: 2604.07635 · v1 · submitted 2026-04-08 · 📊 stat.ML · cs.LG· stat.AP

Variational Approximated Restricted Maximum Likelihood Estimation for Spatial Data

Debjoy Thakur This is my paper

Pith reviewed 2026-05-10 16:50 UTC · model grok-4.3

classification 📊 stat.ML cs.LGstat.AP

keywords spatial statisticsvariational inferencerestricted maximum likelihoodICAR modelsGaussian processesevidence lower boundcoordinate ascentspatial random effects

0 comments

The pith

VREML approximates restricted maximum likelihood for spatial ICAR data and becomes exact under Gaussian assumptions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops a variational restricted maximum likelihood method, called VREML, to estimate parameters in spatial models that use Gaussian intrinsic conditional autoregressive structures. Classical REML estimation requires repeated inversion and factorization of large sparse precision matrices, which grows expensive as the number of locations increases. VREML replaces that step with a Gaussian variational distribution and an evidence lower bound on the restricted likelihood, then optimizes the bound with a coordinate-ascent algorithm that jointly updates random effects and variance components. The authors prove that the bound increases at every iteration and that the chosen variational family recovers the exact posterior when the model is Gaussian ICAR, removing any approximation error at the posterior level. Numerical comparisons show VREML produces more accurate estimates than both maximum likelihood and INLA while running faster on large spatial grids.

Core claim

We propose a variational restricted maximum likelihood (VREML) framework that approximates the intractable marginal likelihood using a Gaussian variational distribution. By constructing an evidence lower bound (ELBO) on the restricted likelihood, we derive a computationally efficient coordinate-ascent algorithm for jointly estimating the spatial random effects and variance components. We theoretically establish the monotone convergence of ELBO and mathematically exhibit that the variational family is exact under Gaussian ICAR settings, which is an indication of nullifying approximation error at the posterior level. We empirically establish the supremacy of our VREML over MLE and INLA.

What carries the argument

A Gaussian variational family that constructs and maximizes an evidence lower bound (ELBO) on the restricted likelihood through coordinate ascent.

If this is right

The ELBO is guaranteed to increase at each coordinate-ascent step until a stationary point is reached.
Under Gaussian ICAR models the variational posterior coincides exactly with the true posterior, so no additional error is introduced.
Parameter estimates are obtained without repeated inversion of the full precision matrix.
Numerical tests on spatial data show VREML recovers variance components more accurately than MLE or INLA while using less computation time.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The exactness result may hold for other Gaussian conditional autoregressive specifications that share the same precision-matrix structure.
The coordinate-ascent scheme could be extended to non-stationary or multivariate spatial fields by choosing a richer but still tractable variational family.
Because the method avoids matrix factorizations, it may enable routine analysis of spatial datasets with tens or hundreds of thousands of locations that remain out of reach for classical REML.

Load-bearing premise

The variational family is exact under Gaussian ICAR settings with no approximation error at the posterior level, and the ELBO construction plus coordinate ascent yields reliable parameter estimates without bias from the variational choice.

What would settle it

On small Gaussian ICAR datasets, compare the posterior obtained from the variational coordinate-ascent procedure with the exact posterior obtained by direct REML; any systematic difference in the estimated variance components or random effects would falsify the exactness claim.

Figures

Figures reproduced from arXiv: 2604.07635 by Debjoy Thakur.

**Figure 2.** Figure 2: Comparison of prediction performance for spatial random effect [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

**Figure 3.** Figure 3: RMSE comparison for error variance with increasing sample size. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

**Figure 4.** Figure 4: Spatial variation of observed and predicted EPCAM gene count. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

read the original abstract

This research considers a scalable inference for spatial data modeled through Gaussian intrinsic conditional autoregressive (ICAR) structures. The classical estimation method, restricted maximum likelihood (REML), requires repeated inversion and factorization of large, sparse precision matrices, which makes this computation costly. To sort this problem out, we propose a variational restricted maximum likelihood (VREML) framework that approximates the intractable marginal likelihood using a Gaussian variational distribution. By constructing an evidence lower bound (ELBO) on the restricted likelihood, we derive a computationally efficient coordinate-ascent algorithm for jointly estimating the spatial random effects and variance components. In this article, we theoretically establish the monotone convergence of ELBO and mathematically exhibit that the variational family is exact under Gaussian ICAR settings, which is an indication of nullifying approximation error at the posterior level. We empirically establish the supremacy of our VREML over MLE and INLA.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

VREML gives exact recovery of the restricted likelihood for Gaussian ICAR models via a Gaussian variational family, which is the cleanest part, though the overall gains look incremental.

read the letter

The main thing to know is that this paper sets up a variational approximation to REML specifically for Gaussian ICAR spatial models and shows that the approximation becomes exact when the variational distribution is also Gaussian. Because the joint model is jointly Gaussian, the variational family can match the true conditional posterior exactly, so the ELBO equals the restricted log-likelihood with no gap. That is a neat theoretical observation for this narrow model class and removes the usual worry about variational bias at the posterior level. They also derive a coordinate-ascent algorithm that jointly updates the spatial effects and variance components while avoiding repeated large-matrix inversions, and they prove monotone ELBO convergence, which follows directly from standard variational arguments but is still useful to have stated for this setting. Empirically they claim outperformance over MLE and INLA, which would matter if the speed or accuracy edge holds on realistic large grids. The soft spots are modest but real. The exactness result is tied tightly to Gaussian ICAR, so it does not extend to non-Gaussian observations or other covariance structures without reintroducing approximation error. The empirical comparisons are asserted in the abstract without numbers, simulation details, or scaling curves, so it is hard to judge whether the reported supremacy is robust or sensitive to tuning. The method stays within existing variational and spatial-stats toolkits rather than introducing a new paradigm. This paper is for spatial statisticians or geostatisticians who already work with ICAR models and need faster REML on big datasets. A reader comfortable with variational inference will see the value in the exactness argument and the coordinate-ascent derivation. It deserves peer review because the theoretical claim is specific and checkable, and the computational motivation is practical even if the advance is incremental.

Referee Report

0 major / 0 minor

Summary. The manuscript proposes VREML, a variational approximation to restricted maximum likelihood (REML) estimation for Gaussian spatial data with intrinsic conditional autoregressive (ICAR) structures. It constructs an evidence lower bound (ELBO) on the restricted likelihood using a Gaussian variational family, derives coordinate-ascent updates for jointly estimating spatial random effects and variance components, proves monotone ELBO convergence, mathematically shows that the Gaussian variational family is exact (nullifying approximation error) for Gaussian ICAR models, and reports empirical outperformance relative to MLE and INLA.

Significance. If the exactness result holds, VREML supplies a computationally scalable procedure that is equivalent to exact REML for the Gaussian ICAR class while avoiding repeated large-matrix inversions; the monotone convergence guarantee is standard for coordinate ascent but useful here. The empirical comparisons, if reproducible, would support adoption in spatial statistics where REML remains the default but is often intractable.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive and accurate summary of our manuscript, as well as for the recommendation of minor revision. The significance assessment correctly identifies the potential value of the exactness result and monotone convergence guarantee for scalable REML in the Gaussian ICAR setting. No major comments were raised in the report.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper constructs an ELBO on the restricted likelihood via standard variational principles and derives a coordinate-ascent optimizer for the Gaussian ICAR model. Monotone ELBO convergence follows from the generic properties of coordinate ascent on a concave bound and requires no self-referential step. The exactness result—that a Gaussian variational family recovers the true restricted likelihood exactly when the model is jointly Gaussian—is a direct algebraic consequence of matching means and covariances in the Gaussian case; it is exhibited mathematically from the model definition rather than fitted to data or smuggled via self-citation. No equation reduces to its own input by construction, no parameter is renamed as a prediction, and the central claims remain independent of any load-bearing self-citation chain.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

Limited information available from abstract alone; the approach rests on standard variational inference and spatial statistics assumptions rather than new free parameters or invented entities.

free parameters (1)

variance components
Estimated jointly via coordinate ascent; specific initialization or scaling choices not detailed in abstract.

axioms (2)

domain assumption Gaussian intrinsic conditional autoregressive (ICAR) structure for spatial random effects
Core modeling assumption stated in abstract for which the exactness result is claimed.
standard math Existence of a valid evidence lower bound on the restricted likelihood
Follows from standard variational inference theory applied to the REML objective.

pith-pipeline@v0.9.0 · 5446 in / 1375 out tokens · 47845 ms · 2026-05-10T16:50:22.563341+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

1 extracted references · 1 canonical work pages

[1]

Concentration of tempered posteriors and of their variational approximations.Annals of Statistics, 48(3), 2020

P Alquier and JA Ridgway. Concentration of tempered posteriors and of their variational approximations.Annals of Statistics, 48(3), 2020. Prateek Bansal, Rico Krueger, and Daniel J Graham. Fast bayesian esti- mation of spatial count data models.Computational Statistics & Data Analysis, 157:107152, 2021. Julian Besag and Charles Kooperberg. On conditional ...

work page 2020

[1] [1]

Concentration of tempered posteriors and of their variational approximations.Annals of Statistics, 48(3), 2020

P Alquier and JA Ridgway. Concentration of tempered posteriors and of their variational approximations.Annals of Statistics, 48(3), 2020. Prateek Bansal, Rico Krueger, and Daniel J Graham. Fast bayesian esti- mation of spatial count data models.Computational Statistics & Data Analysis, 157:107152, 2021. Julian Besag and Charles Kooperberg. On conditional ...

work page 2020