pith. sign in

arxiv: 2604.13563 · v1 · submitted 2026-04-15 · 🧮 math.NA · cs.NA· math.ST· stat.TH

Covariance-Informed Subspace: an Adaptive Gradient-Free Input Dimension Reduction Method for Bayesian Inference

Nad\`ege Polette , Olivier Le Ma\^itre (CMAP) , Pierre Sochala (ASNR) , Alexandrine Gesret This is my paper

Pith reviewed 2026-05-10 12:44 UTC · model grok-4.3

classification 🧮 math.NA cs.NAmath.STstat.TH
keywords inferencegradientindicatormethodsubspacesapproximatebayesiancase
0
0 comments X

The pith

A covariance-ratio-based gradient-free method identifies likelihood-informed subspaces for dimension reduction in Bayesian inference, yielding better posterior approximations in linear Gaussian settings and practical results in nonlinear high-dimensional applications.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Bayesian inference tries to update beliefs about unknown parameters using data, but when the parameters are fields with millions of values, like underground water flow or air pollution levels, the math becomes impossible to handle directly. The usual fix is to split the huge parameter space into two parts: one part that the data actually tells us about and one part that stays controlled by our prior beliefs. Most existing ways to find the data-informed part need the slope, or gradient, of how well the model fits the data. This paper avoids that by looking at how much the spread of possible values changes from before seeing the data to after. They turn that change into a simple number that flags which directions matter. In simple linear cases where everything is Gaussian, combining this number with a rough version of the data fit gives a closer match to the true updated beliefs. They then show how to approximate the same idea when the relationships are more complicated and test it on real problems from groundwater and the atmosphere.

Core claim

We show that, in the linear Gaussian case, this indicator combined with an approximate likelihood leads to a better posterior approximation. The method is then extended to nonlinear cases, and strategies to approximate the posterior covariance are detailed. We demonstrate the effectiveness of this DR through two high-dimensional inference problems arising from groundwater and atmospheric applications.

Load-bearing premise

The central claim rests on the assumption that an approximate likelihood or covariance estimate can be obtained reliably enough to identify informed directions without gradients, and that this approximation preserves the quality of the posterior in both linear and nonlinear regimes.

Figures

Figures reproduced from arXiv: 2604.13563 by Alexandrine Gesret, Nad\`ege Polette, Olivier Le Ma\^itre (CMAP), Pierre Sochala (ASNR).

Figure 1
Figure 1. Figure 1: Groundwater case - (top) Few samples of log f and (bottom) their corresponding solutions. (left) is the true log-field (log ftrue) and its solution (htrue). (bottom left) Black circles are sensor locations and red stars are selected spatial locations. and the wMC approximation of the posterior covariance CP (35). A draw according to the prior is equivalent to consider that the whole space is non-informed (… view at source ↗
Figure 2
Figure 2. Figure 2: Groundwater case - (left) Convergence of the Förstner distance for the estimation of [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Groundwater case - (left) Eigenvalues obtained solving the gradient-based (black stars) and [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Groundwater case - Log-coefficient fields obtained using the fourth first eigenvectors of [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Groundwater case - Convergence of the iterative [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Groundwater case - Distribution of the log-field value at three selected locations (see [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Groundwater case - MC approximations according to the reduced dimension r of (left) the expectation, (middle) the variance considering the square root of the weights over the approximate posterior distribution. Computation using 100 draws in the informed subspace, each one completed with 100 draws in the non-informed subspace, which corresponds to a total of 10, 000 draws. (right) Autocorrelation function … view at source ↗
Figure 8
Figure 8. Figure 8: GOMOS case - Observations: light transmissions at three altitudes (25km, 50km and [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: GOMOS case - Log-profiles of the four gas and their distributions. (top row) Prior [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: (left) Eigenvalue of the CIS eigenproblem using the final covariance approximation ob￾tained with the CIS-SMC algorithm. We recall that a small eigenvalue indicates an informed direc￾tion. (right) Cumulative contribution of the CIS modes to each gas. Gradient-free (CIS method) Gradient-based (GIS method) Model solver Forward model Forward model + Adjoint method or FD approximations Number of MCMC evaluati… view at source ↗
Figure 11
Figure 11. Figure 11: Illustration of inference cases where the posterior variance is not smaller than the prior [PITH_FULL_IMAGE:figures/full_fig_p024_11.png] view at source ↗
read the original abstract

This paper addresses the challenge of dimension reduction (DR) in Bayesian inference of high-resolution two-or three-dimensional fields, where a priori parametrizations require a large number of terms. The underlying idea is common to state-of-the-art methods in which the parameter space is decomposed into two subspaces, one informed by the likelihood and one constrained by the prior. DR techniques generally use gradient information from the log-likelihood to derive the corresponding subspaces. However, the gradient may be unavailable or expensive to compute accurately, for instance in the case of simulation-based inference. Inspired by approaches based on likelihood-informed subspaces, we develop a new DR method tailored for settings where gradient computation is not feasible. More specifically, we propose a gradient-free indicator for determining whether a direction is informed by the data. This indicator is derived from the posterior-to-prior covariance ratio introduced in Spantini et al. (2015). We show that, in the linear Gaussian case, this indicator combined with an approximate likelihood leads to a better posterior approximation. The method is then extended to nonlinear cases, and strategies to approximate the posterior covariance are detailed. We demonstrate the effectiveness of this DR through two high-dimensional inference problems arising from groundwater and atmospheric applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript proposes a gradient-free dimension reduction technique for high-dimensional Bayesian inference called the Covariance-Informed Subspace method. It introduces an indicator derived from the posterior-to-prior covariance ratio (inspired by Spantini et al. 2015) to identify likelihood-informed directions without requiring gradients. The paper shows that, in the linear Gaussian case, combining this indicator with an approximate likelihood yields an improved posterior approximation; it then extends the approach to nonlinear settings via strategies for approximating the posterior covariance and demonstrates the method on two high-dimensional applications from groundwater and atmospheric modeling.

Significance. If the central claims hold, the work would provide a practical tool for dimension reduction in simulation-based or gradient-unavailable Bayesian inference settings, where standard likelihood-informed subspace methods cannot be applied directly. The explicit handling of the linear Gaussian case and the provision of covariance approximation strategies for nonlinear extension represent a clear methodological contribution, with potential impact on fields requiring efficient inference over high-resolution fields.

major comments (3)
  1. [Abstract and §3] Abstract and §3 (linear Gaussian analysis): The claim that the indicator combined with an approximate likelihood produces a 'better posterior approximation' is not supported by any reported quantitative metrics (e.g., posterior error norms, KL divergence, or coverage probabilities) or baseline comparisons in the provided description; without these, the improvement cannot be verified as load-bearing for the method's validity.
  2. [§4] §4 (nonlinear extension): The strategies for approximating the posterior covariance are central to extending the indicator beyond the linear Gaussian case, yet the manuscript provides no sensitivity analysis or error bounds showing how approximation error in the covariance propagates to mis-ranking of informed directions; this directly affects whether the subspace reliably captures the target posterior in nonlinear regimes.
  3. [§5] §5 (applications): The demonstrations on the groundwater and atmospheric problems report effectiveness but omit quantitative error metrics, error bars, or comparisons against full-space inference or alternative DR methods, leaving the practical advantage of the gradient-free indicator unquantified.
minor comments (1)
  1. [Methods] Notation for the covariance ratio indicator should be introduced with an explicit equation number in the methods section to improve traceability from the abstract claim.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thorough and constructive review. The comments highlight opportunities to strengthen the quantitative support for our claims. We address each major point below and will revise the manuscript to incorporate the suggested additions.

read point-by-point responses

  1. Referee: [Abstract and §3] Abstract and §3 (linear Gaussian analysis): The claim that the indicator combined with an approximate likelihood produces a 'better posterior approximation' is not supported by any reported quantitative metrics (e.g., posterior error norms, KL divergence, or coverage probabilities) or baseline comparisons in the provided description; without these, the improvement cannot be verified as load-bearing for the method's validity.

    Authors: We agree that explicit quantitative metrics would make the improvement more verifiable. Section 3 contains a theoretical analysis demonstrating that the covariance-ratio indicator, paired with an approximate likelihood, reduces the posterior approximation error relative to the prior in the linear Gaussian case. To address the concern, we will add numerical experiments in the revised manuscript, reporting KL divergences, posterior error norms, and comparisons against baseline methods. revision: yes

  2. Referee: [§4] §4 (nonlinear extension): The strategies for approximating the posterior covariance are central to extending the indicator beyond the linear Gaussian case, yet the manuscript provides no sensitivity analysis or error bounds showing how approximation error in the covariance propagates to mis-ranking of informed directions; this directly affects whether the subspace reliably captures the target posterior in nonlinear regimes.

    Authors: We acknowledge the value of quantifying robustness to covariance approximation error. The current §4 describes practical approximation strategies (e.g., ensemble-based estimates) but does not include propagation analysis. In the revision we will add a sensitivity study, with both theoretical error bounds where possible and numerical experiments showing the effect of covariance error on direction ranking and subspace fidelity. revision: yes

  3. Referee: [§5] §5 (applications): The demonstrations on the groundwater and atmospheric problems report effectiveness but omit quantitative error metrics, error bars, or comparisons against full-space inference or alternative DR methods, leaving the practical advantage of the gradient-free indicator unquantified.

    Authors: We agree that stronger quantitative evidence would better demonstrate practical utility. The current applications illustrate qualitative behavior on high-dimensional problems, but lack the requested metrics. We will revise §5 to include posterior error metrics, error bars from multiple runs, and direct comparisons to full-space inference and alternative dimension-reduction approaches. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation relies on external citation and introduces independent approximations

full rationale

The paper explicitly derives its gradient-free indicator from the posterior-to-prior covariance ratio introduced in the external reference Spantini et al. (2015). It then provides a demonstration that this indicator plus an approximate likelihood improves the posterior in the linear Gaussian case, followed by an extension to nonlinear regimes via detailed strategies for approximating the posterior covariance. These steps add new methodological content (the specific indicator application and approximation tactics) that does not reduce by the paper's own equations to a self-definition, a fitted parameter renamed as prediction, or a load-bearing self-citation chain. The two application demonstrations supply external empirical checks. No load-bearing derivation step collapses to its inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach relies on the posterior-to-prior covariance ratio from Spantini et al. (2015) as the core indicator, plus standard Gaussian assumptions for the linear case and unspecified approximation strategies for the nonlinear case. No new free parameters or invented entities are introduced in the abstract.

axioms (1)
  • domain assumption The posterior-to-prior covariance ratio serves as a valid indicator of data-informed directions even when only an approximate likelihood is available.
    Invoked to justify the gradient-free indicator in both linear and nonlinear settings.

pith-pipeline@v0.9.0 · 5544 in / 1389 out tokens · 47939 ms · 2026-05-10T12:44:46.770735+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Likelihood-informed dimension reduction across tempered Bayesian posteriors

    stat.CO 2026-05 unverdicted novelty 7.0

    Introduces α-LIS, a provable generalization of likelihood-informed subspaces to α-tempered posteriors with practical extensions for limited noisy data and unavailable gradients.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    A tutorial on adaptive MCMC.Statistics and Computing, 18(4):343–373, December 2008

    Christophe Andrieu and Johannes Thoms. A tutorial on adaptive MCMC.Statistics and Computing, 18(4):343–373, December 2008

    work page 2008

  2. [2]

    On the optimality of conditional expectation as a Bregman predictor

    Arindam Banerjee, Xin Guo, and Hui Wang. On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory, 51(7):2664–2669, 2005

    work page 2005

  3. [3]

    On the Convergence of Adaptive Sequential Monte Carlo Methods.The Annals of Applied Probability, 26(2):1111–1146, 2016

    Alexandros Beskos, Ajay Jasra, Nikolas Kantas, and Alexandre Thiery. On the Convergence of Adaptive Sequential Monte Carlo Methods.The Annals of Applied Probability, 26(2):1111–1146, 2016

    work page 2016

  4. [4]

    Dimension reduction via score ratio matching

    Michael Brennan, Ricardo Baptista, and Youssef Marzouk. Dimension reduction via score ratio matching. InNeurIPS 2022 Workshop on Score-Based Methods, 2022

    work page 2022

  5. [5]

    Covariance-based MCMC for high-dimensional Bayesian updating with Sequential Monte Carlo.Probabilistic Engineering Mechanics, 77:103667, July 2024

    Barbara Carrera and Iason Papaioannou. Covariance-based MCMC for high-dimensional Bayesian updating with Sequential Monte Carlo.Probabilistic Engineering Mechanics, 77:103667, July 2024

    work page 2024

  6. [6]

    Coupled input-output dimension reduction: Application to goal-oriented bayesian experimental design and global sensitivity analysis

    Qiao Chen, Élise Arnaud, Ricardo Baptista, and Olivier Zahm. Coupled input-output dimension reduction: Application to goal-oriented bayesian experimental design and global sensitivity analysis. SIAM Journal on Scientific Computing, 47(5):A2403–A2430, 2025

    work page 2025

  7. [7]

    Constantine.Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies

    Paul G. Constantine.Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies. Society for Industrial and Applied Mathematics, Philadelphia, PA, March 2015

    work page 2015

  8. [8]

    Constantine, Carson Kent, and Tan Bui-Thanh

    Paul G. Constantine, Carson Kent, and Tan Bui-Thanh. Accelerating MCMC with active subspaces. SIAM Journal on Scientific Computing, 38(5):A2779–A2805, January 2016

    work page 2016

  9. [9]

    Marzouk, Antti Solonen, and Alessio Spantini

    Tiangang Cui, James Martin, Youssef M. Marzouk, Antti Solonen, and Alessio Spantini. Likelihood-informed dimension reduction for nonlinear inverse problems.Inverse Problems, 30(11):114015, October 2014

    work page 2014

  10. [10]

    Tiangang Cui, Youssef Marzouk, and Karen Willcox. Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction.Journal of Computational Physics, 315:363–387, June 2016

    work page 2016

  11. [11]

    Marzouk, and Karen E

    Tiangang Cui, Youssef M. Marzouk, and Karen E. Willcox. Data-driven model reduction for the Bayesian solution of inverse problems.International Journal for Numerical Methods in Engineering, 102(5):966–990, 2015

    work page 2015

  12. [12]

    Tiangang Cui and Xin T. Tong. A unified performance analysis of likelihood-informed subspace methods. Bernoulli, 28(4):2788–2815, November 2022

    work page 2022

  13. [13]

    Data-free likelihood-informed dimension reduction of Bayesian inverse problems.Inverse Problems, 37(4):045009, March 2021

    Tiangang Cui and Olivier Zahm. Data-free likelihood-informed dimension reduction of Bayesian inverse problems.Inverse Problems, 37(4):045009, March 2021

    work page 2021

  14. [14]

    PhD thesis, Toulouse, ISAE, 2022

    Maxime El Masri.Échantillonnage Préférentiel En Grande Dimension via Des Projections Dans Un Sous-Espace de Petite Dimension. PhD thesis, Toulouse, ISAE, 2022

    work page 2022

  15. [15]

    A Metric for Covariance Matrices

    Wolfgang Förstner and Boudewijn Moonen. A Metric for Covariance Matrices. In Erik W. Grafarend, Friedrich W. Krumm, and Volker S. Schwarze, editors,Geodesy-The Challenge of the 3rd Millennium, pages 299–309. Springer, Berlin, Heidelberg, 2003

    work page 2003

  16. [16]

    Ribeiro, Niklas Wahlström, and Thomas B

    Daniel Gedon, Antônio H. Ribeiro, Niklas Wahlström, and Thomas B. Schön. Invertible Kernel PCA With Random Fourier Features.IEEE Signal Processing Letters, 30:563–567, 2023

    work page 2023

  17. [17]

    Le Maître, Ibrahim Hoteit, and Omar M

    Loïc Giraldi, Olivier P. Le Maître, Ibrahim Hoteit, and Omar M. Knio. Optimal projection of observations in a Bayesian setting.Computational Statistics & Data Analysis, 124:252–276, August 2018

    work page 2018

  18. [18]

    Solving Bayesian Inverse Problems via Variational Autoencoders

    Hwan Goh, Sheroze Sheriffdeen, Jonathan Wittmer, and Tan Bui-Thanh. Solving Bayesian Inverse Problems via Variational Autoencoders. InProceedings of the 2nd Mathematical and Scientific Machine Learning Conference, pages 386–425. PMLR, April 2022. 24

    work page 2022

  19. [19]

    An adaptive Metropolis algorithm.Bernoulli

    Heikki Haario, Eero Saksman, and Johanna Tamminen. An adaptive Metropolis algorithm.Bernoulli. Official Journal of the Bernoulli Society for Mathematical Statistics and Probability, 7(2):223–242, 2001

    work page 2001

  20. [20]

    Kaipio and Erkki Somersalo.Statistical and Computational Inverse Problems, volume 160 of Applied Mathematical Sciences

    Jari P. Kaipio and Erkki Somersalo.Statistical and Computational Inverse Problems, volume 160 of Applied Mathematical Sciences. Springer, New York, NY, 2005

    work page 2005

  21. [21]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes.International Conference on Learning Representations, 2013

    work page 2013

  22. [22]

    Li, Youssef Marzouk, and Olivier Zahm

    Mathew T.C. Li, Youssef Marzouk, and Olivier Zahm. Principal feature detection viaΦ-Sobolev inequalities. Bernoulli, 30(4):2979–3003, 2024

    work page 2024

  23. [23]

    Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems.SIAM Journal on Scientific Computing, 32(5):2523–2542, January 2010

    Chad Lieberman, Karen Willcox, and Omar Ghattas. Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems.SIAM Journal on Scientific Computing, 32(5):2523–2542, January 2010

    work page 2010

  24. [24]

    An investigation into the distribution of ratios of particle solver-based likelihoods, 2026

    Emil Løvbak and Sebastian Krumscheid. An investigation into the distribution of ratios of particle solver-based likelihoods, 2026

    work page 2026

  25. [25]

    Marzouk and H

    Y. Marzouk and H. Najm. Dimensionality Reduction and Polynomial Chaos Acceleration of Bayesian Inference in Inverse Problems.Journal of Computational Physics, 228:1862–1902, April 2009

    work page 1902

  26. [26]

    Change of measure for Bayesian field inversion with hierarchical hyperparameters sampling.Journal of Computational Physics, 529:113888, May 2025

    Nadège Polette, Olivier Le Maître, Pierre Sochala, and Alexandrine Gesret. Change of measure for Bayesian field inversion with hierarchical hyperparameters sampling.Journal of Computational Physics, 529:113888, May 2025

    work page 2025

  27. [27]

    Nonlinear Component Analysis as a Kernel Eigenvalue Problem.Neural Computation, 10(5):1299–1319, 1998

    Bernard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Nonlinear Component Analysis as a Kernel Eigenvalue Problem.Neural Computation, 10(5):1299–1319, 1998

    work page 1998

  28. [28]

    Triangle: Engineering a 2D quality mesh generator and Delaunay triangulator

    Jonathan Richard Shewchuk. Triangle: Engineering a 2D quality mesh generator and Delaunay triangulator. In Ming C. Lin and Dinesh Manocha, editors,Applied Computational Geometry Towards Geometric Engineering, pages 203–222, Berlin, Heidelberg, 1996. Springer

    work page 1996

  29. [29]

    Optimal Low-rank Approximations of Bayesian Linear Inverse Problems.SIAM Journal on Scientific Computing, 37(6):A2451–A2487, January 2015

    Alessio Spantini, Antti Solonen, Tiangang Cui, James Martin, Luis Tenorio, and Youssef Marzouk. Optimal Low-rank Approximations of Bayesian Linear Inverse Problems.SIAM Journal on Scientific Computing, 37(6):A2451–A2487, January 2015

    work page 2015

  30. [30]

    Enhanced uncertainty quantification variational autoencoders for the solution of bayesian inverse problems, 2025

    Andrea Tonini and Luca Dede’. Enhanced uncertainty quantification variational autoencoders for the solution of bayesian inverse problems, 2025

    work page 2025

  31. [31]

    Fast surrogate modeling using dimensionality reduction in model inputs and field output: Application to additive manufacturing

    Manav Vohra, Paromita Nath, Sankaran Mahadevan, and Yung-Tsun Tina Lee. Fast surrogate modeling using dimensionality reduction in model inputs and field output: Application to additive manufacturing. Reliability Engineering & System Safety, 201:106986, September 2020

    work page 2020

  32. [32]

    Certified dimension reduction in nonlinear Bayesian inverse problems.Mathematics of Computation, 91(336):1789–1835, April 2022

    Olivier Zahm, Tiangang Cui, Kody Law, Alessio Spantini, and Youssef Marzouk. Certified dimension reduction in nonlinear Bayesian inverse problems.Mathematics of Computation, 91(336):1789–1835, April 2022

    work page 2022

  33. [33]

    Åke Björck and Gene H. Golub. Numerical methods for computing angles between linear subspaces. Mathematics of Computation, 27(123):579–594, 1973. 25

    work page 1973