REVIEW 3 minor 2 cited by
Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances
T0 review · 0 major / 3 minor · reviewed 2026-05-19 · grok-4.3
Pith's one-line read Dimension and model reduction exploit data-informed subspaces to approximate posteriors efficiently in linear Bayesian inverse problems with rank-deficient priors.
desk verdict This paper gives workable dimension reduction for general linear Bayesian inverses with rank-deficient priors plus targeted model reduction for linear dynamical system initial conditions, backed by error bounds and numerics. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The data-informed low-dimensional subspace of the parameter space, which carries the argument by letting the prior and posterior be represented and sampled only within that subspace, together with reduced forward operators for dynamical systems that lower the cost of each likelihood evaluation.
What would settle it
A concrete counter-example in which the Wasserstein distance or covariance error between the reduced posterior and the exact posterior exceeds the paper's derived bound on a linear test problem with known ground-truth posterior would falsify the approximation guarantees.
Extended reading notes
Core claim
For linear Bayesian inverse problems with rank-deficient prior covariances, restricting inference to the low-dimensional subspace informed by the data yields posterior approximations with provable accuracy guarantees; for the special case of initial-condition inference in linear dynamical systems, replacing the forward map with a reduced-order model achieves similar efficiency while preserving the same guarantees.
Load-bearing premise
The data inform only a low-dimensional subspace of the parameter space and the forward model can be accurately approximated by a reduced model without introducing errors that propagate into the posterior in uncontrolled ways.
Editorial extensions
If this is right
- Posterior sampling becomes feasible at far lower cost by operating only in the informed directions rather than the full space.
- For dynamical-system initial-condition problems the number of expensive forward evaluations drops while the posterior remains close to the unreduced one.
- Error bounds supplied by the theory let practitioners choose reduction dimensions that meet a target accuracy level.
- The same framework applies directly to many engineering and science settings where priors are singular because of conservation laws or discretization choices.
Reading between the lines
- The subspace idea might carry over to mildly nonlinear forward maps if the effective dimension of the data-informed region stays small locally.
- Pairing the reductions with existing MCMC or ensemble samplers could produce practical codes for real-time data assimilation.
- Studying how the rank deficiency interacts with the choice of reduced basis could yield adaptive algorithms that pick the smallest sufficient subspace automatically.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript develops dimension reduction techniques for general linear Bayesian inverse problems with rank-deficient prior covariances, exploiting the low-dimensional data-informed subspace via range-nullspace decomposition of the prior covariance and its pseudo-inverse. It also introduces model reduction approaches targeted at initial-condition inference for linear dynamical systems, deriving posterior approximation bounds that control contributions from the data-informed subspace and reduced-model errors. Theoretical guarantees are provided alongside numerical experiments demonstrating accuracy and efficiency for linear forward operators.
Significance. If the bounds hold, the work offers a practical advance for high-dimensional Bayesian inference by directly addressing rank-deficient priors common in applications such as initial-condition estimation. The explicit control of approximation errors in the relevant norms and the separation of dimension versus model reduction represent clear strengths, with the numerical confirmation for linear cases supporting applicability.
minor comments (3)
- [§2.1] §2.1: the notation for the pseudo-inverse and the orthogonal decomposition of the prior covariance could be accompanied by a brief remark on how the nullspace component is handled in sampling to improve readability.
- [Numerical experiments] Table 1 and Figure 3: the reported error norms and computational timings would benefit from explicit statement of the parameter dimensions and the number of Monte Carlo samples used, to allow direct comparison with unreduced baselines.
- [Introduction] The abstract states that model reduction is specific to initial-condition inference; a short sentence in the introduction clarifying why the same reduction does not immediately extend to general parameter inference would avoid potential reader confusion.
Simulated Author's Rebuttal
We thank the referee for their positive assessment of the manuscript, accurate summary of our dimension and model reduction techniques for linear Bayesian inverse problems with rank-deficient prior covariances, and recommendation for minor revision. We appreciate the recognition of the practical advances, theoretical guarantees, and numerical experiments.
Circularity Check
No significant circularity; new reduction techniques with independent theoretical guarantees
full rationale
The paper proposes dimension and model reduction methods for linear Bayesian inverse problems with rank-deficient priors, deriving approximation bounds from the range-nullspace decomposition of the prior covariance via its pseudo-inverse and explicit control of data-informed subspace contributions. These steps are self-contained mathematical constructions that do not reduce to fitted parameters renamed as predictions, self-definitional loops, or load-bearing self-citations whose validity depends on the current work. The central claims rest on standard linear algebra and posterior analysis rather than circular reductions, consistent with the reader's assessment of independent content.
Assumptions & free parameters
assumptions (2)
- domain assumption The forward model is linear and the prior covariance is rank-deficient but positive semi-definite.
- domain assumption Data are informative only in a low-dimensional subspace of the parameter space.
Cite this review
Pith. "Pith review of Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances." pith.science (2026). https://pith.science/paper/2506.23892
@misc{pith2026250623892,
author = {Pith},
title = {Pith review of: Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances},
year = {2026},
howpublished = {\url{https://pith.science/paper/2506.23892}},
note = {Machine review of arXiv:2506.23892}
}
read the original abstract
Bayesian inverse problems use observed data to update a prior probability distribution for an unknown state or parameter of a scientific system to a posterior distribution conditioned on the data. In many applications, the unknown parameter is high-dimensional, making computation of the posterior expensive due to the need to sample in a high-dimensional space and the need to evaluate an expensive high-dimensional forward model relating the unknown parameter to the data. However, inverse problems often exhibit low-dimensional structure due to the fact that the available data are only informative in a low-dimensional subspace of the parameter space. Dimension reduction approaches exploit this structure by restricting inference to the low-dimensional subspace informed by the data, which can be sampled more efficiently. Further computational cost reductions can be achieved by replacing expensive high-dimensional forward models with cheaper lower-dimensional reduced models. In this work, we propose new dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances, which arise in many practical inference settings. The dimension reduction approach is applicable to general linear Bayesian inverse problems whereas the model reduction approaches are specific to the problem of inferring the initial condition of a linear dynamical system. We provide theoretical approximation guarantees as well as numerical experiments demonstrating the accuracy and efficiency of the proposed approaches.
Forward citations
Cited by 2 Pith papers
-
Error bounds for approximate posteriors from likelihood-informed reduced-order models
Derives error bounds on the root prior-preconditioned Hessian, posterior covariance, and mean for a Petrov-Galerkin reduced-order model, with exact posterior recovery at the intrinsic dimension.
-
Likelihood-informed Model Reduction for Bayesian Inference of Static Structural Loads
Projecting a linear static PDE model onto the likelihood-informed subspace yields a reduced-order model that matches full-order Bayesian posterior accuracy at one-tenth to one-hundredth the dimension.
Reference graph
Works this paper leans on
-
[2]
Antoulas, A.C.: Approximation of Large-Scale Dynamical Systems. SIAM, Philadelphia (2005). https://doi.org/10.1137/1.9780898718713
-
[3]
Bardsley, J.M.: Computational Uncertainty Quantification for Inverse Problems. SIAM, Philadelphia (2018). https://doi.org/10.1137/1.9781611975383
-
[4]
Brasseur, P., Ballabrera-Poy, J., Verron, J.: Assimilation of altimetric data in the mid-latitude oceans using the Singular Evolutive Extended Kalman filter with an eddy-resolving, primitive equation model. J. Mar. Syst. 22(4), 269–294 (1999) https://doi.org/10.1016/S0924-7963(99)00044-5
-
[5]
Beattie, C., Gugercin, S., Mehrmann, V.: Model reduction for systems with inhomogeneous initial conditions. Syst. Control Lett. 99, 99–106 (2017) https: //doi.org/10.1016/j.sysconle.2016.11.007
-
[6]
Bui-Thanh, T., Ghattas, O., Martin, J., Stadler, G.: A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion. SIAM J. Sci. Comput.35(6), 2494– 2523 (2013) https://doi.org/10.1137/12089586X
-
[7]
Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduc- tion methods for parametric dynamical systems. SIAM Rev. 57(4), 483–531 (2015) https://doi.org/10.1137/130932715
-
[8]
Buehner, M., McTaggart-Cowan, R., Heilliette, S.: An ensemble Kalman fil- ter for numerical weather prediction based on variational data assimilation: VarEnKF. Mon. Weather Rev. 145(2), 617–635 (2017) https://doi.org/10.1175/ MWR-D-16-0106.1
work page 2017
-
[9]
Benner, P., Ohlberger, M., Cohen, A., Willcox, K. (eds.): Model Reduc- tion and Approximation. SIAM, Philadelphia (2017). https://doi.org/10.1137/1. 9781611974829 21
work page doi:10.1137/1 2017
Show all 51 references
-
[10]
GAMM-Mitt
Benner, P., Saak, J.: Numerical solution of large and sparse continuous time algebraic matrix Riccati and Lyapunov equations: A state of the art survey. GAMM-Mitt. 36(1), 32–52 (2013) https://doi.org/10.1002/gamm.201310003
2013 doi
-
[11]
(eds.) Error Statistics in Data Assimilation: Estimation and Modelling, pp
Buehner, M.: In: Lahoz, W., Khattatov, B., Menard, R. (eds.) Error Statistics in Data Assimilation: Estimation and Modelling, pp. 93–112. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-540-74703-1 5
2010 doi
-
[12]
WIREs Climate Change 9(5), 535 (2018) https://doi.org/10.1002/wcc.535
Carrassi, A., Bocquet, M., Bertino, L., Evensen, G.: Data assimilation in the geosciences: An overview of methods, issues, and perspectives. WIREs Climate Change 9(5), 535 (2018) https://doi.org/10.1002/wcc.535
2018 doi
-
[13]
Chapman and Hall/CRC, New York (2009)
Carlin, B.P., Louis, T.A.: Bayesian Methods for Data Analysis. Chapman and Hall/CRC, New York (2009). https://doi.org/10.1201/b14884
2009 doi
-
[14]
Inverse Probl
Cui, T., Martin, J., Marzouk, Y.M., Solonen, A., Spantini, A.: Likelihood- informed dimension reduction for nonlinear inverse problems. Inverse Probl. 30(11), 114015 (2014) https://doi.org/10.1088/0266-5611/30/11/114015
2014 doi
-
[15]
Inverse Probl
Cui, T., Zahm, O.: Data-free likelihood-informed dimension reduction of Bayesian inverse problems. Inverse Probl. 37(4), 045009 (2021) https://doi.org/10.1088/ 1361-6420/abeafb
2021
-
[16]
In: Handbook of Uncertainty Quantification, pp
Dashti, M., Stuart, A.M.: The Bayesian approach to inverse problems. In: Handbook of Uncertainty Quantification, pp. 311–428. Springer, Cham (2017). https://link.springer.com/referenceworkentry/10.1007/978-3-319-12385-1 7
2017 doi
-
[17]
PAMM 24(4), 202400051 (2024) https://doi.org/10.1002/pamm
Freitag, M.A., K¨ onig, J., Qian, E.: Inference-oriented balanced truncation for quadratic dynamical systems: Formulation for Bayesian smoothing and model sta- bility analysis. PAMM 24(4), 202400051 (2024) https://doi.org/10.1002/pamm. 202400051
2024 doi
-
[18]
In: Gra- farend, E.W., Krumm, F.W., Schwarze, V.S
F¨ orstner, W., Moonen, B.: A metric for covariance matrices. In: Gra- farend, E.W., Krumm, F.W., Schwarze, V.S. (eds.) Geodesy-The Challenge of the 3rd Millennium, pp. 299–309. Springer, Berlin, Heidelberg (2003). https://link.springer.com/chapter/10.1007/978-3-662-05296-9 31
2003 doi
-
[19]
Flath, H.P., Wilcox, L.C., Ak¸ celik, V., Hill, J., Van Bloemen Waanders, B., Ghat- tas, O.: Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations. SIAM J. Sci. Comput. 33(1), 407–432 (20...
2011 doi
-
[20]
Gugercin, S., Antoulas, A.C., Beattie, C.: H2 model reduction for large-scale linear dynamical systems. SIAM J. Matrix Anal. Appl. 30(2), 609–638 (2008) https://doi.org/10.1137/060666123 22
2008 doi
-
[21]
Galbally, D., Fidkowski, K., Willcox, K., Ghattas, O.: Non-linear model reduc- tion for uncertainty quantification in large-scale inverse problems. Int. J. Numer. Methods Eng. 81(12), 1581–1608 (2010) https://doi.org/10.1002/nme.2746
2010 doi
-
[22]
Internat
Gawronski, W., Juang, J.-N.: Model reduction in limited time and frequency inter- vals. Internat. J. Systems Sci. 21(2), 349–376 (1990) https://doi.org/10.1080/ 00207729008910366
1990
-
[23]
Springer, Cham (2021)
Heard, N.: An Introduction to Bayesian Inference, Methods and Computation. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-82808-0
2021 doi
-
[24]
SIAM, Philadel- phia (2008)
Higham, N.J.: Functions of Matrices: Theory and Computation. SIAM, Philadel- phia (2008). https://doi.org/10.1137/1.9780898717778
2008 doi
-
[25]
Haben, S.A., Lawless, A.S., Nichols, N.K.: Conditioning and preconditioning of the variational data assimilation problem. Comput. Fluids 46(1), 252–256 (2011) https://doi.org/10.1016/j.compfluid.2010.11.025
2011 doi
-
[26]
Houtekamer, P.L., Mitchell, H.L., Pellerin, G., Buehner, M., Charron, M., Spacek, L., Hansen, B.: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Weather Rev.133(3), 604–620 (2005) https: //doi.org/10.1175/MWR-2864.1
2005 doi
-
[27]
Automatica 47(3), 559–564 (2011) https://doi.org/10.1016/j.automatica.2010.12.002
Heinkenschloss, M., Reis, T., Antoulas, A.C.: Balanced truncation model reduc- tion for systems with inhomogeneous initial conditions. Automatica 47(3), 559–564 (2011) https://doi.org/10.1016/j.automatica.2010.12.002
2011 doi
-
[28]
Springer, Cham (2015)
Hesthaven, J.S., Rozza, G., Stamm, B.: Certified Reduced Basis Methods for Parametrized Partial Differential Equations. Springer, Cham (2015). https://doi. org/10.1007/978-3-319-22470-1
2015 doi
-
[29]
K¨ onig, J., Freitag, M.A.: Time-limited balanced truncation for data assim- ilation problems. J. Sci. Comput. 97(47) (2023) https://doi.org/10.1007/ s10915-023-02358-4
2023
-
[30]
PAMM 23(3), 202300019 (2023) https://doi.org/10.1002/pamm.202300019
K¨ onig, J., Freitag, M.A.: Time-limited balanced truncation within incremental four-dimensional variational data assimilation. PAMM 23(3), 202300019 (2023) https://doi.org/10.1002/pamm.202300019
2023 doi
-
[31]
Kitanidis, P.K.: Compressed state Kalman filter for large systems. Adv. Water Resour. 76, 120–126 (2015) https://doi.org/10.1016/j.advwatres.2014.12.010
2015 doi
-
[32]
K¨ urschner, P.: Balanced truncation model order reduction in limited time inter- vals for large systems. Adv. Comput. Math. 44(6), 1821–1844 (2018) https: //doi.org/10.1007/s10444-018-9608-6
2018 doi
-
[33]
Livings, D.M., Dance, S.L., Nichols, N.K.: Unbiased ensemble square root filters. 23 Phys. D: Nonlinear Phenom. 237(8), 1021–1028 (2008) https://doi.org/10.1016/ j.physd.2008.01.005
2008
-
[34]
Lipponen, A., Seppanen, A., Kaipio, J.P.: Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition. J. Electron. Imaging 22(2), 023008 (2013) https://doi.org/10.1117/1.JEI.22.2. 023008
2013 doi
-
[35]
Texts in Applied Mathematics
Law, K., Stuart, A., Zygalakis, K.: Data Assimilation: A Mathematical Introduc- tion. Texts in Applied Mathematics. Springer, Cham (2015). https://doi.org/10. 1007/978-3-319-20325-6
2015
-
[36]
SIAM Rev
Lieberman, C., Willcox, K.: Goal-oriented inference: Approach, linear theory, and application to advection diffusion. SIAM Rev. 55(3), 493–519 (2013) https: //doi.org/10.1137/130913110
2013 doi
-
[37]
Lieberman, C., Willcox, K., Ghattas, O.: Parameter and state model reduction for large-scale statistical inverse problems. SIAM J. Sci. Comput. 32(5), 2523–2542 (2010) https://doi.org/10.1137/090775622
2010 doi
-
[38]
IEEE Trans
Moore, B.: Principal component analysis in linear systems: Controllability, observ- ability, and model reduction. IEEE Trans. Automat. Control 26(1), 17–32 (1981) https://doi.org/10.1109/TAC.1981.1102568
1981 doi
-
[39]
IEEE Trans
Mullis, C., Roberts, R.: Synthesis of minimum roundoff noise fixed point digital filters. IEEE Trans. Circuits Syst.23(9), 551–562 (1976) https://doi.org/10.1109/ TCS.1976.1084254
1976
-
[40]
Martin, J., Wilcox, L.C., Burstedde, C., Ghattas, O.: A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion. SIAM J. Sci. Comput. 34(3), 1460–1487 (2012) https: //doi.org/10.1137/110845598
2012 doi
-
[41]
(eds.) Data Assimilation for Numerical Weather Prediction: A Review
Navon, I.M.: In: Park, S.K., Xu, L. (eds.) Data Assimilation for Numerical Weather Prediction: A Review. Springer, Berlin, Heidelberg (2009). https://doi. org/10.1007/978-3-540-71056-1 2
2009 doi
-
[42]
Asia Pac
Nguyen, N., Cheong Khoo, B., Willcox, K.: Model order reduction for Bayesian approach to inverse problems. Asia Pac. J. Comput. Engin 1(2) (2014) https: //doi.org/10.1186/2196-1166-1-2
2014 doi
-
[43]
Qian, E., Tabeart, J.M., Beattie, C., Gugercin, S., Jiang, J., Kramer, P.R., Narayan, A.: Model reduction for linear dynamical systems via balancing for Bayesian inference. J. Sci. Comput. 91(29) (2022) https://doi.org/10.1007/ s10915-022-01798-8
2022
-
[44]
SIAM Rev
Rozier, D., Birol, F., Cosme, E., Brasseur, P., Brankart, J.M., Verron, J.: A 24 reduced-order Kalman filter for data assimilation in physical oceanography. SIAM Rev. 49(3), 449–465 (2007) https://doi.org/10.1137/050635717
2007 doi
-
[45]
Spantini, A., Solonen, A., Cui, T., Martin, J., Tenorio, L., Marzouk, Y.: Opti- mal low-rank approximations of Bayesian linear inverse problems. SIAM J. Sci. Comput. 37(6), 2451–2487 (2015) https://doi.org/10.1137/140977308
2015 doi
-
[46]
Schr¨ oder, C., Voigt, M.: Balanced truncation model reduction with a priori error bounds for LTI systems with nonzero initial value. J. Comput. Appl. Math. 420, 114708 (2023) https://doi.org/10.1016/j.cam.2022.114708
2023 doi
-
[47]
SIAM, Philadelphia (2005)
Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Esti- mation. SIAM, Philadelphia (2005). https://doi.org/10.1137/1.9780898717921
2005 doi
-
[48]
Tuan Pham, D., Verron, J., Christine Roubaud, M.: A singular evolutive extended Kalman filter for data assimilation in oceanography. J. Mar. Syst. 16(3), 323–340 (1998) https://doi.org/10.1016/S0924-7963(97)00109-7
1998 doi
-
[49]
van Leeuwen, P.J.: Nonlinear data assimilation in geosciences: an extremely effi- cient particle filter. Q. J. R. Meteorol. Soc. 136(653), 1991–1999 (2010) https: //doi.org/10.1002/qj.699
1991 doi
-
[50]
Stochastic Hydrol
Verlaan, M., Heemink, A.W.: Tidal flow forecasting using reduced rank square root filters. Stochastic Hydrol. Hydraul. 11, 349–368 (1997) https://doi.org/10. 1007/BF02427924
1997
-
[51]
Wang, J., Zabaras, N.: Using Bayesian statistics in the estimation of heat source in radiation. Int. J. Heat Mass Transf. 48(1), 15–29 (2005) https://doi.org/10. 1016/j.ijheatmasstransfer.2004.08.009
2005
-
[52]
Zahm, O., Cui, T., Law, K., Spantini, A., Marzouk, Y.: Certified dimension reduc- tion in nonlinear Bayesian inverse problems. Math. Comput.91, 1789–1835 (2022) https://doi.org/10.1090/MCOM/3737 25
2022 doi
Reviewed May 19, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.