pith. sign in

arxiv: 2506.23892 · v2 · submitted 2025-06-30 · 🧮 math.NA · cs.NA· cs.SY· eess.SY

Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances

Josie K\"onig , Elizabeth Qian , Melina A. Freitag This is my paper

Pith reviewed 2026-05-19 07:29 UTC · model grok-4.3

classification 🧮 math.NA cs.NAcs.SYeess.SY
keywords Bayesian inverse problemsdimension reductionmodel reductionrank-deficient priorslinear dynamical systemsposterior approximationnumerical analysis
0
0 comments X

The pith

Dimension and model reduction exploit data-informed subspaces to approximate posteriors efficiently in linear Bayesian inverse problems with rank-deficient priors.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Bayesian inverse problems often involve high-dimensional unknowns, but data typically constrain only a low-dimensional part of the space. This paper develops dimension reduction that restricts sampling to the subspace informed by the data, applicable to general linear problems even when the prior covariance has deficient rank. For inferring initial conditions in linear dynamical systems it adds model reduction that replaces expensive forward simulations with cheaper lower-dimensional versions. Both methods include theoretical guarantees that bound how much the approximate posterior can differ from the full one. The payoff is feasible computation of posteriors that would otherwise require prohibitive sampling and model evaluations.

Core claim

For linear Bayesian inverse problems with rank-deficient prior covariances, restricting inference to the low-dimensional subspace informed by the data yields posterior approximations with provable accuracy guarantees; for the special case of initial-condition inference in linear dynamical systems, replacing the forward map with a reduced-order model achieves similar efficiency while preserving the same guarantees.

What carries the argument

The data-informed low-dimensional subspace of the parameter space, which carries the argument by letting the prior and posterior be represented and sampled only within that subspace, together with reduced forward operators for dynamical systems that lower the cost of each likelihood evaluation.

If this is right

  • Posterior sampling becomes feasible at far lower cost by operating only in the informed directions rather than the full space.
  • For dynamical-system initial-condition problems the number of expensive forward evaluations drops while the posterior remains close to the unreduced one.
  • Error bounds supplied by the theory let practitioners choose reduction dimensions that meet a target accuracy level.
  • The same framework applies directly to many engineering and science settings where priors are singular because of conservation laws or discretization choices.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The subspace idea might carry over to mildly nonlinear forward maps if the effective dimension of the data-informed region stays small locally.
  • Pairing the reductions with existing MCMC or ensemble samplers could produce practical codes for real-time data assimilation.
  • Studying how the rank deficiency interacts with the choice of reduced basis could yield adaptive algorithms that pick the smallest sufficient subspace automatically.

Load-bearing premise

The data inform only a low-dimensional subspace of the parameter space and the forward model can be accurately approximated by a reduced model without introducing errors that propagate into the posterior in uncontrolled ways.

What would settle it

A concrete counter-example in which the Wasserstein distance or covariance error between the reduced posterior and the exact posterior exceeds the paper's derived bound on a linear test problem with known ground-truth posterior would falsify the approximation guarantees.

read the original abstract

Bayesian inverse problems use observed data to update a prior probability distribution for an unknown state or parameter of a scientific system to a posterior distribution conditioned on the data. In many applications, the unknown parameter is high-dimensional, making computation of the posterior expensive due to the need to sample in a high-dimensional space and the need to evaluate an expensive high-dimensional forward model relating the unknown parameter to the data. However, inverse problems often exhibit low-dimensional structure due to the fact that the available data are only informative in a low-dimensional subspace of the parameter space. Dimension reduction approaches exploit this structure by restricting inference to the low-dimensional subspace informed by the data, which can be sampled more efficiently. Further computational cost reductions can be achieved by replacing expensive high-dimensional forward models with cheaper lower-dimensional reduced models. In this work, we propose new dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances, which arise in many practical inference settings. The dimension reduction approach is applicable to general linear Bayesian inverse problems whereas the model reduction approaches are specific to the problem of inferring the initial condition of a linear dynamical system. We provide theoretical approximation guarantees as well as numerical experiments demonstrating the accuracy and efficiency of the proposed approaches.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript develops dimension reduction techniques for general linear Bayesian inverse problems with rank-deficient prior covariances, exploiting the low-dimensional data-informed subspace via range-nullspace decomposition of the prior covariance and its pseudo-inverse. It also introduces model reduction approaches targeted at initial-condition inference for linear dynamical systems, deriving posterior approximation bounds that control contributions from the data-informed subspace and reduced-model errors. Theoretical guarantees are provided alongside numerical experiments demonstrating accuracy and efficiency for linear forward operators.

Significance. If the bounds hold, the work offers a practical advance for high-dimensional Bayesian inference by directly addressing rank-deficient priors common in applications such as initial-condition estimation. The explicit control of approximation errors in the relevant norms and the separation of dimension versus model reduction represent clear strengths, with the numerical confirmation for linear cases supporting applicability.

minor comments (3)
  1. [§2.1] §2.1: the notation for the pseudo-inverse and the orthogonal decomposition of the prior covariance could be accompanied by a brief remark on how the nullspace component is handled in sampling to improve readability.
  2. [Numerical experiments] Table 1 and Figure 3: the reported error norms and computational timings would benefit from explicit statement of the parameter dimensions and the number of Monte Carlo samples used, to allow direct comparison with unreduced baselines.
  3. [Introduction] The abstract states that model reduction is specific to initial-condition inference; a short sentence in the introduction clarifying why the same reduction does not immediately extend to general parameter inference would avoid potential reader confusion.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive assessment of the manuscript, accurate summary of our dimension and model reduction techniques for linear Bayesian inverse problems with rank-deficient prior covariances, and recommendation for minor revision. We appreciate the recognition of the practical advances, theoretical guarantees, and numerical experiments.

Circularity Check

0 steps flagged

No significant circularity; new reduction techniques with independent theoretical guarantees

full rationale

The paper proposes dimension and model reduction methods for linear Bayesian inverse problems with rank-deficient priors, deriving approximation bounds from the range-nullspace decomposition of the prior covariance via its pseudo-inverse and explicit control of data-informed subspace contributions. These steps are self-contained mathematical constructions that do not reduce to fitted parameters renamed as predictions, self-definitional loops, or load-bearing self-citations whose validity depends on the current work. The central claims rest on standard linear algebra and posterior analysis rather than circular reductions, consistent with the reader's assessment of independent content.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on standard assumptions from Bayesian inverse problems and linear algebra; no new free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption The forward model is linear and the prior covariance is rank-deficient but positive semi-definite.
    Invoked in the setup of the inverse problem throughout the abstract.
  • domain assumption Data are informative only in a low-dimensional subspace of the parameter space.
    Core motivation for dimension reduction stated in the abstract.

pith-pipeline@v0.9.0 · 5764 in / 1303 out tokens · 22697 ms · 2026-05-19T07:29:57.853143+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 51 canonical work pages

  1. [2]

    SIAM, Philadelphia (2005)

    Antoulas, A.C.: Approximation of Large-Scale Dynamical Systems. SIAM, Philadelphia (2005). https://doi.org/10.1137/1.9780898718713

    work page doi:10.1137/1.9780898718713 2005

  2. [3]

    SIAM, Philadelphia (2018)

    Bardsley, J.M.: Computational Uncertainty Quantification for Inverse Problems. SIAM, Philadelphia (2018). https://doi.org/10.1137/1.9781611975383

    work page doi:10.1137/1.9781611975383 2018

  3. [4]

    Brasseur, P., Ballabrera-Poy, J., Verron, J.: Assimilation of altimetric data in the mid-latitude oceans using the Singular Evolutive Extended Kalman filter with an eddy-resolving, primitive equation model. J. Mar. Syst. 22(4), 269–294 (1999) https://doi.org/10.1016/S0924-7963(99)00044-5

    work page doi:10.1016/s0924-7963(99)00044-5 1999

  4. [5]

    Beattie, C., Gugercin, S., Mehrmann, V.: Model reduction for systems with inhomogeneous initial conditions. Syst. Control Lett. 99, 99–106 (2017) https: //doi.org/10.1016/j.sysconle.2016.11.007

    work page doi:10.1016/j.sysconle.2016.11.007 2017

  5. [6]

    Bui-Thanh, T., Ghattas, O., Martin, J., Stadler, G.: A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion. SIAM J. Sci. Comput.35(6), 2494– 2523 (2013) https://doi.org/10.1137/12089586X

    work page doi:10.1137/12089586x 2013

  6. [7]

    Benner, S

    Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduc- tion methods for parametric dynamical systems. SIAM Rev. 57(4), 483–531 (2015) https://doi.org/10.1137/130932715

    work page doi:10.1137/130932715 2015

  7. [8]

    Buehner, M., McTaggart-Cowan, R., Heilliette, S.: An ensemble Kalman fil- ter for numerical weather prediction based on variational data assimilation: VarEnKF. Mon. Weather Rev. 145(2), 617–635 (2017) https://doi.org/10.1175/ MWR-D-16-0106.1

    work page 2017

  8. [9]

    Dolean, P

    Benner, P., Ohlberger, M., Cohen, A., Willcox, K. (eds.): Model Reduc- tion and Approximation. SIAM, Philadelphia (2017). https://doi.org/10.1137/1. 9781611974829 21

    work page doi:10.1137/1 2017

  9. [10]

    GAMM-Mitt

    Benner, P., Saak, J.: Numerical solution of large and sparse continuous time algebraic matrix Riccati and Lyapunov equations: A state of the art survey. GAMM-Mitt. 36(1), 32–52 (2013) https://doi.org/10.1002/gamm.201310003

    work page doi:10.1002/gamm.201310003 2013

  10. [11]

    (eds.) Error Statistics in Data Assimilation: Estimation and Modelling, pp

    Buehner, M.: In: Lahoz, W., Khattatov, B., Menard, R. (eds.) Error Statistics in Data Assimilation: Estimation and Modelling, pp. 93–112. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-540-74703-1 5

    work page doi:10.1007/978-3-540-74703-1 2010

  11. [12]

    WIREs Climate Change , author =

    Carrassi, A., Bocquet, M., Bertino, L., Evensen, G.: Data assimilation in the geosciences: An overview of methods, issues, and perspectives. WIREs Climate Change 9(5), 535 (2018) https://doi.org/10.1002/wcc.535

    work page doi:10.1002/wcc.535 2018

  12. [13]

    Chapman and Hall/CRC, New York (2009)

    Carlin, B.P., Louis, T.A.: Bayesian Methods for Data Analysis. Chapman and Hall/CRC, New York (2009). https://doi.org/10.1201/b14884

    work page doi:10.1201/b14884 2009

  13. [14]

    Inverse Probl

    Cui, T., Martin, J., Marzouk, Y.M., Solonen, A., Spantini, A.: Likelihood- informed dimension reduction for nonlinear inverse problems. Inverse Probl. 30(11), 114015 (2014) https://doi.org/10.1088/0266-5611/30/11/114015

    work page doi:10.1088/0266-5611/30/11/114015 2014

  14. [15]

    Inverse Probl

    Cui, T., Zahm, O.: Data-free likelihood-informed dimension reduction of Bayesian inverse problems. Inverse Probl. 37(4), 045009 (2021) https://doi.org/10.1088/ 1361-6420/abeafb

    work page 2021

  15. [16]

    Dashti and A

    Dashti, M., Stuart, A.M.: The Bayesian approach to inverse problems. In: Handbook of Uncertainty Quantification, pp. 311–428. Springer, Cham (2017). https://link.springer.com/referenceworkentry/10.1007/978-3-319-12385-1 7

    work page doi:10.1007/978-3-319-12385-1 2017

  16. [17]

    PAMM 24(4), 202400051 (2024) https://doi.org/10.1002/pamm

    Freitag, M.A., K¨ onig, J., Qian, E.: Inference-oriented balanced truncation for quadratic dynamical systems: Formulation for Bayesian smoothing and model sta- bility analysis. PAMM 24(4), 202400051 (2024) https://doi.org/10.1002/pamm. 202400051

    work page doi:10.1002/pamm 2024

  17. [18]

    In: Gra- farend, E.W., Krumm, F.W., Schwarze, V.S

    F¨ orstner, W., Moonen, B.: A metric for covariance matrices. In: Gra- farend, E.W., Krumm, F.W., Schwarze, V.S. (eds.) Geodesy-The Challenge of the 3rd Millennium, pp. 299–309. Springer, Berlin, Heidelberg (2003). https://link.springer.com/chapter/10.1007/978-3-662-05296-9 31

    work page doi:10.1007/978-3-662-05296-9 2003

  18. [19]

    Flath, H.P., Wilcox, L.C., Ak¸ celik, V., Hill, J., Van Bloemen Waanders, B., Ghat- tas, O.: Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations. SIAM J. Sci. Comput. 33(1), 407–432 (2011) https://doi.org/10.1137/090780717

    work page doi:10.1137/090780717 2011

  19. [20]

    Gugercin, S., Antoulas, A.C., Beattie, C.: H2 model reduction for large-scale linear dynamical systems. SIAM J. Matrix Anal. Appl. 30(2), 609–638 (2008) https://doi.org/10.1137/060666123 22

    work page doi:10.1137/060666123 2008

  20. [21]

    Galbally, D., Fidkowski, K., Willcox, K., Ghattas, O.: Non-linear model reduc- tion for uncertainty quantification in large-scale inverse problems. Int. J. Numer. Methods Eng. 81(12), 1581–1608 (2010) https://doi.org/10.1002/nme.2746

    work page doi:10.1002/nme.2746 2010

  21. [22]

    Internat

    Gawronski, W., Juang, J.-N.: Model reduction in limited time and frequency inter- vals. Internat. J. Systems Sci. 21(2), 349–376 (1990) https://doi.org/10.1080/ 00207729008910366

    work page 1990

  22. [23]

    Springer, Cham (2021)

    Heard, N.: An Introduction to Bayesian Inference, Methods and Computation. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-82808-0

    work page doi:10.1007/978-3-030-82808-0 2021

  23. [24]

    2008 , publisher =

    Higham, N.J.: Functions of Matrices: Theory and Computation. SIAM, Philadel- phia (2008). https://doi.org/10.1137/1.9780898717778

    work page doi:10.1137/1.9780898717778 2008

  24. [25]

    Haben, S.A., Lawless, A.S., Nichols, N.K.: Conditioning and preconditioning of the variational data assimilation problem. Comput. Fluids 46(1), 252–256 (2011) https://doi.org/10.1016/j.compfluid.2010.11.025

    work page doi:10.1016/j.compfluid.2010.11.025 2011

  25. [26]

    Houtekamer, P.L., Mitchell, H.L., Pellerin, G., Buehner, M., Charron, M., Spacek, L., Hansen, B.: Atmospheric data assimilation with an ensemble Kalman filter: Results with real observations. Mon. Weather Rev.133(3), 604–620 (2005) https: //doi.org/10.1175/MWR-2864.1

    work page doi:10.1175/mwr-2864.1 2005

  26. [27]

    Automatica 47(3), 559–564 (2011) https://doi.org/10.1016/j.automatica.2010.12.002

    Heinkenschloss, M., Reis, T., Antoulas, A.C.: Balanced truncation model reduc- tion for systems with inhomogeneous initial conditions. Automatica 47(3), 559–564 (2011) https://doi.org/10.1016/j.automatica.2010.12.002

    work page doi:10.1016/j.automatica.2010.12.002 2011

  27. [28]

    Springer, Cham (2015)

    Hesthaven, J.S., Rozza, G., Stamm, B.: Certified Reduced Basis Methods for Parametrized Partial Differential Equations. Springer, Cham (2015). https://doi. org/10.1007/978-3-319-22470-1

    work page doi:10.1007/978-3-319-22470-1 2015

  28. [29]

    K¨ onig, J., Freitag, M.A.: Time-limited balanced truncation for data assim- ilation problems. J. Sci. Comput. 97(47) (2023) https://doi.org/10.1007/ s10915-023-02358-4

    work page 2023

  29. [30]

    PAMM 23(3), 202300019 (2023) https://doi.org/10.1002/pamm.202300019

    K¨ onig, J., Freitag, M.A.: Time-limited balanced truncation within incremental four-dimensional variational data assimilation. PAMM 23(3), 202300019 (2023) https://doi.org/10.1002/pamm.202300019

    work page doi:10.1002/pamm.202300019 2023

  30. [31]

    Kitanidis, P.K.: Compressed state Kalman filter for large systems. Adv. Water Resour. 76, 120–126 (2015) https://doi.org/10.1016/j.advwatres.2014.12.010

    work page doi:10.1016/j.advwatres.2014.12.010 2015

  31. [32]

    K¨ urschner, P.: Balanced truncation model order reduction in limited time inter- vals for large systems. Adv. Comput. Math. 44(6), 1821–1844 (2018) https: //doi.org/10.1007/s10444-018-9608-6

    work page doi:10.1007/s10444-018-9608-6 2018

  32. [33]

    Livings, D.M., Dance, S.L., Nichols, N.K.: Unbiased ensemble square root filters. 23 Phys. D: Nonlinear Phenom. 237(8), 1021–1028 (2008) https://doi.org/10.1016/ j.physd.2008.01.005

    work page 2008

  33. [34]

    Lipponen, A., Seppanen, A., Kaipio, J.P.: Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition. J. Electron. Imaging 22(2), 023008 (2013) https://doi.org/10.1117/1.JEI.22.2. 023008

    work page doi:10.1117/1.jei.22.2 2013

  34. [35]

    Texts in Applied Mathematics

    Law, K., Stuart, A., Zygalakis, K.: Data Assimilation: A Mathematical Introduc- tion. Texts in Applied Mathematics. Springer, Cham (2015). https://doi.org/10. 1007/978-3-319-20325-6

    work page 2015

  35. [36]

    SIAM Rev

    Lieberman, C., Willcox, K.: Goal-oriented inference: Approach, linear theory, and application to advection diffusion. SIAM Rev. 55(3), 493–519 (2013) https: //doi.org/10.1137/130913110

    work page doi:10.1137/130913110 2013

  36. [37]

    Lieberman, C., Willcox, K., Ghattas, O.: Parameter and state model reduction for large-scale statistical inverse problems. SIAM J. Sci. Comput. 32(5), 2523–2542 (2010) https://doi.org/10.1137/090775622

    work page doi:10.1137/090775622 2010

  37. [38]

    IEEE Trans

    Moore, B.: Principal component analysis in linear systems: Controllability, observ- ability, and model reduction. IEEE Trans. Automat. Control 26(1), 17–32 (1981) https://doi.org/10.1109/TAC.1981.1102568

    work page doi:10.1109/tac.1981.1102568 1981

  38. [39]

    IEEE Trans

    Mullis, C., Roberts, R.: Synthesis of minimum roundoff noise fixed point digital filters. IEEE Trans. Circuits Syst.23(9), 551–562 (1976) https://doi.org/10.1109/ TCS.1976.1084254

    work page arXiv 1976

  39. [40]

    Martin, J., Wilcox, L.C., Burstedde, C., Ghattas, O.: A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion. SIAM J. Sci. Comput. 34(3), 1460–1487 (2012) https: //doi.org/10.1137/110845598

    work page doi:10.1137/110845598 2012

  40. [41]

    (eds.) Data Assimilation for Numerical Weather Prediction: A Review

    Navon, I.M.: In: Park, S.K., Xu, L. (eds.) Data Assimilation for Numerical Weather Prediction: A Review. Springer, Berlin, Heidelberg (2009). https://doi. org/10.1007/978-3-540-71056-1 2

    work page doi:10.1007/978-3-540-71056-1 2009

  41. [42]

    Asia Pac

    Nguyen, N., Cheong Khoo, B., Willcox, K.: Model order reduction for Bayesian approach to inverse problems. Asia Pac. J. Comput. Engin 1(2) (2014) https: //doi.org/10.1186/2196-1166-1-2

    work page doi:10.1186/2196-1166-1-2 2014

  42. [43]

    Qian, E., Tabeart, J.M., Beattie, C., Gugercin, S., Jiang, J., Kramer, P.R., Narayan, A.: Model reduction for linear dynamical systems via balancing for Bayesian inference. J. Sci. Comput. 91(29) (2022) https://doi.org/10.1007/ s10915-022-01798-8

    work page 2022

  43. [44]

    SIAM Rev

    Rozier, D., Birol, F., Cosme, E., Brasseur, P., Brankart, J.M., Verron, J.: A 24 reduced-order Kalman filter for data assimilation in physical oceanography. SIAM Rev. 49(3), 449–465 (2007) https://doi.org/10.1137/050635717

    work page doi:10.1137/050635717 2007

  44. [45]

    Spantini, A., Solonen, A., Cui, T., Martin, J., Tenorio, L., Marzouk, Y.: Opti- mal low-rank approximations of Bayesian linear inverse problems. SIAM J. Sci. Comput. 37(6), 2451–2487 (2015) https://doi.org/10.1137/140977308

    work page doi:10.1137/140977308 2015

  45. [46]

    Schr¨ oder, C., Voigt, M.: Balanced truncation model reduction with a priori error bounds for LTI systems with nonzero initial value. J. Comput. Appl. Math. 420, 114708 (2023) https://doi.org/10.1016/j.cam.2022.114708

    work page doi:10.1016/j.cam.2022.114708 2023

  46. [47]

    Tarantola , Inverse Problem Theory and Methods for Model Parameter Estimation , Other Titles in Applied Mathematics, Society for Industrial and Applied Mathematics, Jan

    Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Esti- mation. SIAM, Philadelphia (2005). https://doi.org/10.1137/1.9780898717921

    work page doi:10.1137/1.9780898717921 2005

  47. [48]

    Tuan Pham, D., Verron, J., Christine Roubaud, M.: A singular evolutive extended Kalman filter for data assimilation in oceanography. J. Mar. Syst. 16(3), 323–340 (1998) https://doi.org/10.1016/S0924-7963(97)00109-7

    work page doi:10.1016/s0924-7963(97)00109-7 1998

  48. [49]

    van Leeuwen, P.J.: Nonlinear data assimilation in geosciences: an extremely effi- cient particle filter. Q. J. R. Meteorol. Soc. 136(653), 1991–1999 (2010) https: //doi.org/10.1002/qj.699

    work page doi:10.1002/qj.699 1991

  49. [50]

    Stochastic Hydrol

    Verlaan, M., Heemink, A.W.: Tidal flow forecasting using reduced rank square root filters. Stochastic Hydrol. Hydraul. 11, 349–368 (1997) https://doi.org/10. 1007/BF02427924

    work page 1997

  50. [51]

    Wang, J., Zabaras, N.: Using Bayesian statistics in the estimation of heat source in radiation. Int. J. Heat Mass Transf. 48(1), 15–29 (2005) https://doi.org/10. 1016/j.ijheatmasstransfer.2004.08.009

    work page 2005

  51. [52]

    Zahm, O., Cui, T., Law, K., Spantini, A., Marzouk, Y.: Certified dimension reduc- tion in nonlinear Bayesian inverse problems. Math. Comput.91, 1789–1835 (2022) https://doi.org/10.1090/MCOM/3737 25

    work page doi:10.1090/mcom/3737 2022