Adaptive particle-based approximations of the Gibbs posterior for inverse problems
Pith reviewed 2026-05-25 11:00 UTC · model grok-4.3
The pith
A sequential Monte Carlo method with adaptive reduced basis surrogates approximates the Gibbs posterior for inverse problems governed by PDEs.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles. To manage the computational cost of propagating increasing numbers of particles through the loss function, we employ a recently developed local reduced basis method to build an efficient surrogate loss function that is used in the Gibbs update formula in place of the true loss. We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model in an efficient manner.
What carries the argument
The local reduced basis surrogate for the loss function, used in the SMC particle approximation to the Gibbs posterior.
Load-bearing premise
The local reduced basis surrogate accurately approximates the true loss function sufficiently well that the particle-based Gibbs update remains reliable.
What would settle it
Running the method on a simple PDE problem with a computable exact posterior and observing large discrepancies between the surrogate-based and true-loss versions, even when surrogate error stays inside the stated bounds, would falsify the reliability claim.
Figures
read the original abstract
In this work, we adopt a general framework based on the Gibbs posterior to update belief distributions for inverse problems governed by partial differential equations (PDEs). The Gibbs posterior formulation is a generalization of standard Bayesian inference that only relies on a loss function connecting the unknown parameters to the data. It is particularly useful when the true data generating mechanism (or noise distribution) is unknown or difficult to specify. The Gibbs posterior coincides with Bayesian updating when a true likelihood function is known and the loss function corresponds to the negative log-likelihood, yet provides subjective inference in more general settings. We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles. To manage the computational cost of propagating increasing numbers of particles through the loss function, we employ a recently developed local reduced basis method to build an efficient surrogate loss function that is used in the Gibbs update formula in place of the true loss. We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model in an efficient manner. We demonstrate the efficiency of our approach through several numerical examples.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a sequential Monte Carlo (SMC) particle approximation to the Gibbs posterior for Bayesian-style inference in PDE-governed inverse problems. It replaces the true loss function in the Gibbs update with a local reduced-basis surrogate, derives error bounds that quantify the effect of surrogate error on the particle weights and resulting measure, and introduces an adaptive strategy for constructing the surrogate. The approach is illustrated on several numerical examples.
Significance. If the derived error bounds are rigorous and the adaptive surrogate construction controls the error efficiently, the work supplies a practical, theoretically grounded method for uncertainty quantification in settings where a full likelihood is unavailable or expensive to evaluate. The explicit propagation of surrogate error into the particle approximation is a clear strength, as is the focus on computational efficiency for PDE models.
major comments (2)
- [§4] §4 (error analysis): the bound on the total variation distance between the true and approximate Gibbs measures (presumably Eq. (12) or (13)) is stated to depend on the surrogate error in the loss; however, the proof sketch does not make explicit how the Lipschitz constant of the exponential map interacts with the particle degeneracy that occurs when the surrogate error is not uniformly small across the support of the prior. A concrete counter-example or additional assumption would strengthen the claim that the bound remains useful for adaptive refinement.
- [§5.2] §5.2 (adaptive surrogate algorithm): the criterion for adding new basis functions is described as controlling the surrogate error below a user tolerance, but it is not shown that this tolerance can be chosen a priori from the error bound in §4 without knowledge of the unknown posterior. This creates a potential circularity for the claimed efficiency gain.
minor comments (2)
- [§3] The notation for the local reduced-basis space (e.g., the dependence on the current particle set) is introduced in §3 but used without re-definition in the error analysis of §4; a short notational table would improve readability.
- Figure 3 caption states that the surrogate error is plotted against number of basis functions, but the x-axis label is missing the log scale that is used in the text discussion.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive major comments. We respond point-by-point below and have revised the manuscript to address the concerns where possible.
read point-by-point responses
-
Referee: [§4] §4 (error analysis): the bound on the total variation distance between the true and approximate Gibbs measures (presumably Eq. (12) or (13)) is stated to depend on the surrogate error in the loss; however, the proof sketch does not make explicit how the Lipschitz constant of the exponential map interacts with the particle degeneracy that occurs when the surrogate error is not uniformly small across the support of the prior. A concrete counter-example or additional assumption would strengthen the claim that the bound remains useful for adaptive refinement.
Authors: We agree that the interaction between the Lipschitz constant of the exponential map and potential particle degeneracy merits explicit treatment in the proof. The original derivation assumes a uniform bound on the surrogate error (ensured by the adaptive construction in §5), which controls the weight degeneracy in a manner analogous to standard SMC analyses. In the revision we expand the proof of the TV bound to include an intermediate step deriving the effect of the Lipschitz factor on the Radon-Nikodym derivative between the true and surrogate Gibbs measures, and we add a remark clarifying that the bound remains informative provided the surrogate error is controlled below a threshold proportional to the inverse of the Lipschitz constant. No counter-example is needed because the stated assumptions already preclude the degeneracy regime highlighted by the referee. revision: yes
-
Referee: [§5.2] §5.2 (adaptive surrogate algorithm): the criterion for adding new basis functions is described as controlling the surrogate error below a user tolerance, but it is not shown that this tolerance can be chosen a priori from the error bound in §4 without knowledge of the unknown posterior. This creates a potential circularity for the claimed efficiency gain.
Authors: The referee correctly identifies that a direct a-priori link between the §4 error bound and the tolerance parameter is not fully spelled out. The tolerance is user-specified in the original text, with the error bound supplying only a post-hoc guarantee. In the revised manuscript we insert a short subsection explaining a practical a-priori selection procedure: run a short pilot SMC chain with a coarse global surrogate to obtain a rough estimate of the support of the Gibbs posterior, then set the tolerance from the §4 bound using this pilot measure in place of the unknown posterior. This removes the circularity while preserving the adaptive efficiency of the local reduced-basis construction. revision: yes
Circularity Check
No significant circularity; derivation self-contained
full rationale
The paper adopts the established Gibbs posterior framework for inverse problems, employs standard SMC particle approximation, substitutes a local reduced basis surrogate into the loss, and derives error bounds quantifying the surrogate's effect on the resulting measure. These bounds are presented as newly obtained quantities that address the reliability of the substitution rather than being defined by or reducing to any fitted parameters or self-citations within the paper. The adaptive surrogate construction is an efficiency proposal, not a renaming or self-referential fit. No load-bearing step reduces by construction to the paper's own inputs; the central claims rest on independent derivation of approximation error.
Axiom & Free-Parameter Ledger
axioms (2)
- standard math Sequential Monte Carlo particles converge to the target Gibbs posterior under standard assumptions on the loss and proposal distributions.
- domain assumption Local reduced basis methods produce surrogates whose error can be bounded in a manner compatible with the Gibbs update.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles... We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
the Gibbs posterior formulation is a generalization of standard Bayesian inference that only relies on a loss function
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
On the properties of variational approximations of Gibbs posteriors
Pierre Alquier, James Ridgway, and Nicolas Chopin. On the properties of variational approximations of Gibbs posteriors. The Journal of Machine Learning Research , 17(1):8374–8414, 2016
work page 2016
-
[2]
Inverse modeling of the ocean and atmosphere
Andrew F Bennett. Inverse modeling of the ocean and atmosphere . Cambridge University Press, 2005
work page 2005
-
[3]
Sequential Monte Carlo for Bayesian computation
JM Bernardo, MJ Bayarri, JO Berger, AP Dawid, D Heckerman, AFM Smith, and M West. Sequential Monte Carlo for Bayesian computation. In Bayesian Statistics 8: Proceedings of the Eighth Valencia International Meeting, June 2-6, 2006 , volume 8, page 115. Oxford University Press, USA, 2007
work page 2006
-
[4]
Sequential Monte Carlo methods for Bayesian elliptic inverse problems
Alexandros Beskos, Ajay Jasra, Ege A Muzaffer, and Andrew M Stuart. Sequential Monte Carlo methods for Bayesian elliptic inverse problems. Statistics and Computing , 25(4):727–737, 2015
work page 2015
-
[5]
A general framework for updating belief distributions
Pier Giovanni Bissiri, Chris C Holmes, and Stephen G Walker. A general framework for updating belief distributions. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 78(5):1103– 1130, 2016. ADAPTIVE PARTICLE-BASED APPROXIMATIONS OF THE GIBBS POSTERIOR FOR INVERSE PROB- LEMS 33
work page 2016
-
[6]
Tan Bui-Thanh, Omar Ghattas, James Martin, and Georg Stadler. A computational framework for infinite-dimensional Bayesian inverse problems Part I: The linearized case, with application to global seismic inversion. SIAM Journal on Scientific Computing , 35(6):A2494–A2523, 2013
work page 2013
-
[7]
A simple method using Morozov’s discrepancy principle for solving inverse scattering problems
David Colton, Michele Piana, and Roland Potthast. A simple method using Morozov’s discrepancy principle for solving inverse scattering problems. Inverse Problems, 13(6):1477, 1997
work page 1997
-
[8]
Accelerating asymptotically exact MCMC for computationally intensive models via local approximations
Patrick R Conrad, Youssef M Marzouk, Natesh S Pillai, and Aaron Smith. Accelerating asymptotically exact MCMC for computationally intensive models via local approximations. Journal of the American Statistical Association, 111(516):1591–1607, 2016
work page 2016
-
[9]
Patrick Raymond Conrad. Accelerating Bayesian inference in computationally expensive computer models using local and global approximations . PhD thesis, Massachusetts Institute of Technology, 2014
work page 2014
-
[10]
Approximation of Bayesian inverse problems for PDEs
Simon L Cotter, Massoumeh Dashti, and Andrew M Stuart. Approximation of Bayesian inverse problems for PDEs. SIAM Journal on Numerical Analysis , 48(1):322–345, 2010
work page 2010
-
[11]
MCMC methods for functions: modifying old algorithms to make them faster
Simon L Cotter, Gareth O Roberts, Andrew M Stuart, and David White. MCMC methods for functions: modifying old algorithms to make them faster. Statistical Science, pages 424–446, 2013
work page 2013
-
[12]
Tiangang Cui, Youssef M Marzouk, and Karen E Willcox. Data-driven model reduction for the Bayesian solution of inverse problems.International Journal for Numerical Methods in Engineering, 102(5):966– 990, 2015
work page 2015
-
[13]
Sequential Monte Carlo samplers
Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 68(3):411–436, 2006
work page 2006
-
[14]
An introduction to sequential Monte Carlo methods
Arnaud Doucet, Nando De Freitas, and Neil Gordon. An introduction to sequential Monte Carlo methods. In Sequential Monte Carlo Methods in Practice , pages 3–14. Springer, 2001
work page 2001
-
[15]
Isabell M Franck and PS Koutsourelakis. Sparse Variational Bayesian approximations for nonlinear inverse problems: Applications in nonlinear elastography. Computer Methods in Applied Mechanics and Engineering, 299:215–244, 2016
work page 2016
-
[16]
M Frangos, Y Marzouk, K Willcox, and B van Bloemen Waanders. Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [chapter 7]. 2010
work page 2010
-
[17]
Markov chain Monte Carlo: stochastic simulation for Bayesian inference
Dani Gamerman and Hedibert F Lopes. Markov chain Monte Carlo: stochastic simulation for Bayesian inference. Chapman and Hall/CRC, 2006
work page 2006
-
[18]
Reduced order models for random functions
M Grigoriu. Reduced order models for random functions. application to stochastic problems. Applied Mathematical Modelling, 33(1):161–175, 2009
work page 2009
-
[19]
A method for solving stochastic equations by reduced order models and local approxi- mations
Mircea Grigoriu. A method for solving stochastic equations by reduced order models and local approxi- mations. Journal of Computational Physics , 231(19):6495–6513, 2012
work page 2012
-
[20]
Nikolas Kantas, Alexandros Beskos, and Ajay Jasra. Sequential Monte Carlo Methods for High- Dimensional Inverse Problems: A Case Study for the Navier–Stokes Equations. SIAM/ASA Journal on Uncertainty Quantification, 2(1):464–489, 2014
work page 2014
-
[21]
Bayesian calibration of computer models
Marc C Kennedy and Anthony O’Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 63(3):425–464, 2001
work page 2001
-
[22]
Phaedon-Stelios Koutsourelakis. A novel Bayesian strategy for the identification of spatially varying material properties and model validation: an application to static elastography. International Journal for Numerical Methods in Engineering , 91(3):249–268, 2012
work page 2012
-
[23]
Proposals which speed up function-space MCMC
Kody JH Law. Proposals which speed up function-space MCMC. Journal of Computational and Applied Mathematics, 262:127–138, 2014
work page 2014
-
[24]
Adaptive construction of surrogates for the Bayesian solution of inverse problems
Jinglai Li and Youssef M Marzouk. Adaptive construction of surrogates for the Bayesian solution of inverse problems. SIAM Journal on Scientific Computing , 36(3):A1163–A1186, 2014
work page 2014
-
[25]
Andrea Manzoni, Stefano Pagani, and Toni Lassila. Accurate solution of Bayesian inverse uncertainty quantification problems combining reduced basis methods and reduction error models. SIAM/ASA Journal on Uncertainty Quantification , 4(1):380–412, 2016
work page 2016
-
[26]
A stochastic collocation approach to Bayesian inference in inverse problems
Youssef Marzouk and Dongbin Xiu. A stochastic collocation approach to Bayesian inference in inverse problems. 2009
work page 2009
-
[27]
Dimensionality reduction and polynomial chaos acceleration of bayesian inference in inverse problems
Youssef M Marzouk and Habib N Najm. Dimensionality reduction and polynomial chaos acceleration of bayesian inference in inverse problems. Journal of Computational Physics , 228(6):1862–1902, 2009
work page 1902
-
[28]
Stochastic spectral methods for efficient Bayesian solution of inverse problems
Youssef M Marzouk, Habib N Najm, and Larry A Rahn. Stochastic spectral methods for efficient Bayesian solution of inverse problems. Journal of Computational Physics , 224(2):560–586, 2007
work page 2007
-
[29]
Can local particle filters beat the curse of dimensionality? 34 Z
Patrick Rebeschini, Ramon Van Handel, et al. Can local particle filters beat the curse of dimensionality? 34 Z. ZOU The Annals of Applied Probability , 25(5):2809–2866, 2015
work page 2015
-
[30]
Otmar Scherzer. The use of Morozov’s discrepancy principle for Tikhonov regularization for solving nonlinear ill-posed problems. Computing, 51(1):45–60, 1993
work page 1993
-
[31]
Inverse problems: a Bayesian perspective
Andrew M Stuart. Inverse problems: a Bayesian perspective. Acta Numerica, 19:451–559, 2010
work page 2010
-
[32]
Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image
Nicholas Syring and Ryan Martin. Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image. arXiv preprint arXiv:1606.08400 , 2016
-
[33]
A Bayesian inference approach to the inverse heat conduction problem
Jingbo Wang and Nicholas Zabaras. A Bayesian inference approach to the inverse heat conduction problem. International Journal of Heat and Mass Transfer , 47(17-18):3927–3941, 2004
work page 2004
-
[34]
An Adaptive Sampling Approach for Solving PDEs with Uncertain Inputs and Evaluating Risk
Zilong Zou, Drew Kouri, and Wilkins Aquino. An Adaptive Sampling Approach for Solving PDEs with Uncertain Inputs and Evaluating Risk. In 19th AIAA Non-Deterministic Approaches Conference , page 1325, 2017
work page 2017
-
[35]
An adaptive local reduced basis method for solving PDEs with uncertain inputs and evaluating risk
Zilong Zou, Drew Kouri, and Wilkins Aquino. An adaptive local reduced basis method for solving PDEs with uncertain inputs and evaluating risk. Computer Methods in Applied Mechanics and Engineering, 2018
work page 2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.