pith. sign in

arxiv: 1907.01551 · v1 · pith:FPCADFSVnew · submitted 2019-07-02 · 📊 stat.CO · cs.LG

Adaptive particle-based approximations of the Gibbs posterior for inverse problems

Pith reviewed 2026-05-25 11:00 UTC · model grok-4.3

classification 📊 stat.CO cs.LG
keywords Gibbs posteriorsequential Monte Carloreduced basis surrogateinverse problemsPDEparticle approximationadaptive surrogateerror bounds
0
0 comments X

The pith

A sequential Monte Carlo method with adaptive reduced basis surrogates approximates the Gibbs posterior for inverse problems governed by PDEs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a method to approximate the Gibbs posterior for parameters in PDE inverse problems using sequential Monte Carlo particles. It substitutes an adaptive local reduced basis surrogate for the expensive loss function in the Gibbs update. Error bounds for the approximation are derived, and an adaptive strategy for building the surrogate is proposed. This matters for settings where the noise distribution is unknown or hard to model, allowing inference based solely on a loss function.

Core claim

We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles. To manage the computational cost of propagating increasing numbers of particles through the loss function, we employ a recently developed local reduced basis method to build an efficient surrogate loss function that is used in the Gibbs update formula in place of the true loss. We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model in an efficient manner.

What carries the argument

The local reduced basis surrogate for the loss function, used in the SMC particle approximation to the Gibbs posterior.

Load-bearing premise

The local reduced basis surrogate accurately approximates the true loss function sufficiently well that the particle-based Gibbs update remains reliable.

What would settle it

Running the method on a simple PDE problem with a computable exact posterior and observing large discrepancies between the surrogate-based and true-loss versions, even when surrogate error stays inside the stated bounds, would falsify the reliability claim.

Figures

Figures reproduced from arXiv: 1907.01551 by Harbir Antil, Sayan Mukherjee, Wilkins Aquino, Zilong Zou.

Figure 1
Figure 1. Figure 1: The local reduced basis method with two random parameters. The surrogate [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of the posterior distribution of the parameters computed by Algorithm [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: SMC iteration 0, with a loss weight W0 = 0. 8.2. 2D advection diffusion equation. In the second example, we consider the simultane￾ous identification of the diffusivity constant and unknown source for a 2D advection-diffusion [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: SMC iteration 1, with a loss weight W1 = 0.0782. 9 1 0 0.5 1 9 2 0 0.5 1 (a) The true parameters (black), the particles (red) and the atoms for local RB (blue) 9 1 0 0.5 1 0 0.5 1 true val prior posterior (b) CDF plot for ξ1 9 2 0 0.5 1 0 0.5 1 true val prior posterior (c) CDF plot for ξ2 [PITH_FULL_IMAGE:figures/full_fig_p024_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: SMC iteration 2, with a loss weight W2 = 1.43. problem. Let D = (0, 1)2 . We consider the following problem, −∇ · (κ(ξ ∗ )∇u(x, ξ∗ )) + v(x) · ∇u(x, ξ∗ ) = f(x, ξ∗ (8.3a) ) x ∈ D u(x, ξ∗ (8.3b) ) = 0 x ∈ Γd κ(ξ ∗ )∇u(x, ξ∗ (8.3c) ) · n = 0 x ∈ Γn where Γd := [0, 1] × {0} and Γn := ∂D \ Γd. The unknown parameters ξ ∗ are included in the diffusivity constant κ(ξ ∗ ) and the source term f(x, ξ∗ ). In particul… view at source ↗
Figure 6
Figure 6. Figure 6: SMC iteration 3, with a loss weight W3 = 16.7. Iteration 0 1 2 3 Total number of PDE solves 0 50 100 150 200 [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The accumulative number of PDE solves at each iteration of the SMC Algorithm [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The diffusivity, advection and source fields of the 2D advection-diffusion equation. [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The noise-free solution and the noisy measurements. [PITH_FULL_IMAGE:figures/full_fig_p027_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of the posterior distribution of the parameters computed by Algorithm [PITH_FULL_IMAGE:figures/full_fig_p028_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The evolution of the particles and the local RB atoms at each iteration of SMC [PITH_FULL_IMAGE:figures/full_fig_p029_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: The accumulative number of PDE solves at each iteration of the SMC Algorithm [PITH_FULL_IMAGE:figures/full_fig_p030_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: The boundary condition, and true material properties for the simple elastography [PITH_FULL_IMAGE:figures/full_fig_p030_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: The displacement fields and the noisy measurements for the layered material. [PITH_FULL_IMAGE:figures/full_fig_p031_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: The displacement fields and the noisy measurements for the material with a hard [PITH_FULL_IMAGE:figures/full_fig_p031_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: The mean and standard deviation of Gibbs posterior computed using data with [PITH_FULL_IMAGE:figures/full_fig_p031_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: The mean and standard deviation of Gibbs posterior computed using data with [PITH_FULL_IMAGE:figures/full_fig_p032_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: The total number of local RB atoms upon solving the Gibbs posterior for both [PITH_FULL_IMAGE:figures/full_fig_p032_18.png] view at source ↗
read the original abstract

In this work, we adopt a general framework based on the Gibbs posterior to update belief distributions for inverse problems governed by partial differential equations (PDEs). The Gibbs posterior formulation is a generalization of standard Bayesian inference that only relies on a loss function connecting the unknown parameters to the data. It is particularly useful when the true data generating mechanism (or noise distribution) is unknown or difficult to specify. The Gibbs posterior coincides with Bayesian updating when a true likelihood function is known and the loss function corresponds to the negative log-likelihood, yet provides subjective inference in more general settings. We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles. To manage the computational cost of propagating increasing numbers of particles through the loss function, we employ a recently developed local reduced basis method to build an efficient surrogate loss function that is used in the Gibbs update formula in place of the true loss. We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model in an efficient manner. We demonstrate the efficiency of our approach through several numerical examples.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper develops a sequential Monte Carlo (SMC) particle approximation to the Gibbs posterior for Bayesian-style inference in PDE-governed inverse problems. It replaces the true loss function in the Gibbs update with a local reduced-basis surrogate, derives error bounds that quantify the effect of surrogate error on the particle weights and resulting measure, and introduces an adaptive strategy for constructing the surrogate. The approach is illustrated on several numerical examples.

Significance. If the derived error bounds are rigorous and the adaptive surrogate construction controls the error efficiently, the work supplies a practical, theoretically grounded method for uncertainty quantification in settings where a full likelihood is unavailable or expensive to evaluate. The explicit propagation of surrogate error into the particle approximation is a clear strength, as is the focus on computational efficiency for PDE models.

major comments (2)
  1. [§4] §4 (error analysis): the bound on the total variation distance between the true and approximate Gibbs measures (presumably Eq. (12) or (13)) is stated to depend on the surrogate error in the loss; however, the proof sketch does not make explicit how the Lipschitz constant of the exponential map interacts with the particle degeneracy that occurs when the surrogate error is not uniformly small across the support of the prior. A concrete counter-example or additional assumption would strengthen the claim that the bound remains useful for adaptive refinement.
  2. [§5.2] §5.2 (adaptive surrogate algorithm): the criterion for adding new basis functions is described as controlling the surrogate error below a user tolerance, but it is not shown that this tolerance can be chosen a priori from the error bound in §4 without knowledge of the unknown posterior. This creates a potential circularity for the claimed efficiency gain.
minor comments (2)
  1. [§3] The notation for the local reduced-basis space (e.g., the dependence on the current particle set) is introduced in §3 but used without re-definition in the error analysis of §4; a short notational table would improve readability.
  2. Figure 3 caption states that the surrogate error is plotted against number of basis functions, but the x-axis label is missing the log scale that is used in the text discussion.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive major comments. We respond point-by-point below and have revised the manuscript to address the concerns where possible.

read point-by-point responses
  1. Referee: [§4] §4 (error analysis): the bound on the total variation distance between the true and approximate Gibbs measures (presumably Eq. (12) or (13)) is stated to depend on the surrogate error in the loss; however, the proof sketch does not make explicit how the Lipschitz constant of the exponential map interacts with the particle degeneracy that occurs when the surrogate error is not uniformly small across the support of the prior. A concrete counter-example or additional assumption would strengthen the claim that the bound remains useful for adaptive refinement.

    Authors: We agree that the interaction between the Lipschitz constant of the exponential map and potential particle degeneracy merits explicit treatment in the proof. The original derivation assumes a uniform bound on the surrogate error (ensured by the adaptive construction in §5), which controls the weight degeneracy in a manner analogous to standard SMC analyses. In the revision we expand the proof of the TV bound to include an intermediate step deriving the effect of the Lipschitz factor on the Radon-Nikodym derivative between the true and surrogate Gibbs measures, and we add a remark clarifying that the bound remains informative provided the surrogate error is controlled below a threshold proportional to the inverse of the Lipschitz constant. No counter-example is needed because the stated assumptions already preclude the degeneracy regime highlighted by the referee. revision: yes

  2. Referee: [§5.2] §5.2 (adaptive surrogate algorithm): the criterion for adding new basis functions is described as controlling the surrogate error below a user tolerance, but it is not shown that this tolerance can be chosen a priori from the error bound in §4 without knowledge of the unknown posterior. This creates a potential circularity for the claimed efficiency gain.

    Authors: The referee correctly identifies that a direct a-priori link between the §4 error bound and the tolerance parameter is not fully spelled out. The tolerance is user-specified in the original text, with the error bound supplying only a post-hoc guarantee. In the revised manuscript we insert a short subsection explaining a practical a-priori selection procedure: run a short pilot SMC chain with a coarse global surrogate to obtain a rough estimate of the support of the Gibbs posterior, then set the tolerance from the §4 bound using this pilot measure in place of the unknown posterior. This removes the circularity while preserving the adaptive efficiency of the local reduced-basis construction. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained

full rationale

The paper adopts the established Gibbs posterior framework for inverse problems, employs standard SMC particle approximation, substitutes a local reduced basis surrogate into the loss, and derives error bounds quantifying the surrogate's effect on the resulting measure. These bounds are presented as newly obtained quantities that address the reliability of the substitution rather than being defined by or reducing to any fitted parameters or self-citations within the paper. The adaptive surrogate construction is an efficiency proposal, not a renaming or self-referential fit. No load-bearing step reduces by construction to the paper's own inputs; the central claims rest on independent derivation of approximation error.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The approach rests on standard convergence properties of SMC and approximation properties of reduced basis methods; no free parameters, invented entities, or ad-hoc axioms are introduced in the abstract.

axioms (2)
  • standard math Sequential Monte Carlo particles converge to the target Gibbs posterior under standard assumptions on the loss and proposal distributions.
    Invoked implicitly when SMC is used to approximate the Gibbs posterior.
  • domain assumption Local reduced basis methods produce surrogates whose error can be bounded in a manner compatible with the Gibbs update.
    Required for the substitution of the surrogate loss into the Gibbs formula and for the derived error bounds.

pith-pipeline@v0.9.0 · 5720 in / 1297 out tokens · 35972 ms · 2026-05-25T11:00:24.076474+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages

  1. [1]

    On the properties of variational approximations of Gibbs posteriors

    Pierre Alquier, James Ridgway, and Nicolas Chopin. On the properties of variational approximations of Gibbs posteriors. The Journal of Machine Learning Research , 17(1):8374–8414, 2016

  2. [2]

    Inverse modeling of the ocean and atmosphere

    Andrew F Bennett. Inverse modeling of the ocean and atmosphere . Cambridge University Press, 2005

  3. [3]

    Sequential Monte Carlo for Bayesian computation

    JM Bernardo, MJ Bayarri, JO Berger, AP Dawid, D Heckerman, AFM Smith, and M West. Sequential Monte Carlo for Bayesian computation. In Bayesian Statistics 8: Proceedings of the Eighth Valencia International Meeting, June 2-6, 2006 , volume 8, page 115. Oxford University Press, USA, 2007

  4. [4]

    Sequential Monte Carlo methods for Bayesian elliptic inverse problems

    Alexandros Beskos, Ajay Jasra, Ege A Muzaffer, and Andrew M Stuart. Sequential Monte Carlo methods for Bayesian elliptic inverse problems. Statistics and Computing , 25(4):727–737, 2015

  5. [5]

    A general framework for updating belief distributions

    Pier Giovanni Bissiri, Chris C Holmes, and Stephen G Walker. A general framework for updating belief distributions. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 78(5):1103– 1130, 2016. ADAPTIVE PARTICLE-BASED APPROXIMATIONS OF THE GIBBS POSTERIOR FOR INVERSE PROB- LEMS 33

  6. [6]

    A computational framework for infinite-dimensional Bayesian inverse problems Part I: The linearized case, with application to global seismic inversion

    Tan Bui-Thanh, Omar Ghattas, James Martin, and Georg Stadler. A computational framework for infinite-dimensional Bayesian inverse problems Part I: The linearized case, with application to global seismic inversion. SIAM Journal on Scientific Computing , 35(6):A2494–A2523, 2013

  7. [7]

    A simple method using Morozov’s discrepancy principle for solving inverse scattering problems

    David Colton, Michele Piana, and Roland Potthast. A simple method using Morozov’s discrepancy principle for solving inverse scattering problems. Inverse Problems, 13(6):1477, 1997

  8. [8]

    Accelerating asymptotically exact MCMC for computationally intensive models via local approximations

    Patrick R Conrad, Youssef M Marzouk, Natesh S Pillai, and Aaron Smith. Accelerating asymptotically exact MCMC for computationally intensive models via local approximations. Journal of the American Statistical Association, 111(516):1591–1607, 2016

  9. [9]

    Accelerating Bayesian inference in computationally expensive computer models using local and global approximations

    Patrick Raymond Conrad. Accelerating Bayesian inference in computationally expensive computer models using local and global approximations . PhD thesis, Massachusetts Institute of Technology, 2014

  10. [10]

    Approximation of Bayesian inverse problems for PDEs

    Simon L Cotter, Massoumeh Dashti, and Andrew M Stuart. Approximation of Bayesian inverse problems for PDEs. SIAM Journal on Numerical Analysis , 48(1):322–345, 2010

  11. [11]

    MCMC methods for functions: modifying old algorithms to make them faster

    Simon L Cotter, Gareth O Roberts, Andrew M Stuart, and David White. MCMC methods for functions: modifying old algorithms to make them faster. Statistical Science, pages 424–446, 2013

  12. [12]

    Data-driven model reduction for the Bayesian solution of inverse problems.International Journal for Numerical Methods in Engineering, 102(5):966– 990, 2015

    Tiangang Cui, Youssef M Marzouk, and Karen E Willcox. Data-driven model reduction for the Bayesian solution of inverse problems.International Journal for Numerical Methods in Engineering, 102(5):966– 990, 2015

  13. [13]

    Sequential Monte Carlo samplers

    Pierre Del Moral, Arnaud Doucet, and Ajay Jasra. Sequential Monte Carlo samplers. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 68(3):411–436, 2006

  14. [14]

    An introduction to sequential Monte Carlo methods

    Arnaud Doucet, Nando De Freitas, and Neil Gordon. An introduction to sequential Monte Carlo methods. In Sequential Monte Carlo Methods in Practice , pages 3–14. Springer, 2001

  15. [15]

    Sparse Variational Bayesian approximations for nonlinear inverse problems: Applications in nonlinear elastography

    Isabell M Franck and PS Koutsourelakis. Sparse Variational Bayesian approximations for nonlinear inverse problems: Applications in nonlinear elastography. Computer Methods in Applied Mechanics and Engineering, 299:215–244, 2016

  16. [16]

    Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [chapter 7]

    M Frangos, Y Marzouk, K Willcox, and B van Bloemen Waanders. Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [chapter 7]. 2010

  17. [17]

    Markov chain Monte Carlo: stochastic simulation for Bayesian inference

    Dani Gamerman and Hedibert F Lopes. Markov chain Monte Carlo: stochastic simulation for Bayesian inference. Chapman and Hall/CRC, 2006

  18. [18]

    Reduced order models for random functions

    M Grigoriu. Reduced order models for random functions. application to stochastic problems. Applied Mathematical Modelling, 33(1):161–175, 2009

  19. [19]

    A method for solving stochastic equations by reduced order models and local approxi- mations

    Mircea Grigoriu. A method for solving stochastic equations by reduced order models and local approxi- mations. Journal of Computational Physics , 231(19):6495–6513, 2012

  20. [20]

    Sequential Monte Carlo Methods for High- Dimensional Inverse Problems: A Case Study for the Navier–Stokes Equations

    Nikolas Kantas, Alexandros Beskos, and Ajay Jasra. Sequential Monte Carlo Methods for High- Dimensional Inverse Problems: A Case Study for the Navier–Stokes Equations. SIAM/ASA Journal on Uncertainty Quantification, 2(1):464–489, 2014

  21. [21]

    Bayesian calibration of computer models

    Marc C Kennedy and Anthony O’Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 63(3):425–464, 2001

  22. [22]

    A novel Bayesian strategy for the identification of spatially varying material properties and model validation: an application to static elastography

    Phaedon-Stelios Koutsourelakis. A novel Bayesian strategy for the identification of spatially varying material properties and model validation: an application to static elastography. International Journal for Numerical Methods in Engineering , 91(3):249–268, 2012

  23. [23]

    Proposals which speed up function-space MCMC

    Kody JH Law. Proposals which speed up function-space MCMC. Journal of Computational and Applied Mathematics, 262:127–138, 2014

  24. [24]

    Adaptive construction of surrogates for the Bayesian solution of inverse problems

    Jinglai Li and Youssef M Marzouk. Adaptive construction of surrogates for the Bayesian solution of inverse problems. SIAM Journal on Scientific Computing , 36(3):A1163–A1186, 2014

  25. [25]

    Accurate solution of Bayesian inverse uncertainty quantification problems combining reduced basis methods and reduction error models

    Andrea Manzoni, Stefano Pagani, and Toni Lassila. Accurate solution of Bayesian inverse uncertainty quantification problems combining reduced basis methods and reduction error models. SIAM/ASA Journal on Uncertainty Quantification , 4(1):380–412, 2016

  26. [26]

    A stochastic collocation approach to Bayesian inference in inverse problems

    Youssef Marzouk and Dongbin Xiu. A stochastic collocation approach to Bayesian inference in inverse problems. 2009

  27. [27]

    Dimensionality reduction and polynomial chaos acceleration of bayesian inference in inverse problems

    Youssef M Marzouk and Habib N Najm. Dimensionality reduction and polynomial chaos acceleration of bayesian inference in inverse problems. Journal of Computational Physics , 228(6):1862–1902, 2009

  28. [28]

    Stochastic spectral methods for efficient Bayesian solution of inverse problems

    Youssef M Marzouk, Habib N Najm, and Larry A Rahn. Stochastic spectral methods for efficient Bayesian solution of inverse problems. Journal of Computational Physics , 224(2):560–586, 2007

  29. [29]

    Can local particle filters beat the curse of dimensionality? 34 Z

    Patrick Rebeschini, Ramon Van Handel, et al. Can local particle filters beat the curse of dimensionality? 34 Z. ZOU The Annals of Applied Probability , 25(5):2809–2866, 2015

  30. [30]

    The use of Morozov’s discrepancy principle for Tikhonov regularization for solving nonlinear ill-posed problems

    Otmar Scherzer. The use of Morozov’s discrepancy principle for Tikhonov regularization for solving nonlinear ill-posed problems. Computing, 51(1):45–60, 1993

  31. [31]

    Inverse problems: a Bayesian perspective

    Andrew M Stuart. Inverse problems: a Bayesian perspective. Acta Numerica, 19:451–559, 2010

  32. [32]

    Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image

    Nicholas Syring and Ryan Martin. Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image. arXiv preprint arXiv:1606.08400 , 2016

  33. [33]

    A Bayesian inference approach to the inverse heat conduction problem

    Jingbo Wang and Nicholas Zabaras. A Bayesian inference approach to the inverse heat conduction problem. International Journal of Heat and Mass Transfer , 47(17-18):3927–3941, 2004

  34. [34]

    An Adaptive Sampling Approach for Solving PDEs with Uncertain Inputs and Evaluating Risk

    Zilong Zou, Drew Kouri, and Wilkins Aquino. An Adaptive Sampling Approach for Solving PDEs with Uncertain Inputs and Evaluating Risk. In 19th AIAA Non-Deterministic Approaches Conference , page 1325, 2017

  35. [35]

    An adaptive local reduced basis method for solving PDEs with uncertain inputs and evaluating risk

    Zilong Zou, Drew Kouri, and Wilkins Aquino. An adaptive local reduced basis method for solving PDEs with uncertain inputs and evaluating risk. Computer Methods in Applied Mechanics and Engineering, 2018