Pith. sign in

REVIEW 2 major objections 5 minor 79 references

Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DF-GMVI makes derivative-free Bayesian inference stable by keeping Gaussian-mixture covariances positive and captures multimodal, curved posteriors.

desk verdict DF-GMVI is a genuine algorithmic contribution—stable derivative-free mixture VI—but the stability guarantee does not establish posterior accuracy, and the Hessian quadrature's PSD projection can bias the fixed point. read the letter →

arxiv 2501.04259 v3 pith:6FSR2MWB submitted 2025-01-08 cs.LG cs.NAmath.NA

Baojun Che , Yifan Chen , Zhenghao Huan , Daniel Zhengyu Huang , Weijie Wang This is my paper · ORCID
classification cs.LGcs.NAmath.NA MSC 62F1565M3290C56
keywords BayesianinverseproblemsvariationalinferenceGaussianmixturederivative-freemethodsFisher-Raogradientflownaturalcovariancepositivitymultimodalposteriors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces DF-GMVI, a derivative-free variational inference algorithm that approximates Bayesian posteriors with Gaussian mixtures when the forward model is expensive and gradients are unavailable. The method discretizes the Fisher-Rao natural gradient flow using two specialized quadrature rules: one evaluates the least-squares potential and its gradient and Hessian through finite differences at $\sigma$ points, and one treats the mixture entropy terms consistently. The paper's central claim is that the resulting updates keep every mixture covariance positive definite whenever the time step satisfies $0<\Delta t<1$, and that the scheme is affine invariant under triangular coordinate transformations. Because of this stability, the algorithm can use large time steps without covariance breakdown and, in the reported experiments, captures multiple, overlapping, infinitely many, and curved modes in up to 100 dimensions, including a Navier-Stokes initial-condition recovery. The practical motivation is that many large-scale inverse problems have black-box or nondifferentiable forward models, so a stable derivative-free posterior approximation matters.

What carries the argument

The machinery is the Fisher-Rao (natural) gradient flow restricted to Gaussian mixture variational families, together with a pair of derivative-free quadrature rules. Definition 4.1 generates $2N_\theta+1$ $\sigma$ points per Gaussian component and approximates the expectation of $\Phi_R$ and its gradient by mean-point and finite-difference formulas; the Hessian expectation is kept in the positive semidefinite form $\sqrt{C}^{-T}(6\operatorname{Diag}(A^T A)+B^T B)\sqrt{C}^{-1}$. Definition 4.3 approximates the entropy terms consistently at the same mean points and replaces the negative-definite part of the log-mixture Hessian by $-C_k^{-1}$, which is exact in weighted average. The load-bearing identity is the covariance update $C_k^{-1}(t+\Delta t)=(1-\Delta t)C_k^{-1}(t)+\Delta t$ times a positive semidefinite matrix, which is what yields Proposition 5.1 and enables $O(1)$ time steps.

What would settle it

Set a target posterior with known non-Gaussian form in which the dropped cross term $2\operatorname{Diag}(A^T c)$ in the Hessian expectation is large, for example by giving $F$ a strong quadratic interaction along a single coordinate. If DF-GMVI's converged density differs from the best $K$-component Gaussian mixture fit by a total-variation distance much larger than the same distance for a gradient-based natural-gradient flow using the exact Hessian, then the truncation in Definition 4.1 is the limiting error; repeating over targets with increasing dropped-term magnitude would map the accuracy boundary of the method.

Watch

Extended reading notes

Core claim

The central discovery is that the instability of Gaussian-mixture variational inference, where covariance matrices lose positive definiteness under large time steps, can be removed by choosing consistent and curvature-aware quadrature rules. In Definition 4.1 the paper approximates expectations of $\Phi_R(\theta)=\tfrac12 F(\theta)^T F(\theta)$ at $2N_\theta+1$ $\sigma$ points per Gaussian component, using finite differences to form matrices $B$ and $A$. The Hessian expectation is approximated as $\sqrt{C}^{-T}(6\operatorname{Diag}(A^T A)+B^T B)\sqrt{C}^{-1}$, keeping only the positive semidefinite part of the exact Gaussian-Newton-type expression. Combining this with a Hessian correction for the log-mixture term in Definition 4.3 makes the covariance update $C_k^{-1}(t+\Delta t)=(1-\Delta t)C_k^{-1}(t)+\Delta t\cdot(\text{positive semidefinite terms})$, so Proposition 5.1 gives $C_k$ positive definite for every $0<\Delta t<1$. Proposition 5.2 establishes affine invariance of the continuous natural gradient flow and a restricted (lower-triangular) affine invariance for the discretized algorithm when Cholesky square roots are used. The paper's claim is that this stability and invariance, together with linear cost in the dimension for forward-model evaluations, makes DF-GMVI a reliable derivative-free tool for multimodal Bayesian inverse problems.

Load-bearing premise

The load-bearing premise is that the curvature kept by the derivative-free quadrature is close enough to the true curvature that the updates settle near the true posterior; the paper proves the updates stay numerically stable, not that they land on the right answer.

Editorial extensions

If this is right

  • Users can set $\Delta t=0.5$ and run hundreds of iterations with no time-step tuning while preserving covariance positivity, as the paper does in all reported experiments.
  • Derivative-free posterior approximation costs only $(2N_\theta+1)K$ forward-model evaluations per iteration, and those evaluations are embarrassingly parallel, making high-dimensional inverse problems feasible.
  • Because the entropy and cross-entropy terms are treated with consistent quadrature, overlapping modes are resolved in a way that the earlier Kalman-based derivative-free method could not, according to the paper's experiments.
  • The method captures posterior shapes that typically break gradient-free samplers: narrow curved banana valleys, circular ridges with infinitely many modes, and multimodal 100-dimensional marginals.
  • The Navier-Stokes experiment indicates that a 128-parameter PDE inverse problem with a symmetric bimodal posterior can be solved by a derivative-free variational method that converges in fewer than 50 iterations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of Proposition 5.1 is that any quadrature whose Hessian surrogate is positive semidefinite and whose entropy Hessian term is replaced by the exact weighted average $-C_k^{-1}$ would enjoy the same stability guarantee, since the proof uses only positivity, not the particular finite-difference formula.
  • The stability guarantee does not by itself control the gap between the stationary point of (4.9) and the true posterior; accuracy still rests on the surrogate Hessian's fidelity, so the most informative future experiment would compare DF-GMVI's stationary KL divergence against the best achievable $K$-component mixture approximation on targets with controlled curvature.
  • The block-diagonal Fisher information approximation inherited from earlier work assumes well-separated modes; for strongly overlapping components this approximation may be the limiting error even though covariance positivity and consistent entropy treatment fix the overlap artifacts observed in the earlier method.
  • Because the algorithm is affine invariant under triangular transformations, its empirical convergence rates on anisotropic posteriors should persist under linear reparameterizations, which suggests the reported banana-shaped-case results are not artifacts of a particular coordinate system.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper develops Derivative Free Gaussian Mixture Variational Inference (DF-GMVI), a variational method for Bayesian inverse problems where the forward model is expensive and gradients are unavailable. The method discretizes a Fisher-Rao natural gradient flow over Gaussian mixture variational families, using two derivative-free quadrature rules: one for the least-squares potential ΦR(θ)=1/2 F(θ)^T F(θ) (Definition 4.1) and one for the Gaussian-mixture entropy terms (Definition 4.3). The main theoretical claims are covariance positivity for time steps Δt∈(0,1) (Proposition 5.1) and affine invariance, restricted to lower-triangular linear transformations when the Cholesky square root is used (Proposition 5.2). Numerical experiments cover 1D and 2D targets, 100-dimensional versions of those targets, comparisons with other variational and MCMC methods, and a 128-parameter Navier-Stokes initial-condition recovery problem with a bimodal posterior.

Significance. If the accuracy of the quadrature approximation can be established, DF-GMVI would be a practically valuable contribution: it combines derivative-free updates, a deliberate treatment of entropy and cross-entropy consistency, a simple stability guarantee, and a large-scale PDE application. The paper has clear strengths: Proposition 5.1 is elementary and machine-checkable; the affine-invariance analysis is explicit; the numerical study is broad and includes comparisons with BBVI, natural-gradient and Wasserstein-gradient methods, and affine-invariant MCMC; and the code is publicly available. The principal gap is that the stability guarantee (Proposition 5.1) is not accompanied by an accuracy statement for the surrogate Hessian, so the evidence for the central claim of accurate posterior approximation is currently empirical and partly heuristic.

major comments (2)
  1. [§4, Definition 4.1 and Eq. (4.5)] The PSD projection in the Hessian quadrature is load-bearing for posterior accuracy, but no accuracy estimate is provided. The exact expected Hessian of the quadratic surrogate contains the terms 2Diag(A^T c) and the off-diagonal column interactions 2Σ_{l≠j} a_j^T a_l; Eq. (4.5) replaces these with 6Diag(A^T A)+B^T B. These dropped terms are not negligible in general: for a scalar quadratic F(θ)=aθ²+bθ+c, the omitted term is 2ac, which can be large and negative when the residual at the mode is large, biasing the stationary covariance toward overconfidence. Because the fixed point of (4.9) is defined through the quadrature-approximated gradients and Hessians, a biased Hessian changes the fixed point itself, not merely the convergence path. Proposition 5.1 guarantees only positivity, and Section 7 explicitly defers convergence and accuracy analysis for general posteriors. The abstract's accuracy claim therefore rests on the heuristic PSD projection in Definition 4.1, compounded by the unquantified block-diagonal Fisher information approximation (C.16) introduced in Section 2.1. Please add an error bound for the surrogate Hessian under stated smoothness assumptions, or an explicit quantitative check that the discarded curvature terms are controlled on the problem class considered, or soften the accuracy claim to match the proven stability guarantee.
  2. [§6.2, 100-dimensional tests (text near Figure 4)] The claim that DF-GMVI remains accurate in 100 dimensions is supported only by two-dimensional marginal densities of the first two coordinates. The remaining 98 coordinates θc are coupled to θ through the matrix K and contribute to the posterior, but no reference comparison is reported for them. Total variation on the 2D marginal cannot detect errors in the other 98 dimensions, so the statements that performance is 'robust across different dimensionalities' and that DF-GMVI avoids the deterioration seen for BBVI/MCMC in 100D are not fully supported. A comparison in at least one additional coordinate direction, a one-dimensional slice or marginal of θc, or a summary statistic such as the marginal variance or RMSE in θc would make the high-dimensional claim credible.
minor comments (5)
  1. [Abstract and Proposition 5.2] The abstract states that DF-GMVI 'guarantees covariance positivity and affine invariance' without qualification, while Proposition 5.2 establishes affine invariance only for invertible lower-triangular transformations when the Cholesky square root is used. The abstract should be qualified to match this restriction.
  2. [§4, after Eq. (4.5)] The text says the positive definite part of the expectation is retained, but 6Diag(A^T A)+B^T B is only positive semidefinite in general; 'positive semidefinite part' would be more accurate.
  3. [§6.1 and Figure 2] The caption of Figure 2 lists the bottom-row noise variance as 1.5², while the text defines Case D as η∼N(0, 2.0²). The caption should be corrected.
  4. [§1.2 and §6] The contribution statement 'requires no hyperparameter tuning' is too strong given that α, Δt, K, and w_min are user-set parameters. Appendix B studies sensitivity, but the paper does not show that the method is insensitive for all problem classes; please soften the wording.
  5. [§6.2, comparison setup] The text says Figure 5 shows results over 500 iterations, while Section 6 states that all tests run for 200 iterations; this inconsistency in reported iteration counts should be reconciled so the comparison is unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the covariance-positivity guarantee is an explicitly designed algebraic property of the algorithm, and the accuracy claims rest on independent numerical benchmarks rather than on fitted inputs or self-citation chains.

full rationale

The derivation chain is self-contained for the claims it actually proves. Proposition 5.1 is an algebraic consequence of the DF-GMVI update (4.9) combined with the deliberately PSD-projected Hessian quadrature (4.5) and the Hessian correction in Definition 4.3; the paper explicitly states at Definition 4.1 that it retains only the positive definite part to ensure covariance positivity, so the theorem is a design guarantee, not a prediction disguised as a discovery. Proposition 5.2 is proved directly from the quadrature definitions in Appendix A, so it does not reduce to an external citation. The block-diagonal Fisher information approximation (C.16) is inherited from [15,50], and [15] shares authors with the present paper, but this is an explicitly stated modeling approximation rather than a load-bearing uniqueness theorem or an ansatz smuggled in as external proof; the stability and invariance theorems do not depend on [15]'s empirical results. Numerical claims are benchmarked against reference densities at fixed hyperparameters (Delta t = 0.5, alpha = 1e-3), with no fitting of parameters to the reference posterior, so no fitted input is renamed as a prediction. The skeptical concern that dropping 2Diag(A^T c) and column-interaction terms in (4.5) can bias the stationary covariance is a legitimate correctness and robustness issue, not a circularity; Section 7 itself defers a thorough convergence analysis, and Proposition 5.1 does not assert closeness to the true posterior. Accordingly, the paper shows no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

DF-GMVI's stability guarantee follows from the quadrature's PSD structure, but its accuracy depends on the block-diagonal FIM approximation (from prior work) and on the unproven surrogate Hessian accuracy. These are the load-bearing assumptions beyond the paper's explicit claims.

free parameters (4)
  • alpha (finite-difference step) = 1e-3
    Step size for sigma-point quadrature in Definition 4.1; chosen by hand, tested in Appendix B.
  • Delta_t (time step) = 0.5
    Forward Euler step in (4.9); chosen by hand, tested in Appendix B. Prop 5.1 allows any value in (0,1).
  • K (mixture components) = 2,5,10,20,40 in tests
    Number of Gaussian components; user-set per problem; Section 6.2 notes small K may miss modes.
  • w_min (weight floor) = 1e-8
    Lower bound on mixture weights during normalization in Section 4; default value, exponential recovery mechanism.
assumptions (5)
  • domain assumption Posterior has nonlinear least-squares form Φ_R(θ)=0.5||F(θ)||^2 (1.3).
    Structure of Bayesian inverse problems with Gaussian noise and Gaussian prior; Definition 4.1 relies on this form.
  • domain assumption Fisher information matrix for Gaussian mixtures is approximated by block-diagonal form (C.16), assuming components are well separated.
    Inherited from [15,50]; used to obtain (2.3); not re-derived in this paper.
  • domain assumption F is smooth enough for central finite differences with α=1e-3 to approximate gradient and curvature.
    Required for Definition 4.1; supported only by empirical sensitivity study, no error bound.
  • ad hoc to paper Mean-point consistency at component means is a sufficient target for quadrature design.
    Guideline from Section 3; motivates Definitions 4.1 and 4.3 but is not a theorem.
  • ad hoc to paper Retaining only the positive semidefinite part of the quadratic-model Hessian expectation preserves enough curvature for accurate posterior approximation.
    Design choice in the paragraph after (4.5); Prop 5.1 does not quantify accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems." pith.science (2026). https://pith.science/paper/6FSR2MWB

@misc{pith2026250104259,
  author       = {Pith},
  title        = {Pith review of: Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6FSR2MWB}},
  note         = {Machine review of arXiv:2501.04259}
}
read the original abstract

This paper is concerned with the approximation of probability distributions known up to normalization constants, with a focus on Bayesian inference for large-scale inverse problems in scientific computing. In this context, key challenges include costly repeated evaluations of forward models, multimodality, and inaccessible gradients for the forward model. To address them, we develop a variational inference framework that combines Fisher-Rao natural gradient with specialized quadrature rules to enable derivative free updates of Gaussian mixture variational families. The resulting method, termed Derivative Free Gaussian Mixture Variational Inference (DF-GMVI), guarantees covariance positivity and affine invariance, offering a stable and efficient framework for approximating complex posterior distributions. The effectiveness of DF-GMVI is demonstrated through numerical experiments on challenging scenarios, including distributions with multiple modes, infinitely many modes, and curved modes in spaces with up to 100 dimensions. The method's practicality is further demonstrated in a large-scale application, where it successfully recovers the initial conditions of the Navier-Stokes equations from solution data at positive times.

Figures

Figures reproduced from arXiv: 2501.04259 by the authors.

Figure 1
Figure 1. Quadrature rule comparison for Gaussian mixture and circular-shaped target [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Results for the 1D bimodal problem with Σ [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Multi-dimensional problems with dimensionality 2, arranged from Case A to [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Multi-dimensional problems with dimensionality 100, arranged from Case A [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Comparison study for multi-dimensional problems with dimensionality 2, ar [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Comparison study for multi-dimensional problems with dimensionality 100 [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: The vorticity field ω at T = 0.25 and T = 0.5 and observations ω([x(1), x(2)] T )− ω([2π−x(1), x(2)] T ) at 56 equidistant points (solid black dots). Their mirroring points are marked (empty black dots). itself [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: The true initial vorticity field ω0(x; θref), and recovered initial vorticity fields ω0(x; mk) obtained by DF-GMVI. 0 50 100 150 200 Iterations 0.4 0.6 0.8 1.0 1.2 1.4 Rel. error of (x) mode 1 mode 2 mode 3 mode 4 mode 5 0 50 100 150 200 Iterations 10 2 10 3 R mode 1 m…
Figure 9
Figure 9. Figure 9: Navier-Stokes flow problem: the relative errors of the initial vorticity field, the [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Navier-Stokes flow problem: the true Karhunen-Loeve expansion parameters [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Comparison for different choices of α and ∆t on the Case E of multi￾dimensional problems with dimensionality 2. The left figure shows the total variation between the reference density and the densities estimated by the DF-GMVI over the iterations using different α ∈ {…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 66 canonical work pages

  1. [1]

    Alsup and B

    T. Alsup and B. Peherstorfer , Context-aware surrogate modeling for balancing approx- imation and sampling costs in multifidelity importance sampling and bayesian inverse problems, SIAM/ASA Journal on Uncertainty Quantification, 11 (2023), pp. 285–319

    work page 2023

  2. [2]

    Amari , Natural gradient works efficiently in learning , Neural computation, 10 (1998), pp

    S.-I. Amari , Natural gradient works efficiently in learning , Neural computation, 10 (1998), pp. 251–276

    work page 1998

  3. [3]

    Arasaratnam and S

    I. Arasaratnam and S. Haykin , Cubature kalman filters , IEEE Transactions on automatic control, 54 (2009), pp. 1254–1269

    work page 2009

  4. [4]

    M. J. Berger, P. Colella, et al. , Local adaptive mesh refinement for shock hydrodynamics , Journal of computational Physics, 82 (1989), pp. 64–84

    work page 1989

  5. [5]

    Beskos, A

    A. Beskos, A. Jasra, E. A. Muzaffer, and A. M. Stuart , Sequential monte carlo methods for bayesian elliptic inverse problems , Statistics and Computing, 25 (2015), pp. 727–737

    work page 2015

  6. [6]

    D. M. Blei, A. Kucukelbir, and J. D. McAuliffe , Variational inference: A review for statisticians, Journal of the American statistical Association, 112 (2017), pp. 859–877

    work page 2017

  7. [7]

    Borker, D

    R. Borker, D. Huang, S. Grimberg, C. F arhat, P. A very, and J. Rabinovitch , Mesh adaptation framework for embedded boundary methods for computational fluid dynamics 22 and fluid-structure interaction, International Journal for Numerical Methods in Fluids, 90 (2019), pp. 389–424

    work page 2019

  8. [8]

    C. J. T. Braak , A markov chain monte carlo version of the genetic algorithm differential evolution: easy bayesian computing for real parameter spaces , Statistics and Computing, 16 (2006), pp. 239–249

    work page 2006

Show all 79 references
  1. [9]

    Burger, M

    M. Burger, M. Erbar, F. Hoffmann, D. Matthes, and A. Schlichting , Covariance- modulated optimal transport and gradient flows , arXiv preprint arXiv:2302.07773, (2023)

    2023 arXiv

  2. [10]

    Cao and D

    S. Cao and D. Z. Huang, Bayesian calibration for large-scale fluid structure interaction prob- lems under embedded/immersed boundary framework, International Journal for Numerical Methods in Engineering, (2022)

    2022

  3. [11]

    Carrillo and U

    J. Carrillo and U. V aes , Wasserstein stability estimates for covariance-preconditioned fokker–planck equations, Nonlinearity, 34 (2021), p. 2275

    2021

  4. [12]

    J. A. Carrillo, Y. Chen, D. Z. Huang, J. Huang, and D. Wei , Fisher-rao gradient flow: geodesic convexity and functional inequalities , arXiv preprint arXiv:2407.15693, (2024)

    2024

  5. [13]

    J. A. Carrillo, K. Craig, and F. S. Patacchini , A blob method for diffusion , Calculus of Variations and Partial Differential Equations, 58 (2019), pp. 1–53

    2019

  6. [14]

    Y. Chen, D. Z. Huang, J. Huang, S. Reich, and A. M. Stuart , Sampling via gradient flows in the space of probability measures , arXiv preprint arXiv:2310.03597, (2023)

    2023 arXiv

  7. [15]

    Y. Chen, D. Z. Huang, J. Huang, S. Reich, and A. M. Stuart , Efficient, multimodal, and derivative-free bayesian inference with fisher–rao gradient flows , Inverse Problems, 40 (2024), p. 125001

    2024

  8. [16]

    Cleary, A

    E. Cleary, A. Garbuno-Inigo, S. Lan, T. Schneider, and A. M. Stuart , Calibrate, emu- late, sample , arXiv preprint arXiv:2001.03689, (2020)

    2020 arXiv

  9. [17]

    Detommaso, T

    G. Detommaso, T. Cui, Y. Marzouk, A. Spantini, and R. Scheichl , A stein variational newton method, Advances in Neural Information Processing Systems, 31 (2018)

    2018

  10. [18]

    M. Z. Diao, K. Balasubramanian, S. Chewi, and A. Salim , Forward-backward gaussian variational inference via jko in the bures-wasserstein space , in International Conference on Machine Learning, PMLR, 2023, pp. 7960–7991

    2023

  11. [19]

    Domingo-Enrich and A.-A

    C. Domingo-Enrich and A.-A. Pooladian , An explicit expansion of the kullback-leibler divergence along its fisher-rao gradient flow , Trans. Mach. Learn. Res., 2023 (2023), https://api.semanticscholar.org/CorpusID:257102914

    2023

  12. [20]

    Doucet, A

    A. Doucet, A. M. Johansen, et al. , A tutorial on particle filtering and smoothing: Fifteen years later, Handbook of nonlinear filtering, 12 (2009), p. 3

    2009

  13. [21]

    Elfring, E

    J. Elfring, E. Torta, and R. V an De Molengraft , Particle filters: A hands-on tutorial , Sensors, 21 (2021), p. 438

    2021

  14. [22]

    G. Evensen , Sequential data assimilation with a nonlinear quasi-geostrophic model using monte carlo methods to forecast error statistics , Journal of Geophysical Research: Oceans, 99 (1994), pp. 10143–10162

    1994

  15. [23]

    Foreman-Mackey, D

    D. Foreman-Mackey, D. W. Hogg, D. Lang, and J. Goodman , emcee: the mcmc hammer , Publications of the Astronomical Society of the Pacific, 125 (2013), p. 306

    2013

  16. [24]

    Z. Gao, L. Yan, and T. Zhou , Adaptive operator learning for infinite-dimensional bayesian inverse problems, SIAM/ASA Journal on Uncertainty Quantification, 12 (2024), pp. 1389– 1423

    2024

  17. [25]

    Garbuno-Inigo, F

    A. Garbuno-Inigo, F. Hoffmann, W. Li, and A. M. Stuart, Interacting langevin diffusions: Gradient structure and ensemble kalman sampler , SIAM Journal on Applied Dynamical Systems, 19 (2020), pp. 412–441

    2020

  18. [26]

    Gayrard, A

    V. Gayrard, A. Bovier, M. Eckhoff, and M. Klein , Metastability in reversible diffusion processes i: Sharp asymptotics for capacities and exit times , Journal of the European Mathematical Society, 6 (2004), pp. 399–424

    2004

  19. [27]

    Gayrard, A

    V. Gayrard, A. Bovier, and M. Klein , Metastability in reversible diffusion processes ii: Precise asymptotics for small eigenvalues , Journal of the European Mathematical Society, 7 (2005), pp. 69–99

    2005

  20. [28]

    Gelman, W

    A. Gelman, W. R. Gilks, and G. O. Roberts , Weak convergence and optimal scaling of random walk metropolis algorithms , The annals of applied probability, 7 (1997), pp. 110– 120

    1997

  21. [29]

    C. J. Geyer , Practical markov chain monte carlo , Statistical science, (1992), pp. 473–483

    1992

  22. [30]

    M. B. Giles , Multilevel monte carlo methods , Acta numerica, 24 (2015), pp. 259–328

    2015

  23. [31]

    Goodman and J

    J. Goodman and J. Weare , Ensemble samplers with affine invariance , Communications in applied mathematics and computational science, 5 (2010), pp. 65–80

    2010

  24. [32]

    L. Guo, H. Wu, Y. W ang, W. Zhou, and T. Zhou, Ib-uq: Information bottleneck based uncer- tainty quantification for neural function regression and neural operator learning , Journal of Computational Physics, 510 (2024), p. 113089. DERIV ATIVE FREE V ARIATIONAL INFERENCE 23

    2024

  25. [33]

    Hairer and J

    M. Hairer and J. C. Mattingly, Ergodicity of the 2d navier-stokes equations with degenerate stochastic forcing, Annals of Mathematics, (2006), pp. 993–1032

    2006

  26. [34]

    Han and Q

    J. Han and Q. Liu , Stein variational gradient descent without gradient , in International Con- ference on Machine Learning, PMLR, 2018, pp. 1900–1908

    2018

  27. [35]

    M. D. Hoffman, D. M. Blei, C. W ang, and J. Paisley , Stochastic variational inference , Journal of Machine Learning Research, (2013)

    2013

  28. [36]

    D. Z. Huang, P. A very, C. F arhat, J. Rabinovitch, A. Derkevorkian, and L. D. Pe- terson, Modeling, simulation and validation of supersonic parachute inflation dynamics during mars landing , in AIAA Scitech 2020 Forum, 2020, p. 0313

    2020

  29. [37]

    D. Z. Huang, D. De Santis, and C. F arhat, A family of position-and orientation-independent embedded boundary methods for viscous flow and fluid–structure interaction problems , Journal of Computational Physics, 365 (2018), pp. 74–104

    2018

  30. [38]

    D. Z. Huang, J. Huang, S. Reich, and A. M. Stuart , Efficient derivative-free bayesian inference for large-scale inverse problems , Inverse Problems, 38 (2022), p. 125006

    2022

  31. [39]

    D. Z. Huang, T. Schneider, and A. M. Stuart , Iterated kalman methodology for inverse problems, Journal of Computational Physics, 463 (2022), p. 111262

    2022

  32. [40]

    T. Huix, A. Korba, A. Durmus, and E. Moulines , Theoretical guarantees for variational in- ference with fixed-variance mixture of gaussians , arXiv preprint arXiv:2406.04012, (2024)

    2024 arXiv

  33. [41]

    A. H. Jazwinski , Stochastic processes and filtering theory , Courier Corporation, 2007

    2007

  34. [42]

    J. Jia, P. Li, and D. Meng , Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems , SIAM Journal on Numerical Analysis, 60 (2022), pp. 2225–2252

    2022

  35. [43]

    Jordan, D

    R. Jordan, D. Kinderlehrer, and F. Otto, The variational formulation of the fokker–planck equation, SIAM journal on mathematical analysis, 29 (1998), pp. 1–17

    1998

  36. [44]

    Julier, J

    S. Julier, J. Uhlmann, and H. Durrant-Whyte , A new method for the nonlinear transfor- mation of means and covariances in filters and estimators , IEEE Transactions on Auto- matic Control, 45 (2000), pp. 477–482, https://doi.org/10.1109/9.847726

    2000 doi

  37. [45]

    S. J. Julier and J. K. Uhlmann , New extension of the kalman filter to nonlinear systems , in Signal processing, sensor fusion, and target recognition VI, vol. 3068, International Society for Optics and Photonics, 1997, pp. 182–193

    1997

  38. [46]

    S. J. Julier, J. K. Uhlmann, and H. F. Durrant-Whyte , A new approach for filtering nonlinear systems , in Proceedings of 1995 American Control Conference-ACC’95, vol. 3, IEEE, 1995, pp. 1628–1632

    1995

  39. [47]

    Kaipio and E

    J. Kaipio and E. Somersalo , Statistical and computational inverse problems , vol. 160, Springer Science & Business Media, 2006

    2006

  40. [48]

    Lambert, S

    M. Lambert, S. Chewi, F. Bach, S. Bonnabel, and P. Rigollet , Variational inference via wasserstein gradient flows, Advances in Neural Information Processing Systems, 35 (2022), pp. 14434–14447

    2022

  41. [49]

    Y. Li, Y. W ang, and L. Yan , Surrogate modeling for bayesian inverse problems based on physics-informed neural networks , Journal of Computational Physics, 475 (2023), p. 111841

    2023

  42. [50]

    W. Lin, M. E. Khan, and M. Schmidt, Fast and simple natural-gradient variational inference with mixture of exponential-family approximations, in International Conference on Machine Learning, PMLR, 2019, pp. 3992–4002

    2019

  43. [51]

    Liu , Stein variational gradient descent as gradient flow , Advances in neural information processing systems, 30 (2017)

    Q. Liu , Stein variational gradient descent as gradient flow , Advances in neural information processing systems, 30 (2017)

    2017

  44. [52]

    T. Liu, P. Ghosal, K. Balasubramanian, and N. Pillai , Towards understanding the dy- namics of gaussian-stein variational gradient descent , Advances in Neural Information Processing Systems, 36 (2024)

    2024

  45. [53]

    Lopez-Gomez, C

    I. Lopez-Gomez, C. Christopoulos, H. L. Langeland Ervik, O. R. Dunbar, Y. Cohen, and T. Schneider, Training physics-based machine-learning parameterizations with gradient- free ensemble kalman methods, Journal of Advances in Modeling Earth Systems, 14 (2022), p. e2022MS003105

    2022

  46. [54]

    H. Lu, J. Jia, and D. Meng , Sequential monte carlo with gaussian mixture distributions for infinite-dimensional statistical inverse problems , arXiv preprint arXiv:2503.16028, (2025)

    2025

  47. [55]

    Y. Lu, D. Slep ˇcev, and L. W ang, Birth–death dynamics for sampling: global convergence, approximations and their asymptotics , Nonlinearity, 36 (2023), p. 5731, https://doi.org/ 10.1088/1361-6544/acf988, https://dx.doi.org/10.1088/1361-6544/acf988

    2023 doi

  48. [56]

    Martens, New insights and perspectives on the natural gradient method, Journal of Machine Learning Research, 21 (2020), pp

    J. Martens, New insights and perspectives on the natural gradient method, Journal of Machine Learning Research, 21 (2020), pp. 1–76, http://jmlr.org/papers/v21/17-678.html

    2020

  49. [57]

    Maurais and Y

    A. Maurais and Y. Marzouk , Sampling in unit time with kernel fisher-rao flow , arXiv pre- print arXiv:2401.03892, (2024). 24

    2024 arXiv

  50. [58]

    Mo¨es, J

    N. Mo¨es, J. Dolbow, and T. Belytschko, A finite element method for crack growth without remeshing, International journal for numerical methods in engineering, 46 (1999), pp. 131– 150

    1999

  51. [59]

    J. B. Nagel and B. Sudret , A unified framework for multilevel uncertainty quantification in bayesian inverse problems , Probabilistic Engineering Mechanics, 43 (2016), pp. 68–84

    2016

  52. [60]

    D. H. Nguyen, T. Sakurai, and H. Mamitsuka , Wasserstein gradient flow over variational parameter space for variational inference , arXiv preprint arXiv:2310.16705, (2023)

    2023 arXiv

  53. [61]

    Opper and C

    M. Opper and C. Archambeau , The variational gaussian approximation revisited , Neural computation, 21 (2009), pp. 786–792

    2009

  54. [62]

    C. S. Peskin, Numerical analysis of blood flow in the heart , Journal of computational physics, 25 (1977), pp. 220–252

    1977

  55. [63]

    Ranganath, S

    R. Ranganath, S. Gerrish, and D. Blei , Black box variational inference , in Artificial intel- ligence and statistics, PMLR, 2014, pp. 814–822

    2014

  56. [64]

    C. R. Rao, Information and the accuracy attainable in the estimation of statistical parameters , Reson. J. Sci. Educ, 20 (1945), pp. 78–90

    1945

  57. [65]

    M. K. Sen and P. L. Stoffa, Global optimization methods in geophysical inversion, Cambridge University Press, 2013

    2013

  58. [66]

    Smith , Sequential Monte Carlo methods in practice , Springer Science & Business Media, 2013

    A. Smith , Sequential Monte Carlo methods in practice , Springer Science & Business Media, 2013

    2013

  59. [67]

    A. M. Stuart , Inverse problems: a bayesian perspective , Acta numerica, 19 (2010), pp. 451– 559

    2010

  60. [68]

    Z. Tan, C. M. Kaul, K. G. Pressel, Y. Cohen, T. Schneider, and J. Teixeira, An extended eddy-diffusivity mass-flux scheme for unified representation of subgrid-scale turbulence and convection, Journal of Advances in Modeling Earth Systems, 10 (2018), pp. 770–800

    2018

  61. [69]

    Tebaldi, R

    C. Tebaldi, R. L. Smith, D. Nychka, and L. O. Mearns , Quantifying uncertainty in pro- jections of regional climate change: A Bayesian approach to the analysis of multimodel ensembles, Journal of Climate, 18 (2005), pp. 1524–1540

    2005

  62. [70]

    J. A. Vrugt, C. J. ter Braak, C. G. Diks, B. A. Robinson, J. M. Hyman, and D. Hig- don, Accelerating markov chain monte carlo simulation by differential evolution with self- adaptive randomized subspace sampling , International journal of nonlinear sciences and numerical simul...

    2009

  63. [71]

    M. J. W ainwright, M. I. Jordan, et al. , Graphical models, exponential families, and vari- ational inference, Foundations and Trends® in Machine Learning, 1 (2008), pp. 1–305

    2008

  64. [72]

    Welandawe, M

    M. Welandawe, M. R. Andersen, A. Vehtari, and J. H. Huggins, A framework for improv- ing the reliability of black-box variational inference, Journal of Machine Learning Research, 25 (2024), pp. 1–71

    2024

  65. [73]

    Wibisono, Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem, in Conference on learning theory, PMLR, 2018, pp

    A. Wibisono, Sampling as optimization in the space of measures: The langevin dynamics as a composite optimization problem, in Conference on learning theory, PMLR, 2018, pp. 2093– 3027

    2018

  66. [74]

    Yan and T

    L. Yan and T. Zhou, An adaptive surrogate modeling based on deep neural networks for large- scale bayesian inverse problems , Communications in Computational Physics, 28 (2020), pp. 2180–2205, https://doi.org/10.4208/cicp.OA-2020-0186

    2020 doi

  67. [75]

    Yan and T

    L. Yan and T. Zhou, Stein variational gradient descent with local approximations , Computer Methods in Applied Mechanics and Engineering, 386 (2021), p. 114087

    2021

  68. [76]

    Yatunin, S

    D. Yatunin, S. Byrne, C. Kawczynski, S. Kandala, G. Bozzola, A. Sridhar, Z. Shen, A. Jaruga, J. Sloan, J. He, et al., The climate modeling alliance atmosphere dynamical core: Concepts, numerics, and scaling , Authorea Preprints, (2025)

    2025

  69. [77]

    Zhu and A

    J.-J. Zhu and A. Mielke , Kernel approximation of fisher-rao gradient flows , arXiv preprint arXiv:2410.20622, (2024). Appendix A. Proof of Proposition 5.2. Proof. We note that ∇θΦR(θ) = T T ∇eθ eΦR(eθ), ∇θ∇θΦR(θ) = T T ∇eθ∇eθ eΦR(eθ)T

    2024 arXiv

  70. [78]

    We simplify the notation by eNk(eθ) = N (eθ, fmk, fCk), ρGM ea (eθ) = PK k=1 ewk eNk(eθ)

    To prove the affine invariance of the natural gradient flow defined by (2.3), it suffices to verify that under the affine mapping ewk = wk, emk = T mk + d, eCk = T CkT T , ewk, emk and eCk also satisfy the corresponding equation (2.3). We simplify the notation by eNk(eθ) = N (...

  71. [79]

    Here in the last equality, we can tolerate a constant difference since we will normalize wk(t + ∆t)K k=1 after the update (4.9)

    To prove that the DF-GMVI algorithm (4.9) is T -invariant, it suffices to show that the quadrature rules defined in Definition 4.1 and Definition 4.3 satisfy the following equations for any T ∈ T: (A.1) QR eNk ∇eθ∇eθ log ρGM ea + ∇eθ∇eθ eΦR = T −T QRNk ∇θ∇θ log ρGM a + ∇θ∇θΦR ...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.