REVIEW 4 major objections 5 minor 32 references
Greedy Stein Variational Gradient Descent: An algorithmic approach for wave prospection problems
T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read This paper proposes G-SVGD, a variational inference algorithm that appends a weighted gradient-norm term to the ELBO and optimizes the step size against the combined loss, claiming faster posterior approximation in wave prospection models.
desk verdict A clearly-specified SVGD variant whose custom loss is never shown to track posterior fidelity; the acceleration claim rests on circular comparisons. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the loss function $L(\Theta,q) = (1-\omega)\lVert \nabla_x \log(\Theta,y)\rVert_\infty - \omega\,\mathrm{ELBO}(q)$, which blends the supremum norm of the log-posterior gradient across particles with the evidence lower bound of a kernel density estimate $q$. Minimizing this loss over the step size $\alpha$ at each iteration steers particles along the SVGD direction $\phi$ while controlling the trade-off between mode-seeking ($\omega$ near 0) and distribution-shape alignment ($\omega$ near 1) through KDE. The five-point finite-difference operator and adjoint method enter as supporting machinery that makes the gradient evaluations accurate enough for the loss to be meaningful. The paper's convergence argument rests on the condition $\lVert \nabla_x \log(\Theta,y)\rVert_\infty = -\mathrm{ELBO}(q)$ derived from minimizing $L$ with respect to $\omega$, which it takes as the stopping criterion.
What would settle it
Run G-SVGD, A-SVGD, and t-walk on a Bayesian model with a known Gaussian-mixture posterior and known true parameters; if G-SVGD's 200-particle output has substantially larger KL divergence to the true posterior than the t-walk output, for comparable wall-clock time, the claim that the loss drives convergence to the posterior would be contradicted. A cheaper check: on a unimodal Gaussian posterior, verify whether the stopping condition $\lVert \nabla \log p\rVert_\infty = -\mathrm{ELBO}$ holds at convergence while minimizing $L$ actually decreases KL divergence.
Extended reading notes
Core claim
On the paper's own terms, the discovery is an algorithm rather than a theorem: replacing SVGD's hand-set or per-parameter learning rate with a global step size $\alpha^* = \arg\min_{0\le \alpha\le \xi} L(\Phi(\alpha), q_\alpha)$, where $L(\Theta,q) = (1-\omega)\lVert \nabla_x \log(\Theta,y)\rVert_\infty - \omega\,\mathrm{ELBO}(q)$, yields a Greedy Stein Variational Gradient Descent that drives particles toward local modes while approximating the local density with kernel density estimation. In the high-contrast experiment, G-SVGD reached a posterior approximation comparable to t-walk with 200 samples where t-walk needed 10,000, and it converged faster than SVGD-ADAM across $\omega = \{0, 0.5, 1\}$. The paper interprets this as evidence that the loss function provides an effective stopping condition and step-size selection rule, and that the method is well suited to wave-prospection models where gradient evaluation is computationally expensive.
Load-bearing premise
The argument assumes that the loss $L(\Theta,q)$ is a valid measure of how close the particle set is to the posterior, so that minimizing it drives particles to the posterior; the paper derives a balance condition on $\omega$ but never connects $L$ to any divergence from the posterior, and if $L$ does not measure posterior fidelity the acceleration and accuracy claims lack a foundation.
Editorial extensions
If this is right
- Gradient-expensive inverse problems would become tractable with far fewer forward solves: 200 particles versus 10,000 MCMC samples in the demonstrated high-contrast case.
- Uncertainty quantification in wave prospection can be done with a global, adaptive step size and an explicit stopping rule, both of which the original SVGD leaves unspecified.
- The weight $\omega$ acts as a dial between mode-hunting and shape-matching, giving practitioners a practical trade-off for multimodal posteriors.
- The five-point operator with its error bound and node-per-wavelength criterion gives a concrete recipe for discretizing the wave forward model accurately enough for gradient-based inference.
Reading between the lines
- The condition $\lVert \nabla \log p\rVert_\infty = -\mathrm{ELBO}$ suggests a cheap convergence diagnostic, but the paper does not prove that this condition is sufficient for convergence to the target distribution; a toy experiment with a known posterior would clarify whether the balance point tracks KL divergence.
- If the loss is used as a stopping rule, then any bias introduced by KDE smoothing, which the paper lists as a limitation, will leak directly into the inferred posterior width; comparing G-SVGD output with a Gaussian mixture of known covariance would quantify that bias.
- The 50x sample reduction over t-walk is demonstrated in a 2-parameter high-contrast model; in the 15-parameter low-contrast spline model the reference methods were not run, so the efficiency claim is not yet established at higher dimension.
- The same loss could be tested as a step-size selector for other particle-based samplers, since the mechanism is agnostic to the forward model; that would be a quick extension the paper does not pursue.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Greedy Stein Variational Gradient Descent (G-SVGD), a modification of SVGD for Bayesian inference in wave prospection problems. The core novelty is a loss function L(Θ,q) = (1−ω)∥∇θ log π(Θ|y)∥∞ − ω ELBO(q), used to select a global step size at each iteration and to define a stopping rule. The method is applied to two 1D wave-equation models (high-contrast and low-contrast stratified media), with derivatives computed by a five-point finite-difference operator and gradients of the log posterior via the adjoint method. Numerical comparisons are made against SVGD with ADAM and the t-walk MCMC sampler.
Significance. If the central claims were established, G-SVGD would be a valuable contribution to variational inference for PDE-constrained inverse problems, particularly for expensive forward solvers. The paper contains some useful components: a careful derivation of five-point finite-difference error bounds, a clear treatment of the high-contrast layered wave model with reflection/transmission coefficients, and transparent acknowledgement of algorithmic limitations in Section 4.4. However, the central algorithmic innovation—the loss function in Eq. (35)—is not shown to measure posterior fidelity, and the convergence claims in the abstract are not supported by any theorem. The numerical comparisons are also circular because all methods are evaluated on the same loss that G-SVGD is explicitly designed to minimize. These issues concern the core contribution, not peripheral presentation.
major comments (4)
- [§2.7, Eqs. (35)–(38)] The derivation of the loss-function minimum is flawed. For fixed Θ and q, L is affine in ω, so ∂²ω L = 0 and the minimum over ω ∈ [0,1] occurs at an endpoint; the interior stationarity condition ∥∇x log(Θ,y)∥∞ = −ELBO(q) in Eq. (38) does not identify a minimum. More fundamentally, minimizing ∥∇x log(Θ,y)∥∞ drives particles to zeros of the log-posterior gradient, i.e., modes or flat regions, and imposes no constraint on the shape of q. No argument in §2.7–§2.9 connects minimizers of L to the posterior or to a controlled Gaussian local approximation, so the claim that G-SVGD samples the posterior is unsupported.
- [§2.6, Eqs. (32)–(34)] The ELBO is computed using the closed-form KL divergence between two Gaussians in Eq. (32), but q is a kernel density estimate and p is the posterior, which is not generally Gaussian. Therefore Eq. (34) is not a valid lower bound on the log marginal likelihood; it is an ad hoc plug-in estimator. Consequently, the ω = 1 case does not actually maximize the ELBO, and the reported log(−ELBO) comparisons in Figures 12–14 are not measuring what they claim to measure.
- [§3.2.3, Figures 9–11] The convergence comparison evaluates G-SVGD, A-SVGD, and t-walk using the same loss function L that G-SVGD is explicitly designed to minimize. Lower values of L for G-SVGD are therefore partly by construction and do not demonstrate improved posterior accuracy. The manuscript reports no independent posterior diagnostic (e.g., posterior predictive checks, a reference posterior from long MCMC, or a two-sample test). Additionally, the claim in §4.2 that t-walk needs 10,000 samples while G-SVGD needs only 200 for 'similar accuracy' omits both wall-clock runtime and the number of forward PDE solves, so the computational-cost advantage is not established.
- [Abstract and §1] The abstract and introduction claim 'theoretical guarantees of convergence,' but no convergence theorem is stated or proved. The inequality chain in Eq. (39) of §2.8 assumes the KL reduction result for SVGD and then substitutes it into the proposed loss; it does not prove that the proposed step-size rule converges to a posterior sample. The theoretical claim should either be substantiated with a precise statement and proof or removed.
minor comments (5)
- [§2.4.2, Eq. (5) vs. Eq. (26)] The boundary condition u(L,t) = −c_L ∂x u(0,t) in system (5) appears dimensionally inconsistent and does not match the absorbing boundary condition used in the numerical system (26); please clarify the correct boundary condition and notation.
- [§2.9, Algorithm 2] The notation KDE_{Θ,ω}(θ) is undefined: the KDE bandwidth is not specified and does not depend on ω in the description. Please specify the bandwidth selection rule and whether ω enters the KDE.
- [Throughout] Several typos and formatting issues: 'sufficientelly' (§2.5.1), 'mínimum' (§2.7), 'Gaussain' (§2.9), 'completly' and 'becuase' (§4.2), and the broken cross-reference in Figure 1. These should be corrected.
- [§3.2.3, Fig. 8] The histograms in Figure 8 do not show the true posterior or a reference posterior, so it is difficult to assess which method is more accurate; adding a contour or density overlay of a long-run MCMC reference would help.
- [§4.4, Limitations] The limitations section acknowledges sensitivity to ω and dependence on KDE quality, but these limitations are not connected to the numerical claims; please discuss how the reported speedups would change under different ω and bandwidth choices.
Circularity Check
Convergence comparisons in Figs. 9–11 reuse the exact loss function L that G-SVGD minimizes, so the reported speedup is partly by construction; no independent posterior diagnostic is reported.
-
fitted input called prediction
[Section 2.9, Algorithm 2 line 8, and Section 3.2.3]
"Find the optimal step size: αℓ ← arg min α∈(0,ξ) L(Φ, qα). ... they can be compared with the results provided by the t-walk approximation and A-SVGD by evaluating the final numerical results using the same loss function."
G-SVGD's only adaptive ingredient is the step size, chosen at every iteration as the argmin of L. The convergence comparison is then made by evaluating all methods with that same L. Minimizing L better than methods that do not minimize L is therefore expected by construction. The paper presents this lower L as evidence of accelerated convergence and improved posterior approximation, but no independent posterior fidelity measure (e.g., Wasserstein distance, coverage, or distance to a reference posterior) is reported. The comparison in Figures 9–11 thus measures the algorithm against its own training objective.
-
self definitional
[Section 4.2, Contributions and Problems Solved]
"This suggests that the loss function serves as an effective measure for approximating the posterior distribution, as minimizing it leads to convergence of the initial sample towards a sample from the posterior distribution."
The conclusion that minimizing L yields a sample from the posterior is the load-bearing premise, and it is justified by observing that G-SVGD attains lower L values than the t-walk and A-SVGD. But Algorithm 2 defines its output only as 'a set of particles that minimizes the loss function L.' Without an external diagnostic connecting low L to posterior fidelity, the statement equates the algorithm's own objective with posterior convergence by definition. The histogram comparison with t-walk is qualitative and does not quantify posterior error, so the central acceleration and accuracy claim rests on the same loss that the method is built to minimize.
full rationale
The central claim 'G-SVGD accelerates convergence and offers improved performance' is supported primarily by Figures 9–11, where all methods are scored with the loss L(Θ,q) = (1−ω)‖∇logπ(Θ|y)‖∞ − ω ELBO(q). Since Algorithm 2 chooses every step size by minimizing exactly this L, lower L for G-SVGD is partly a tautology. The paper never establishes that L measures posterior fidelity: Eq. (38) is derived from an affine-in-ω function and is not a valid interior minimum condition, and Eq. (32) inserts the Gaussian KL formula while q is a KDE test distribution, so the 'ELBO' term is not a valid lower bound. Those issues are correctness concerns, but the circular step is the self-evaluation: the method's fitted step size and its reported convergence metric are the same objective. Some independent content exists: the five-point operator analysis, the wave prospection forward models, and the t-walk comparison using 200 vs 10,000 samples provide non-circular evidence for computational savings in terms of sample count, though runtime and forward-solve counts are omitted. The self-citations (Capistran et al. 2012, 2020; Rojo García 2020) are used for modeling and initialization choices and are not load-bearing for the main posterior-approximation claim, so they do not raise the score further. Overall, the central quantitative performance claim is partially circular, meriting a score of 6 rather than a higher score because the algorithm does have independent algorithmic content and a qualitative comparison to MCMC.
Assumptions & free parameters
free parameters (5)
- omega (loss weight) =
0, 0.5, 1 (chosen by user)
- RBF kernel bandwidth h =
not specified
- KDE bandwidth =
not specified
- maximum step size xi =
not specified
- stopping tolerance epsilon =
not specified
assumptions (4)
- domain assumption The target posterior density p is smooth and the RBF kernel is in the Stein class, as in Liu and Wang (2016).
- ad hoc to paper The closed-form KL divergence in Eq. (32) applies to q and p.
- ad hoc to paper Minimizing L with respect to omega yields a condition for posterior approximation.
- domain assumption The five-point finite difference operator provides an accurate approximation of the wave equation solution used in the likelihood.
Cite this review
Pith. "Pith review of Greedy Stein Variational Gradient Descent: An algorithmic approach for wave prospection problems." pith.science (2026). https://pith.science/paper/C6NVHGQ2
@misc{pith2026250119370,
author = {Pith},
title = {Pith review of: Greedy Stein Variational Gradient Descent: An algorithmic approach for wave prospection problems},
year = {2026},
howpublished = {\url{https://pith.science/paper/C6NVHGQ2}},
note = {Machine review of arXiv:2501.19370}
}
read the original abstract
In this project, we propose a Variational Inference algorithm to approximate posterior distributions. Building on prior methods, we develop the Gradient-Steered Stein Variational Gradient Descent (G-SVGD) approach. This method introduces a novel loss function that combines a weighted gradient and the Evidence Lower Bound (ELBO) to enhance convergence speed and accuracy. The learning rate is determined through a suboptimal minimization of this loss function within a gradient descent framework. The G-SVGD method is compared against the standard Stein Variational Gradient Descent (SVGD) approach, employing the ADAM optimizer for learning rate adaptation, as well as the Markov Chain Monte Carlo (MCMC) method. We assess performance in two wave prospection models representing low-contrast and high-contrast subsurface scenarios. To achieve robust numerical approximations in the forward model solver, a five-point operator is employed, while the adjoint method improves accuracy in computing gradients of the log posterior. Our findings demonstrate that G-SVGD accelerates convergence and offers improved performance in scenarios where gradient evaluation is computationally expensive. The abstract highlights the algorithm's applicability to wave prospection models and its potential for broader applications in Bayesian inference. Finally, we discuss the benefits and limitations of G-SVGD, emphasizing its contribution to advancing computational efficiency in uncertainty quantification.
Figures
Figures from the paper (14 more)
Reference graph
Works this paper leans on
-
[1]
author Berlinet, A. , author Thomas-Agnan, C. , year 2011 . title Reproducing kernel Hilbert spaces in probability and statistics . publisher Springer Science & Business Media
work page 2011
-
[2]
author Blei, D.M. , author Kucukelbir, A. , author McAuliffe, J.D. , year 2017 . title Variational inference: A review for statisticians . journal Journal of the American Statistical Association volume 112 , pages 859--877
work page 2017
-
[3]
author Brennan, M.C. , author Bigoni, D. , author Zahm, O. , author Spantini, A. , author Marzouk, Y. , year 2020 . title Greedy inference with structure-exploiting lazy maps , in: booktitle Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020) , address Vancouver, Canada
work page 2020
-
[4]
author Buckman, K. , author Sanchez-Vizuet, T. , author Stapf, M. , author Qiu, T. , year 2012 . title One sided wave equations & absorbing boundary conditions note Maroon Team
work page 2012
-
[5]
author Capistran, M. , author Moreles, M.A. , author Pena, J. , year 2012 . title On full seismic waveform inversion by descent methods in a lattice . journal Appl. Math volume 6 , pages 193--203
work page 2012
-
[6]
author Capistran, M.A. , author Chi Vinh, P. , author Thanh Tuan, T. , year 2020 . title Reliability assessment and data inversion using a surrogate model of wave propagation in functionally graded materials . journal Inverse Problems in Science and Engineering volume 28 , pages 614--636
work page 2020
-
[7]
author Christen, J.A. , author Fox, C. , year 2010 . title A general purpose sampling algorithm for continuous distributions (the t-walk)
work page 2010
- [8]
Show all 32 references
-
[9]
, author Neklydov, K
author Egorov, E. , author Neklydov, K. , author Kostoev, R. , author Burnaev, E. , year 2019 . title Maxentropy pursuit variational inference . journal arXiv preprint arXiv:1905.07855
2019 arXiv
-
[10]
, author Wolfe, P
author Frank, M. , author Wolfe, P. , year 1956 . title An algorithm for quadratic programming . journal Naval Research Logistics Quarterly volume 3 , pages 95--110
1956
-
[11]
, author Cui, Z
author Futami, F. , author Cui, Z. , author Sato, I. , author Sugiyama, M. , year 2019 . title Bayesian posterior approximation via greedy particle optimization , in: booktitle Proceedings of the AAAI Conference on Artificial Intelligence , pp. pages 3606--3613
2019
-
[12]
, author Wang, X
author Guo, F. , author Wang, X. , author Fan, K. , author Broderick, T. , author Dunson, D. , year 2016 . title Boosting variational inference , in: booktitle Advances in Neural Information Processing Systems (NIPS) , address Barcelona, Spain
2016
-
[13]
, author Resnick, R
author Halliday, D. , author Resnick, R. , author Walker, J. , year 2013 . title Fundamentals of physics . publisher John Wiley & Sons
2013
-
[14]
, year 2017
author Igel, H. , year 2017 . title Computational seismology: a practical introduction . publisher Oxford University Press
2017
-
[15]
, author Ba, J
author Kingma, D.P. , author Ba, J. , year 2014 . title Adam: A method for stochastic optimization . journal arXiv preprint arXiv:1412.6980
2014 arXiv
-
[16]
, year 2016
author Langtangen, H.P. , year 2016 . title A primer on scientific programming with Python . publisher Springer
2016
-
[17]
, author Linge, S
author Langtangen, H.P. , author Linge, S. , year 2016 . title Finite difference methods for wave equations . journal Journal of Numerical Methods
2016
-
[18]
, author Linge, S
author Langtangen, H.P. , author Linge, S. , year 2017 . title Finite difference computing with PDEs: a modern software approach . publisher Springer Nature
2017
-
[19]
, author Lee, J
author Liu, Q. , author Lee, J. , author Jordan, M. , year 2016 . title A kernelized stein discrepancy for goodness-of-fit tests , in: booktitle International Conference on Machine Learning , organization PMLR . pp. pages 276--284
2016
-
[20]
, author Wang, D
author Liu, Q. , author Wang, D. , year 2016 . title Stein variational gradient descent: A general purpose bayesian inference algorithm . journal Advances in Neural Information Processing Systems volume 29
2016
-
[21]
, author Khanna, R
author Locatello, F. , author Khanna, R. , author Ghosh, J. , author R \"a tsch, G. , year 2018 . title Boosting variational inference: An optimization perspective , in: booktitle Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTA...
2018
-
[22]
, author Morken, K
author Lyche, T. , author Morken, K. , year 2008 . title Spline methods draft . journal Department of Informatics, Center of Mathematics for Applications, University of Oslo, Oslo , pages 3--8
2008
-
[23]
, year 1998
author Moczo, P. , year 1998 . title Introduction to Modeling Seismic Wave Propagation by the Finite-Difference Methods . publisher Disaster Prevention Research Institute, Kyoto University
1998
-
[24]
, year 2017
author Owen, A.B. , year 2017 . title A randomized halton algorithm in r . journal arXiv preprint arXiv:1706.02808
2017 arXiv
-
[25]
, author Sacco, R
author Quarteroni, A. , author Sacco, R. , author Saleri, F. , year 2006 . title Numerical mathematics . volume volume 37 . publisher Springer Science & Business Media
2006
-
[26]
, author Valli, A
author Quarteroni, A. , author Valli, A. , year 2008 . title Numerical approximation of partial differential equations . volume volume 23 . publisher Springer Science & Business Media
2008
-
[27]
, year 2020
author Rojo García, R. , year 2020 . title Diseño óptimo en problemas inversos lineales
2020
-
[28]
, author Ley, C
author R nning, O. , author Ley, C. , author Al-Sibahi, A.S. , author Hamelryck, T. , year 2023 . title Elbo-ing stein mixtures . journal arXiv preprint arXiv:2306.09917
2023 arXiv
-
[29]
, year 2016
author Ruder, S. , year 2016 . title An overview of gradient descent optimization algorithms . journal arXiv preprint arXiv:1609.04747
2016 arXiv
-
[30]
, year 2019
author Shearer, P.M. , year 2019 . title Introduction to seismology . publisher Cambridge University Press
2019
-
[31]
, author Wysession, M
author Stein, S. , author Wysession, M. , year 2009 . title An introduction to seismology, earthquakes, and earth structure . publisher John Wiley & Sons
2009
-
[32]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.