Semi-Implicit Variational Inference via Kernelized Path Gradient Descent

Bernd Bischl; David R\"ugamer; Tobias Pielok

REVIEW 4 major objections 5 minor 16 references

Semi-Implicit Variational Inference via Kernelized Path Gradient Descent

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes a kernelized path-gradient estimator for semi-implicit variational inference, proves it has lower variance than Stein-based score-gradient estimators, and shows the importance-sampling variant matches state-of-the-art…

desk verdict The variance-reduction proof misses the covariance with the shared target-score term, so the central theoretical claim as stated is not established; the method and importance-sampling twist are still worth a serious look. read the letter →

arxiv 2506.05088 v1 pith:AGJWS35V submitted 2025-06-05 cs.LG stat.CO

Tobias Pielok , Bernd Bischl , David Rügamer This is my paper

classification cs.LGstat.CO

keywords semi-implicitvariationalinferencekernelizedpathgradientimportancesamplingSteindescentvariancereductionBayesiankerneldiscrepancy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Semi-implicit variational inference (SIVI) approximates a posterior with a mixture distribution: draw a latent variable, push it through a network, and use the output to parameterize a simple conditional distribution. This paper proposes replacing the Stein-identity estimate of the score-gradient difference with the closed-form conditional score of the semi-implicit layer, giving a Kernelized Path Gradient (KPG) that provably has lower Monte Carlo variance for Gaussian kernels and Gaussian conditionals. An importance-sampling variant, KPG-IS, learns a mixture proposal for the latent variable, reducing bias while keeping the support condition needed for unbiased gradient estimation. The authors show that KPG and amortized Stein variational gradient descent minimize the same objective, while KPG exploits the semi-implicit structure for stability. On high-dimensional diffusion, Bayesian logistic regression, and toy benchmarks, KPG-IS matches or improves on state-of-the-art SIVI methods with lower training cost.

What carries the argument

The machinery is the Kernelized Path Gradient (KPG): the Monte Carlo plug-in of $\Delta_{\mathrm{SI},k}$ into the reparameterized pathwise gradient, replacing the Stein-identity term $-\nabla_{z'}k(z,z')$ with the conditional score $k(z,z')\nabla_{z'}\log q_{z|\epsilon}(z'|\epsilon')$. This substitution does the main work: the semi-implicit construction makes the conditional score available in closed form, so the estimator avoids the boundary assumptions of Stein's identity and needs only a continuous kernel. The importance-sampling extension replaces the $\epsilon$ draws in the outer expectation with draws from the learned mixture proposal $\tau_{\epsilon|z}$, whose support contains that of $p_\epsilon$ by construction, keeping the score-gradient estimator consistent while reducing bias in function space.

What would settle it

Run KPG and the Stein estimator on a semi-implicit Gaussian target with known posterior, using kernel widths around the median-heuristic value; if the empirical trace of $V[\hat{s}_{\mathrm{STEIN},k}(z)]-V[\hat{s}_{\mathrm{SI},k}(z)]$ is negative in a regime that satisfies Proposition 3.2's sufficient condition, the lower-variance claim is falsified. A direct estimate of the correlation between $k(z,z')^2$ and $\beta$ on the same simulations would also settle the upper-bound argument.

Watch

Extended reading notes

Core claim

The central object is the kernelized score-gradient difference $\Delta_{\mathrm{SI},k}(z)=\mathbb{E}_{z',\epsilon'\sim q_{z,\epsilon}}[k(z,z')(\nabla_{z'}\log q_{z|\epsilon}(z'|\epsilon')-\nabla_{z'}\log p_z(z'))]$, which equals the RKHS-restricted score difference $\Delta_k(z)$ through the kernel trick for semi-implicit distributions. Feeding a Monte Carlo estimate of $\Delta_{\mathrm{SI},k}$ into the pathwise gradient $\mathbb{E}_{\epsilon,\eta}[\Delta_{\mathrm{SI},k}(h_\phi(\epsilon,\eta))\cdot\nabla_\phi h_\phi(\epsilon,\eta)]$ defines KPG. The paper proves that for a Gaussian kernel and Gaussian conditional, the trace of the covariance difference between the Stein and KPG score-gradient estimators equals $\frac{1}{n}\mathbb{E}_{\epsilon',\eta'}[k(z,\mathrm{diag}(\sigma_{\epsilon'})\eta'+\mu_{\epsilon'})^2(\|\mathrm{diag}(\sigma_{\epsilon'})\eta'+\mu_{\epsilon'}-z\|_2^2/\sigma_k^4-\|\mathrm{diag}(\sigma_{\epsilon'})^{-1}\eta'\|_2^2)]$, and gives sufficient conditions under which this quantity is nonnegative, establishing lower variance for KPG. The KPG-IS variant replaces draws of $\epsilon$ in the outer expectation with draws from a learned mixture proposal $\tau_{\epsilon|z}=\alpha(z)p_\epsilon+(1-\alpha(z))\tilde{\tau}_{\epsilon|z}$, which preserves the support condition $\operatorname{supp}(\tau_{\epsilon|z})\supset\operatorname{supp}(p_\epsilon)$ and is trained to minimize the expected negative log-likelihood; the paper shows $p_\epsilon$ and $q_{\epsilon|z}$ are the upper and lower bounds of this objective. Empirically, KPG-IS reaches log marginal likelihood 74528 versus KSIVI's 74504 on the 100-dimensional diffusion benchmark and outperforms all compared kernel-based SIVI variants on banana, x-shaped, and multimodal benchmarks, while amortized SVGD diverges on Bayesian logistic regression under the same hyperparameters.

Load-bearing premise

The lower-variance guarantee depends on assuming a negative correlation between the squared kernel weight and a distance-based expression in Proposition 3.3, plus a sufficient condition on the kernel width that is argued but not proven and never checked when the width is chosen by the median heuristic.

Editorial extensions

If this is right

Amortized SVGD and KPG minimize the same kernel-smoothed KL objective, so differences in behavior are attributable to gradient variance and bias, not to a different training objective.
KPG does not need the boundary or vanishing-at-infinity condition required for Stein's identity; a continuous kernel suffices.
KPG-IS's mixture proposal keeps the support of the proposal a superset of the support of the latent distribution, so the importance-weighted score-gradient estimator stays consistent throughout optimization.
On the 100-dimensional diffusion benchmark, KPG-IS reaches log marginal likelihood 74528 versus KSIVI's 74504 while converging in fewer iterations, and the KPG ablation shows a per-iteration cost of $2.5\times10^{-3}$ s versus KSIVI's $1.4\times10^{-2}$ s.
Amortized SVGD diverged on the Bayesian logistic regression benchmark under the same hyperparameters used for KPG, which the paper attributes to the higher variance of Stein-based gradient estimates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The structural variance reduction should carry over to non-Gaussian reparameterizable conditionals, since the kernel trick only requires the conditional score; testing KPG with, for example, Student-$t$ conditionals would separate the Gaussian-specific variance formula from the general mechanism.
The proposal objective in Eq. 23 can be read as learning an amortized posterior over the latent $\epsilon$; the same update could serve as a low-variance gradient estimator in other hierarchical or doubly stochastic variational settings.
Because the kernel width is chosen by the median heuristic rather than by the sufficient condition of Proposition 3.2, an online or adaptive scheme for $\sigma_k$ would be a natural extension to make the variance guarantee operational.
The reported divergence of amortized SVGD on logistic regression suggests gradient variance, not model capacity, is the bottleneck in that benchmark; a controlled experiment varying only the score-gradient estimator on the same SIVI model would isolate that effect.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know about this paper: the advertised lower-gradient-variance result does not follow from the proof as written. Propositions 3.1–3.3 only compare the q-score sub-estimators s_STEIN and s_SI. But the gradient estimator actually used is Δ = ŝ – t̂, where t̂ is the target-score term estimated on the same samples. The variance of Δ depends on the covariance between ŝ and t̂, and that covariance term is never bounded. Even if every assumption in Prop. 3.2 and 3.3 holds, the claim that “our semi-implicit approach achieves lower gradient variance” is unsupported. That is a real gap, not a nitpick.

What is genuinely new: the pathwise use of the conditional score instead of Stein’s identity inside the kernelized estimator, and the importance-sampling proposal with a learned mixture coefficient α. The derivation of the proposal loss and the bias-variance tradeoff discussion are thoughtful. The variance computation for the Gaussian case is a useful reference, even if it only covers the sub-estimator. Empirically, the diffusion benchmark shows KPG-IS matching KSIVI, and the lower-cost KPG variant is much faster, though with somewhat lower log marginal likelihood. The extra benchmarks in Appendix B are a plus.

Soft spots, in proportion: the negative-correlation assumption in Prop. 3.3 is plausible but not proven; the sufficient condition on σ_k is not checked when using the median heuristic; Table 1 has no uncertainty or repeats for the main metric; code is mentioned but not linked; there are typos (e.g., “log log” in Algorithm 1, a garbled expression in Eq. 21). None of these are fatal, but together with the proof gap they make the theory section the weakest part. The empirical gains over KSIVI are modest, so the practical contribution is incremental, not a breakthrough.

Who this is for: people actively working on SIVI, amortized SVGD, or kernel-based score estimators. The paper is not circular—it evaluates on external benchmarks—and the writing is generally clear, aside from the proof issue.

My call: send it to peer review. The method is plausible, the empirical results are promising enough to warrant scrutiny, and the theory can be fixed either by restricting the claim to the q-score sub-estimator or by actually computing the covariance terms. A referee should push on the variance claim hard, but the paper deserves the effort.

Referee Report

4 major / 5 minor

Summary. The paper introduces Kernelized Path Gradient (KPG), an estimator for semi-implicit variational inference that replaces the Stein-identity score term with the conditional score of the semi-implicit hierarchy, and KPG-IS, an importance-sampling variant with a learnable proposal. The authors claim a theoretical connection to amortized Stein variational gradient descent, assert that the new estimator has provably lower gradient variance, and state that its function-space bias is benign. The paper also reports empirical comparisons on a 100-dimensional conditional diffusion benchmark, Bayesian logistic regression, and several toy targets, claiming parity or improvement over KSIVI with better training efficiency. The theoretical analysis in Propositions 3.1–3.3 compares the variance of the q-score sub-estimators, while the algorithms and experiments additionally include a shared target-score term; the main advertised variance guarantee is therefore not established as written.

Significance. A genuinely lower-variance, low-bias kernelized gradient estimator for semi-implicit variational inference would be a useful contribution: the use of the conditional score avoids boundary assumptions, and the importance-correction idea is interesting. The paper also ships a reproducible-looking comparison against current SIVI baselines. However, the central theoretical claim is load-bearing for the method's motivation, and the proof as written covers only a sub-estimator; the benign-bias claim is asserted rather than derived. These issues make the current version unsuitable for acceptance without substantive revision, but the gap appears addressable either by extending the proof or by restricting the claims.

major comments (4)

[Section 3, Eqs. (13), (18), (20), Algorithms 1–2] The variance-reduction claim compares only the q-score sub-estimators s_STEIN and s_SI, whereas the gradient estimator actually used in Algorithms 1 and 2 is Δ̂ = ŝ − t̂, with t̂ = (1/n)Σ k(z,z_i)∇ log p(z_i) evaluated on the same samples as ŝ. For vector-valued estimators, V(Δ̂) = V(ŝ) + V(t̂) − 2Cov(ŝ,t̂). Propositions 3.1–3.3 bound only V(ŝ_STEIN) − V(ŝ_SI) and never bound the covariance difference −2(Cov(ŝ_STEIN,t̂) − Cov(ŝ_SI,t̂)). Since t̂ is random and shares samples with ŝ, a lower-variance q-score sub-estimator does not imply a lower-variance gradient estimator. The advertised claim in the abstract, 'our semi-implicit approach achieves lower gradient variance', is therefore not supported by the presented proof; the proof must either compute or bound the covariance terms, or the claim must be restricted to the q-score sub-estimator.
[Propositions 3.2 and 3.3, Eq. (19)] The upper bound in Prop. 3.3 assumes that k(z,z')² and β = ∥diag(σ_ε)η + μ_ε − z∥²/σ_k⁴ − ∥diag(σ_ε)⁻¹η∥² are negatively correlated. The text argues this is plausible because the two quantities are monotone in opposite directions in ∥z−z'∥, but β also depends on σ_ε and η, and no calculation of the actual correlation under q_{z,ε} is given. Moreover, the sufficient condition of Prop. 3.2 is not checked in the experiments: Section 3.2 sets σ_k by the median heuristic, so even the weaker claim of lower variance for ŝ_SI versus ŝ_STEIN is conditional on an unverified scale condition. A corrected presentation needs either a verified sufficient condition, an empirical check of the condition, or a statement that the variance bound is heuristic.
[Abstract, Section 3, Section 6] The paper states that the method's bias in function space is benign, but no formal definition, theorem, or proof of this claim appears anywhere in the manuscript. The discussion around Eq. (20) only argues informally that small σ_k reduces RKHS bias while increasing variance; it does not quantify the bias of Δ_SI,k, state conditions under which the bias vanishes as σ_k → 0, or compare it with the bias of the Stein-based estimator. Since 'benign bias' is presented as a contribution, this gap should be closed by a precise statement or the claim should be removed.
[Appendix E and Section 5.2] The efficient variant described in Appendix E replaces the samples ε_{i,j} ∼ τ_{ε|z} used in Algorithm 2 by samples ε_j ∼ p_ε and sets α close to one. As written, this changes the proposal distribution of the importance-weighting identity in Eq. (21) and is not shown to preserve unbiasedness; drawing from p_ε while using weights p_ε(ε)/τ_{ε|z}(ε|z) does not estimate the original expectation. Because the Bayesian logistic regression experiments in Section 5.2 explicitly rely on this variant, the reported empirical results there are not clearly aligned with the exact KPG-IS algorithm, and the efficiency claim for those experiments is not supported without additional justification.

minor comments (5)

[Algorithm 1] The expression 'log logpz' in the definition of eΔ appears to be a typo for 'log pz'.
[Eq. (21)] The notation in the score term uses ∇_z log q_{z|ε}(z'|ε), but the differentiation variable should be z'; please make the distinction between z and z' explicit.
[Table 1] The main diffusion benchmark reports log marginal likelihood and timings without standard errors, even though the text mentions replications in Appendix D.2; adding error bars or a table of repeated-run statistics would make the claims more robust.
[Section 5.1] The sentence 'one obtained a 100-dimensional latent variable' should read 'one obtains'.
[Section 3.1, Eq. (23)] The definition of optimal proposal via minimizing expected negative log-likelihood is sensible, but the text should clarify that this optimality notion is with respect to the proposal-fitting objective only and does not directly imply optimality for the kernelized gradient estimator's variance or bias.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the KPG estimator is derived from an explicit score-based objective and is evaluated against external benchmarks, with no fitted value relabeled as a prediction.

full rationale

The paper's derivation chain is self-contained rather than circular. The KPG estimator is constructed from an explicit kernelized difference of score gradients, using the kernel trick for semi-implicit distributions introduced in the cited prior work [1], and is then substituted into the pathwise gradient estimator. No parameter is fitted to a subset of the data and then reported as a prediction of that same subset; the median heuristic for the kernel width is a standard tuning choice, not a relabeled output. The variance comparison in Propositions 3.1-3.3 concerns the Monte Carlo estimators of the q-score sub-estimators, and while the proof omits the covariance between those sub-estimators and the shared target-score term, that is a derivational gap or an unverified technical assumption, not a circular reduction. The paper contains no load-bearing self-citations: the references to prior work on KSIVI, amortized SVGD, and SIVI are all to external authors, and no uniqueness theorem or ansatz is imported from the present authors' own prior work. The empirical evaluation uses external benchmarks with ground truth from SGLD or known data-generating processes, so the central claims do not reduce to the method's own definitions. The reader's noted concern about the negative-correlation assumption and the missing covariance term may undermine the strength of the variance-reduction proof, but it does not make the derivation circular. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central variance-reduction claim rests on domain-restricted and partly unproven assumptions: Gaussian kernel and conditional, negative correlation, and benign RKHS bias. The practical method also assumes the alternating optimization converges. The free parameters are the kernel width and the mixture floor alpha, both hand-chosen rather than derived.

free parameters (2)

Kernel width sigma_k = Median heuristic, set per iteration
Controls the RKHS and enters the variance bound; the paper does not show that the median heuristic satisfies the sufficient condition in Proposition 3.2.
Mixture coefficient floor alpha in KPG-IS = 0.99 in the Bayesian logistic regression; unspecified elsewhere
Guarantees the support of the proposal and sets the bias-variance tradeoff; it is a hand-set hyperparameter and not derived from first principles.

assumptions (4)

domain assumption Gaussian kernel and conditional Gaussian likelihood for Propositions 3.1 to 3.3
The variance reduction proofs are restricted to this setting; the method uses a Gaussian kernel and Gaussian conditional in experiments, so this restricts the analysis rather than the general method.
ad hoc to paper Negative correlation between k(z,z')^2 and beta in Proposition 3.3
Needed to upper-bound the variance difference; only a plausibility argument based on monotonicity in ||z-z'|| is given, not a proof.
ad hoc to paper The RKHS bias is benign
Stated in the abstract and Section 3; no formal bound or empirical measure of the bias introduced by restricting the score difference to an RKHS is provided.
ad hoc to paper Alternating optimization of the proposal parameters theta and the variational parameters phi converges
KPG-IS relies on alternating updates between the proposal and the variational model; no convergence analysis is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Semi-Implicit Variational Inference via Kernelized Path Gradient Descent." pith.science (2026). https://pith.science/paper/AGJWS35V

@misc{pith2026250605088,
  author       = {Pith},
  title        = {Pith review of: Semi-Implicit Variational Inference via Kernelized Path Gradient Descent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGJWS35V}},
  note         = {Machine review of arXiv:2506.05088}
}

read the original abstract

Semi-implicit variational inference (SIVI) is a powerful framework for approximating complex posterior distributions, but training with the Kullback-Leibler (KL) divergence can be challenging due to high variance and bias in high-dimensional settings. While current state-of-the-art semi-implicit variational inference methods, particularly Kernel Semi-Implicit Variational Inference (KSIVI), have been shown to work in high dimensions, training remains moderately expensive. In this work, we propose a kernelized KL divergence estimator that stabilizes training through nonparametric smoothing. To further reduce the bias, we introduce an importance sampling correction. We provide a theoretical connection to the amortized version of the Stein variational gradient descent, which estimates the score gradient via Stein's identity, showing that both methods minimize the same objective, but our semi-implicit approach achieves lower gradient variance. In addition, our method's bias in function space is benign, leading to more stable and efficient optimization. Empirical results demonstrate that our method outperforms or matches state-of-the-art SIVI methods in both performance and training efficiency.

Figures

Figures reproduced from arXiv: 2506.05088 by the authors.

**Figure 1.** Convergence speed comparison between the current state-of-the-art method KSIVI and our proposal KPG-IS. Our contributions In this work, we propose a novel approach for performing inference with semiimplicit variational distributions by combining kernelized score estimation with pathwise gradients. Our contributions are as follows: (i) We introduce the Kernelized Path Gradient (KPG), a method that leverages the re… view at source ↗

**Figure 2.** Comparison of posterior quality of different models (facets) for the diffusion process. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** A scatter plot of all pairwise correlation coefficients [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Comparison of marginal and pairwise density estimates for [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Density contour plots for the DGP and each method from one representative repetition. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: STEIN: Marginal and pairwise density estimates for [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Convergence speed comparison between the current state-of-the-art method KSIVI and our [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 14 canonical work pages

[1]

Kernel semi-implicit variational inference

Ziheng Cheng, Longlin Yu, Tianyu Xie, Shiyue Zhang, and Cheng Zhang. Kernel semi-implicit variational inference. In Forty-first International Conference on Machine Learning , 2024. URL https://openreview.net/forum?id=w5oUo0LhO1

work page 2024
[2]

A stein variational newton method

Gianluca Detommaso, Tiangang Cui, Alessio Spantini, Youssef Marzouk, and Robert Scheichl. A stein variational newton method. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 9187–9197, Red Hook, NY , USA, 2018. Curran Associates Inc

work page 2018
[3]

Uci machine learning repository, 2017

Dheeru Dua and Casey Graff. Uci machine learning repository, 2017. URL http://archive.ics.uci. edu/ml

work page 2017
[4]

Learning to draw samples with amortized stein variational gradient descent

Yihao Feng, Dilin Wang, and Qiang Liu. Learning to draw samples with amortized stein variational gradient descent. In Gal Elidan, Kristian Kersting, and Alexander Ihler, editors, Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI 2017, Sydney, Australia, August 11-15, 2017. AUAI Press, 2017. URL http://auai.org/uai20...

work page 2017
[5]

Kingma and Max Welling

Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http://arxiv.org/abs/1312.6114

arXiv 2014
[6]

Yingzhen Li and Richard E. Turner. Gradient estimators for implicit models. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=SJi9WOeRb

work page 2018
[7]

Particle semi-implicit variational inference

Jen Ning Lim and Adam Michael Johansen. Particle semi-implicit variational inference. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview. net/forum?id=p3gMGkHMkM

work page 2024
[8]

Stein variational gradient descent: A general purpose bayesian inference algorithm

Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Gar- nett, editors, Advances in Neural Information Processing Systems , volume 29. Curran Asso- ciates, Inc., 2016. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/ b3ba8f1bee1238...

work page 2016

Show all 16 references

[9]

Pytorch: An imperative style, high-performance deep learning library

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

2019
[10]

Variational inference with normalizing flows

Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 1530–1538, Lille, France...

2015
[11]

Sticking the landing: Simple, lower-variance gradient estimators for variational inference

Geoffrey Roeder, Yuhuai Wu, and David K Duvenaud. Sticking the landing: Simple, lower-variance gradient estimators for variational inference. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Informatio...

2017
[12]

Importance weighted hierarchical variational infer- ence

Artem Sobolev and Dmitry P Vetrov. Importance weighted hierarchical variational infer- ence. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d 'Alché-Buc, E. Fox, and R. Gar- nett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Asso- ciates, In...

2019
[13]

Titsias and Francisco Ruiz

Michalis K. Titsias and Francisco Ruiz. Unbiased implicit variational inference. In Kamalika Chaudhuri and Masashi Sugiyama, editors, Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learni...

2019
[14]

Bayesian learning via stochastic gradient langevin dynamics

Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In International Conference on Machine Learning, 2011. URL https://api.semanticscholar.org/ CorpusID:2178983. 10

2011
[15]

Semi-implicit variational inference

Mingzhang Yin and Mingyuan Zhou. Semi-implicit variational inference. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5660–5669. PMLR, 10–15 Jul 2018....

2018
[16]

Semi-implicit variational inference via score matching

Longlin Yu and Cheng Zhang. Semi-implicit variational inference via score matching. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum? id=sd90a2ytrt. 11 A Proofs A.1 Variance comparison of score gradient estimators Fir...

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Kernel semi-implicit variational inference

Ziheng Cheng, Longlin Yu, Tianyu Xie, Shiyue Zhang, and Cheng Zhang. Kernel semi-implicit variational inference. In Forty-first International Conference on Machine Learning , 2024. URL https://openreview.net/forum?id=w5oUo0LhO1

work page 2024

[2] [2]

A stein variational newton method

Gianluca Detommaso, Tiangang Cui, Alessio Spantini, Youssef Marzouk, and Robert Scheichl. A stein variational newton method. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 9187–9197, Red Hook, NY , USA, 2018. Curran Associates Inc

work page 2018

[3] [3]

Uci machine learning repository, 2017

Dheeru Dua and Casey Graff. Uci machine learning repository, 2017. URL http://archive.ics.uci. edu/ml

work page 2017

[4] [4]

Learning to draw samples with amortized stein variational gradient descent

Yihao Feng, Dilin Wang, and Qiang Liu. Learning to draw samples with amortized stein variational gradient descent. In Gal Elidan, Kristian Kersting, and Alexander Ihler, editors, Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI 2017, Sydney, Australia, August 11-15, 2017. AUAI Press, 2017. URL http://auai.org/uai20...

work page 2017

[5] [5]

Kingma and Max Welling

Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http://arxiv.org/abs/1312.6114

arXiv 2014

[6] [6]

Yingzhen Li and Richard E. Turner. Gradient estimators for implicit models. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=SJi9WOeRb

work page 2018

[7] [7]

Particle semi-implicit variational inference

Jen Ning Lim and Adam Michael Johansen. Particle semi-implicit variational inference. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview. net/forum?id=p3gMGkHMkM

work page 2024

[8] [8]

Stein variational gradient descent: A general purpose bayesian inference algorithm

Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Gar- nett, editors, Advances in Neural Information Processing Systems , volume 29. Curran Asso- ciates, Inc., 2016. URL https://proceedings.neurips.cc/paper_files/paper/2016/file/ b3ba8f1bee1238...

work page 2016