Pith. sign in

REVIEW 4 major objections 6 minor 47 references

Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Refining a variational posterior with MCMC sampler steps yields a tighter lower bound and learns the sampler's step size by backpropagation.

desk verdict Empirical winner, theoretical miscalibration: the sampler-in-the-guide idea works, but the claimed tighter ELBO is not a valid bound because the log-Jacobian term is dropped. read the letter →

arxiv 1908.09744 v4 pith:3VV5AOMW submitted 2019-08-26 cs.LG stat.ML

Victor Gallego , David Rios Insua This is my paper · ORCID
classification cs.LGstat.ML
keywords variationalinferenceprobabilisticprogrammingMarkovchainMonteCarlostochasticgradientLangevindynamicsevidencelowerboundimplicitdistributionsautomaticdifferentiationBayesianclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes building variational posteriors by starting from a standard tractable guide and pushing sampled latent values through several iterations of an MCMC-type sampler such as stochastic gradient Langevin dynamics. Its claim is that optimizing the resulting refined evidence lower bound gives a posterior approximation closer to the true posterior than the original guide alone, while automatic differentiation through the sampler steps automatically tunes the sampler's step size. This matters because it turns two separate tasks—designing a flexible variational family and hand-tuning an MCMC sampler—into one end-to-end optimization that can be implemented in any probabilistic programming system supporting algorithmic differentiation. The paper also supplies several approximations for the entropy of the implicit refined density and demonstrates the approach on funnel densities, state-space models, variational autoencoders, and a Bayesian classifier.

What carries the argument

The load-bearing object is the refined variational approximation, the compound distribution $q_{\varphi,\eta}(z|x)=\int Q_{\eta,T}(z|z_0)\,q_{0,\varphi}(z_0|x)\,dz_0$, in which a tractable guide is evolved by a sampler transition. The load-bearing simplification is the particle approximation of that transition: approximating $Q$ by a sum of Dirac deltas sets the transition entropy to zero, reducing the refined ELBO to an expectation under the original guide of the log joint evaluated at the perturbed latent point. The paper also introduces MC, Gaussian, Fokker-Planck, and change-of-variable (VIS-D) approximations that track some portion of the lost entropy, and two automatic-differentiation modes, Full AD and Fast AD, that respectively do or do not differentiate through the sampler perturbation. This machinery converts an intractable implicit density into a computable objective that remains differentiable in both the guide and sampler parameters.

What would settle it

Compute the exact pushed-forward density for a one-dimensional Gaussian target after one SGD step, including the log-Jacobian term, and compare its ELBO with the particle-approximation ELBO over a range of step sizes; the tighter-bound claim collapses at the first step size where the two bounds disagree by a non-negligible amount.

Watch

Extended reading notes

Core claim

The central discovery is that a compound distribution $q_{\varphi,\eta}(z|x)=\int Q_{\eta,T}(z|z_0)\,q_{0,\varphi}(z_0|x)\,dz_0$, formed by evolving an initial variational guide through $T$ steps of an MCMC-like transition, can be used as the variational family even though the refined density is implicit. Using a Dirac-delta particle approximation for the transition, the entropy of the flow is dropped and the objective becomes an expectation under the original guide of the log joint evaluated at the perturbed latent point, with the perturbation being one steepest-ascent sampler step. The paper argues that for moderate step sizes this refined bound is tighter than the standard ELBO of the original guide, and that the framework permits tuning both the guide parameters and the sampler parameters by backpropagation. The underlying SG-MCMC sampler remains valid in the limit $T\to\infty$, so the learned initial distribution and step size can be reused at test time.

Load-bearing premise

The particle and Gaussian approximations quietly ignore the randomness that the sampler itself contributes to the refined posterior, and the refined lower bound stands only if that discarded contribution is negligible.

Editorial extensions

If this is right

  • Optimizing the refined ELBO with $T>0$ gives tighter lower bounds and posterior approximations with better coverage than the original guide, as demonstrated on funnel, HMM, DLM, and VAE experiments.
  • The sampler's step size can be learned by gradient descent on the refined ELBO, removing the need to hand-tune that hyperparameter.
  • Differentiating through $T$ sampler steps costs $O(mT)$ in the number of sampler hyperparameters, keeping the approach practical when those hyperparameters are few.
  • Full AD and Fast AD offer a trade-off: differentiating through the sampler steps tunes the sampler, while stopping gradients is cheaper and still improves on plain variational inference.
  • At test time, a learned initial distribution combined with a few extra sampler steps can produce large accuracy gains, such as 99.8% accuracy on MNIST classification with a Bayesian cVAE classifier.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the tighter-bound claim holds generally, the refined ELBO could be used as a convergence diagnostic for the sampler: one could measure how much the bound improves per sampler step and use that to choose an adaptive number of refinement steps.
  • The dropped Jacobian term suggests a cheap correction for deterministic flows: estimating $\log\left|\det\left(I+\eta\nabla^2\log p\right)\right|$ along sampled trajectories would restore lower-bound validity without full Jacobian storage, bridging the particle and change-of-variable approximations.
  • The compound-distribution construction could likely be extended to discrete latent variables by replacing the diffusion with a differentiable relaxation, although the entropy approximation would then need a different treatment.
  • The large improvement from $T=0$ to $T=5$ in the VAE experiments hints that most of the modelling power comes from just a few refinement steps, so a training schedule that anneals $T$ during optimization could give both fast early progress and precise final approximations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces a variational inference framework (VIS) that refines a tractable guide q0,φ(z|x) by applying T steps of a sampler Q_{η,T}(z|z0), such as SGLD or SGD, and then optimizes an ELBO-like objective with respect to both the guide parameters φ and the sampler parameters η. Several approximations to the entropy term are proposed: a particle/Dirac approximation (VIS-P), a pathwise Monte Carlo approximation (VIS-MC), a Gaussian approximation (VIS-G), an exact change-of-variable formulation for deterministic flows (VIS-D, not used in experiments), and a Fokker-Planck approximation (VIS-FP). The claimed theoretical contributions are an unbiasedness argument, a "refined ELBO" that is asserted to be tighter than the standard ELBO, and a Taylor-expansion interpretation of the refined gradient. Experiments cover a funnel density, hidden Markov and dynamic linear models, VAEs on MNIST and fashion-MNIST, and a conditional VAE used as a deep Bayes classifier, with code released online.

Significance. If the theoretical claims were sound, this would be a practically valuable framework: it is simple to implement, compatible with probabilistic programming, and it demonstrates how to tune sampler hyperparameters by automatic differentiation. The paper deserves credit for releasing runnable code and for evaluating the method on standard benchmarks; the reported gains, e.g., MNIST test log-likelihood of -82.74 for VIS-5-10 versus -100.91 for a plain VAE, are substantial and worth explaining. However, the central mathematical assertion—that the proposed "refined ELBO" is a valid, tighter lower bound—is not substantiated and is false as stated for the Dirac/SGD and Gaussian approximations. Because the entire title and Section 4 rest on this claim, the theoretical contribution needs major reworking, even though the empirical approach may remain useful as a heuristic.

major comments (4)
  1. [Section 4.2, Eq. (4)] The claim that the Dirac-delta/SGD refined objective is a tighter ELBO than the original is unproven and, as stated, false. For a deterministic flow F(z)=z+η∇log p(x,z), the density transform is q_{φ,η}(z)=q0(F^{-1}(z))|det J_{F^{-1}}(z)|, so the entropy term is not equal to E_{q0}[log q0]. The paper drops log|det J_F| when it sets E_{q_{φ,η}}[log q_{φ,η}] = E_{q0}[log q0] in VIS-P and VIS-G. Concretely, take d=1, log p(x,z) = -z^2/2 + const, and q0(z)=N(0,1). Then F(z)=(1-η)z, the original ELBO equals log p(x) because q0 is the prior, the paper's objective equals log p(x)+η(2-η)/2, which for η=0.5 is log p(x)+0.375, exceeding log p(x); the true refined ELBO including log(1-η) is log p(x)+η(2-η)/2+log(1-η), which is below log p(x). Thus Eq. (4) is not a valid lower bound for VIS-P/VIS-G, and the "straightforward" tightness proof must either include the Jacobian term or be restricted to a properly formulated pathwise bound.
  2. [Section 3.2, VIS-P and VIS-G] The approximations set the transition entropy to zero and identify E_{q_{φ,η}}[log q_{φ,η}] with E_{q0,φ}[log q0,φ], but no justification is given for why the omitted change-of-variable term is negligible. This is internally inconsistent with the paper's own VIS-D paragraph, which correctly states that deterministic flows require the change-of-variable formula. Since VIS-P and VIS-G are presented as general ELBO strategies, the authors need to specify under which conditions the zero-entropy replacement is valid, or, if it is not valid, explicitly label these objectives as heuristic rather than as ELBOs.
  3. [Section 3.2, VIS-MC] The pathwise MC approximation is written as q_{φ,η}(z_T|x) = ∏_{i=1}^T q_η(z_i|z_{i-1}) q0,φ(z0|x), but the right-hand side is a joint density on z_{0:T}, not a density on z_T. If the authors intend to optimize an ELBO on the extended space q(z_{0:T}), then the objective is a valid lower bound—though a looser one than the marginal ELBO—and this should be stated explicitly. If they intend Eq. (4) with the marginal q(z_T), the displayed density is wrong. The VAE experiments rely on VIS-MC, so this ambiguity affects the interpretation of the main empirical results.
  4. [Section 4.2, VIS-FP] The statement that "for the VIS-FP case ... KL(q_{φ,η}(z|x)||p(z|x)) ≤ KL(q0,φ(z|x)||p(z|x))" is not generally true for the discretized deterministic flow in Appendix A, Eq. (6). The Fokker-Planck equivalence holds for the continuous-time SDE, not for the Euler step, and no proof is supplied for the discrete case. Since VIS-FP is not used in the experiments this is less central, but it is part of the theoretical analysis and should be corrected or removed.
minor comments (6)
  1. [Section 5.2, HMM model equation] The HMM joint distribution is written as p(z_{1:τ}, x_{1:τ}, θ) = ∏_{t=1}^{τ} p(x_t|z_t,θ_em) p(x_t|x_{t-1},θ_tr) p(θ); the second factor should presumably be p(z_t|z_{t-1},θ_tr), not another p(x_t|·).
  2. [Figure 1 caption] The caption says the evolution of the negative ELBO is shown at the "bottom", while the text says the trajectories are shown at the "top"; the placement is inconsistent and should be corrected.
  3. [Section 4.3] The expression for the refined gradient, ∇_z log p(x,z) + η ∇_z log p(x,z)^T ∇_z^2 log p(x,z), is notationally ambiguous with respect to row/column vectors; the authors should specify the convention and derive the second-order term consistently.
  4. [Section 4.1] The heading "Unbiasedness" is misleading: no estimator unbiasedness is proved. The paragraph only argues that consistency results of SG-MCMC apply as T→∞, which is an asymptotic statement, not finite-sample unbiasedness.
  5. [Section 3.2, VIS-G] The phrase "By treating the gradient terms as points" is vague; the authors should specify how the covariance σ_φ(x) is used after T deterministic steps and why the resulting distribution is Gaussian with mean z_T.
  6. [Section 5.4 and Appendix B.3.2] The experiments report that sampler parameters are tuned via ∇_η rELBO, but Appendix B.3.2 gives only the initial value η=5e-5; the adaptation schedule and its effect on training should be described.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: VIS is built by composing q0 with a sampler, self-citation [33] is optional, and the reported benchmarks are held-out; the omitted log-Jacobian term is a correctness gap, not circularity.

full rationale

I find no circular step in the paper's derivation chain. The refined variational approximation is defined directly as q_phi,eta(z|x) = integral Q_eta,T(z|z0) q_0,phi(z0|x) dz0, and the subsequent ELBO approximations are proposals for optimizing that object, not fitted parameters renamed as predictions. The main theoretical claim in Section 4.2 is a tightness assertion about the SGD-refined objective, but it is asserted rather than derived; the paper does not reduce it to an input by construction. The only self-citation, reference [33] by the same authors, is used to offer an optional repulsive-force sampler and is not load-bearing for the central approximation or experiments. The empirical evaluations use held-out metrics such as test log-likelihood and classification accuracy, so the experimental claims are not forced by construction. I also explicitly note a validity concern that is not circular: Section 3.2 (VIS-P and VIS-G) and Section 4.2 drop the change-of-variable/log-det-Jacobian term for the deterministic SGD flow, while the VIS-D paragraph itself states that deterministic flows require that term. This is an internal inconsistency and a possible correctness gap, but it is not an instance of a result being equivalent to its inputs by definition or by self-citation. Under the stated hard rules, such a correctness concern does not raise the circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method depends on standard mathematical tools (Euler-Maruyama, Fokker-Planck, reverse KL) plus an unsupported assumption that the zero-entropy particle approximation preserves the lower-bound property. The not-yet-justified assumption is the main source of soundness risk.

free parameters (3)
  • sampler step size eta = learned via gradient descent in Full AD; set to 0.001 for VAE, 5e-5 for cVAE
    Central method tunes this hyperparameter to speed up mixing; in several experiments it is set by hand.
  • number of refinement steps T = 1, 2, 5, or 10 in experiments
    Controls how many sampler iterations are backpropagated through; chosen by hand per experiment.
  • RBF kernel bandwidth gamma in VIS-FP = not specified in experiments
    Appears in the Fokker-Planck and SVGD approximation in Appendix A; no value or sensitivity study is given.
assumptions (4)
  • standard math Euler-Maruyama discretization of Langevin dynamics is a valid approximation for small step size.
    Used in Section 3.1 to derive the SGLD sampler inside the variational guide.
  • standard math The deterministic flow in Proposition 1 has the same Fokker-Planck equation as SGLD.
    Proven in Appendix A; it is a standard equivalence.
  • ad hoc to paper The Dirac-delta approximation of the transition Q with zero entropy still yields a valid lower bound for the ELBO.
    Introduced in Section 3.2 (particle approximation) and used in Section 4.2 to claim a tighter bound; no proof is given and the missing Jacobian term makes the objective not a valid ELBO for deterministic flows.
  • domain assumption For finite T, the refined variational distribution is closer to the true posterior than the initial guide.
    Stated in Section 3 ('As T increases, the variational approximation will be closer to the exact posterior') without a finite-T theorem; with approximate entropy it is not generally true.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs." pith.science (2026). https://pith.science/paper/3VV5AOMW

@misc{pith2026190809744,
  author       = {Pith},
  title        = {Pith review of: Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3VV5AOMW}},
  note         = {Machine review of arXiv:1908.09744}
}
read the original abstract

A framework to boost the efficiency of Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation. We call it the refined variational approximation. Its strength lies both in ease of implementation and automatically tuning of the sampler parameters to speed up mixing time using automatic differentiation. Several strategies to approximate \emph{evidence lower bound} (ELBO) computation are introduced. Experimental evidence of its efficient performance is shown solving an influence diagram in a high-dimensional space using a conditional variational autoencoder (cVAE) as a deep Bayes classifier; an unconditional VAE on density estimation tasks; and state-space models for time-series data.

Figures

Figures reproduced from arXiv: 1908.09744 by the authors.

Figure 1
Figure 1. In the top, we show the trajectories of the lower [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 1
Figure 1. Bottom: evolution of the negative ELBO loss [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Results of rELBO optimization for state-space [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Top row: original images. Bottom row: recon [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 5
Figure 5. Figure 5: Influence Diagram for the deep Bayes classifier. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 4
Figure 4. Figure 4: Model architecture for the cVAE. The VAE model is implemented with PyTorch [41]. The prior distribution p(z) for the latent variables z ∈ R 10 is a standard factorized Gaussian. The decoder distribution pθ(x|z) and the encoder distribution (initial variational approxim…
Figure 6
Figure 6. Figure 6: Model architecture for the cVAE. The cVAE model is implemented with PyTorch [41]. The prior distribution p(z) for the latent variables z ∈ R 10 is a standard factorized Gaussian. The decoder distribu￾tion pθ(x|y, z) and the encoder distribution (initial varia￾tional ap…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 37 canonical work pages

  1. [1]

    Winbugs-a bayesian modelling framework: concepts, structure, and extensibility

    David J Lunn, Andrew Thomas, Nicky Best, and David Spiegelhalter. Winbugs-a bayesian modelling framework: concepts, structure, and extensibility. Statistics and com- puting, 10(4):325–337, 2000

    work page 2000

  2. [2]

    Stan: A probabilistic programming language

    Bob Carpenter, Andrew Gelman, Matthew D Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell. Stan: A probabilistic programming language. Journal of statisti- cal software, 76(1), 2017

    work page 2017

  3. [3]

    Simple, distributed, and accelerated probabilistic programming

    Dustin Tran, Matthew W Hoffman, Dave Moore, Christo- pher Suter, Srinivas Vasudevan, and Alexey Radul. Simple, distributed, and accelerated probabilistic programming. In Advances in Neural Information Processing Systems , pages 7609–7620, 2018

    work page 2018

  4. [4]

    Pyro: Deep universal probabilistic programming

    Eli Bingham, Jonathan P Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D Goodman. Pyro: Deep universal probabilistic programming. arXiv preprint arXiv:1810.09538, 2018

    arXiv

  5. [5]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    arXiv 2013

  6. [6]

    A tutorial on hidden markov models and selected applications in speech recognition

    Lawrence R Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceed- ings of the IEEE, 77(2):257–286, 1989

    work page 1989

  7. [7]

    Particle Markov chain monte carlo methods

    Christophe Andrieu, Arnaud Doucet, and Roman Holen- stein. Particle Markov chain monte carlo methods. Jour- nal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3):269–342, 2010

    work page 2010

  8. [8]

    Mcmc using hamiltonian dynamics

    Radford M Neal et al. Mcmc using hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, 2(11):2, 2011

    work page 2011

Show all 47 references
  1. [9]

    Automatic differentiation variational inference

    Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M Blei. Automatic differentiation variational inference. The Journal of Machine Learning Research, 18(1):430–474, 2017

    2017

  2. [10]

    Approx- imate inference for deep latent gaussian mixtures

    Eric Nalisnick, Lars Hertel, and Padhraic Smyth. Approx- imate inference for deep latent gaussian mixtures. 2016

    2016

  3. [11]

    Markov chain monte carlo and variational inference: Bridging the gap

    Tim Salimans, Diederik Kingma, and Max Welling. Markov chain monte carlo and variational inference: Bridging the gap. In International Conference on Ma- chine Learning, pages 1218–1226, 2015

    2015

  4. [12]

    The variational gaussian process

    Dustin Tran, Rajesh Ranganath, and David M Blei. The variational gaussian process. arXiv preprint arXiv:1511.06499, 2015

    2015 arXiv

  5. [13]

    A new approach to probabilistic programming inference

    Frank Wood, Jan Willem Meent, and Vikash Mansinghka. A new approach to probabilistic programming inference. In Artificial Intelligence and Statistics, pages 1024–1032, 2014

    2014

  6. [14]

    Turing: a language for flexible probabilistic inference

    Hong Ge, Kai Xu, and Zoubin Ghahramani. Turing: a language for flexible probabilistic inference. In Interna- tional Conference on Artificial Intelligence and Statistics, AISTATS 2018, 9-11 April 2018, Playa Blanca, Lanzarote, Canary Islands, Spain, pages 1682–1690, 2018

    2018

  7. [15]

    A general framework for the parametrization of hierarchical models

    Omiros Papaspiliopoulos, Gareth O Roberts, and Martin Sköld. A general framework for the parametrization of hierarchical models. Statistical Science , pages 59–73, 2007

    2007

  8. [16]

    Neutra- lizing bad geometry in hamiltonian monte carlo using neural transport

    Matthew D Hoffman, Pavel Sountsov, Joshua Dillon, Ian Langmore, Dustin Tran, and Srinivas Vasudevan. Neutra- lizing bad geometry in hamiltonian monte carlo using neural transport. 2018

    2018

  9. [17]

    Neural network renormaliza- tion group

    Shuo-Hui Li and Lei Wang. Neural network renormaliza- tion group. Phys. Rev. Lett., 121:260601, Dec 2018

    2018

  10. [18]

    Transport map accelerated markov chain monte carlo

    Matthew Parno and Youssef Marzouk. Transport map accelerated markov chain monte carlo. arXiv preprint arXiv:1412.5492, 2014

    2014 arXiv

  11. [19]

    Variational infer- ence with normalizing flows

    Danilo Rezende and Shakir Mohamed. Variational infer- ence with normalizing flows. In International Conference on Machine Learning, pages 1530–1538, 2015

    2015

  12. [20]

    Continuous-time flows for efficient inference and density estimation, 2018

    Changyou Chen, Chunyuan Li, Liqun Chen, Wenlin Wang, Yunchen Pu, and Lawrence Carin. Continuous-time flows for efficient inference and density estimation, 2018

    2018

  13. [21]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Ad- vances in neural information processing systems , pages 2672–2680, 2014

    2014

  14. [22]

    Variational inference using implicit distri- butions

    Ferenc Huszár. Variational inference using implicit distri- butions. arXiv preprint arXiv:1702.08235, 2017

    2017 arXiv

  15. [23]

    Unbiased im- plicit variational inference

    Michalis K Titsias and Francisco Ruiz. Unbiased im- plicit variational inference. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 167–176, 2019

    2019

  16. [24]

    Semi-implicit varia- tional inference

    Mingzhang Yin and Mingyuan Zhou. Semi-implicit varia- tional inference. arXiv preprint arXiv:1805.11183, 2018

    2018 arXiv

  17. [25]

    Learning deep latent gaussian mod- els with markov chain monte carlo

    Matthew D Hoffman. Learning deep latent gaussian mod- els with markov chain monte carlo. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1510–1519. JMLR. org, 2017

    2017

  18. [26]

    Learning to draw samples with amortized stein variational gradient descent

    Yihao Feng, Dilin Wang, and Qiang Liu. Learning to draw samples with amortized stein variational gradient descent. arXiv preprint arXiv:1707.06626, 2017

    2017 arXiv

  19. [27]

    In- ference suboptimality in variational autoencoders

    Chris Cremer, Xuechen Li, and David Duvenaud. In- ference suboptimality in variational autoencoders. arXiv preprint arXiv:1801.03558, 2018

    2018 arXiv

  20. [28]

    A contrastive di- vergence for combining variational inference and mcmc

    Francisco Ruiz and Michalis Titsias. A contrastive di- vergence for combining variational inference and mcmc. In International Conference on Machine Learning, pages 5537–5545, 2019

    2019

  21. [29]

    Bayesian learning via stochastic gradient langevin dynamics

    Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Machine Learn- ing (ICML-11), pages 681–688, 2011

    2011

  22. [30]

    Preconditioned stochastic gradient langevin dynamics for deep neural networks

    Chunyuan Li, Changyou Chen, David Carlson, and Lawrence Carin. Preconditioned stochastic gradient langevin dynamics for deep neural networks. In Thirtieth AAAI Conference on Artificial Intelligence, 2016

    2016

  23. [31]

    High-order stochastic gradient thermostats for bayesian learning of deep models

    Chunyuan Li, Changyou Chen, Kai Fan, and Lawrence Carin. High-order stochastic gradient thermostats for bayesian learning of deep models. In Thirtieth AAAI Con- ference on Artificial Intelligence, 2016

    2016

  24. [32]

    Adageo: Adaptive geometric learning for optimization and sampling

    Gabriele Abbati, Alessandra Tosi, Michael Osborne, and Seth Flaxman. Adageo: Adaptive geometric learning for optimization and sampling. In International Conference on Artificial Intelligence and Statistics , pages 226–234, 2018

    2018

  25. [33]

    Stochastic gra- dient mcmc with repulsive forces

    Victor Gallego and David Rios Insua. Stochastic gra- dient mcmc with repulsive forces. arXiv preprint arXiv:1812.00071, 2018

    2018 arXiv

  26. [34]

    Stochastic gradient descent as approximate bayesian in- ference

    Stephan Mandt, Matthew D Hoffman, and David M Blei. Stochastic gradient descent as approximate bayesian in- ference. The Journal of Machine Learning Research , 18(1):4873–4907, 2017

    2017

  27. [35]

    Stein variational gradient descent: A general purpose bayesian inference algorithm

    Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In Advances In Neural Information Processing Systems, pages 2378–2386, 2016

    2016

  28. [36]

    Early stopping as nonparametric variational inference

    David Duvenaud, Dougal Maclaurin, and Ryan Adams. Early stopping as nonparametric variational inference. In Artificial Intelligence and Statistics , pages 1070–1077, 2016

    2016

  29. [37]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  30. [38]

    Forward and reverse gradient-based hyperparameter optimization

    Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1165–1173. JMLR. org, 2017

    2017

  31. [39]

    Coupled variational bayes via optimization embedding

    Bo Dai, Hanjun Dai, Niao He, Weiyang Liu, Zhen Liu, Jianshu Chen, Lin Xiao, and Le Song. Coupled variational bayes via optimization embedding. In Advances in Neural Information Processing Systems, pages 9690–9700, 2018

    2018

  32. [40]

    Implicit deep latent variable models for text generation

    Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. Implicit deep latent variable models for text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage...

    2019

  33. [41]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017

    2017

  34. [42]

    Fundamentals of Kalman filtering: a practical approach

    Paul Zarchan and Howard Musoff. Fundamentals of Kalman filtering: a practical approach . American In- stitute of Aeronautics and Astronautics, Inc., 2013

    2013

  35. [43]

    Strictly proper scoring rules, prediction, and estimation

    Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, 2007

    2007

  36. [44]

    Atmospheric carbon dioxide record from mauna loa

    Charles D Keeling. Atmospheric carbon dioxide record from mauna loa. 2005

    2005

  37. [45]

    Influence dia- grams

    Ronald A Howard and James E Matheson. Influence dia- grams. Decision Analysis, 2(3):127–143, 2005

    2005

  38. [46]

    Shachter

    Ross D. Shachter. Probabilistic inference and influence diagrams. Operations Research, 36(4):589–604, 1988

    1988

  39. [47]

    A spectral ap- proach to gradient estimation for implicit distributions

    Jiaxin Shi, Shengyang Sun, and Jun Zhu. A spectral ap- proach to gradient estimation for implicit distributions. In International Conference on Machine Learning, pages 4651–4660, 2018. A Fokker-Planck approximation (VIS-FP) The Fokker-Planck equation is a PDE that describes th...

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.