The Kullback-Leibler divergence is the only Bregman divergence whose gradient flow with respect to many popular metrics does not require the normalizing constant of the target distribution π.
arXiv , arxivId =:1905.09863 , file =
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanism to accelerate the mixing of Langevin diffusion. Our algorithm is motivated by its mean field partial differential equation (PDE), which is a Fokker-Planck equation supplemented by a nonlocal birth-death term. This PDE can be viewed as a gradient flow of the Kullback-Leibler divergence with respect to the Wasserstein-Fisher-Rao metric. We prove that under some assumptions the asymptotic convergence rate of the nonlocal PDE is independent of the potential barrier, in contrast to the exponential dependence in the case of the Langevin diffusion. We illustrate the efficiency of the birth-death accelerated Langevin method through several analytical examples and numerical experiments.
citation-role summary
other 1
citation-polarity summary
verdicts
UNVERDICTED 4roles
other 1polarities
unclear 1representative citing papers
Derives MSIP algorithm from MMD gradient flows for weighted quantization, extending mean shift and relating to preconditioned gradient descent and Lloyd's clustering.
Geometric tempering yields exponential convergence bounds for both Wasserstein and Fisher-Rao flows but produces no speedup in the Fisher-Rao metric, with new adaptive schedules derived from the tempered dynamics.
Establishes stability bounds for SHK flows yielding dimension-free controls on log-likelihood ratios and divergences, then applies them to time-dependent Pure-DP and Approximate-DP certificates for exponential-mechanism samplers.
citing papers explorer
-
A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows
The Kullback-Leibler divergence is the only Bregman divergence whose gradient flow with respect to many popular metrics does not require the normalizing constant of the target distribution π.
-
Weighted quantization using MMD: From mean field to mean shift via gradient flows
Derives MSIP algorithm from MMD gradient flows for weighted quantization, extending mean shift and relating to preconditioned gradient descent and Lloyd's clustering.
-
Properties and limitations of geometric tempering for gradient flow dynamics
Geometric tempering yields exponential convergence bounds for both Wasserstein and Fisher-Rao flows but produces no speedup in the Fisher-Rao metric, with new adaptive schedules derived from the tempered dynamics.
-
On the Stability of Spherical Hellinger-Kantorovich Flows and Their Implications for Differential Privacy
Establishes stability bounds for SHK flows yielding dimension-free controls on log-likelihood ratios and divergences, then applies them to time-dependent Pure-DP and Approximate-DP certificates for exponential-mechanism samplers.