REVIEW 5 cited by
Convergence of coordinate ascent variational inference for log-concave measures via optimal transport
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Mean field variational inference (VI) is the problem of finding the closest product (factorized) measure, in the sense of relative entropy, to a given high-dimensional probability measure $\rho$. The well known Coordinate Ascent Variational Inference (CAVI) algorithm aims to approximate this product measure by iteratively optimizing over one coordinate (factor) at a time, which can be done explicitly. Despite its popularity, the convergence of CAVI remains poorly understood. In this paper, we prove the convergence of CAVI for log-concave densities $\rho$. If additionally $\log \rho$ has Lipschitz gradient, we find a linear rate of convergence, and if also $\rho$ is strongly log-concave, we find an exponential rate. Our analysis starts from the observation that mean field VI, while notoriously non-convex in the usual sense, is in fact displacement convex in the sense of optimal transport when $\rho$ is log-concave. This allows us to adapt techniques from the optimization literature on coordinate descent algorithms in Euclidean space.
Forward citations
Cited by 5 Pith papers
-
The Influence Function of Transport-based Quantiles
The influence function of multivariate transport quantiles has a pole-type singularity in dimension ≥2, so contamination near a quantile level yields unbounded first-order sensitivity.
-
Variational Inference for Latent Variable Models in High Dimensions
The paper characterizes the exact regimes where mean-field variational inference is accurate for LDA and MMSB, with tight finite-sample KL bounds.
-
Stability of Mean-Field Variational Inference
The mean-field variational inference optimizer is Lipschitz stable and differentiable in the target potential under strong log-concavity, with an explicit PDE for the derivative.
-
Exponential Convergence of CAVI for Bayesian PCA
CAVI for Bayesian PCA converges exponentially: for k=1 the mean updates are exactly power iteration, and for general k the KL divergence contracts locally under a low-rank condition and a non-singular Hessian assumption.
-
Rotated Mean-Field Variational Inference and Iterative Gaussianization
Iteratively rotating the coordinate system and applying mean-field variational inference builds an invertible transport map that approximates unnormalized target densities more accurately than standard MFVI and at low...
Discussion (0). Sign in to comment.