Pith. sign in

REVIEW 4 cited by

Inference Suboptimality in Variational Autoencoders

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1801.03558 v3 pith:IAVBEVMC submitted 2018-01-10 cs.LG stat.ML

Chris Cremer , Xuechen Li , David Duvenaud This is my paper · ORCID
classification cs.LGstat.ML
keywords inferencevariationalapproximationapproximateautoencoderscomplexitydistributionfactors
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Amortized inference allows latent-variable models trained via variational learning to scale to large datasets. The quality of approximate inference is determined by two factors: a) the capacity of the variational distribution to match the true posterior and b) the ability of the recognition network to produce good variational parameters for each datapoint. We examine approximate inference in variational autoencoders in terms of these factors. We find that divergence from the true posterior is often due to imperfect recognition networks, rather than the limited complexity of the approximating distribution. We show that this is due partly to the generator learning to accommodate the choice of approximation. Furthermore, we show that the parameters used to increase the expressiveness of the approximation play a role in generalizing inference rather than simply improving the complexity of the approximation.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations

    stat.ML 2019-09 conditional novelty 7.0 of 10

    An ISA-style Lp-nested prior over VAE latents improves disentanglement and reduces the reconstruction/regularization trade-off compared to modified-ELBO methods like beta-VAE.

  2. Implicit Deep Latent Variable Models for Text Generation

    cs.LG 2019-08 conditional novelty 6.0 of 10

    Implicit (sample-based) variational posteriors with aggregated-posterior matching beat Gaussian VAEs on text generation and reduce posterior collapse.

  3. Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs

    cs.LG 2019-08 conditional novelty 5.0 of 10

    The paper proposes VIS, a variational guide that embeds several iterations of SGLD or SGD and auto-tunes the step size, claiming tighter ELBOs and faster mixing, with experiments on VAEs and state-space models.

  4. Bayesian Neural Networks: An Introduction and Survey

    stat.ML 2020-06 unverdicted novelty 1.0 of 10

    A survey introducing Bayesian Neural Networks and comparing approximate inference methods to enable uncertainty quantification in neural network predictions.

Pith tools