RECORD
Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows
Machine Learning · Submitted 16 May 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. | background 0.97 | 0.04 | 0.03 | asserted 0.99 |
| s1 | We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. | contribution 1.00 | 0.94 | 0.05 | asserted 1.00 |
| s2 | Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. | contribution 0.95 | 0.95 | 0.08 | asserted 1.00 |
| s3 | In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). | gap 0.59 | 0.79 | 0.18 | asserted 1.00 |
| s4 | To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). | contribution 0.71 | 0.91 | 0.27 | asserted 0.99 |
| s5 | Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy. | result 1.00 | 0.94 | 0.63 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 13 sentences without a confident label
Compares with 3 papers
-
Our work differs from [ref] in that we consider fat-tailed variational approximations $q(z |x)$ rather than priors $p(z)$.
Use of Student's t-Distribution for the Latent Layer in a Coupled Variational Autoencoder · Related Work · 0.86
-
Our work differs from [ref] in that we consider fat-tailed variational approximations $q(z |x)$ rather than priors $p(z)$.
Variational Autoencoder with Embedded Student-$t$ Mixture Model for Authorship Attribution · Related Work · 0.86
-
Although [ref] also considers a StudentT approximate posterior, our work involves a more general variational family which use normalizing flows.
Variational auto-encoders with Student's t-prior · Related Work · 0.97
Cites as support 2 papers
-
In practice, many flow-based methods exhibit Lipschitz-continuity in their transport map either by design [ref] , or as a consequence of choice of architecture and activation function (<Ref>).
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models · Tail behavior of Lipschitz flows · 0.96
-
In practice, many flow-based methods exhibit Lipschitz-continuity in their transport map either by design [ref] , or as a consequence of choice of architecture and activation function (<Ref>).
Residual Flows for Invertible Generative Modeling · Tail behavior of Lipschitz flows · 0.96
Cites as background 4 papers
-
Normalizing flows and other flow based methods have a rich history within variational inference [ref] .
Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization · Related Work · 1.00
-
Recent work in variational autoencoders (VAEs) have considered relaxing Gaussian assumptions to heavier-tailed distributions [ref] .
Variational auto-encoders with Student's t-prior · Related Work · 0.96
-
Recent work in variational autoencoders (VAEs) have considered relaxing Gaussian assumptions to heavier-tailed distributions [ref] .
Variational Autoencoder with Embedded Student-$t$ Mixture Model for Authorship Attribution · Related Work · 0.96
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.