RECORD
Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
Machine Learning · Submitted 14 Jun 2018
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Thus our method establishes a new state-of-the-art for inference in DGPs.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. One of the biggest challenges with these models is that exact inference is intractable. The current state-of-the-art inference method, Variational Inference (VI), employs a Gaussian approximation to the posterior distribution. This can be a potentially poor unimodal approximation of the generally multimodal posterior. In this work, we provide evidence for the non-Gaussian nature of the posterior and we apply the Stochastic Gradient Hamiltonian Monte Carlo method to generate samples. To efficiently optimize the hyperparameters, we introduce the Moving Window MCEM algorithm. This results in significantly better predictions at a lower computational cost than its VI counterpart. Thus our method establishes a new state-of-the-art for inference in DGPs.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. | background 1.00 | 0.06 | 0.03 | asserted 1.00 |
| s1 | One of the biggest challenges with these models is that exact inference is intractable. | background 0.89 | 0.08 | 0.02 | asserted 1.00 |
| s2 | The current state-of-the-art inference method, Variational Inference (VI), employs a Gaussian approximation to the posterior distribution. | background 0.99 | 0.04 | 0.02 | asserted 1.00 |
| s3 | This can be a potentially poor unimodal approximation of the generally multimodal posterior. | background 0.73 | 0.10 | 0.03 | hedged possibility 0.91 |
| s4 | In this work, we provide evidence for the non-Gaussian nature of the posterior and we apply the Stochastic Gradient Hamiltonian Monte Carlo method to generate samples. | contribution 0.87 | 0.95 | 0.12 | asserted 1.00 |
| s5 | To efficiently optimize the hyperparameters, we introduce the Moving Window MCEM algorithm. | method 0.67 | 0.94 | 0.04 | asserted 1.00 |
| s6 | This results in significantly better predictions at a lower computational cost than its VI counterpart. | result 0.99 | 0.92 | 0.29 | asserted 1.00 |
| s7 | Thus our method establishes a new state-of-the-art for inference in DGPs. | result 0.57 | 0.82 | 0.83 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. One of the biggest challenges with these models is that exact inference is intractable. The current state-of-the-art inference method, Variational Inference (VI), employs a Gaussian approximation to the posterior distribution. This can be a potentially poor unimodal approximation of the generally multimodal posterior. In this work, we provide evidence for the non-Gaussian nature of the posterior and we apply the Stochastic Gradient Hamiltonian Monte Carlo method to generate samples. To efficiently optimize the hyperparameters, we introduce the Moving Window MCEM algorithm. This results in significantly better predictions at a lower computational cost than its VI counterpart. Thus our method establishes a new state-of-the-art for inference in DGPs.
Human review
How the literature uses it
3 arXiv papers cite this in 3 sentences
Papers citing it · 2 sentences without a confident label
Cites it as support 1 paper
-
A plausible reason behind this under-fitting resides in the factorized approximate posterior formulation, which was shown to negatively affect predictive performance compared to MCMC inference schemes [ref] .
Distributional Gaussian Processes Layers for Out-of-Distribution Detection · Discussion & Conclusion · 0.99
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.