A decoupled Bayesian neural network that maps a DNN's logits to calibrated probabilities consistently reduces expected calibration error, at the cost of some accuracy loss on complex datasets.
Ergodic Inference: Accelerate Convergence by Optimisation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Statistical inference methods are fundamentally important in machine learning. Most state-of-the-art inference algorithms are variants of Markov chain Monte Carlo (MCMC) or variational inference (VI). However, both methods struggle with limitations in practice: MCMC methods can be computationally demanding; VI methods may have large bias. In this work, we aim to improve upon MCMC and VI by a novel hybrid method based on the idea of reducing simulation bias of finite-length MCMC chains using gradient-based optimisation. The proposed method can generate low-biased samples by increasing the length of MCMC simulation and optimising the MCMC hyper-parameters, which offers attractive balance between approximation bias and computational efficiency. We show that our method produces promising results on popular benchmarks when compared to recent hybrid methods of MCMC and VI.
fields
cs.LG 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks
A decoupled Bayesian neural network that maps a DNN's logits to calibrated probabilities consistently reduces expected calibration error, at the cost of some accuracy loss on complex datasets.