Weight Uncertainty in Neural Networks
read the original abstract
We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
Concrete Problems in AI Safety
The paper categorizes five concrete AI safety problems arising from flawed objectives, costly evaluation, and learning dynamics.
-
Unsupervised Domain Adaptation via Calibrating Uncertainties
A new regularization approach for unsupervised domain adaptation that calibrates Renyi entropy of uncertainties estimated via variational Bayes.
-
A Framework for Variational Inference of Lightweight Bayesian Neural Networks with Heteroscedastic Uncertainties
Framework embeds aleatoric and epistemic uncertainties into BNN parameter variances and applies moment propagation for sampling-free variational inference in lightweight networks.
-
Comparing Semi-Parametric Model Learning Algorithms for Dynamic Model Estimation in Robotics
Semi-parametric Gaussian process regression yields the most accurate inverse dynamics models in most tested robotic scenarios compared to parametric, non-parametric, and other semi-parametric baselines.
-
Quality of Uncertainty Quantification for Bayesian Neural Network Inference
Empirical comparison of ten BNN inference methods shows test log-likelihood can mislead on uncertainty quality and that posterior-structure innovations do not necessarily yield high-quality approximations.
-
Portfolio Optimization Proxies under Label Scarcity and Regime Shifts via Bayesian and Deterministic Students under Semi-Supervised Sandwich Training
A semi-supervised teacher-student framework enables neural networks to proxy CVaR portfolio optimization using synthetic data augmentation for scarce labels and regime shifts.
-
Bayesian Neural Networks: An Introduction and Survey
A survey introducing Bayesian Neural Networks and comparing approximate inference methods to enable uncertainty quantification in neural network predictions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.