Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks

Daniel Ramos; Juan Maro\~nas; Roberto Paredes

REVIEW 4 major objections 4 minor 77 references

Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Decoupled Bayesian neural networks recalibrate deep classifiers, beating temperature scaling on seven benchmarks.

desk verdict Useful decoupled BNN calibrator with consistent ECE gains over TS, but the missing deterministic-MLP control makes the 'Bayesian averaging is the cause' claim under-supported. read the letter →

arxiv 1908.08972 v3 pith:FV6VNEHU submitted 2019-08-23 cs.LG stat.ML

Juan Maroñas , Roberto Paredes , Daniel Ramos This is my paper · ORCID

classification cs.LGstat.ML

keywords calibrationBayesianneuralnetworksdecoupledrecalibrationexpectederrorvariationalinferencetemperaturescalingimageclassificationuncertaintyquantification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes attaching a Bayesian calibration stage, a small Bayesian neural network, after a pre-trained deep classifier, feeding it only the classifier's uncalibrated logits and training it to produce calibrated class probabilities. The authors claim this decoupled Bayesian stage consistently lowers expected calibration error relative to temperature scaling and other state-of-the-art calibrators across several image-classification benchmarks. Because the Bayesian stage averages over many weight samples rather than committing to a single point estimate, it treats calibration as an uncertainty problem rather than a simple one-parameter fix. The broader claim is that complex models can improve calibration without overfitting as long as parameter uncertainty is handled properly.

What carries the argument

The load-bearing object is the decoupled Bayesian calibration stage: a fully-connected Bayesian neural network with ReLU hidden units and a softmax output, whose input is the class-dimensional logit vector produced by a pre-trained deep network and whose output is a calibrated probability vector of the same dimension. Inference uses variational Bayes with a factorized Gaussian posterior approximation, optimized through the evidence lower bound using the reparameterization trick and its local reparameterization variant, and predictions are formed by averaging many softmax outputs over samples from the variational posterior. That Monte Carlo average is the mechanism that expresses parameter uncertainty in the final probabilities, and choosing the number of samples on a validation set is the device that controls the variance under-estimation of mean-field inference.

What would settle it

Construct two groups of test images whose deep-network logit vectors are identically distributed but whose true-class rates differ, for example distinct subpopulations that collapse to the same logits. Since the BNN is trained only on logits, it must assign the same calibrated probability to both groups, so measuring a nonzero expected calibration error within either group after calibration would falsify the logit-sufficiency assumption.

Watch

Extended reading notes

Core claim

At the paper's core is the claim that a Bayesian neural network used as a post-hoc recalibrator, trained on the logit vectors of a frozen, high-accuracy deep network, maps those logits to probabilities whose confidence matches empirical accuracy more closely than the network's own softmax output. The predictive distribution is a Monte Carlo average of softmax outputs over weights drawn from a factorized Gaussian variational posterior, with the number of samples chosen on a validation set to counteract variance under-estimation. In the reported experiments, this BNN recalibrator achieves the lowest expected calibration error among the compared explicit and implicit calibration techniques, including temperature scaling, deep ensembles, Monte Carlo dropout, MMCE, and VWCI, while leaving the original deep network untouched and requiring only one forward pass through it. The authors interpret this as evidence that Bayesian treatment of uncertainty is what makes calibrated predictions reliable, and that the calibration space is not inherently simple.

Load-bearing premise

The calibration stage's input is only the pre-trained network's logit vector, so the method assumes that all miscalibration is already contained in those logits and can be removed by a learned transformation of them alone.

Editorial extensions

If this is right

On the seven image benchmarks reported, the decoupled BNN yields lower expected calibration error than temperature scaling, the strongest explicit baseline, and lower than Monte Carlo dropout and deep ensembles among implicit methods.
A single forward pass through the large deep network plus several light forward passes through the BNN replaces any retraining or modification of the original network, so calibration can be improved post-hoc at modest computational cost.
Because the Bayesian stage is decoupled, it can be upgraded with richer variational approximations or better inference methods without touching the pre-trained network's architecture or weights.
If uncertainty is truly the limiting factor, then better posterior approximations than mean-field should push calibration error lower still, a direct implication the paper draws for future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the paper does not test is to feed the Bayesian stage additional deep-network features, such as penultimate-layer activations, alongside the logits; if expected calibration error drops further, the logit-only input assumption would need revision.
The validation-selected number of Monte Carlo samples behaves like a regularization knob on epistemic uncertainty, so one could study whether its optimal value tracks dataset difficulty or network capacity.
The decoupled design suggests that a single calibrated BNN might be reused across different deep networks trained on the same label space, offering a cheap route to recalibrating a whole family of models.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Juan,

Quick read for your desk: the paper is a real contribution, not a breakthrough. It trains a mean-field variational BNN on the logits of a pre-trained DNN and shows, across a wide set of architectures and datasets, that this consistently lowers ECE compared to temperature scaling and a few other baselines. The breadth of experiments and the fact that code is provided are genuine assets. The discussion of variance under-estimation and the validation-based selection of K is honest and unusually candid for this literature.

Where it is soft: the central attribution. The main tables never compare against a deterministic MLP calibrator of the same architecture trained on the same logits. Table 2 does a Bayesian vs point-estimate linear comparison, but the main method is a multilayer BNN, so that does not isolate the effect of marginalization in the model actually used. The disclosed K-on-validation trick could plausibly be doing much of the work, and the extra tuning step is not applied to the baselines. So the headline claim that Bayesian inference is what drives the calibration gains is under-supported, even though the empirical method itself works. That is a fixable gap, not a fatal flaw: the method is well motivated, the paper is clear about its own limitations, and the accuracy degradation on CIFAR-100/BIRDS/CARS is acknowledged and analyzed.

Also worth noting: no error bars anywhere, and the baseline calibration-data split is not fully explicit. Those are minor-to-moderate. The claim that incorporating uncertainty provides more reliable models is a reasonable empirical conclusion for the settings tested, but it should be softened to 'this particular approximate Bayesian stage plus validation-based K selection.'

Verdict: it deserves a serious referee. The method is simple, clearly described, reproducible in principle, and the empirical comparison is broad. A good referee could ask for the missing control and for error bars, and the paper would be stronger for it. I'd cite it if I worked on post-hoc calibration.

Cheers.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes a decoupled calibration stage for pre-trained DNN classifiers: a Bayesian Neural Network (BNN) trained on the DNN's logits maps them to calibrated logits, with the posterior approximated by mean-field variational inference (MFVI or MFVILR) and predictions averaged over Monte Carlo samples. The claimed contribution is that this Bayesian stage consistently improves expected calibration error (ECE) over temperature scaling and other baselines across seven image classification datasets, while retaining the accuracy of the original DNN in most cases. The paper also contains a toy experiment contrasting Bayesian and point-estimate models, a linear logistic comparison, an analysis of variance under-estimation, and a discussion of the validation-based selection of the number of Monte Carlo samples K.

Significance. If the empirical claims are supported, the work is significant because it challenges the view that calibration spaces for DNNs are inherently simple and that temperature scaling should be preferred over more complex calibrators. The paper provides a broad experimental matrix, including several architectures and datasets, and the authors release code for reproducibility. It also honestly discloses known limitations of the mean-field approximation, including accuracy degradation on fine-grained tasks and the need for validation-based selection of K. However, the central attribution of the calibration gains to Bayesian averaging is weakened by the absence of a deterministic neural calibrator control, by the lack of variance estimates for the reported ECE improvements, and by asymmetric hyperparameter treatment (validation-based K) relative to the baselines.

major comments (4)

[Section 5.6, Tables 6 and 7] The experimental design never compares the proposed BNN calibrator against a deterministic MLP calibrator of identical architecture trained on the same logits with the same optimizer and hyperparameter-selection procedure. Without this control, the reported ECE gains cannot be attributed to Bayesian marginalization as opposed to the increased representational capacity of a learned logit-to-logit map. I ask the authors to add a MAP-trained (or otherwise non-Bayesian) MLP calibrator with the same hidden-layer configuration, the same ELBO-derived loss without the KL term (or an equivalent regularized loss), and the same validation-based selection of predictive samples, and to report ECE and accuracy for that model in Tables 6 and 7.
[Section 4.5 and Section 5.4] The number of Monte Carlo predictive samples K is selected on a validation set to mitigate variance under-estimation, whereas no comparable extra tuning step is applied to temperature scaling or the other baselines. This asymmetry is a plausible source of the ECE improvement and is not controlled for. The authors should either fix K across methods, or apply the same validation-based selection to the baselines (e.g., selecting the temperature on the same validation criterion), and report the sensitivity of ECE to K in the main tables.
[Table 3 and Section 5.5] No error bars, confidence intervals, or repeated-run statistics are reported for any ECE or accuracy value, although the text states that accuracy degradation is 'slight' on CIFAR100, BIRDS, and CARS. Given that Table 3 shows accuracy drops of roughly 2.5, 2.0, and 3.5 points on those tasks, and that several ECE differences between methods in Tables 6 and 7 are small (e.g., SVHN rows), the claims of consistent calibration improvement and only slight accuracy loss require variance estimates over multiple training runs or at least over the selected BNN architectures.
[Section 5.1 and Tables 6-7] It is unclear which data split is used to calibrate each baseline. The text states that a validation set is used to select K for the proposed method but does not explicitly state whether temperature scaling, network ensembles, and the other baselines are fit on the same calibration split or on the training split. The authors should specify this precisely and, if the splits differ, rerun the comparisons on a common calibration set.

minor comments (4)

[Table 4] The table header reads 'MVFI' and 'MVFILR' instead of 'MFVI' and 'MFVILR'; this should be corrected for consistency with the text.
[Section 5.7] The phrase 'miss-classified' appears in the qualitative analysis and should be 'misclassified'.
[Introduction, Section 2] The acronym 'NNL' in the Introduction should be 'NLL' (negative log-likelihood); the paper otherwise uses 'NLL'.
[Section 4.4] Equation (6) is written as an argmax over q of the ELBO with a beta-weighted KL term, but the text later refers to beta as a hyperparameter selected from a set; the authors should clarify whether beta is optimized or grid-searched, and how it interacts with the validation-based selection of K.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the calibration pipeline is an empirical, held-out evaluation rather than a tautological derivation.

full rationale

The paper's central claim is that a decoupled Bayesian stage, trained as a BNN on the logits of a pre-trained DNN, improves calibration on held-out test data. The prediction procedure in Eq. 7 is a Monte Carlo average of softmax outputs over sampled variational parameters, and the reported ECE values in Tables 6 and 7 are computed on test sets after training on separate data. This is an out-of-sample empirical evaluation, not a quantity that is identical to the training objective by construction. The selection of the number of Monte Carlo samples K on a validation set, disclosed in Section 4.5, is a standard hyperparameter choice and is analogous to tuning the temperature parameter of the TS baseline on validation data; it does not amount to fitting the test predictions. No load-bearing self-citation is present: the cited work by one of the authors, reference [19], is used only for background on proper scoring rules and calibration decomposition, and the central framework does not invoke a uniqueness theorem or ansatz imported from the authors' own prior work. The absence of a deterministic MLP calibrator with identical architecture is a legitimate experimental-control concern, but it is not a circularity: the reported ECE gains are empirical findings whose attribution to Bayesian averaging could be questioned, not claims that reduce to their own inputs by definition. Therefore the paper warrants a score of 0 for circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new entities. It relies on standard variational inference machinery and a set of modeling choices (prior, likelihood, variational family) that are common in the BNN literature. The most consequential assumptions are the sufficiency of logits for calibration and the adequacy of the mean-field family after tuning K.

free parameters (3)

beta (KL weight in ELBO) = Selected from {10^-i} for i=0..4 per BNN architecture
Controls the strength of the KL divergence in the ELBO (Eq. 6); chosen per dataset and architecture, affecting the variational posterior and calibration.
K (number of Monte Carlo predictive samples) = Selected on validation set to minimize ECE
Number of samples from the variational posterior used in Eq. 5; the paper tunes K on validation to counteract variance under-estimation (Section 4.5), making final ECE depend on this choice.
BNN architecture (hidden layers and units) = Not specified in main text, provided in supplementary material
The complexity of the calibration network is adapted per dataset and DNN, which influences both calibration and accuracy results.

assumptions (4)

standard math ELBO optimization with the reparameterization trick provides a valid approximate posterior qphi(theta) close to p(theta|O).
Basis of variational inference; used to train the BNN (Eq. 6).
domain assumption The likelihood is categorical with softmax output and the prior on weights is a standard Gaussian.
Modeling choices in Section 4.2 and 4.4; they shape the form of the posterior and the predictive distribution.
domain assumption The logits of the pre-trained DNN are a sufficient input for calibration.
Decoupling assumption in Section 4.1: the Bayesian stage only sees logits, not original inputs; if miscalibration depends on input features beyond the logit values, this could fail.
domain assumption Mean-field Gaussian variational family is sufficient to yield a well-calibrated predictive distribution when K is tuned.
The paper relies on this in experiments; it acknowledges VUE and compensates with K selection (Sections 4.5 and 5.4), implying the family alone is not sufficient.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks." pith.science (2026). https://pith.science/paper/FV6VNEHU

@misc{pith2026190808972,
  author       = {Pith},
  title        = {Pith review of: Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FV6VNEHU}},
  note         = {Machine review of arXiv:1908.08972}
}

read the original abstract

Deep Neural Networks (DNNs) have achieved state-of-the-art accuracy performance in many tasks. However, recent works have pointed out that the outputs provided by these models are not well-calibrated, seriously limiting their use in critical decision scenarios. In this work, we propose to use a decoupled Bayesian stage, implemented with a Bayesian Neural Network (BNN), to map the uncalibrated probabilities provided by a DNN to calibrated ones, consistently improving calibration. Our results evidence that incorporating uncertainty provides more reliable probabilistic models, a critical condition for achieving good calibration. We report a generous collection of experimental results using high-accuracy DNNs in standardized image classification benchmarks, showing the good performance, flexibility and robust behavior of our approach with respect to several state-of-the-art calibration methods. Code for reproducibility is provided.

Figures

Figures reproduced from arXiv: 1908.08972 by the authors.

**Figure 2.** Reliability diagrams [20] for two DNNs trained on two computer vision benchmarks, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Decision thresholds learned by a neural network on a 2-D toy dataset problem where [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: ECE measure on validation and test set varying the number of Monte Carlo Predic [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]

**Figure 5.** Figure 5: Comparison of ECE performance between TS and BNN in test and validation. On [PITH_FULL_IMAGE:figures/full_fig_p028_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 67 canonical work pages

[1]

Huang, et al., Densely connected convolutional networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 2261–2269

G. Huang, et al., Densely connected convolutional networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 2261–2269

work page 2017
[2]

Zagoruyko, et al., Wide residual networks, in: E

S. Zagoruyko, et al., Wide residual networks, in: E. R. H. Richard C. Wilson, W. A. P. Smith (Eds.), Proceedings of the British Machine Vision Conference (BMVC), BMVA Press, 2016, pp. 87.1–87.12. doi: 10.5244/C.30.87

doi:10.5244/c.30.87 2016
[3]

Mikolov, et al., Eﬃcient estimation of word representations in vector space, in: International Conference on Learning Representations, 2013

T. Mikolov, et al., Eﬃcient estimation of word representations in vector space, in: International Conference on Learning Representations, 2013

work page 2013
[4]

T. Mikolov, et al., Distributed representations of words and phrases and their compositionality, in: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, Curran Associates Inc., USA, 2013, pp. 3111–3119. 33

work page 2013
[5]

Vaswani, et al., Attention is all you need, in: I

A. Vaswani, et al., Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran Associates, Inc., 2017, pp. 5998–6008

work page 2017
[6]

Hinton, et al., Deep neural networks for acoustic modelling in speech recognition

G. Hinton, et al., Deep neural networks for acoustic modelling in speech recognition. the shared views of four research groups, IEEE Signal Process- ing Magazine 29 (6) (2012) 82–97. doi:10.1109/MSP.2012.2205597

arXiv 2012
[7]

A. P. Dawid, The well-calibrated Bayesian, Journal of the American Sta- tistical Association 77 (379) (1982) 605–610

work page 1982
[8]

Cohen, et al., Properties and beneﬁts of calibrated classiﬁers, in: Knowl- edge Discovery in Databases: PKDD 2004, Vol

I. Cohen, et al., Properties and beneﬁts of calibrated classiﬁers, in: Knowl- edge Discovery in Databases: PKDD 2004, Vol. 3202 of Lecture Notes in Computer Science, Springer, Heidelberg - Berlin, 2004

work page 2004

Show all 77 references

[9]

Br¨ ummer, Measuring, reﬁning and calibrating speaker and language information extracted from speech, Ph.D

N. Br¨ ummer, Measuring, reﬁning and calibrating speaker and language information extracted from speech, Ph.D. thesis, School of Electrical En- gineering, University of Stellenbosch, Stellenbosch, South Africa, available at http://sites.google.com/site/nikobrummer/ (2010)

2010
[10]

R. Caruana, et al., Intelligible models for healthcare: Predicting pneu- monia risk and hospital 30-day readmission, in: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, ACM, New York, NY, USA, 2015, pp. 1721–1730. do...

2015
[11]

B. Zadrozny, et al., Transforming classiﬁer scores into accurate multiclass probability estimates, Proceeding of the Eight International Conference on Knowledge Discovery and Data Mining (KDD’02) doi:10.1145/775047. 775151

doi
[12]

A. Niculescu-Mizil, et al., Predicting good probabilities with supervised learning, in: Proceedings of the 22nd International Conference on Machine 34 Learning, Bonn, Germany, 2005, pp. 625–632. doi:10.1145/1102351. 1102430

2005 doi
[13]

Gulcehre, et al., On integrating a language model into neural machine translation, Comput

C. Gulcehre, et al., On integrating a language model into neural machine translation, Comput. Speech Lang. 45 (C) (2017) 137–148. doi:10.1016/ j.csl.2017.01.014

2017
[14]

Br¨ ummer, et al., On calibration of language recognition scores, in: Proc

N. Br¨ ummer, et al., On calibration of language recognition scores, in: Proc. of Odyssey, San Juan, Puerto Rico, 2006

2006
[15]

Bojarski, et al., End to end learning for self-driving cars

M. Bojarski, et al., End to end learning for self-driving cars
[16]

Lee, et al., Training conﬁdence-calibrated classiﬁers for detecting out- of-distribution samples, in: International Conference On Learning Repre- sentations, 2018

K. Lee, et al., Training conﬁdence-calibrated classiﬁers for detecting out- of-distribution samples, in: International Conference On Learning Repre- sentations, 2018

2018
[17]

M. H. deGroot, S. E. Fienberg, The comparison and evaluation of forecast- ers, The Statistician 32 (1983) 12–22

1983
[18]

Lakshminarayanan, et al., Simple and scalable predictive uncertainty estimation using deep ensembles, in: I

B. Lakshminarayanan, et al., Simple and scalable predictive uncertainty estimation using deep ensembles, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran Associates, ...

2017
[19]

Ramos, J

D. Ramos, J. Franco-Pedroso, A. Lozano-Diez, J. Gonzalez-Rodriguez, De- constructing cross-entropy for probabilistic binary classiﬁers, Entropy (3) (2018) 208. doi:10.3390/e20030208

2018 doi
[20]

Guo, et al., On calibration of modern neural networks, in: D

C. Guo, et al., On calibration of modern neural networks, in: D. Pre- cup, Y. W. Teh (Eds.), Proceedings of the 34th International Conference on Machine Learning, Vol. 70 of Proceedings of Machine Learning Re- search, PMLR, International Convention Centre, Sydney, Australia, 2...

2017
[21]

Kuleshov, et al., Accurate uncertainties for deep learning using cali- brated regression, in: ICML, Vol

V. Kuleshov, et al., Accurate uncertainties for deep learning using cali- brated regression, in: ICML, Vol. 80 of JMLR Workshop and Conference Proceedings, 2018, pp. 2801–2809

2018
[22]

Kumar, et al., Trainable calibration measures for neural networks from kernel mean embeddings, in: J

A. Kumar, et al., Trainable calibration measures for neural networks from kernel mean embeddings, in: J. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference on Machine Learning, Vol. 80 of Proceedings of Machine Learning Research, PMLR, 2018, pp. 2805–2814

2018
[23]

S. Seo, et al., Learning for single-shot conﬁdence calibration in deep neural networks through stochastic inferences, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 9022–9030. doi:10.1109/CVPR.2019.00924

2019
[24]

Kendall, et al., What uncertainties do we need in bayesian deep learning for computer vision?, in: I

A. Kendall, et al., What uncertainties do we need in bayesian deep learning for computer vision?, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Infor- mation Processing Systems 30, Curran Associates, Inc.,...

2017
[25]

Wu, et al., Fixing variational bayes: Deterministic variational inference for bayesian neural networks, in: International Conference On Learning Representations, 2019

A. Wu, et al., Fixing variational bayes: Deterministic variational inference for bayesian neural networks, in: International Conference On Learning Representations, 2019

2019
[26]

B. Zadrozny, et al., Obtaining calibrated probability estimates from deci- sion trees and naive bayesian classiﬁers, in: Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, Morgan Kauf- mann Publishers Inc., San Francisco, CA, USA, 2001, pp. 609–616

2001
[27]

J. C. Platt, Probabilistic outputs for support vector machines and com- parisons to regularized likelihood methods, in: ADVANCES IN LARGE MARGIN CLASSIFIERS, MIT Press, 1999, pp. 61–74

1999
[28]

M. P. Naeini, et al., Obtaining well calibrated probabilities using bayesian 36 binning, in: Proceedings of the Twenty-Ninth AAAI Conference on Artiﬁ- cial Intelligence, AAAI’15, AAAI Press, 2015, pp. 2901–2907

2015
[29]

Y. Gal, et al., Dropout as a bayesian approximation: Representing model uncertainty in deep learning, in: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, JMLR.org, 2016, pp. 1050–1059

2016
[30]

Pereyra, et al., Regularizing neural networks by penalizing conﬁdent output distributions

G. Pereyra, et al., Regularizing neural networks by penalizing conﬁdent output distributions
[31]

T. Chen, J. Navratil, V. Iyengar, K. Shanmugam, Conﬁdence scoring us- ing whitebox meta-models with linear classiﬁer probes, in: K. Chaudhuri, M. Sugiyama (Eds.), Proceedings of Machine Learning Research, Vol. 89 of Proceedings of Machine Learning Research, PMLR, 2019, pp. 146...

2019
[32]

DeVries, et al., Learning conﬁdence for out-of-distribution detection in neural networks

T. DeVries, et al., Learning conﬁdence for out-of-distribution detection in neural networks
[33]

Y. Gal, et al., Bayesian convolutional neural networks with bernoulli ap- proximate variational inference, in: International Conference On Learning Representations, Workshop track, 2016

2016
[34]

D. P. Kingma, et al., Variational dropout and the local reparameterization trick, in: C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, R. Garnett (Eds.), Advances in Neural Information Processing Systems 28, Curran Associates, Inc., 2015, pp. 2575–2583

2015
[35]

Louizos, et al., Multiplicative normalizing ﬂows for variational Bayesian neural networks, in: D

C. Louizos, et al., Multiplicative normalizing ﬂows for variational Bayesian neural networks, in: D. Precup, Y. W. Teh (Eds.), Proceedings of the 34th International Conference on Machine Learning, Vol. 70 of Proceedings of Machine Learning Research, PMLR, 2017, pp. 2218–2227

2017
[36]

Dinh, et al., Density estimation using real nvp, in: International Con- ference on Learning Representations, 2017

L. Dinh, et al., Density estimation using real nvp, in: International Con- ference on Learning Representations, 2017. 37

2017
[37]

D. J. Rezende, et al., Variational inference with normalizing ﬂows, in: Pro- ceedings of the 32Nd International Conference on Machine Learning - Vol- ume 37, ICML’15, JMLR.org, 2015, pp. 1530–1538

2015
[38]

L. Maaløe, et al., Auxiliary deep generative models, in: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, JMLR.org, 2016, pp. 1445–1454

2016
[39]

Zhang, et al., Variational measure preserving ﬂows, CoRR abs/1805.10377

Y. Zhang, et al., Variational measure preserving ﬂows, CoRR abs/1805.10377. arXiv:1805.10377

arXiv
[40]

R. M. Neal, MCMC using Hamiltonian dynamics, Handbook of Markov Chain Monte Carlo 54 (2010) 113–162

2010
[41]

C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), Springer-Verlag, Berlin, Heidelberg, 2006

2006
[42]

Gal, Uncertainty in deep learning, Ph.D

Y. Gal, Uncertainty in deep learning, Ph.D. thesis, University of Cambridge (2016)

2016
[43]

Snelson, et al., Sparse gaussian processes using pseudo-inputs, in: Y

E. Snelson, et al., Sparse gaussian processes using pseudo-inputs, in: Y. Weiss, B. Sch¨ olkopf, J. C. Platt (Eds.), Advances in Neural Information Processing Systems 18, MIT Press, 2006, pp. 1257–1264

2006
[44]

Havasi, et al., Inference in deep gaussian processes using stochastic gra- dient hamiltonian monte carlo, in: S

M. Havasi, et al., Inference in deep gaussian processes using stochastic gra- dient hamiltonian monte carlo, in: S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett (Eds.), Advances in Neural In- formation Processing Systems 31, Curran Associates, Inc...

2018
[45]

T. Chen, et al., Stochastic gradient hamiltonian monte carlo, in: Proceed- ings of the 31st International Conference on International Conference on Machine Learning - Volume 32, ICML’14, JMLR.org, 2014, pp. II–1683– II–1691. 38

2014
[46]

Betancourt, A conceptual introduction to hamiltonian monte carlo, arxiv:1701.02434 (2017)

M. Betancourt, A conceptual introduction to hamiltonian monte carlo, arxiv:1701.02434 (2017)

2017 arXiv
[47]

D. P. Kingma, et al., Auto-encoding variational bayes, in: International Conference on Learning Representations, 2014

2014
[48]

D. J. Rezende, et al., Stochastic backpropagation and approximate infer- ence in deep generative models, in: E. P. Xing, T. Jebara (Eds.), Proceed- ings of the 31st International Conference on Machine Learning, Vol. 32 of Proceedings of Machine Learning Research, PMLR, Bejing,...

2014
[49]

Wah, et al., The Caltech-UCSD Birds-200-2011 Dataset, Tech

C. Wah, et al., The Caltech-UCSD Birds-200-2011 Dataset, Tech. Rep. CNS-TR-2011-001, California Institute of Technology (2011)

2011
[50]

Krause, M

J. Krause, M. Stark, J. Deng, L. Fei-Fei, 3d object representations for ﬁne-grained categorization, in: 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013

2013
[51]

Krizhevsky, et al., Cifar-100 (canadian institute for advanced research)

A. Krizhevsky, et al., Cifar-100 (canadian institute for advanced research). URL http://www.cs.toronto.edu/~kriz/cifar.html
[52]

Krizhevsky, et al., Cifar-10 (canadian institute for advanced research)

A. Krizhevsky, et al., Cifar-10 (canadian institute for advanced research). URL http://www.cs.toronto.edu/~kriz/cifar.html
[53]

Y. o. Netzer, Reading digits in natural images with unsupervised feature learning
[54]

Cao, others., Vggface2: A dataset for recognising faces across pose and age, in: International Conference on Automatic Face and Gesture Recog- nition, 2018

Q. Cao, others., Vggface2: A dataset for recognising faces across pose and age, in: International Conference on Automatic Face and Gesture Recog- nition, 2018

2018
[55]

Eidinger, et al., Age and gender estimation of unﬁltered faces, Trans

E. Eidinger, et al., Age and gender estimation of unﬁltered faces, Trans. Info. For. Sec. 9 (12) (2014) 2170–2179.doi:10.1109/TIFS.2014.2359646. 39

2014
[56]

Simonyan, et al., Very deep convolutional networks for large-scale im- age recognition, in: International Conference On Learning Representations, 2015

K. Simonyan, et al., Very deep convolutional networks for large-scale im- age recognition, in: International Conference On Learning Representations, 2015

2015
[57]

He, et al., Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp

K. He, et al., Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–

2016
[58]

He, et al., Identity mappings in deep residual networks, in: ECCV, 2016

K. He, et al., Identity mappings in deep residual networks, in: ECCV, 2016

2016
[59]

Chen, et al., Dual path networks, in: I

Y. Chen, et al., Dual path networks, in: I. Guyon, U. V. Luxburg, S. Ben- gio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran Associates, Inc., 2017, pp. 4467–4475

2017
[60]

Xie, et al., Aggregated residual transformations for deep neural net- works, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 5987–5995

S. Xie, et al., Aggregated residual transformations for deep neural net- works, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 5987–5995

2017
[61]

Sandler, et al., Mobilenetv2: Inverted residuals and linear bottlenecks, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

M. Sandler, et al., Mobilenetv2: Inverted residuals and linear bottlenecks, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

2018
[62]

Hu, et al., Squeeze-and-excitation networks, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018) 7132–7141

J. Hu, et al., Squeeze-and-excitation networks, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018) 7132–7141

2018
[63]

D. P. Kingma, J. Ba, Adam: A method for stochastic optimization (2014)

2014
[64]

Goodfellow, et al., Explaining and harnessing adversarial examples, in: International Conference on Learning Representations, 2015

I. Goodfellow, et al., Explaining and harnessing adversarial examples, in: International Conference on Learning Representations, 2015

2015
[65]

Gal, et al., Concrete dropout, in: I

Y. Gal, et al., Concrete dropout, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran Associates, Inc., 2017, pp. 3581–3590. URL http://papers.nips.cc/paper/6949-concret...

2017
[66]

Tran, et al., Calibrating deep convolutional gaussian processes, in: K

G.-L. Tran, et al., Calibrating deep convolutional gaussian processes, in: K. Chaudhuri, M. Sugiyama (Eds.), Proceedings of Machine Learning Re- search, Vol. 89 of Proceedings of Machine Learning Research, PMLR, 2019, pp. 1554–1563

2019
[67]

Milios, othes, Dirichlet-based gaussian processes for large-scale cali- brated classiﬁcation, in: S

D. Milios, othes, Dirichlet-based gaussian processes for large-scale cali- brated classiﬁcation, in: S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett (Eds.), Advances in Neural Information Pro- cessing Systems 31, Curran Associates, Inc., 2018, pp....

2018
[68]

D. P. Kingma, et al., Improved variational inference with inverse autore- gressive ﬂow, in: D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, R. Gar- nett (Eds.), Advances in Neural Information Processing Systems 29, Curran Associates, Inc., 2016, pp. 4743–4751

2016
[69]

Huang, et al., Neural autoregressive ﬂows, in: ICML, 2018

C.-W. Huang, et al., Neural autoregressive ﬂows, in: ICML, 2018

2018
[70]

Globerson, A

Van Den Berg, et al., Sylvester normalizing ﬂows for variational inference, in: A. Globerson, A. Globerson, R. Silva (Eds.), 34th Conference on Uncer- tainty in Artiﬁcial Intelligence 2018, UAI 2018, 34th Conference on Uncer- tainty in Artiﬁcial Intelligence 2018, UAI 2018, As...

2018
[71]

F. V. Agakov, et al., An auxiliary variational method, in: Neural Informa- tion Processing, 11th International Conference, ICONIP 2004, Calcutta, India, November 22-25, 2004, Proceedings, 2004, pp. 561–566

2004
[72]

Ranganath, et al., Hierarchical variational models, in: M

R. Ranganath, et al., Hierarchical variational models, in: M. F. Balcan, K. Q. Weinberger (Eds.), Proceedings of The 33rd International Conference on Machine Learning, Vol. 48 of Proceedings of Machine Learning Research, PMLR, New York, New York, USA, 2016, pp. 324–333

2016
[73]

Cremer, et al., Inference suboptimality in variational autoencoders, in: J

C. Cremer, et al., Inference suboptimality in variational autoencoders, in: J. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference 41 on Machine Learning, Vol. 80 of Proceedings of Machine Learning Research, PMLR, Stockholmsm¨ assan, Stockholm Sweden, 2018, ...

2018
[74]

Shu, et al., Amortized inference regularization, in: S

R. Shu, et al., Amortized inference regularization, in: S. Bengio, H. Wal- lach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett (Eds.), Ad- vances in Neural Information Processing Systems 31, Curran Associates, Inc., 2018, pp. 4393–4402

2018
[75]

Kim, et al., Semi-amortized variational autoencoders, in: J

Y. Kim, et al., Semi-amortized variational autoencoders, in: J. Dy, A. Krause (Eds.), Proceedings of the 35th International Conference on Machine Learning, Vol. 80 of Proceedings of Machine Learning Research, PMLR, Stockholmsm¨ assan, Stockholm Sweden, 2018, pp. 2683–2692

2018
[76]

Knoblauch, J

J. Knoblauch, J. Jewson, T. Damoulas, Generalized variational inference: Three arguments for deriving new posteriors (2019). arXiv:1904.02063. 42

2019 arXiv
[778]

doi:10.1109/CVPR.2016.90

2016 doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Huang, et al., Densely connected convolutional networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 2261–2269

G. Huang, et al., Densely connected convolutional networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) 2261–2269

work page 2017

[2] [2]

Zagoruyko, et al., Wide residual networks, in: E

S. Zagoruyko, et al., Wide residual networks, in: E. R. H. Richard C. Wilson, W. A. P. Smith (Eds.), Proceedings of the British Machine Vision Conference (BMVC), BMVA Press, 2016, pp. 87.1–87.12. doi: 10.5244/C.30.87

doi:10.5244/c.30.87 2016

[3] [3]

Mikolov, et al., Eﬃcient estimation of word representations in vector space, in: International Conference on Learning Representations, 2013

T. Mikolov, et al., Eﬃcient estimation of word representations in vector space, in: International Conference on Learning Representations, 2013

work page 2013

[4] [4]

T. Mikolov, et al., Distributed representations of words and phrases and their compositionality, in: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, Curran Associates Inc., USA, 2013, pp. 3111–3119. 33

work page 2013

[5] [5]

Vaswani, et al., Attention is all you need, in: I

A. Vaswani, et al., Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran Associates, Inc., 2017, pp. 5998–6008

work page 2017

[6] [6]

Hinton, et al., Deep neural networks for acoustic modelling in speech recognition

G. Hinton, et al., Deep neural networks for acoustic modelling in speech recognition. the shared views of four research groups, IEEE Signal Process- ing Magazine 29 (6) (2012) 82–97. doi:10.1109/MSP.2012.2205597

arXiv 2012

[7] [7]

A. P. Dawid, The well-calibrated Bayesian, Journal of the American Sta- tistical Association 77 (379) (1982) 605–610

work page 1982

[8] [8]

Cohen, et al., Properties and beneﬁts of calibrated classiﬁers, in: Knowl- edge Discovery in Databases: PKDD 2004, Vol

I. Cohen, et al., Properties and beneﬁts of calibrated classiﬁers, in: Knowl- edge Discovery in Databases: PKDD 2004, Vol. 3202 of Lecture Notes in Computer Science, Springer, Heidelberg - Berlin, 2004

work page 2004