Pith. sign in

REVIEW 4 major objections 5 minor 72 references

Streamlining Prediction in Bayesian Deep Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that the posterior predictive distribution of a Bayesian neural network can be computed analytically in one forward pass by locally linearising activations and Gaussian-approximating each layer, matching or beating Monte…

desk verdict A useful single-pass recipe for BDL predictions, but the headline uncertainty gains are inflated by a validation-tuned variance scalar applied only to the proposed method. read the letter →

arxiv 2411.18425 v4 pith:D4X5GUXD submitted 2024-11-27 cs.LG

Rui Li , Marcus Klasson , Arno Solin , Martin Trapp This is my paper · ORCID
classification cs.LG MSC 68T0762F15
keywords BayesiandeeplearningposteriorpredictivedistributionlocallinearisationGaussianapproximationsingleforwardpassuncertaintyquantificationLaplacemean-fieldvariationalinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to make Bayesian deep learning predictions cheap and practical by replacing Monte Carlo sampling with an analytic single forward pass. The authors show that by locally linearising activation functions and applying local Gaussian approximations at linear layers, the posterior predictive distribution can be propagated layer-by-layer in closed form. They demonstrate that this streamlined prediction matches or improves on the predictive quality of sampling-based baselines for neural networks including MLPs, Vision Transformers, and GPT-2, while adding little computational overhead. The result matters because it separates the cost of estimating a posterior from the cost of making predictions, which is a bottleneck for real-world Bayesian deep learning.

What carries the argument

The central mechanism is the simultaneous use of local linearisation and local Gaussian approximation at every layer. For a pre-activation $h^{(m)} = W^{(m)} a^{(m-1)} + b^{(m)}$, the method assumes activations of the previous layer and parameters of the current layer are independent (Assumption 3.1) and that each product $a_i^{(m-1)} W_{ki}^{(m)}$ is Gaussian; this yields closed-form mean and covariance for $h^{(m)}$ (Eqs. 2-4). For a nonlinear activation $g$, a first-order Taylor expansion $g(h) \approx g(\mathbb{E}[h]) + J_g|_{h=\mathbb{E}[h]}(h - \mathbb{E}[h])$ propagates the Gaussian through the activation, giving $a \sim \mathcal{N}(g(\mathbb{E}[h]), J^\top \Sigma_h J)$. In attention layers, the query and key are treated deterministically at their means, while only the value projection carries the propagated uncertainty, keeping the path tractable.

What would settle it

Train an MLP whose weight posterior is made heavy-tailed (for instance, by increasing the posterior temperature), then compare the analytic predictive entropy from a single forward pass against a very large Monte Carlo estimate (e.g., 100,000 samples) on an out-of-distribution test set; if the analytic entropy is much lower or higher while sampling is stable, the Gaussian-independence approximation is the cause and the claim of calibration parity fails.

Watch

Extended reading notes

Core claim

The central claim is that the posterior predictive distribution of a Bayesian neural network admits an accurate and tractable approximation through a single deterministic forward pass. The method recursively computes the mean and covariance of activations at every layer: for a linear layer, the pre-activation distribution is obtained from the weight posterior and the previous activation distribution under a local Gaussian assumption; for a nonlinear activation, a first-order Taylor expansion at the pre-activation mean maps the Gaussian distribution to a Gaussian. For classification, a probit approximation yields the predictive class probabilities. Empirical results show that this analytic predictive achieves lower or equal negative log predictive density and expected calibration error compared to Monte Carlo sampling and to the global linearised Laplace approximation, across regression and classification tasks, with near-zero runtime overhead relative to a standard forward pass.

Load-bearing premise

The load-bearing premise is that the propagated uncertainty remains accurate under the twin assumptions that each weight-activation product is Gaussian and that activations and weights are independent, which can fail when activation variances are large or the posterior is heavy-tailed.

Editorial extensions

If this is right

  • Predictions from a Bayesian neural network no longer require sampling: a single forward pass produces both the predictive mean and uncertainty, which is a direct consequence of the closed-form propagation.
  • The method applies to modern architectures including Vision Transformers and GPT-2, and to both Laplace and mean-field variational posteriors, making it compatible with large pre-trained models.
  • The analytic predictive distribution is better calibrated than Monte Carlo sampling in most experiments, with lower NLPD and ECE while maintaining equal accuracy.
  • Because the input distribution can be propagated analytically, sensitivity analysis with respect to inputs becomes tractable in a single pass, as demonstrated by the pixel-sensitivity maps in Sec. 4.3.
  • The method's cost scales with the number of parameters and the posterior covariance structure, allowing further speedups through diagonal or Kronecker-factored approximations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This propagation scheme implies that the predictive uncertainty can be decomposed into contributions from weight uncertainty, bias uncertainty, and input (aleatoric) uncertainty at each layer, which the paper does not explicitly use but would be a natural extension for interpretable uncertainty attribution.
  • If the independence assumption between activations and weights degrades in deeper networks with correlated activations, the analytic covariance will underestimate the true uncertainty; a testable extension would be to compare against sampling on networks trained with increasing depth and width.
  • The method opens the possibility of real-time Bayesian prediction in latency-sensitive applications such as autonomous driving or medical diagnostics, where the cost of many forward passes is currently prohibitive.
  • A similar layer-wise moment propagation could be applied to other posterior families beyond Gaussians (e.g., Student-t or alpha-stable), provided the local linearisation is adjusted accordingly, since the paper already exploits stability under linear maps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a single-forward-pass method for approximate posterior predictive inference in Bayesian deep learning. The method propagates mean and covariance through locally linearised activation functions and local Gaussian approximations at linear layers, supporting full, diagonal, and KFAC posterior covariance structures. It is applied to MLPs, Vision Transformers, and GPT-2, and evaluated on UCI regression, MNIST-like classification, and several large-scale classification and language-understanding tasks. The central claim is that this analytic propagation achieves better or on-par NLPD and ECE compared to Monte Carlo sampling and the global linearised Laplace approximation, with little or no runtime overhead.

Significance. If the empirical comparisons are valid, the contribution is practically useful: it removes the need for MC sampling at prediction time, provides a closed-form route to OOD detection and input-sensitivity analysis, and is supported by an open-source library and detailed derivations. The derivation of moment propagation, including the KFAC covariance block retrieval, is technically sound under the stated assumptions. However, the paper's headline uncertainty comparisons are confounded by a validation-set-fitted variance scaling factor applied only to the proposed method, so the significance of the empirical claims is currently not established.

major comments (4)
  1. [Sec. 4, Posterior approximations paragraph; Sec. 5 Limitations] The predictive variance for 'Ours' is rescaled by a scalar fitted by minimising NLPD on a validation set, while the MC sampling and GLM baselines receive no analogous calibration. Since NLPD and ECE both depend directly on predictive variance, the comparisons in Tables 1, 2, and 4 are not controlled for this free parameter; the Limitations section acknowledges the fitting but does not quantify its effect. Please provide an ablation without the scaling factor, report the fitted scaling values, or apply an equivalent validation-based calibration to the baselines, so that the reported gains can be attributed to the propagation method itself.
  2. [Table 5] On BoolQ, LA Ours has NLPD 0.665 vs 0.656 for LA Sampling and 0.633 for LA GLM; on MRPC, LA Ours has NLPD 0.706 vs 0.658 and 0.609, and ACC drops to 0.416 vs 0.665 and 0.720 for the baselines. The text states that 'our method systematically outperforms sampling,' which is contradicted by these rows. Please revise this claim and discuss the failure modes on these language-understanding tasks.
  3. [Table 1, CAC row] On the CAC regression data set, LA Ours has NLPD 2.662±1.096, which is substantially worse than LA Sampling (1.848±0.055) and GLM (1.281±0.069), with a large standard error. The general claim in Sec. 4.1 that 'our method achieves better NLPD in general' and the Table 1 caption 'better or matching performance' are not supported by this row. Please address this failure case or qualify the claim accordingly.
  4. [Assumption 3.1 and Assumption A1; Sec. A.2] The propagation relies on Assumption 3.1 (independence between activations and parameters) and Assumption A1 (local Gaussianity of each product a_i W_ki). The appendix itself shows the approximation fails to capture skewness for large-magnitude weights, and the manuscript provides no formal error bound or practical diagnostic for when activation variances are large or the posterior is heavy-tailed. Please add quantitative diagnostics, such as comparing propagated moments against MC samples on the actual posteriors used for the ViT/GPT-2 experiments, or a bounded-error statement.
minor comments (5)
  1. [Sec. 3.1, first paragraph] The sentence 'the covariance between the kth and the jth hidden unit' uses 'jth' but Eq. (3) defines the covariance between the kth and lth hidden unit; please correct the index notation.
  2. [Sec. 3.2, Eq. (7)] Treating the query and key as deterministic is a strong approximation that ignores their posterior covariance; the text gives only a brief 'computational reasons' justification. Please state clearly that this can underestimate the attention output variance and discuss the potential impact on uncertainty calibration.
  3. [Sec. 3.3, Eq. (10)] The approximation (B⊗A+λ²I)^{-1} ≈ (B+λI)^{-1}⊗(A+λI)^{-1} is stated without a source or empirical check; please add a reference or a small numerical verification of its accuracy for the models used.
  4. [Tables 2 and 4] ECE values are reported without standard errors or confidence intervals; please state whether these are averages over folds/seeds and how they were aggregated.
  5. [Acknowledgments] The acknowledgment mentions 'Jonas Vestergaard for finding a bug in our code' without indicating whether the bug affected any reported results; please clarify in the manuscript whether the experiments were rerun after the fix.

Circularity Check

1 steps flagged · score 4.0 of 10

Derivation is self-contained; empirical uncertainty comparisons are partially driven by a validation-tuned variance scaling applied only to Ours.

  1. fitted input called prediction [Section 4, 'Posterior approximations' paragraph (p. 6-7); see also Limitations (Sec. 5)]
    "For our method, we fit an additional scaling factor on the predictive variance by minimising the NLPD on a validation set, similar to the pseudo-count used in Ritter et al. (2018)."

    NLPD and ECE are the headline comparison metrics (Tables 1, 2, 4, 5), and ECE is a direct function of the predictive-variance scale. The scaling factor is selected by minimizing NLPD on a validation set, then the same NLPD/ECE values are reported on test as evidence that the method 'achieves better or on-par NLPD and ECE.' MC-sampling and GLM baselines are not given this validation-tuned variance calibration, so the comparison is not controlled: the variance entering the reported predictive distribution is not a pure analytic output of the Sec. 3 derivation but partly a fitted input, tuned to the exact metric being compared. The Limitations confirm the grid-search fitting of this scalar.

full rationale

The core propagation formulas (Eqs. 2-6) are derived from the stated Assumption 3.1 and from local linearisation, with an explicit error analysis in Sec. A.2; no equation reduces to its own input. The KFAC, attention, and residual-connection treatments are stated approximations, not circular steps. Self-citations (Baumann et al., Meronen et al., Scannell et al.) appear only as related work and are not load-bearing. The one substantive circularity-adjacent issue is the validation-set-fitted variance scaling in the experimental protocol: it makes the reported NLPD/ECE gains partially a fitted input rather than a pure prediction of the propagation method, and it is applied asymmetrically relative to the baselines. This compromises the headline empirical comparison but does not invalidate the derivation itself, so the overall circularity is moderate rather than complete.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The method rests on several standard approximations (Gaussian stability, local linearization) and a few ad hoc choices specific to this paper (deterministic Q/K, independence in residuals, KFAC inverse approximation). The only fitted parameter is the predictive variance scaling factor.

free parameters (1)
  • predictive variance scaling factor (pseudo-count) = not reported (grid search on validation set)
    Fit by minimizing NLPD on a validation set; used for all 'Ours' uncertainty results. Without it, NLPD values may differ.
assumptions (5)
  • domain assumption Activations of previous layer and parameters of the m-th layer are independent (Assumption 3.1).
    Needed to factor moments in Eqs. (2)-(4); violates when activations depend on parameters, acknowledged in Limitations.
  • ad hoc to paper Each product a_i W_ki is Gaussian (Assumption A1).
    Allows Gaussian pre-activation; product of non-Gaussian variables is not Gaussian; error estimated empirically in Fig. 5.
  • ad hoc to paper For attention, query and key are treated as deterministic (Eq. 7).
    Avoids softmax distribution propagation, but discards uncertainty in Q and K.
  • ad hoc to paper Residual connections assume independence.
    Stated in Sec. 3.2; may lose correlations across a block.
  • ad hoc to paper KFAC covariance approximation (B⊗A+λ²I)^{-1} ≈ (B+λI)^{-1}⊗(A+λI)^{-1} (Eq. 10).
    Needed to maintain Kronecker structure; approximation quality not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Streamlining Prediction in Bayesian Deep Learning." pith.science (2026). https://pith.science/paper/D4X5GUXD

@misc{pith2026241118425,
  author       = {Pith},
  title        = {Pith review of: Streamlining Prediction in Bayesian Deep Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D4X5GUXD}},
  note         = {Machine review of arXiv:2411.18425}
}
read the original abstract

The rising interest in Bayesian deep learning (BDL) has led to a plethora of methods for estimating the posterior distribution. However, efficient computation of inferences, such as predictions, has been largely overlooked with Monte Carlo integration remaining the standard. In this work we examine streamlining prediction in BDL through a single forward pass without sampling. For this we use local linearisation on activation functions and local Gaussian approximations at linear layers. Thus allowing us to analytically compute an approximation to the posterior predictive distribution. We showcase our approach for both MLP and transformers, such as ViT and GPT-2, and assess its performance on regression and classification tasks. Open-source library: https://github.com/AaltoML/SUQ

Figures

Figures reproduced from arXiv: 2411.18425 by the authors.

Figure 1
Figure 1. Our streamlined approach allows for practical outlier detection and sensitivity analysis. Locally linearising the network function with local Gaussian approximations enables many relevant inference tasks to be solved analytically, helping render BDL a practical tool for downstream tasks. 1 arXiv:2411.18425v4 [cs.LG] 22 Jul 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our approach for different network architectures. In MLPs, we can directly [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Kernel density plots over the predictive entropy from a ViT network finetuned on CIFAR-10 [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Pixel sensitivity maps of an MLP trained on a subset of MNIST digits (classes [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Comparison between Monte-Carlo estimates [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: To retrieve the highlighted submatrix Cov[W[1, :],W[2, :]] of the covariance for W ∈ R 2×3 , we identify the Kronecker blocks that contain the covariance of interest (II, III, V, and VI), explicate those blocks in memory, and then retrieve the relevant submatrix. A.5 D…
Figure 7
Figure 7. Figure 7: NLPD and ACC for MNIST-trained MLP on rotated versions of the MNIST test set. The [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: Kernel density plots over the predictive entropy from an MLP trained on FMNIST (blue, in [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]
Figure 9
Figure 9. Figure 9: Pixel sensitivity of MLP classifiers trained on binary classification tasks ( [PITH_FULL_IMAGE:figures/full_fig_p029_9.png]
Figure 10
Figure 10. Figure 10: Effects of the number of MC samples on performance for LA and MFVI on CIFAR-10 [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]
Figure 11
Figure 11. Figure 11: Effects of the number of MC samples on performance for LA and MFVI on FMNIST. In [PITH_FULL_IMAGE:figures/full_fig_p030_11.png]
Figure 12
Figure 12. Figure 12: Effects of the number of MC samples on performance for LA and MFVI on regression [PITH_FULL_IMAGE:figures/full_fig_p031_12.png]
Figure 13
Figure 13. Figure 13: Estimated divergence from a locally linear function as a function of [PITH_FULL_IMAGE:figures/full_fig_p031_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 61 canonical work pages

  1. [1]

    Post-hoc probabilistic vision-language models

    Anton Baumann, Rui Li, Marcus Klasson, Santeri Mentu, Shyamgopal Karthik, Zeynep Akata, Arno Solin, and Martin Trapp. Post-hoc probabilistic vision-language models. arXiv preprint arXiv:2412.06014, 2024

    arXiv 2024

  2. [2]

    The need for uncertainty quantification in machine-assisted medical decision making

    Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantification in machine-assisted medical decision making. Nature Machine Intelligence, 1 0 (1): 0 20--23, 2019

    2019

  3. [3]

    Variational inference: A review for statisticians

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112 0 (518): 0 859--877, 2017

    2017

  4. [4]

    Weight uncertainty in neural network

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In Proceedings of the 32th International Conference on Machine Learning (ICML), volume 37 of Proceedings of Machine Learning Research, pp.\ 1613--1622. PMLR, 2015

    work page 2015

  5. [5]

    Sample average approximation for black-box variational inference

    Javier Burroni, Justin Domke, and Daniel Sheldon. Sample average approximation for black-box variational inference. In Proceedings of the 40th Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, 2024

    work page 2024

  6. [6]

    Remote sensing image scene classification: Benchmark and state of the art

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105 0 (10): 0 1865--1883, 2017

    2017

  7. [7]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 3606--3613. IEEE Computer Society, 2014

    work page 2014

  8. [8]

    Wide mean-field bayesian neural networks ignore the data

    Beau Coker, Wessel P Bruinsma, David R Burt, Weiwei Pan, and Finale Doshi-Velez. Wide mean-field bayesian neural networks ignore the data. In Proceedings of the twenty fifth International Conference on Artificial Intelligence and Statistics (AISTATS), volume 131 of Proceedings of Machine Learning Research, pp.\ 5276--5333. PMLR, 2022

    work page 2022

Show all 72 references
  1. [9]

    Kronecker-factored approximate curvature (kfac) from scratch

    Felix Dangel, Bálint Mucsányi, Tobias Weber, and Runa Eschenhagen. Kronecker-factored approximate curvature (kfac) from scratch. arXiv, 2025. URL https://github.com/f-dangel/kfac-tutorial

    2025

  2. [10]

    Laplace redux -- effortless B ayesian deep learning

    Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux -- effortless B ayesian deep learning. In Advances in Neural Information Processing Systems 34 (NeurIPS), volume 34, pp.\ 20089--20103. Curran Associates, ...

    2021

  3. [11]

    B ayesian deep learning via subnetwork inference

    Erik Daxberger, Eric Nalisnick, James U Allingham, Javier Antor \'a n, and Jos \'e Miguel Hern \'a ndez-Lobato. B ayesian deep learning via subnetwork inference. In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machin...

    2021

  4. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 248--255. IEEE Computer Society, 2009

    2009

  5. [13]

    Efficient parametric approximations of neural network function space distance

    Nikita Dhawan, Sicong Huang, Juhan Bae, and Roger Baker Grosse. Efficient parametric approximations of neural network function space distance. In Proceedings of the 40th International Conference on Machine Learning (ICML) , Proceedings of Machine Learning Research, pp.\ 7795--...

    2023

  6. [14]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  7. [15]

    Mixtures of L apkace approximations for improved post-hoc uncertainty in deep learning

    Runa Eschenhagen, Erik Daxberger, Philipp Hennig, and Agustinus Kristiadi. Mixtures of L apkace approximations for improved post-hoc uncertainty in deep learning. In NeurIPS workshop on B ayesian deep learning , 2021

    2021

  8. [16]

    On the expressiveness of approximate inference in B ayesian neural networks

    Andrew Foong, David Burt, Yingzhen Li, and Richard Turner. On the expressiveness of approximate inference in B ayesian neural networks. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 15897--15908. Curran Associates, Inc., 2020

    2020

  9. [17]

    B ayesian neural network priors revisited

    Vincent Fortuin, Adri \`a Garriga-Alonso, Sebastian W Ober, Florian Wenzel, Gunnar R \"a tsch, Richard E Turner, Mark van der Wilk, and Laurence Aitchison. B ayesian neural network priors revisited. In International Conference on Learning Representations (ICLR), 2021

    2021

  10. [18]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33th International Conference on Machine Learning (ICML), volume 48 of Proceedings of Machine Learning Research, pp.\ 1050--1059. PMLR, 2016

    2016

  11. [19]

    Deep bayesian active learning with image data

    Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, pp.\ 1183--1192. PMLR, 2017

    2017

  12. [20]

    Yoav Gelberg, Tycho F. A. van der Ouderaa, Mark van der Wilk, and Yarin Gal. Variational inference failures under model symmetries: Permutation invariant posteriors for B ayesian neural networks. In ICML 2024 Workshop on Geometry-grounded Representation Learning and Generative...

    2024

  13. [21]

    Black box variational inference with a deterministic objective: Faster, more accurate, and even more black box

    Ryan Giordano, Martin Ingram, and Tamara Broderick. Black box variational inference with a deterministic objective: Faster, more accurate, and even more black box. Journal of Machine Learning Research, 25 0 (18): 0 1--39, 2024

    2024

  14. [22]

    Tractable approximate G aussian inference for B ayesian neural networks

    James-A Goulet, Luong Ha Nguyen, and Saeid Amiri. Tractable approximate G aussian inference for B ayesian neural networks. Journal of Machine Learning Research, 22 0 (251): 0 1--23, 2021

    2021

  15. [23]

    Training independent subnetworks for robust prediction

    Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew Mingbo Dai, and Dustin Tran. Training independent subnetworks for robust prediction. In International Conference on Learning Representations (ICLR), 2021

    2021

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 770--778. IEEE Computer Society, 2016

    2016

  17. [25]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Kevin Zhao, Sebastian Basart, Jacob Steinhardt, and Dawn Song. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 8340--8349. IEEE , 2021

    2021

  18. [26]

    Scalable marginal likelihood estimation for model selection in deep learning

    Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar R \"a tsch, and Khan Mohammad Emtiyaz. Scalable marginal likelihood estimation for model selection in deep learning. In Proceedings of the 38th International Conference on Machine Learning (ICML), Proceedings of Machine ...

    2021

  19. [27]

    Improving predictions of B ayesian neural nets via local linearization

    Alexander Immer, Maciej Korzepa, and Matthias Bauer. Improving predictions of B ayesian neural nets via local linearization. In Proceedings of the twenty forth International Conference on Artificial Intelligence and Statistics (AISTATS), volume 130 of Proceedings of Machine Le...

    2021

  20. [28]

    Towards scalable B ayesian transformers: Investigating stochastic subset selection for nlp

    Peter JT Kampen, Gustav RS Als, and Michael Riis Andersen. Towards scalable B ayesian transformers: Investigating stochastic subset selection for nlp. In Proceedings of the 40th Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, 2024

    2024

  21. [29]

    From moments of sum to moments of product

    Raymond Kan. From moments of sum to moments of product. Journal of Multivariate Analysis, 99 0 (3): 0 542--554, 2008

    2008

  22. [30]

    The UCI machine learning repository, 2023

    Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. The UCI machine learning repository, 2023. URL : https://archive.ics.uci.edu

    2023

  23. [31]

    Being bayesian, even just a bit, fixes overconfidence in relu networks

    Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just a bit, fixes overconfidence in relu networks. In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pp.\ 5436--54...

    2020

  24. [32]

    Promises and pitfalls of the linearized L apkace in B ayesian optimization

    Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, and Vincent Fortuin. Promises and pitfalls of the linearized L apkace in B ayesian optimization. In Fifth Symposium on Advances in Approximate B ayesian Inference , 2023

    2023

  25. [33]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Toronto, ON, Canada, 2009

    2009

  26. [34]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems 30 (NeurIPS), volume 30, pp.\ 6402--6413. Curran Associates, Inc., 2017

    2017

  27. [35]

    Gradient-based learning applied to document recognition

    Yann LeCun, L \'e on Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

    1998

  28. [36]

    Soft: Softmax-free transformer with linear complexity

    Jiachen Lu, Jinghan Yao, Junge Zhang, Xiatian Zhu, Hang Xu, Weiguo Gao, Chunjing Xu, Tao Xiang, and Li Zhang. Soft: Softmax-free transformer with linear complexity. In Advances in Neural Information Processing Systems 34 (NeurIPS), pp.\ 21297--21309. Curran Associates, Inc., 2021

    2021

  29. [37]

    Information-based objective functions for active data selection

    David JC MacKay. Information-based objective functions for active data selection. Neural Computation, 4 0 (4): 0 590--604, 1992 a

    1992

  30. [38]

    B ayesian interpolation

    David JC MacKay. B ayesian interpolation. Neural computation, 4 0 (3): 0 415--447, 1992 b

    1992

  31. [39]

    B ayesian methods for backpropagation networks

    David JC MacKay. B ayesian methods for backpropagation networks. In Models of Neural Networks III : A ssociation, Generalization, and Representation , pp.\ 211--254. Springer, 1996

    1996

  32. [40]

    Maddox, Pavel Izmailov, Timur Garipov, Dmitry P

    Wesley J. Maddox, Pavel Izmailov, Timur Garipov, Dmitry P. Vetrov, and Andrew Gordon Wilson. A simple baseline for B ayesian uncertainty in deep learning. In Advances in Neural Information Processing Systems 32 (NeurIPS), pp.\ 13132--13143. Curran Associates, Inc., 2019

    2019

  33. [41]

    Optimizing neural networks with K ronecker-factored approximate curvature

    James Martens and Roger Grosse. Optimizing neural networks with K ronecker-factored approximate curvature. In Proceedings of the 32nd International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, pp.\ 2408--2417. PMLR, 2015

    2015

  34. [42]

    Periodic activation functions induce stationarity

    Lassi Meronen, Martin Trapp, and Arno Solin. Periodic activation functions induce stationarity. In Advances in Neural Information Processing Systems 34 (NeurIPS), pp.\ 1673--1685. Curran Associates, Inc., 2021

    2021

  35. [43]

    Fixing overconfidence in dynamic neural networks

    Lassi Meronen, Martin Trapp, Andrea Pilzer, Le Yang, and Arno Solin. Fixing overconfidence in dynamic neural networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp.\ 2680--2690, 2024

    2024

  36. [44]

    Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control

    Rhiannon Michelmore, Matthew Wicker, Luca Laurenti, Luca Cardelli, Yarin Gal, and Marta Kwiatkowska. Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. In IEEE International Conference on Robotics and Automation (ICRA), pp.\ 7344--...

    2020

  37. [45]

    On the distribution of the product of correlated normal random variables

    Saralees Nadarajah and Tibor K Pog \'a ny. On the distribution of the product of correlated normal random variables. Comptes Rendus. Math \'e matique , 354 0 (2): 0 201--204, 2016

    2016

  38. [46]

    On priors for B ayesian neural networks

    Eric Thomas Nalisnick. On priors for B ayesian neural networks . University of California, Irvine, 2018

    2018

  39. [47]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, 2011

    2011

  40. [48]

    Theodore Papamarkou, Maria Skoularidou, Konstantina Palla, Laurence Aitchison, Julyan Arbel, David B. Dunson, Maurizio Filippone, Vincent Fortuin, Philipp Hennig, Jos \' e Miguel Hern \' a ndez - Lobato, Aliaksandr Hubin, Alexander Immer, Theofanis Karaletsos, Mohammad Emtiyaz...

    2024

  41. [49]

    Uncertainty quantification via stable distribution propagation

    Felix Petersen, Aashwin Ananda Mishra, Hilde Kuehne, Christian Borgelt, Oliver Deussen, and Mikhail Yurochkin. Uncertainty quantification via stable distribution propagation. In International Conference on Learning Representations (ICLR), 2024

    2024

  42. [50]

    Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

    Apostolos F Psaros, Xuhui Meng, Zongren Zou, Ling Guo, and George Em Karniadakis. Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons. Journal of Computational Physics, 477: 0 111902, 2023

    2023

  43. [51]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI blog, 2019

    2019

  44. [52]

    A scalable L aplace approximation for neural networks

    Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable L aplace approximation for neural networks. In International Conference on Learning Representations (ICLR), 2018

    2018

  45. [53]

    B ayesian Filtering and Smoothing

    Simo S \"a rkk \"a and Lennart Svensson. B ayesian Filtering and Smoothing . Cambridge University Press, 2023

    2023

  46. [54]

    Function-space parameterization of neural networks for sequential learning

    Aidan Scannell, Riccardo Mereu, Paul Edmund Chang, Ella Tamir, Joni Pajarinen, and Arno Solin. Function-space parameterization of neural networks for sequential learning. In International Conference on Learning Representations (ICLR), 2024

    2024

  47. [55]

    Variational learning is effective for large deep networks

    Yuesong Shen, Nico Daheim, Bai Cong, Peter Nickl, Gian Maria Marconi, Bazan Clement Emile Marcel Raoul, Rio Yokota, Iryna Gurevych, Daniel Cremers, Mohammad Emtiyaz Khan, and Thomas Möllenhoff. Variational learning is effective for large deep networks. In Proceedings of the 41...

    2024

  48. [56]

    Prediction-oriented bayesian active learning

    Freddie Bickford Smith, Andreas Kirsch, Sebastian Farquhar, Yarin Gal, Adam Foster, and Tom Rainforth. Prediction-oriented bayesian active learning. In Proceedings of the twenty sixth International Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of ...

    2023

  49. [57]

    All you need is a good functional prior for B ayesian deep learning

    Ba-Hien Tran, Simone Rossi, Dimitrios Milios, and Maurizio Filippone. All you need is a good functional prior for B ayesian deep learning. Journal of Machine Learning Research, 23 0 (74): 0 1--56, 2022

    2022

  50. [58]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30 (NeurIPS). Curran Associates, Inc., 2017

    2017

  51. [59]

    High-dimensional G aussian sampling: a review and a unifying approach based on a stochastic proximal point algorithm

    Maxime Vono, Nicolas Dobigeon, and Pierre Chainais. High-dimensional G aussian sampling: a review and a unifying approach based on a stochastic proximal point algorithm. SIAM Review, 64 0 (1): 0 3--56, 2022

    2022

  52. [60]

    Superglue: A stickier benchmark for general-purpose language understanding systems

    Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems 32 (NeurIPS), pp.\ ...

    2019

  53. [61]

    Glue: A multi-task benchmark and analysis platform for natural language understanding

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations (ICLR), 2019 b

    2019

  54. [62]

    Florian Wenzel, Kevin Roth, Bastiaan Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the B ayes posterior in deep neural networks really? In Proceedings of the 37th International Conference...

    2020

  55. [63]

    The case for B ayesian deep learning

    Andrew Gordon Wilson. The case for B ayesian deep learning. arXiv preprint arXiv:2001.10995, 2020

    2001 arXiv

  56. [64]

    B ayesian deep learning and a probabilistic perspective of generalization

    Andrew Gordon Wilson and Pavel Izmailov. B ayesian deep learning and a probabilistic perspective of generalization. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 4697--4708. Curran Associates, Inc., 2020

    2020

  57. [65]

    Huggingface's transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R \' e mi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  58. [66]

    Gaussian pre-activations in neural networks: Myth or reality? arXiv preprint arXiv:2205.12379, 2022

    Pierre Wolinski and Julyan Arbel. Gaussian pre-activations in neural networks: Myth or reality? arXiv preprint arXiv:2205.12379, 2022

    2022 arXiv

  59. [67]

    Turner, Jos \' e Miguel Hern \' a ndez - Lobato, and Alexander L

    Anqi Wu, Sebastian Nowozin, Edward Meeds, Richard E. Turner, Jos \' e Miguel Hern \' a ndez - Lobato, and Alexander L. Gaunt. Deterministic variational inference for robust B ayesian neural networks. In International Conference on Learning Representations (ICLR), 2019

    2019

  60. [68]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms

    Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017

    2017 arXiv

  61. [69]

    B ayesian low-rank adaptation for large language models

    Adam X Yang, Maxime Robeyns, Xi Wang, and Laurence Aitchison. B ayesian low-rank adaptation for large language models. In International Conference on Learning Representations (ICLR), 2024

    2024

  62. [70]

    Rubin, and Holger R

    Jiancheng Yang, Rui Shi, Donglai Wei, Zeju Liu, Lin Zhao, Bilian Ke, Ziyang Shi, Yunzhu Li, Xiaoyang Hu, Yang Gao, Ye Xu, Daniel L. Rubin, and Holger R. Roth. Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10 0 ...

    2023 doi

  63. [71]

    u tepage, Hedvig Kjellstr \

    Cheng Zhang, Judith B \"u tepage, Hedvig Kjellstr \"o m, and Stephan Mandt. Advances in variational inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41 0 (8): 0 2008--2026, 2018

    2008

  64. [72]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.