Pith. sign in

REVIEW 4 major objections 6 minor 63 references

Stochastic Weight Sharing for Bayesian Neural Networks

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a mean-field Bayesian neural network's millions of per-weight Gaussian distributions can be replaced by a few thousand shared two-dimensional Gaussian components, and that this preserves accuracy and uncertainty…

desk verdict Genuinely new representation, but the 1000x compression claim falls apart once you count the per-weight cluster index; the core idea still deserves a careful look. read the letter →

arxiv 2505.17856 v1 pith:NHYG3FB6 submitted 2025-05-23 cs.LG cs.AI

Moule Lin , Shuhao Guan , Weipeng Jing , Goetz Botterweck , Andrea Patane This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0762F15
keywords BayesianneuralnetworksvariationalinferenceweightsharingGaussianmixturemodelcompressionquantizationuncertaintyquantificationWassersteindistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the millions of per-weight Gaussian distributions in a mean-field Bayesian neural network can be replaced by a few thousand two-dimensional Gaussian components learned on the mean-variance plane, and that sampling weights from this mixture during training preserves accuracy and uncertainty almost as well as the full network. If true, Bayesian uncertainty quantification becomes practical for very large models: the authors report parameter reductions of three to four orders of magnitude (about 99 percent), a 75 percent smaller model footprint, and successful training of ResNet-101 and Vision Transformer on ImageNet-1k with accuracy within about 2 percent of uncompressed baselines. The paper's central proposal is that stochastic weight sharing is a legitimate way to do Bayesian inference at scale, not merely a compression trick.

What carries the argument

The load-bearing object is the two-dimensional Gaussian Mixture Model learned on the weight distributions' means and standard deviations, $p((\mu,\sigma)) = \sum_{k=1}^{K}\pi_k\,\mathcal{N}((\mu,\sigma)\mid \mu_k,\Sigma_k)$, with each 2D Gaussian acting as a shared stochastic weight for the inlier set. The argument rides on three supporting devices: the outlier/inlier split by mean threshold and top-1-percent gradient; the Wasserstein-2 distance $W_2^2 = \|\mu_i-\mu_j\|_2^2 + \mathrm{Tr}(\Sigma_i+\Sigma_j-2(\Sigma_i^{1/2}\Sigma_j\Sigma_i^{1/2})^{1/2})$ with the merging equations for $\mu_{\mathrm{merged}}$ and $\Sigma_{\mathrm{merged}}$; and $\alpha$-blending, which samples a weight from several nearby components when its Mahalanobis distance exceeds the 95th percentile of $\chi^2_2$. Together these turn the ELBO into an upper bound (Eq. 10) whose minimization the paper shows guarantees improvement of the original variational objective.

What would settle it

Train the same architecture as a full mean-field BNN and as 2DGBNN from the same random initialization (without deterministic pretraining) on ImageNet-scale data, and compare log-loss and expected calibration error on a held-out set and on shifted inputs such as ImageNet-C. If the compressed model's NLL or ECE exceeds the full model's by more than the margins reported here, or if the 99 percent compression disappears without the deterministic initialization, the claim that the GMM represents the posterior is falsified.

Watch

Extended reading notes

Core claim

The central claim is that a mean-field BNN's posterior can be represented by a Gaussian Mixture Model over the $(\mu, \sigma)$ coordinates of its weights, so that each cluster component acts as a shared stochastic weight. Inliers (weights whose mean magnitude is below a threshold and whose gradients are not in the top 1 percent) are clustered with a mini-batch GMM; close components are merged using the Wasserstein-2 distance; weights near cluster boundaries are sampled by $\alpha$-blending among several components; outliers keep their own full distributions. Final training optimizes a variational lower bound that splits into a GMM term and an outliers term. The paper reports that this reduces stored parameters by three to four orders of magnitude (about 99 percent) while keeping accuracy within roughly 2 percent and NLL/ECE comparable to uncompressed state-of-the-art BNNs, and that the reduced representation enables BNN training of ResNet-101 and Vision Transformer.

Load-bearing premise

A few thousand two-dimensional Gaussian blobs, fitted once mostly to weights of a pretrained (often deterministic) network, represent the posterior of millions of inlier weights accurately enough that sampling from the mixture during final training keeps predictions and uncertainty close to the full mean-field BNN; the paper gives no approximation error bound.

Editorial extensions

If this is right

  • Bayesian uncertainty estimates become available for models like ResNet-101 and Vision Transformer, where standard mean-field variational inference is prohibitive.
  • Stored parameters fall to about one percent of the original count, putting BNNs in range for edge and embedded deployment.
  • Training time on ResNet-18/CIFAR-10 drops from about 5–5.5 hours to roughly 1.5 hours total, and inference takes 151.9 seconds versus 882.4 seconds for 1024 images.
  • Accuracy stays within about 2 percent of uncompressed baselines on ImageNet-1k, with comparable NLL and ECE, so the compression does not sacrifice all uncertainty quality.
  • On CIFAR-10 and MNIST, 2DGBNN matches int8-quantized BNNs in accuracy while giving lower calibration error.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the GMM genuinely captures the posterior, the effective cost of a BNN could drop further by quantizing the component parameters and the per-weight cluster index, which the paper notes currently dominates storage on small networks.
  • The large-scale experiments rely on deterministic pretrained networks for initialization, so a fully Bayesian training run from random initialization is not yet demonstrated; a natural test is whether the compression ratio and calibration survive that change.
  • The method frames weight sharing as stochastic vector quantization in $(\mu,\sigma)$ space; one extension would be to structured posteriors such as rank-1 factors or to settings where the codebook must adapt over time.
  • Because out-of-distribution detection is reported in only one setting (CIFAR-10 to SVHN, AUROC 0.887), harder distribution shifts would show whether the shared representation preserves the uncertainty signal that motivates Bayesian methods.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes 2DGBNN, a stochastic weight-sharing scheme for mean-field variational BNNs. The method clusters the (μ, σ) pairs of 'inlier' weights into a 2D Gaussian mixture model, detects outlier weights by magnitude and gradient thresholds, merges nearby components by Wasserstein distance, uses alpha-blending for multi-cluster assignments, and in a final training stage samples inlier weights from the GMM while updating the variational objective. Experiments span MNIST, CIFAR-10/100, and ImageNet-1k with ResNet-18/50/101 and ViT-B-16, reporting accuracy, NLL, ECE, parameter counts, ablations, OOD detection, and timing. The central claim is that this reduces BNN parameter count and model size by orders of magnitude while preserving accuracy and uncertainty estimates comparable to state-of-the-art.

Significance. If the compression claims were valid, the paper would be a practically useful contribution to efficient BNN deployment: it provides a code URL, evaluates on large-scale architectures including ViT, includes ablations and timing comparisons, and addresses a real gap in scalable Bayesian inference. The empirical evidence also shows non-trivial accuracy/uncertainty trade-offs on several benchmarks. However, the significance is currently limited by three concrete problems: the reported parameter reductions omit the per-weight index that Section 5.2 admits must be stored, the variational-bound claim in Eq. (10) is mathematically incorrect as written, and the uncertainty-preservation claim is not tested against the full mean-field posterior the method replaces. These are load-bearing issues for the paper's headline conclusions, not cosmetic presentation points.

major comments (4)
  1. [5.2; Tables 1-4] Tables 1-3 report the compression ratio using only GMM components, ellipses, and outlier parameters, but do not count the per-inlier cluster index that Section 5.2 and Table 4 state is required. For the ResNet-50/ImageNet row of Table 1, a mean-field BNN stores about 51.2M floats (about 205 MB); with K=3,250, each of the roughly 25.6M inlier weights needs at least 12 index bits (about 38 MB), before including GMM parameters, outlier parameters, and the extra assignments introduced by alpha-blending. The resulting end-to-end storage compression is around 5x, not the 99% or '3-4 orders of magnitude' claimed in Section 5.1 and the abstract. The abstract's 75% model-size reduction is broadly consistent with index-inclusive accounting, but the 'approximately 50x parameter compression' claim and all '99%' ratios in Tables 1-3 are not. Please report end-to-end storage with the index included and re-state all compression claims accordingly.
  2. [4.3.1 / Eq. (10)] With F(D,q) defined in Eq. (8) as the ELBO, the inequality in Eq. (10) gives the right-hand side as a lower bound on the approximate ELBO, not an upper bound, because KL is convex in the first argument for a fixed prior: KL(Σ π_k N_k || p) ≤ Σ π_k KL(N_k || p). Even if the bound direction were reversed, minimizing an upper bound does not by itself guarantee a decrease of the original objective. The sentence 'the resulting value loss function, \hat F is an upper bound on the original loss ... its minimisation ... guarantees the improvement of the latter' is therefore unsupported. Please correct the sign or the definitions, and clarify whether \hat F is an objective to be maximized or a loss to be minimized.
  3. [5.1; Appendix F] The uncertainty-preservation claim is not directly tested: no experiment compares the predictive distribution of a model sampling from the GMM with the predictive distribution of the full mean-field BNN posterior it replaces. The large-scale experiments in Section 5.1 initialize from deterministic pretrained networks (Appendix F), so the reported NLL and ECE measure a particular approximation pipeline rather than the fidelity of the GMM representation of a learned BNN posterior. Please add head-to-head comparisons (for example, the same architecture and sample count with a mean-field VI baseline) and quantify the approximation error of the GMM on the inlier weight distribution, or explicitly limit the claims to the tested pipeline.
  4. [Tables 2, 3, 9] Several table entries are externally impossible or internally inconsistent, which undermines the empirical comparison as published. The ABNN row in Table 2 (CIFAR-100) reports ECE=4.5, which is outside the unit interval for a calibration error; the ResNet-101 row in Table 3 (CIFAR-10) repeats exactly the same #Outliers=45240, #Ellipses=348, #Gaussians=3199, and #Parameters=0.052M as the ResNet-101 row in Table 2 (CIFAR-100), despite very different accuracy values; and Table 9 reports ECE=0.936 identical to NLL for ResNet-101 on ImageNet-1k. Please correct or explain these entries and verify all reported ECE and NLL values.
minor comments (6)
  1. [4.2 / Eq. (3)] Equation (3) says 'min ∑ log(...)', but the GMM is learned by maximum likelihood; the objective should be maximized or the negative log-likelihood minimized.
  2. [1; Contributions] The contribution list and introduction refer to 'ResNet-10', while the experiments use ResNet-101; this typo should be corrected.
  3. [Appendix I.5] Appendix I.5 is internally inconsistent: the text says K=2000 clusters and N_min=30, while Table 14 reports K=6000 and N_min=20, and Section 4.2 states N_min=30; these values should be harmonized.
  4. [5] The paper states that each experiment is run three times, but Tables 1-4 report only point estimates; standard deviations or error bars should be provided for the central metrics.
  5. [5.2 / Table 4] Table 4 mixes units in the #Parameters column, with some entries in millions, some in MB, and parenthetical MB values that are not defined in the caption; a single storage metric should be used throughout.
  6. [4.3 / Eq. (7)] The mixing coefficients α_k in Eq. (7) are defined as unnormalized pdf values; the paper should specify how they are normalized so that the mixture sums to one.

Circularity Check

1 steps flagged · score 4.0 of 10

Compression claim is an accounting artifact (per-weight index omitted), while accuracy and uncertainty results are independent measurements; partial circularity only.

  1. self definitional [Section 5.1 (Table 1) and Section 5.2 (Table 4)]
    "We observe that, in all cases, our method successfully reduces the number of parameters by 3 or 4 orders of magnitudes. ... Notice, however, that in small NNs (like the one here analysed) our technique incurs significant storage overhead in that we need to keep an index (encoded in uint8) that assigns each inlier weight to its cluster. When this value is added (size reported in brackets in the Table) quantisation has a significant advantage over our storage requirements."

    Tables 1-3 report '#Parameters' and compression ratios counting only GMM components and outlier weights; the per-inlier-weight cluster index needed to reconstruct every shared weight is excluded. For ResNet-50 on ImageNet, 25.6M inlier weights with K=3,250 require at least 12 bits per inlier (~38 MB) before counting GMM/outlier storage, so actual storage compression is about 5x, not 99% or 3-4 orders of magnitude. The headline reduction is forced by the definition of '#Parameters' rather than by the method; the paper's own Section 5.2 explicitly admits that once the index is included, quantisation 'has a significant advantage' over 2DGBNN.

full rationale

No self-citation chain or imported uniqueness theorem is load-bearing, and the accuracy, NLL, and ECE results are empirical measurements against external baselines, so the uncertainty claims are not derived from fitted parameters. The main definitional issue is the compression metric: the per-weight cluster index is omitted from the count, making the '3-4 orders of magnitude' reduction true only by accounting convention, not by storage reality. Appendix E tunes hyperparameters such as tau_w on the same benchmark before reporting results, which is a selection-bias concern rather than derivation circularity. Appendix F also shows that the large-scale results initialize from deterministic pretrained networks, so the GMM is not learned from a fully Bayesian run; this limits interpretation but is not circular. These limitations should be weighed in correctness review, but the core empirical comparison retains independent content, hence a moderate rather than high circularity score.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method rests on several hand-set thresholds and on the fidelity of a GMM approximation to the true variational posterior. The paper reports no error bars and no approximation error analysis. The theoretical justification in Eq. (10) contains a bound-direction error, and the large-scale experiments rely on pretrained deterministic initializations, meaning the central claims are only partially supported by the presented evidence.

free parameters (6)
  • Outlier mean threshold tau_w = 0.2
    Selected by grid search on ResNet-20/CIFAR-10 (Appendix E, Table 8); applied uniformly to all experiments. The paper notes it is the most critical and sensitive parameter.
  • Wasserstein merge threshold T_W = 1e-2 (Appendix H text states 1.5e-7)
    Distance below which Gaussian components are candidates for merging; the two stated values are inconsistent.
  • Gradient percentile threshold P_g = top 1%
    Weights with gradient magnitude in the top 1% are treated as outliers (Section 4.1).
  • Minimum cluster size N_min = 30 (Algorithm 1) / 20 (Table 14)
    Clusters with fewer weights are dissolved and their members reclassified as outliers; inconsistent values reported.
  • Number of GMM components K = 2000 (initial) / 6000 (Table 14)
    User-chosen number of clusters for k-means initialization of the GMM; values inconsistent across the paper.
  • Number of alpha-blend neighbors k = 5
    Weights outside the Mahalanobis threshold are assigned to k nearest Gaussian components (Appendix I.5).
assumptions (5)
  • domain assumption Mean-field variational posterior: q(w) = product of independent Gaussians
    Standard BNN assumption; the paper relies on it to define the 2D (mu, sigma) representation and the ELBO decomposition.
  • domain assumption w_in and w_out are pairwise independent in the variational distribution
    Stated in Section 4.3.1: 'This is true for the variational distribution but it is an approximation for the true posterior.' Enables the KL split in Eq. (8)-(10).
  • standard math KL divergence is convex in the first argument
    Used in Eq. (10) to bound KL(sum pi_k N_k || p) by sum pi_k KL(N_k || p). The direction then contradicts the paper's 'upper bound' wording.
  • ad hoc to paper A GMM with k ~ 2000-6000 2D Gaussian components can adequately represent the posterior of millions of inlier weights
    No approximation error is measured; the entire method's fidelity depends on this representational assumption.
  • ad hoc to paper Pretrained deterministic network parameters can serve as the prior centering for the BNN posterior
    Appendix F: deterministic networks are used as prior for the centring of GMMs. Replaces the stated Gaussian prior with a learned initialization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stochastic Weight Sharing for Bayesian Neural Networks." pith.science (2026). https://pith.science/paper/NHYG3FB6

@misc{pith2026250517856,
  author       = {Pith},
  title        = {Pith review of: Stochastic Weight Sharing for Bayesian Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NHYG3FB6}},
  note         = {Machine review of arXiv:2505.17856}
}
read the original abstract

While offering a principled framework for uncertainty quantification in deep learning, the employment of Bayesian Neural Networks (BNNs) is still constrained by their increased computational requirements and the convergence difficulties when training very deep, state-of-the-art architectures. In this work, we reinterpret weight-sharing quantization techniques from a stochastic perspective in the context of training and inference with Bayesian Neural Networks (BNNs). Specifically, we leverage 2D adaptive Gaussian distributions, Wasserstein distance estimations, and alpha blending to encode the stochastic behaviour of a BNN in a lower dimensional, soft Gaussian representation. Through extensive empirical investigation, we demonstrate that our approach significantly reduces the computational overhead inherent in Bayesian learning by several orders of magnitude, enabling the efficient Bayesian training of large-scale models, such as ResNet-101 and Vision Transformer (VIT). On various computer vision benchmarks including CIFAR10, CIFAR100, and ImageNet1k. Our approach compresses model parameters by approximately 50x and reduces model size by 75, while achieving accuracy and uncertainty estimations comparable to the state-of-the-art.

Figures

Figures reproduced from arXiv: 2505.17856 by the authors.

Figure 1
Figure 1. Weight distribution for the BNN prior to stochastic-sharing. Panel (a) shows the density plot for the second convo [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Left: Posterior distribution of weights in ResNet-18 with CIFAR-10 using 2D Gaussian Bayesian Neural Network [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Initial 2D Gaussian of the second convolutional layer of ResNet-18 in CIFAR-10 dataset. Different colors represent [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 56 canonical work pages

  1. [1]

    M., Schmeink, A., and Genewein, T

    Achterhold, J., Koehler, J. M., Schmeink, A., and Genewein, T. (2018). Variational network quantization. In International conference on learning representations

    work page 2018

  2. [2]

    and Carlier, G

    Agueh, M. and Carlier, G. (2011). Barycenters in the wasserstein space. SIAM Journal on Mathematical Analysis , 43(2):904--924

    work page 2011

  3. [3]

    Beckers, J., Van Erp, B., Zhao, Z., Kondrashov, K., and De Vries, B. (2023). Principled pruning of bayesian neural networks through variational free energy minimization. IEEE Open Journal of Signal Processing

    work page 2023

  4. [4]

    Bharadiya, J. P. (2023). A review of bayesian machine learning principles, methods, and applications. International Journal of Innovative Science and Research Technology , 8(5):2033--2038

    work page 2023

  5. [5]

    Billah, M. E. and Javed, F. (2022). Bayesian convolutional neural network-based models for diagnosis of blood cancer. Applied Artificial Intelligence , 36(1):2011688

    work page 2022

  6. [6]

    Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015). Weight uncertainty in neural networks. In Proceedings of the 32nd International Conference on Machine Learning , pages 1613--1622

    work page 2015

  7. [7]

    Bonnet, D., Hirtzlin, T., Majumdar, A., Dalgaty, T., Esmanhotto, E., Meli, V., Castellani, N., Martin, S., Nodin, J.-F., Bourgeois, G., et al. (2023). Bringing uncertainty quantification to the extreme-edge with memristor-based bayesian neural networks. Nature Communications , 14(1):7530

    work page 2023

  8. [8]

    and Chang, S.-T

    Chien, J.-T. and Chang, S.-T. (2023). Bayesian asymmetric quantized neural networks. Pattern Recognition , 139:109463

    work page 2023

Show all 63 references
  1. [9]

    Chizat, L., Roussillon, P., L \'e ger, F., Vialard, F.-X., and Peyr \'e , G. (2020). Faster wasserstein distance estimation with the sinkhorn divergence. Advances in Neural Information Processing Systems , 33:2257--2269

    2020

  2. [10]

    De Palma, G., Marvian, M., Trevisan, D., and Lloyd, S. (2021). The quantum wasserstein distance of order 1. IEEE Transactions on Information Theory , 67(10):6627--6643

    2021

  3. [11]

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248--255. IEEE

    2009

  4. [12]

    G., Shamsi, A., Guo, X.-Y., Mohammadi, A., Alinejad-Rokny, H., Sejdinovic, D., Ranasinghe, D

    Doan, B. G., Shamsi, A., Guo, X.-Y., Mohammadi, A., Alinejad-Rokny, H., Sejdinovic, D., Ranasinghe, D. C., and Abbasnejad, E. (2024). Bayesian low-rank learning (bella): A practical approach to bayesian neural networks. arXiv preprint arXiv:2407.20891

    2024 arXiv

  5. [13]

    Dong, R., Tan, Z., Wu, M., Zhang, L., and Ma, K. (2022). Finding the task-optimal low-bit sub-distribution in deep neural networks. In International Conference on Machine Learning , pages 5343--5359. PMLR

    2022

  6. [14]

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Repres...

    2021

  7. [15]

    Dusenberry, M., Jerfel, G., Wen, Y., Ma, Y., Snoek, J., Heller, K., Lakshminarayanan, B., and Tran, D. (2020). Efficient and scalable bayesian neural nets with rank-1 factors. In International conference on machine learning , pages 2782--2792. PMLR

    2020

  8. [16]

    Ferianc, M., Maji, P., Mattina, M., and Rodrigues, M. (2021). On the effects of quantisation on model uncertainty in bayesian neural networks. In Uncertainty in Artificial Intelligence , pages 929--938. PMLR

    2021

  9. [17]

    Forsberg, H., Lind \'e n, J., Hjorth, J., M nefjord, T., and Daneshtalab, M. (2020). Challenges in using neural networks in safety-critical applications. In 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC) , pages 1--7. IEEE

    2020

  10. [18]

    Franchi, G., Bursuc, A., Aldea, E., Dubuisson, S., and Bloch, I. (2023). Encoding the latent posterior of bayesian neural networks for uncertainty quantification. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2023

  11. [19]

    Franchi, G., Laurent, O., Legu \'e ry, M., Bursuc, A., Pilzer, A., and Yao, A. (2024). Make me a bnn: A simple strategy for estimating bayesian uncertainty from pre-trained models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 121...

    2024

  12. [20]

    and Ghahramani, Z

    Gal, Y. and Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning , pages 1050--1059. PMLR

    2016

  13. [21]

    Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. (2017). On calibration of modern neural networks. In International conference on machine learning , pages 1321--1330. PMLR

    2017

  14. [22]

    Guo, Y. (2018). A survey on methods and theories of quantized neural networks. arXiv preprint arXiv:1808.04752

    2018 arXiv

  15. [23]

    He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 770--778

    2016

  16. [24]

    and Kalchbrenner, N

    Heek, J. and Kalchbrenner, N. (2019). Bayesian inference for large scale image classification. arXiv preprint arXiv:1908.03491

    2019 arXiv

  17. [25]

    Hern \'a ndez-Lobato, J. M. and Adams, R. (2015). Probabilistic backpropagation for scalable learning of bayesian neural networks. In International conference on machine learning , pages 1861--1869. PMLR

    2015

  18. [26]

    Hinton, G. E. and Neal, R. M. (1995). Bayesian learning for neural networks , volume 118. Springer Science & Business Media

    1995

  19. [27]

    M., Patel, L., Bhattacharya, A., and Pati, D

    Jacobs, P. M., Patel, L., Bhattacharya, A., and Pati, D. (2023). Memory efficient and minimax distribution estimation under wasserstein distance using bayesian histograms. arXiv preprint arXiv:2307.10099

    2023 arXiv

  20. [28]

    Kim, K., Ma, E.-Y., Choi, J., and Kim, H. (2023). Inverse-reference priors for fisher regularization of bayesian neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 8264--8272

    2023

  21. [29]

    Kong, I., Yang, D., Lee, J., Ohn, I., Baek, G., and Kim, Y. (2023). Masked bayesian neural networks: Theoretical guarantee and its posterior inference. In International Conference on Machine Learning , pages 17462--17491. PMLR

    2023

  22. [30]

    Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto

    2009

  23. [31]

    Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems , 30

    2017

  24. [32]

    and Vehtari, A

    Lampinen, J. and Vehtari, A. (2001). Bayesian approach for neural networks—review and case studies. Neural networks , 14(3):257--274

    2001

  25. [33]

    LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278--2324

    1998

  26. [34]

    Li, J., Miao, Z., Qiu, Q., and Zhang, R. (2024). Training bayesian neural networks with sparse subspace variational inference. arXiv preprint arXiv:2402.11025

    2024 arXiv

  27. [35]

    Li, M., Zhang, T., Chen, Y., and Smola, A. J. (2014). Efficient mini-batch training for stochastic optimization. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 661--670

    2014

  28. [36]

    R., Subedar, M., Sanghavi, V., Arunachalam, M., Tickoo, O., Iyer, R., and Kandemir, M

    Lin, J.-L., Krishnan, R., Ranipa, K. R., Subedar, M., Sanghavi, V., Arunachalam, M., Tickoo, O., Iyer, R., and Kandemir, M. T. (2023). Quantization for bayesian deep learning: Low-precision characterization and robustness. In 2023 IEEE International Symposium on Workload Chara...

    2023

  29. [37]

    Louizos, C., Ullrich, K., and Welling, M. (2017). Bayesian compression for deep learning. Advances in neural information processing systems , 30

    2017

  30. [38]

    T., Del Campo, F., and Zamarr \'o n, C

    Marcos, J., Hornero, R., Alvarez, D., Nabney, I. T., Del Campo, F., and Zamarr \'o n, C. (2010). The classification of oximetry signals using bayesian neural networks to assist in the detection of obstructive sleep apnoea syndrome. Physiological measurement , 31(3):375

    2010

  31. [39]

    Michelmore, R., Wicker, M., Laurenti, L., Cardelli, L., Gal, Y., and Kwiatkowska, M. (2020). Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. In 2020 IEEE international conference on robotics and automation (ICRA) , pages 7344--7...

    2020

  32. [40]

    P., Tancik, M., Barron, J

    Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. (2021). Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM , 65(1):99--106

    2021

  33. [41]

    Minka, T. P. (2001). Expectation propagation for approximate bayesian inference. In Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (UAI) , pages 362--369

    2001

  34. [42]

    Nguyen, V.-A., Vuong, T.-L., Phan, H., Do, T.-T., Phung, D., and Le, T. (2024). Flat seeking bayesian neural networks. Advances in Neural Information Processing Systems , 36

    2024

  35. [43]

    Nowlan, S. J. and Hinton, G. E. (2018). Simplifying neural networks by soft weight sharing. In The mathematics of generalization , pages 373--394. CRC Press

    2018

  36. [44]

    Park, N., Lee, T., and Kim, S. (2021). Vector quantized bayesian neural network inference for data streams. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9322--9330

    2021

  37. [45]

    Perrin, M., Guicquero, W., Paille, B., and Sicard, G. (2024). Hardware-aware bayesian neural architecture search of quantized cnns. IEEE Embedded Systems Letters

    2024

  38. [46]

    C., Nguyen, C

    Pham, V. C., Nguyen, C. C., Le, T., Phung, D., Carneiro, G., and Do, T.-T. (2024). Model and feature diversity for bayesian neural networks in mutual learning. Advances in Neural Information Processing Systems , 36

    2024

  39. [47]

    and Pernkopf, F

    Roth, W. and Pernkopf, F. (2018). Bayesian neural networks with weight sharing using dirichlet processes. IEEE transactions on pattern analysis and machine intelligence , 42(1):246--252

    2018

  40. [48]

    and Jennings, E

    Sharma, H. and Jennings, E. (2021). Bayesian neural networks at scale: a performance analysis and pruning study. The Journal of Supercomputing , 77(4):3811--3839

    2021

  41. [49]

    Soudry, D., Hubara, I., and Meir, R. (2014). Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights. Advances in neural information processing systems , 27

    2014

  42. [50]

    N., and Tickoo, O

    Subedar, M., Krishnan, R., Kashyap, S. N., and Tickoo, O. (2021). Quantization of bayesian neural networks and its effect on quality of uncertainty. In Workshop on Uncertainty and Robustness in Deep Learning, ICML

    2021

  43. [51]

    and Dasmahapatra, S

    Subia-Waud, C. and Dasmahapatra, S. (2024). Probabilistic weight fixing: Large-scale training of neural network weight uncertainties for quantisation. Advances in Neural Information Processing Systems , 36

    2024

  44. [52]

    Swiatkowski, J., Roth, K., Veeling, B., Tran, L., Dillon, J., Snoek, J., Mandt, S., Salimans, T., Jenatton, R., and Nowozin, S. (2020). The k-tied normal distribution: A compact parameterization of gaussian mean field posteriors in bayesian neural networks. In International co...

    2020

  45. [53]

    Takatsu, A. (2011). Wasserstein geometry of gaussian measures

    2011

  46. [54]

    Treiss, A., Walk, J., and K \"u hl, N. (2021). An uncertainty-based human-in-the-loop system for industrial tool wear analysis. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track: European Conference, ECML PKDD 2020, Ghent, Belgium, S...

    2021

  47. [55]

    Ullrich, K., Meeds, E., and Welling, M. (2017). Soft weight-sharing for neural network compression. arXiv preprint arXiv:1702.04008

    2017 arXiv

  48. [56]

    A., Wang, Y., Blankevoort, T., and Welling, M

    Van Baalen, M., Louizos, C., Nagel, M., Amjad, R. A., Wang, Y., Blankevoort, T., and Welling, M. (2020). Bayesian bits: Unifying quantization and pruning. Advances in neural information processing systems , 33:5741--5752

    2020

  49. [57]

    and Lampinen, J

    Vehtari, A. and Lampinen, J. (1999). Bayesian neural networks for industrial applications. In SMCia/99 Proceedings of the 1999 IEEE Midnight-Sun Workshop on Soft Computing Methods in Industrial Applications (Cat. No. 99EX269) , pages 63--68. IEEE

    1999

  50. [58]

    and Teh, Y

    Welling, M. and Teh, Y. W. (2011). Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning (ICML) , pages 681--688

    2011

  51. [59]

    Wicker, M., Laurenti, L., Patane, A., Paoletti, N., Abate, A., and Kwiatkowska, M. (2024). Probabilistic reach-avoid for bayesian neural networks. Artificial Intelligence , page 104132

    2024

  52. [60]

    Yang, Y., Bamler, R., and Mandt, S. (2020a). Variational bayesian quantization. In International Conference on Machine Learning , pages 10670--10680. PMLR

    2020

  53. [61]

    Yang, Z., Wang, Y., Han, K., Xu, C., Xu, C., Tao, D., and Xu, C. (2020b). Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems , 33:4091--4102

    2020

  54. [62]

    u tepage, J., Kjellstr \

    Zhang, C., B \"u tepage, J., Kjellstr \"o m, H., and Mandt, S. (2018). Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence , 41(8):2008--2026

    2018

  55. [63]

    Zhang, R., Li, C., Zhang, J., Chen, C., and Wilson, A. G. (2019). Cyclical stochastic gradient mcmc for bayesian deep learning. arXiv preprint arXiv:1902.03932

    2019 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.