Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new prior for Bayesian neural networks—the R2D2 prior—shrinks irrelevant weights toward zero while preserving the weights that carry strong signals, with a variational Gibbs inference algorithm and a near-minimax contraction guarantee.

desk verdict A useful BNN prior and inference recipe with strong empirical work, but the variational Gibbs sampler is not shown to be variational, leaving the theory and the experiments disconnected. read the letter →

arxiv 2505.18280 v1 pith:DQUMTHGH submitted 2025-05-23 cs.LG cs.AIstat.ML

Tsai Hor Chan , Dora Yan Zhang , Guosheng Yin , Lequan Yu This is my paper · ORCID
classification cs.LGcs.AIstat.ML MSC 62F1562G2068T0762J07
keywords BayesianneuralnetworkR2D2priorshrinkagevariationalinferenceGibbsuncertaintyestimationposteriorconcentrationsparsedeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes R2D2-Net, a Bayesian neural network that places the R2-induced Dirichlet Decomposition (R2D2) prior on every network weight. The central claim is that this prior shrinks weights of irrelevant or noisy features toward zero while leaving weights carrying strong signals intact, which ordinary Gaussian priors cannot do and which lighter-tailed shrinkage priors such as the Horseshoe do too aggressively. To fit the model, the authors develop a variational Gibbs inference algorithm that alternates Gibbs updates of the shrinkage parameters with gradient-based updates of the weights, and they derive closed-form KL divergences for the shrinkage parameters. They also prove that the R2D2 posterior achieves a near-minimax contraction rate under a polynomial-boundedness condition on the true weights. If the claims hold, BNNs using this prior would suffer less variance inflation, support deeper architectures, and give more reliable uncertainty estimates on out-of-distribution inputs.

What carries the argument

The carrying object is the R2D2 prior, defined by the scale-mixture representation $\beta_j \mid \psi_j,\phi_j,\omega \sim N(0, \psi_j \phi_j \omega \sigma^2/2)$ with $\psi_j \sim \mathrm{Exp}(1/2)$, $\phi \sim \mathrm{Dir}(a_\pi,\dots,a_\pi)$, $\omega \mid \xi \sim \mathrm{Ga}(a,\xi)$, $\xi \sim \mathrm{Ga}(b,1)$. Its marginal density decays like $O(|\beta|^{-(1+2b)})$ in the tails and concentrates at zero like $O(|\beta|^{-(1-2a_\pi)})$, which the paper compares favorably against the Horseshoe, Horseshoe+, Dirichlet-Laplace, and generalized double Pareto priors. The inference machinery is a variational Gibbs algorithm that alternately samples $\psi$, $\omega$, $\xi$, and $\phi$ from their conditional posteriors and updates the reparameterized weight means and variances by back-propagating the ELBO, with closed-form KL divergences for the shrinkage parameters.

What would settle it

Train R2D2-Net on a small synthetic regression with known sparse ground-truth weights and compare its posterior to the Hamiltonian Monte Carlo oracle posterior used by the paper as ground truth: if the variational Gibbs posterior's mean and variance do not track the HMC oracle, or if the ELBO does not monotonically improve during the Gibbs updates, then the algorithm is not performing variational inference and Theorem 1's contraction guarantee does not apply to the implemented procedure.

Watch

Extended reading notes

Core claim

Placing the R2D2 prior on the weights of a Bayesian neural network makes the posterior contract toward sparse solutions while staying concentrated enough on large coefficients to preserve predictive features. The paper states this as: the R2D2 prior has the highest concentration rate at zero among the compared global-local shrinkage priors and the heaviest tail, so irrelevant coefficients are shrunk toward zero and key features are not over-shrunk. The variational Gibbs algorithm treats the conditional posteriors of the shrinkage parameters as variational updates and back-propagates the ELBO through reparameterized weights to learn the per-neuron variances. The theoretical result, Theorem 1, asserts that under sparsity and polynomial-boundedness conditions the posterior contracts in Hellinger distance at a near-minimax rate $\epsilon_n^2 = O(\varpi_n^2) + O([r_n L_n \log n + r_n \log \bar H + s_n \log D_n]/n)$, matching the spike-and-slab rate.

Load-bearing premise

The load-bearing premise is that the hybrid variational Gibbs algorithm, which alternates Gibbs samples of the shrinkage parameters with gradient updates of the weights, actually converges to the target posterior and maximizes the evidence lower bound; the paper does not prove this convergence.

Editorial extensions

If this is right

  • On image classification benchmarks (CIFAR-10, CIFAR-100, TinyImageNet), R2D2-Net reports higher accuracy and AUROC than Gaussian, Horseshoe, and spike-and-slab BNN designs, in some cases matching or exceeding the frequentist baseline.
  • For out-of-distribution detection, the paper reports that R2D2-Net's entropy-based uncertainty scores outperform the compared Bayesian and non-Bayesian baselines on natural and medical image datasets.
  • The variational Gibbs updates plus closed-form KL divergences make the ELBO more faithful than the Gaussian-KL approximation used in standard mean-field BNNs, which the paper argues reduces variance inflation.
  • Theorem 1 gives a near-minimax posterior contraction rate under the R2D2 prior, placing it on par with spike-and-slab priors for sparse deep learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The shrinkage-versus-preservation tradeoff suggests R2D2-Net could double as a pruning method: weights whose posterior mass concentrates at zero could be removed after training, yielding compressed networks without a separate sparsification pass.
  • The variational Gibbs update is not derived from a mean-field variational objective, so a stricter derivation (or a correction) might be needed before the empirical results can be tied to the contraction theorem; one test is to check whether the Gibbs steps decrease the ELBO at every iteration.
  • If the R2D2 prior's heavy-tail property transfers to attention weights, the same construction could give Bayesian transformers a principled way to prune attention heads, an extension the paper flags as non-trivial.
  • The per-neuron variance parameter $\sigma_{jl}$ is learned by back-propagation rather than set by regression MSE; a direct comparison against the original R2D2 setting with layer-shared variance would isolate how much of the gain comes from the prior versus the per-neuron variance.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes R2D2-Net, a Bayesian neural network in which the R2D2 shrinkage prior is placed on the weights. The central claims are that this prior has the highest concentration at zero and the heaviest tail among global–local shrinkage priors, so it shrinks irrelevant weights while preserving strong features; that the proposed 'variational Gibbs' algorithm, combining conditional sampling of shrinkage parameters with gradient-based updates of weights, provides accurate posterior approximation; and that the resulting posterior attains a near-minimax contraction rate (Theorem 1). The authors derive analytical KL divergences for part of the variational objective, and they report extensive experiments on simulations, CIFAR-10/100, TinyImageNet, and a medical OOD benchmark, where R2D2-Net often improves classification accuracy and OOD detection over Gaussian, Horseshoe, and spike-and-slab baselines.

Significance. If the claims were established, the paper would be a useful contribution to Bayesian deep learning: it combines a theoretically motivated shrinkage prior with a scalable inference procedure and provides extensive empirical comparisons, with code released at a public repository. The breadth of experiments, including synthetic studies, larger architectures, and a medical imaging OOD task, is a clear strength, as is the attempt to provide an analytical ELBO and posterior concentration analysis. However, the current manuscript does not establish the connection between the proposed inference algorithm and the theoretical posterior concentration result, and the empirical setting does not match the hyperparameter regimes required by the theory. These gaps are load-bearing because the empirical wins are attributed to the R2D2 prior and the variational Gibbs procedure, while Theorem 1 concerns the exact posterior under assumptions not met by the experiments.

major comments (4)
  1. [Section 4.2 / Algorithm 1] The variational Gibbs algorithm is not derived from any variational objective. In Section 4.2, the shrinkage parameters ψ, ω, ξ, and φ are sampled from their full conditional posterior distributions conditioning on the current stochastic weight draws, while w and ρ are updated by backpropagating the ELBO in Eq. (1). No variational family is defined for the shrinkage parameters, and the alternating updates are not shown to increase or even to preserve the ELBO. This is therefore not a mean-field coordinate-ascent procedure, and the output of Algorithm 1 is not established as an approximation of the R2D2 posterior. Consequently, Theorem 1, which is a statement about the exact posterior under the R2D2 prior, is not connected to the distribution produced by the algorithm used in the experiments. The Limitations paragraph in Section 8 acknowledges that the Gibbs sampling may not handle multimodal posteriors, but it does not address this more basic issue. I would need either a derivation of the updates as coordinate ascent on a well-defined variational family with a monotone ELBO, or a comparison against exact HMC on a small BNN showing that Algorithm 1's output tracks the exact posterior.
  2. [Appendix, Proof of Theorem 1] The proof of Theorem 1 is not complete and contains notational and algebraic problems. The proof verifies condition (4) of Theorem 2 by discussing the tail density of the R2D2 prior, but condition (4) is a statement about log(1/π_b), the prior mass or minimum density, not the tail decay; the link between the two is not provided. In the verification of condition (5), the display ends with the inequality 1 − k_n^{a_π} ≤ D_n^{−(1+u)}, which is not derived and appears inconsistent with the preceding algebra, since the left side tends to 1 as k_n → 0. There is also a notation mismatch in Theorem 2, where the terms ar L and ar H are used interchangeably. As written, the appendix does not establish the minimax contraction rate claimed in Theorem 1.
  3. [Theorem 1 / Supplementary Section .4] The hyperparameters used in the experiments are incompatible with the theoretical assumptions and with the paper's own tail claims. Theorem 1 requires b to satisfy E_n/(L_n log n + log \bar H)^{1/2} ≲ b ≲ n^α, where E_n is polynomially growing in n, so b must grow with n. The experiments use a fixed b = 0.5 (Supplementary Section .4). Moreover, Table 1 gives the R2D2 tail decay as O(1/|β|^{1+2b}); with b = 0.5 this is O(1/β^2), identical to the Horseshoe tail. Therefore the abstract's and Section 1's claims that the R2D2 prior has the heaviest tail and avoids over-shrinkage compared to the Horseshoe are not realized under the reported default setting. Similarly, with a_π = 0.6 the concentration-at-zero exponent 1−2a_π is negative, so the claimed divergence at zero in Table 1 does not occur. The authors should either choose hyperparameters satisfying the stated rates or qualify the 'highest concentration / heaviest tail' claims to the parameter regimes in which they hold.
  4. [Algorithm 1] The pseudocode of Algorithm 1 is inconsistent with the text in Section 4.2. Lines 10–14, which sample ω_l, ξ_l, ψ_jl, and T_jl, are inside the inner loop 'for w_jl in w_l do', so the global shrinkage parameters ω_l and ξ_l are resampled once for every weight in the layer, with only the last draw retained. Section 4.2 specifies that ω_l and ξ_l are sampled once per layer. This ambiguity makes the published algorithm impossible to reproduce unambiguously and should be corrected in the pseudocode, with the layer-wise updates moved outside the per-weight loop.
minor comments (5)
  1. [Section 2, Eq. (1)] The displayed definition of KL(q∥π) appears to contain typos: it reads KL(q∥π) = E_{q∈Q}[log p(θ|·)] + H[π(θ)], which is not the Kullback–Leibler divergence; it should presumably be E_q[log q(θ)] − E_q[log π(θ)] (or the equivalent entropy form). Please correct this.
  2. [Section 5, Condition A.2.2] The notation s_n is used for 'the input dimension of γ*', but s_n was not introduced earlier and the phrase is ambiguous; please define s_n precisely (e.g., the number of nonzero input coordinates or the number of active input dimensions) and relate it to D_n.
  3. [Table 2 / Appendix KL derivation] In Table 2, the KL divergence for ψ_jl is written in terms of ψ, but the variational posterior is placed on ψ^{-1} (Reciprocal InvGaussian). The appendix derivation later introduces Y = 1/ψ; the table and the derivation should be written in the same variable, and the resulting closed form should be stated explicitly.
  4. [Section 6.1, Scenario 3] The text says that in Scenario 3 'shrinkage methods are expected to underperform as they shrink noise features to zeros', but Table 3 shows R2D2-Net performing best in this scenario; please clarify what 'underperform' means here or remove the sentence, which currently contradicts the reported results.
  5. [Throughout] The appendix organization is confusing: Section A and then Sections C and D are used nontransparently, and several places refer only to 'the supplementary materials' without section numbers. Please unify the numbering and make all forward references explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the R2D2 prior's theoretical properties are externally established mathematical results, and the paper's empirical and theoretical claims are independently evaluated.

full rationale

The proposed R2D2-Net is a BNN whose weights are assigned the R2D2 prior of Zhang et al. [60]. Yan Dora Zhang is a co-author of both [60] and the present paper, so there is self-citation, but the cited content is not an unverified premise: the tail decay and concentration-at-zero rates in Table 1 are asymptotic properties of the R2D2 density restated in Lemma 1 (appendix) and are externally checkable mathematical facts with stated hyperparameter conditions; they are not fitted in this paper. Theorem 1 is proved by verifying the prior-mass and tail conditions of the general contraction theorem of Sun et al. [47,48] for the R2D2 density; it invokes an external theorem and does not assume the conclusion. The variational Gibbs algorithm in Section 4.2 borrows Gibbs conditionals from Zhang et al., but this is a computational design choice, not a circular reduction: no fitted value is renamed as a prediction, the ELBO/KL derivations are analytic identities, and the empirical comparisons (Tables 5-8, OOD detection, simulations) are evaluated against standard benchmarks and baselines with fixed architectures. The admitted limitation that Gibbs sampling 'may not tackle the multimodal posteriors satisfactorily' concerns correctness/convergence of the proposed sampler, not circularity. Thus the derivation chain is self-contained: the prior's properties come from published math, the contraction theorem comes from an external general theorem, and the empirical claims are benchmark comparisons.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on the R2D2 prior properties and the training algorithm. The prior's concentration and tail rates come from the authors' earlier work [60]; Theorem 1 assumes regularity conditions A.1-A.3 and uses the contraction theorem of Sun et al. [48]. The algorithm additionally assumes that learned per-weight variances do not break the prior or the contraction proof.

free parameters (3)
  • a_pi = 0.6
    Dirichlet concentration parameter for local shrinkage phi; directly controls the concentration rate at zero. Set by grid search, reported in the appendix.
  • b = 0.5
    Gamma shape for the global shrinkage scale xi; controls tail heaviness. Set by grid search; with b=0.5 the R2D2 tail exponent matches the Horseshoe prior.
  • rho_0 = N(-3, 0.1^2)
    Initial log-softplus variance parameter for weights. The paper notes the method is more sensitive to rho_0 than to a_pi or b.
assumptions (4)
  • domain assumption R2D2 prior density properties, including concentration at zero and tail decay, as stated in Zhang et al. [60].
    The main motivation for choosing R2D2 over other priors relies on the asymptotic rates in Table 1, cited from [60] and not re-derived in this paper.
  • domain assumption Regularity conditions A.1-A.3: bounded input, existence of a sparse DNN approximation, polynomial bounded true weights, and Lipschitz activation.
    Theorem 1 requires these conditions; A.2.3 (polynomial bound on true weights) is explicitly acknowledged as stronger than typical assumptions.
  • standard math The general prior contraction theorem of Sun et al. [48] (Theorem 2 in the appendix) holds and can be applied to the R2D2 prior.
    The proof of Theorem 1 verifies prior conditions of this existing theorem rather than establishing a new contraction result from first principles.
  • ad hoc to paper Per-weight variances sigma_jl learned by backpropagation do not invalidate the R2D2 prior properties or the contraction proof.
    The theory is stated for a fixed prior on weights, while the algorithm learns individual sigma_jl through rho_jl. The paper does not reconcile this gap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior." pith.science (2026). https://pith.science/paper/DQUMTHGH

@misc{pith2026250518280,
  author       = {Pith},
  title        = {Pith review of: Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DQUMTHGH}},
  note         = {Machine review of arXiv:2505.18280}
}
read the original abstract

Bayesian neural networks (BNNs) treat neural network weights as random variables, which aim to provide posterior uncertainty estimates and avoid overfitting by performing inference on the posterior weights. However, the selection of appropriate prior distributions remains a challenging task, and BNNs may suffer from catastrophic inflated variance or poor predictive performance when poor choices are made for the priors. Existing BNN designs apply different priors to weights, while the behaviours of these priors make it difficult to sufficiently shrink noisy signals or they are prone to overshrinking important signals in the weights. To alleviate this problem, we propose a novel R2D2-Net, which imposes the R^2-induced Dirichlet Decomposition (R2D2) prior to the BNN weights. The R2D2-Net can effectively shrink irrelevant coefficients towards zero, while preventing key features from over-shrinkage. To approximate the posterior distribution of weights more accurately, we further propose a variational Gibbs inference algorithm that combines the Gibbs updating procedure and gradient-based optimization. This strategy enhances stability and consistency in estimation when the variational objective involving the shrinkage parameters is non-convex. We also analyze the evidence lower bound (ELBO) and the posterior concentration rates from a theoretical perspective. Experiments on both natural and medical image classification and uncertainty estimation tasks demonstrate satisfactory performance of our method.

Figures

Figures reproduced from arXiv: 2505.18280 by the authors.

Figure 1
Figure 1. An illustrative comparison of priors with different [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed R2D2-Net with the yellow part representing the graphical model of each neuron and the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Prediction mean and confidence intervals of R2D2-Net at test time on [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Ablation studies of our method to different hyperparameters. We run the three simulation scenarios (S1–S3) with an [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Density plots of the weight samples of Gaussian BNN, [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Five largest-norm convolutional filters of the R2D2- [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: In-distribution (healthy) and OOD (unhealthy) sam [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Comparison of training curves. .3 Additional Details on Model Architectures Compose Network Architecture with R2D2 Layers. With the given marginal weight distributions in Eq. (1) from the main text, we can construct the layers of the R2D2-Net. Specifically, we consider…
Figure 9
Figure 9. Figure 9: Example of the R2D2 LeNet architecture. Each convolutional or linear layer is replaced by its R2D2 design (i.e., R2D2 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Marginal densities of typical global–local shrinkage priors [ [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 52 canonical work pages

  1. [1]

    A systematic comparison of bayesian deep learning robustness in diabetic retinopathy tasks

    SebastianFarquhar AngelosFilos, AidanN Gomez, and TimG J Rudner. A systematic comparison of bayesian deep learning robustness in diabetic retinopathy tasks. stat, 1050:22, 2019

    work page 2019

  2. [2]

    Posterior consistency in linear models under shrinkage priors.Biometrika, 100 (4):1011–1018, 2013

    Artin Armagan, David B Dunson, Jaeyong Lee, Wa- heed U Bajwa, and Nate Strawn. Posterior consistency in linear models under shrinkage priors.Biometrika, 100 (4):1011–1018, 2013

    work page 2013

  3. [3]

    The horseshoe+ estimator of ultra-sparse signals.Bayesian Analysis, 12(4):1105–1131, 2017

    Anindya Bhadra, Jyotishka Datta, Nicholas G Polson, and Brandon Willard. The horseshoe+ estimator of ultra-sparse signals.Bayesian Analysis, 12(4):1105–1131, 2017

    work page 2017

  4. [4]

    Dirichlet–laplace priors for optimal shrinkage.Journal of the American Statistical Association, 110(512):1479–1490, 2015

    Anirban Bhattacharya, Debdeep Pati, Natesh S Pillai, and David B Dunson. Dirichlet–laplace priors for optimal shrinkage.Journal of the American Statistical Association, 110(512):1479–1490, 2015

    work page 2015

  5. [5]

    Graph neural networks with convolutional arma filters.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3496–3507, 2021

    Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. Graph neural networks with convolutional arma filters.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3496–3507, 2021

    work page 2021

  6. [6]

    Optimal approximation with sparsely connected deep neural networks.SIAM Journal on Mathematics of Data Science, 1(1):8–45, 2019

    Helmut Bolcskei, Philipp Grohs, Gitta Kutyniok, and Philipp Petersen. Optimal approximation with sparsely connected deep neural networks.SIAM Journal on Mathematics of Data Science, 1(1):8–45, 2019

    2019

  7. [7]

    Triple the gamma—a unifying shrinkage prior for variance and variable selection in sparse state space and tvp models.Econometrics, 8(2):20, 2020

    Annalisa Cadonna, Sylvia Frühwirth-Schnatter, and Peter Knaus. Triple the gamma—a unifying shrinkage prior for variance and variable selection in sparse state space and tvp models.Econometrics, 8(2):20, 2020

    work page 2020

  8. [8]

    Handling sparsity via the horseshoe

    Carlos M Carvalho, Nicholas G Polson, and James G Scott. Handling sparsity via the horseshoe. InArtificial Intelligence and Statistics, pages 73–80. PMLR, 2009

    work page 2009

Show all 63 references
  1. [9]

    Fastgcn: Fast learning with graph convolu-tional networks via impor- tance sampling

    Jie Chen, Tengfei Ma, and Cao Xiao. Fastgcn: Fast learning with graph convolu-tional networks via impor- tance sampling. InInternational Conference on Learning Representations. International Conference on Learning Representations, ICLR, 2018

    2018

  2. [10]

    Stochastic gradient hamiltonian monte carlo

    Tianqi Chen, Emily Fox, and Carlos Guestrin. Stochastic gradient hamiltonian monte carlo. InInternational Conference on Machine Learning, pages 1683–1691. PMLR, 2014

    2014

  3. [11]

    Diffusive gibbs sampling

    Wenlin Chen, Mingtian Zhang, Brooks Paige, José Miguel Hernández-Lobato, and David Barber. Diffusive gibbs sampling. InProceedings of the 41st Inter- national Conference on Machine Learning, pages 7731–7747,

  4. [12]

    Learning graph convolutional networks for multi- label recognition and applications.IEEE Transactions on JOURNAL OF LATEX CLASS FILES, VOL

    Zhaomin Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Learning graph convolutional networks for multi- label recognition and applications.IEEE Transactions on JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 13 Pattern Analysis and Machine Intelligence, 2021

    2015

  5. [13]

    Efficient and scalable bayesian neural nets with rank-1 factors

    Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snoek, Katherine Heller, Balaji Lakshmi- narayanan, and Dustin Tran. Efficient and scalable bayesian neural nets with rank-1 factors. InInternational Conference on Machine Learning, pages 2782–2792. PMLR, 2020

    2020

  6. [14]

    Radial bayesian neural networks: Beyond discrete support in large-scale bayesian deep learning

    Sebastian Farquhar, Michael A Osborne, and Yarin Gal. Radial bayesian neural networks: Beyond discrete support in large-scale bayesian deep learning. InInter- national Conference on Artificial Intelligence and Statistics, pages 1352–1362. PMLR, 2020

    2020

  7. [15]

    Encoding the latent posterior of bayesian neural networks for uncertainty quantification.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    Gianni Franchi, Andrei Bursuc, Emanuel Aldea, Séver- ine Dubuisson, and Isabelle Bloch. Encoding the latent posterior of bayesian neural networks for uncertainty quantification.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  8. [16]

    Dropout as a bayesian approximation: Representing model uncer- tainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncer- tainty in deep learning. InInternational Conference on Machine Learning, pages 1050–1059. PMLR, 2016

    2016

  9. [17]

    The gr2d2 estimator for the precision matrices.Briefings in Bioinformatics, 23(6):bbac426, 2022

    Dailin Gan, Guosheng Yin, and Yan Dora Zhang. The gr2d2 estimator for the precision matrices.Briefings in Bioinformatics, 23(6):bbac426, 2022

    2022

  10. [18]

    Topology- aware graph pooling networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4512– 4518, 2021

    Hongyang Gao, Yi Liu, and Shuiwang Ji. Topology- aware graph pooling networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4512– 4518, 2021

    2021

  11. [19]

    A bayesian approach to recurrence in neural networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8):2527–2537, 2020

    Philip N Garner and Sibo Tong. A bayesian approach to recurrence in neural networks.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8):2527–2537, 2020

    2020

  12. [20]

    Model selection in bayesian neural networks via horse- shoe priors.J

    Soumya Ghosh, Jiayu Yao, and Finale Doshi-Velez. Model selection in bayesian neural networks via horse- shoe priors.J. Mach. Learn. Res., 20(182):1–46, 2019

    2019

  13. [21]

    Riemann mani- fold langevin and hamiltonian monte carlo methods

    Mark Girolami and Ben Calderhead. Riemann mani- fold langevin and hamiltonian monte carlo methods. Journal of the Royal Statistical Society Series B: Statistical Methodology, 73(2):123–214, 2011

    2011

  14. [22]

    Structured shrink- age priors.Journal of Computational and Graphical Statis- tics, 33(1):1–14, 2024

    Maryclare Griffin and Peter D Hoff. Structured shrink- age priors.Journal of Computational and Graphical Statis- tics, 33(1):1–14, 2024

    2024

  15. [23]

    Forecasting macroe- conomic data with bayesian vars: Sparse or dense? it depends!International Journal of Forecasting, 2025

    Luis Gruber and Gregor Kastner. Forecasting macroe- conomic data with bayesian vars: Sparse or dense? it depends!International Journal of Forecasting, 2025. URL https://doi.org/10.1016/j.ijforecast.2025.02.001

    2025 doi

  16. [24]

    Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding

    Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Confer- ence Track P...

    2016

  17. [25]

    Stochastic variational inference.Journal of Machine Learning Research, 2013

    Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference.Journal of Machine Learning Research, 2013

    2013

  18. [26]

    Hands-on bayesian neural networks-a tutorial for deep learning users

    Laurent Valentin Jospin. Hands-on bayesian neural networks-a tutorial for deep learning users. 2020

    2020

  19. [27]

    What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Processing Systems, 30, 2017

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Processing Systems, 30, 2017

    2017

  20. [28]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  21. [29]

    Imagenet classification with deep convolutional neural networks.Advances in Neural Information Processing Systems, 25, 2012

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks.Advances in Neural Information Processing Systems, 25, 2012

    2012

  22. [30]

    Simple and scalable predictive un- certainty estimation using deep ensembles.Advances in Neural Information Processing Systems, 30, 2017

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive un- certainty estimation using deep ensembles.Advances in Neural Information Processing Systems, 30, 2017

    2017

  23. [31]

    Handwritten digit recognition with a back-propagation network.Advances in Neural Informa- tion Processing Systems, 2, 1989

    Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence Jackel. Handwritten digit recognition with a back-propagation network.Advances in Neural Informa- tion Processing Systems, 2, 1989

    1989

  24. [32]

    Graphmax for text genera- tion.Journal of Artificial Intelligence Research, 78:823–848, 2023

    Bin Liu and Guosheng Yin. Graphmax for text genera- tion.Journal of Artificial Intelligence Research, 78:823–848, 2023

    2023

  25. [33]

    Bayesian compression for deep learning.Advances in Neural Information Processing Systems, 30, 2017

    Christos Louizos, Karen Ullrich, and Max Welling. Bayesian compression for deep learning.Advances in Neural Information Processing Systems, 30, 2017

    2017

  26. [34]

    Predictive uncertainty estimation via prior networks.Advances in Neural Information Processing Systems, 31, 2018

    Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks.Advances in Neural Information Processing Systems, 31, 2018

    2018

  27. [35]

    The ridgelet prior: A covariance function ap- proach to prior specification for bayesian neural net- works.Journal of Machine Learning Research, 22(157):1–57, 2021

    Takuo Matsubara, Chris J Oates, and François-Xavier Briol. The ridgelet prior: A covariance function ap- proach to prior specification for bayesian neural net- works.Journal of Machine Learning Research, 22(157):1–57, 2021

    2021

  28. [36]

    Variational dropout sparsifies deep neural networks

    Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov. Variational dropout sparsifies deep neural networks. InInternational Conference on Machine Learning, pages 2498–2507. PMLR, 2017

    2017

  29. [37]

    Stochastic gradient markov chain monte carlo.Journal of the American Statistical Association, 116(533):433–450, 2021

    Christopher Nemeth and Paul Fearnhead. Stochastic gradient markov chain monte carlo.Journal of the American Statistical Association, 116(533):433–450, 2021

    2021

  30. [38]

    Sparsity information and regularization in the horseshoe and other shrinkage priors.Electronic Journal of Statistics, 11(2):5018–5051, 2017

    Juho Piironen and Aki Vehtari. Sparsity information and regularization in the horseshoe and other shrinkage priors.Electronic Journal of Statistics, 11(2):5018–5051, 2017

    2017

  31. [39]

    Posterior concentration for sparse deep learning.Advances in Neural Information Processing Systems, 31, 2018

    Nicholas G Polson and Veronika Roˇ cková. Posterior concentration for sparse deep learning.Advances in Neural Information Processing Systems, 31, 2018

    2018

  32. [40]

    Interpretable outcome prediction with sparse bayesian neural networks in intensive care.arXiv preprint arXiv:1905.02599, 29:34, 2019

    Anna-Lena Popkes, Hiske Overweg, Ari Ercole, Yingzhen Li, José Miguel Hernández-Lobato, Yordan Zaykov, and Cheng Zhang. Interpretable outcome prediction with sparse bayesian neural networks in intensive care.arXiv preprint arXiv:1905.02599, 29:34, 2019

    1905 arXiv

  33. [41]

    Tractable function-space variational infer- ence in bayesian neural networks.Advances in Neural Information Processing Systems, 35:22686–22698, 2022

    Tim GJ Rudner, Zonghao Chen, Yee Whye Teh, and Yarin Gal. Tractable function-space variational infer- ence in bayesian neural networks.Advances in Neural Information Processing Systems, 35:22686–22698, 2022

    2022

  34. [42]

    Evidential deep learning to quantify classification uncer- tainty.Advances in Neural Information Processing Systems, 31, 2018

    Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncer- tainty.Advances in Neural Information Processing Systems, 31, 2018

    2018

  35. [43]

    A comprehensive guide to bayesian convolutional neural network with variational inference.arXiv preprint arXiv:1901.02731, 2019

    Kumar Shridhar, Felix Laumann, and Marcus Liwicki. A comprehensive guide to bayesian convolutional neural network with variational inference.arXiv preprint arXiv:1901.02731, 2019. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 14

    1901 arXiv

  36. [44]

    Understanding measures of uncertainty for adversarial example detection

    Lewis Smith and Yarin Gal. Understanding measures of uncertainty for adversarial example detection. In Amir Globerson and Ricardo Silva, editors,Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6...

    2018

  37. [45]

    Generalized dropout.arXiv preprint arXiv:1611.06791, 2016

    Suraj Srinivas and R Venkatesh Babu. Generalized dropout.arXiv preprint arXiv:1611.06791, 2016

    2016 arXiv

  38. [47]

    Consistent sparse deep learning: Theory and computation.Journal of the American Statistical Association, 117(540):1981–1995, 2022

    Yan Sun, Qifan Song, and Faming Liang. Consistent sparse deep learning: Theory and computation.Journal of the American Statistical Association, 117(540):1981–1995, 2022

    1981

  39. [48]

    Learning sparse deep neural networks with a spike-and-slab prior

    Yan Sun, Qifan Song, and Faming Liang. Learning sparse deep neural networks with a spike-and-slab prior. Statistics & Probability letters, 180:109246, 2022

    2022

  40. [49]

    Collapsed variational bounds for bayesian neural networks.Advances in Neural Information Processing Systems, 34:25412–25426, 2021

    Marcin Tomczak, Siddharth Swaroop, Andrew Foong, and Richard Turner. Collapsed variational bounds for bayesian neural networks.Advances in Neural Information Processing Systems, 34:25412–25426, 2021

    2021

  41. [50]

    All you need is a good functional prior for bayesian deep learning.Journal of Machine Learning Research, 23(74):1–56, 2022

    Ba-Hien Tran, Simone Rossi, Dimitrios Milios, and Maurizio Filippone. All you need is a good functional prior for bayesian deep learning.Journal of Machine Learning Research, 23(74):1–56, 2022

    2022

  42. [51]

    Graph attention networks

    Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. InInternational Conference on Learn- ing Representations, 2018

    2018

  43. [52]

    Sparse bayesian learning for end-to-end eeg decoding

    Wenlong Wang, Feifei Qi, David Wipf, Chang Cai, Tianyou Yu, Yuanqing Li, Zhuliang Yu, and Wei Wu. Sparse bayesian learning for end-to-end eeg decoding. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2023

    2023

  44. [53]

    Semi-supervised classification with graph convolutional networks

    Max Welling and Thomas N Kipf. Semi-supervised classification with graph convolutional networks. InJ. International Conference on Learning Representations (ICLR 2017), 2016

    2017

  45. [55]

    Bayesian learning via stochastic gradient langevin dynamics

    Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. InProceedings of the 28th International Conference on Machine Learning (ICML-11), pages 681–688. Citeseer, 2011

    2011

  46. [56]

    Fashion- mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion- mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    2017

  47. [57]

    How powerful are graph neural networks? In International Conference on Learning Representations, 2018

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2018

    2018

  48. [58]

    Scalable stochastic gradient riemannian langevin dynamics in non-diagonal metrics.Transactions on machine learning research, 2023(8), 2023

    Hanlin Yu, Marcelo Hartmann, Bernardo Williams Moreno Sanchez, and Arto Klami. Scalable stochastic gradient riemannian langevin dynamics in non-diagonal metrics.Transactions on machine learning research, 2023(8), 2023

    2023

  49. [59]

    Scalable stochastic gradient riemannian langevin dynamics in non-diagonal metrics.Transactions on Machine Learning Research, 2023

    Hanlin Yu, Marcelo Hartmann, Bernardo Williams, and Arto Klami. Scalable stochastic gradient riemannian langevin dynamics in non-diagonal metrics.Transactions on Machine Learning Research, 2023

    2023

  50. [60]

    Bayesian regression using a prior on the model fit: The r2-d2 shrinkage prior.Journal of the American Statistical Association, pages 1–13, 2020

    Yan Dora Zhang, Brian P Naughton, Howard D Bondell, and Brian J Reich. Bayesian regression using a prior on the model fit: The r2-d2 shrinkage prior.Journal of the American Statistical Association, pages 1–13, 2020

    2020

  51. [61]

    Robust graph representation learning via neural sparsification

    Cheng Zheng, Bo Zong, Wei Cheng, Dongjin Song, Jingchao Ni, Wenchao Yu, Haifeng Chen, and Wei Wang. Robust graph representation learning via neural sparsification. InInternational Conference on Machine Learning, pages 11458–11468. PMLR, 2020. Tsai Hor Chanreceived the BSc degr...

    2020

  52. [64]

    For all sufficiently largen,P ∗ n p[d(pw, pµ∗ )>4ϵ n|D]≥2e −nϵ2 n/4 o ≤2e −nϵ2 n/4

  53. [65]

    log 1 ψjl √ 2π exp (1−µψ jl)2 2ψjlµ !# −E q(ψjl |·) log 1 2 e− 1 2 ψjl =Eq(Y|·)

    For all sufficiently largen,E ∗ D {p[d(pw, pµ∗ )>4ϵ n|D]} ≤4e−nϵ2 n/2, where P∗ and E∗ denote the respective probability measure and expectation with respect to the data D, and p[·] represents the posterior distribution. Lemma 1.(Density of the R2D2 prior [60]) The density of ...

    2015

  54. [2021]

    His research focuses on medical image analysis, computer vision, machine learning and AI in healthcare

    He is currently an Assistant Professor in Department of Statistics and Actuarial Science, University of Hong Kong. His research focuses on medical image analysis, computer vision, machine learning and AI in healthcare. APPENDIX JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUG...

    2015

  55. [2024]

    URL https://dl.acm.org/doi/10.5555/3692070. 3692373

    doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.