Pith. sign in

REVIEW 4 major objections 4 minor 26 references

Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Any pretrained classifier can be retrofitted with uncertainty by attaching a probabilistic head to an intermediate layer selected with neural-collapse metrics, avoiding retraining.

desk verdict A practical retrofit for deterministic UQ that mostly delivers on its claims, but the layer-selection heuristic needs stronger evidence before the no-retraining promise generalizes. read the letter →

arxiv 2501.04816 v1 pith:UQG4W7EZ submitted 2025-01-08 cs.LG stat.ML

Felix Jimenez , Matthias Katzfuss This is my paper · ORCID
classification cs.LGstat.ML
keywords deterministicuncertaintyquantificationout-of-distributiondetectionneuralcollapseprobabilisticskipconnectionsspectralnormalizationintermediatelayersfeaturesingle-passinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deterministic uncertainty quantification (UQ) normally requires a feature extractor that is both sensitive and smooth, and existing single-pass methods enforce this by retraining with spectral normalization. This paper proposes probabilistic skip connections (PSCs): instead of retraining, use two neural-collapse statistics to find an intermediate layer of an already-trained network that already has both properties. A distance-aware probabilistic model is fit to a projected version of that layer's features, giving epistemic and aleatoric uncertainty in one forward pass. On CIFAR-10 and Dirty-MNIST experiments, PSC-equipped networks match or exceed the in-distribution calibration and out-of-distribution detection of spectral-normalized networks, and the approach also works on networks without residual connections. The practical payoff is that uncertainty estimates can be added to existing classifiers with a lightweight post-hoc fit on training data.

What carries the argument

The probabilistic skip connection is a distance-aware probabilistic head attached to the output of a chosen intermediate layer instead of the network's penultimate layer. The layer is selected by a collapse-accuracy trade-off rule: compute $\mathrm{NC}_1 = \operatorname{Tr}(\Sigma_W)/\operatorname{Tr}(\Sigma_T)$ and $\mathrm{NC}_4$ (nearest-centroid accuracy) at every layer, then take the deepest layer with $\mathrm{NC}_1 > 0.2$ while $\mathrm{NC}_4$ remains high, optionally combining the layer just before and after collapse. $\mathrm{NC}_1$ and $\mathrm{NC}_4$ act as proxies for the sensitivity and smoothness bounds of Eq. (1). The intermediate feature map is reshaped, channel-wise whitened, and projected with a tensor factorisation of its channel-wise covariance to a low-dimensional vector, on which the probabilistic head is fit.

What would settle it

Measure the empirical sensitivity constant $L_1$ of Eq. (1) at the selected candidate layer on an out-of-distribution benchmark; if $L_1$ is effectively zero for OOD inputs even though $\mathrm{NC}_1$ stays above 0.2 on the training set, the neural-collapse proxy has failed and PSC uncertainty would be overconfident.

Watch

Extended reading notes

Core claim

The paper's central claim is that the bi-Lipschitz condition underlying deterministic UQ does not need to be enforced during training: a normally trained network contains an intermediate layer that is both sensitive and smooth, because sensitivity and smoothness trade off across depth. The neural-collapse metric $\mathrm{NC}_1$ (the ratio of within-class to total feature variance) is used as a proxy for sensitivity, and $\mathrm{NC}_4$ (nearest-centroid accuracy) as a proxy for smoothness; the candidate layer is the deepest one before collapse sets in that still has high $\mathrm{NC}_4$. Placing a distance-aware head—quadratic discriminant analysis for out-of-distribution detection and a linear model with a Gaussian posterior approximation for predictions—on projected features from this layer yields uncertainty quality comparable to spectral-normalized training. Experiments on ResNet-50, ResNet-18, Wide-ResNet-28-10, and VGG-16 support the claim, including a VGG-16 network without residual connections where the penultimate-layer feature space is poorly structured.

Load-bearing premise

The load-bearing premise is that an intermediate layer picked by neural-collapse statistics on training and validation data will also be both sensitive and smooth for unseen out-of-distribution inputs, so that distances in that feature space remain meaningful for inputs the network has never seen.

Editorial extensions

If this is right

  • Any pretrained classification network can be given epistemic and aleatoric uncertainty estimates via a post-hoc fit on training data, with no change to the network architecture or training procedure.
  • Deterministic UQ methods such as DDU and SNGP-style distance-aware heads become usable on networks that lack residual connections, where the penultimate-layer feature space is typically too collapsed for reliable uncertainty.
  • Spectral normalization is demoted from a requirement to one possible way of satisfying the sensitivity-smoothness condition, because the same effect is achievable by layer selection.
  • The collapse-accuracy trade-off can also serve as a measure of a network's effective depth, identifying how early the classifier has already committed to its decision boundary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to apply the same collapse-accuracy rule to language models or other non-vision architectures, where spectral normalization is rarely used and retraining is expensive; the projection step would need to be adapted to token-level representations.
  • The fixed $\mathrm{NC}_1 > 0.2$ cutoff is a plausible weak point; a data-driven variant that picks the layer maximizing validation out-of-distribution separation could make the rule more robust across architectures.
  • If the proxy holds broadly, PSCs effectively give a cheap drop-in UQ retrofit for deployed models, relevant for production systems that cannot afford ensembles or Bayesian inference at inference time.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces probabilistic skip connections (PSCs) as a post-hoc method for deterministic uncertainty quantification (UQ) in classification neural networks. Rather than retraining a network with spectral normalization (SN) to satisfy a bi-Lipschitz condition, the authors propose using neural-collapse metrics (NC1 and NC4) to identify an intermediate layer that is both sensitive and smooth. They then project that layer's activations to a low-dimensional space via Tucker decomposition and fit a distance-aware probabilistic model (quadratic discriminant analysis for OOD detection and a KFAC-Laplace linear model for in-distribution UQ). Experiments on CIFAR-10, Dirty-MNIST, and FashionMNIST with ResNet-18, ResNet-50, VGG-16, and Wide-ResNet-28-10 suggest that PSCs can match or exceed DDU with spectral normalization, while also applying to architectures without residual connections.

Significance. If the central claims hold, PSCs would be a practically valuable contribution: they promise a drop-in retrofit for existing pretrained classifiers, require no retraining, and extend deterministic UQ to non-residual architectures. The paper provides pseudo-code for the full pipeline, reports 10-seed error bars in the main in-distribution UQ table (Table 4), and includes a robustness study of the projection step. However, the layer-selection rule that the method's success depends on is under-specified and is validated only on a single architecture/dataset pair (ResNet-50/CIFAR-10) in Section 5.1. The empirical evidence for the headline OOD claim is also thinner than the text suggests, with no error bars in Table 2 and narrow baseline coverage. These gaps currently limit the strength of the conclusions.

major comments (4)
  1. [Section 4.1, Section 5.1] The layer-selection rule in Section 4.1 is not precisely specified. The criterion "taking both the layer before and after collapse if NC1 is especially close to epsilon" admits subjective judgment, and the only validation that NC1/NC4 serve as proxies for the bi-Lipschitz condition is a single architecture/dataset pair (ResNet-50 on CIFAR-10 in Section 5.1) with no out-of-distribution inputs. Since the paper's central claim is that PSCs can be dropped into any pretrained classifier, the authors need a fully specified selection rule or a systematic study across multiple architectures, datasets, and OOD shifts to support generality.
  2. [Section 5.2, Table 2] Table 2 reports AUROC values for OOD detection and aleatoric/epistemic separation without any error bars or repeated-seed variability, in contrast to the 10-seed results in Table 4. These point estimates are the primary evidence that PSCs "match or exceed" DDU with spectral normalization. Without variability estimates it is impossible to judge whether the reported differences are meaningful; the authors should report means and standard errors over multiple seeds or otherwise justify the single-run comparison.
  3. [Section 5.3, Table 3] The evaluation that projection preserves sensitivity and smoothness is circular: the projection dimensions c_proj and d_proj are chosen to keep NC1 and NC4 nearly unchanged, and the same metrics are then reported in Table 3 as evidence of preservation. The robustness study in Figure 5 also uses NC1/NC4 as the sole outcome. To break this loop, the authors should validate the projection using external metrics, such as OOD detection AUROC or in-distribution NLL, as a function of the projection dimensions.
  4. [Section 5, overall baselines] The headline claim that PSCs "match or exceed existing single-pass methods requiring training modifications" is only tested against a single baseline, DDU with spectral normalization (Table 2), and each model is evaluated on only one OOD source (FashionMNIST for Dirty-MNIST). No comparisons to SNGP, DUQ, or other deterministic UQ methods are reported in the experiments, and there is no evaluation on more diverse OOD benchmarks (e.g., SVHN, LSUN, CIFAR-100 vs CIFAR-10). The stated breadth of the claim is therefore not supported by the presented experimental evidence.
minor comments (4)
  1. [Section 5.2] The text says both that a multinomial logistic regression model is used to "replace the original network predictions" and that "in-distribution (iD) predictions use network outputs." This is contradictory or at least unclear; please clarify which outputs are used for iD predictions and for predictive entropy.
  2. [Section 4.1] The value of epsilon = 0.2 is presented as a fixed cutoff, but the phrase "especially close to epsilon" leaves the rule ambiguous; a precise operationalization (e.g., a tolerance on |NC1 - epsilon|) would make the method reproducible.
  3. [Figure 3] The caption and text refer to "boxes" highlighting layers that meet the criteria, but no boxes are actually visible in the figure; please add the annotation or remove the reference.
  4. [Table 1] The abbreviation "Jacob." in the "F.C. Soln." row for DUQ is not defined in the caption or the body; please spell out "Jacobian regularization" or provide a reference.

Circularity Check

1 steps flagged · score 4.0 of 10

One supporting claim is self-validating: the projection dimensions are chosen to keep NC1/NC4 unchanged, and Tab. 3 then reports NC1/NC4 as unchanged. The central OOD/UQ claim is benchmarked externally (AUROC/NLL/ECE), so the circularity is partial rather than total.

  1. fitted input called prediction [Sec. 4.2 (projection guidance) and Sec. 5.3 / Tab. 3]
    "we recommend setting cproj and dproj to be the smallest values such that neither N C1 and N C4 change dramatically. ... Tab. 3 presents the values ofN C1 and N C4 computed before and after projection. ... the results show that both before and after projection, there is no collapse (N C1 > ϵ), and the nearest-centroid accuracy remains nearly unchanged."

    The projection hyperparameters cproj and dproj are selected by the rule 'smallest values such that neither N C1 and N C4 change dramatically.' The validation in Tab. 3 then reports that after projection, 'there is no collapse (N C1 > ϵ), and the nearest-centroid accuracy remains nearly unchanged.' The target quantities of the experiment are the same quantities used by the selection rule, so the 'preservation' claim is not an independent prediction: any choice satisfying the recommendation automatically produces the reported outcome with respect to the metrics used as evidence. The robustness sweep in Fig. 5 mitigates but does not remove the loop, because success is still evaluated in terms of the same NC1/NC4 criterion rather than an external OOD/UQ metric.

full rationale

The core derivation is not circular: the layer-selection rule in Sec. 4.1 chooses a layer using NC1/NC4 on training/validation data, while the headline claims (OOD detection and iD UQ matching spectral-normalized training) are tested on held-out and OOD data via feature-density AUROC, predictive-entropy AUROC, NLL, and ECE in Secs. 5.2 and 5.4. These benchmarks do not reduce to the selection metrics, so the central UQ/OOD result has independent content. The one place where the paper's validation loop closes is Sec. 5.3: Sec. 4.2 recommends choosing cproj and dproj as the smallest values that leave NC1 and NC4 'not change dramatically,' and the 'Projecting preserves feature geometry' experiment then reports that NC1 and NC4 are essentially unchanged (Tab. 3). With respect to the metrics used as evidence, that claim is true by construction; the robustness sweep in Fig. 5 provides some independent support, but the headline statement in Tab. 3 remains a selection rule restated as a finding. I also checked the other listed patterns: the citation to the authors' own DVE work [10] is not load-bearing (the intermediate-layer idea is independently supported by external references [2,7,21], and PSC does not build on DVE), and no uniqueness theorem is imported from the authors. The NC1/NC4-to-bi-Lipschitz mapping is presented as an analogy ('akin to', 'resembling'), not an identity, so it is better characterized as a validity risk than as a circular step.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a hand-chosen collapse cutoff, empirically tuned projection dimensions, and several domain assumptions about what neural-collapse metrics measure. No new physical entities are introduced. The method itself is a combination of known components reframed for classification UQ.

free parameters (5)
  • Neural collapse cutoff epsilon = 0.2
    Chosen by hand in Section 4.1 as the threshold for feature variability suppression; the paper gives no data-driven justification and it directly determines which layers are candidate PSC layers.
  • Channel projection dimension c_proj = dataset-dependent (e.g., 50-250 in Fig. 5)
    Chosen empirically in Section 5.3 as the smallest value such that NC1 and NC4 do not change dramatically; the UQ results could depend on it.
  • Spatial projection dimension d_proj = dataset-dependent (up to 15 in Fig. 5)
    Same as c_proj; chosen empirically in Section 5.3 to preserve neural-collapse metrics.
  • Optional final PCA dimension = unspecified in paper
    In Appendix B.2, an additional PCA-based projection is applied for the DDU case study, but its dimension is not reported; this affects the AUROC values in Table 2.
  • Spectral norm constant for SN baselines = 3.0
    Taken from DDU training hyperparameters (Table 5); used for baselines, not for PSC itself.
assumptions (5)
  • domain assumption A pretrained classifier contains at least one intermediate layer whose features are both sensitive and smooth enough for distance-aware UQ.
    The paper's central mechanism; Section 4.1 states the assumption ('Assuming such a subnetwork exists') and Section 5.1 provides evidence only for ResNet-50 on CIFAR-10.
  • domain assumption NC1 = Tr(Sigma_W)/Tr(Sigma_T) with cutoff 0.2 reliably indicates the absence of feature collapse.
    Section 4.1 introduces the 0.2 cutoff as a proxy; no calibration or theoretical justification is given.
  • domain assumption High nearest-centroid accuracy (NC4) of an intermediate layer implies semantic smoothness that supports OOD detection.
    Section 4.1 'Why use NC1 and NC4?' argues this heuristically; no formal link to the bi-Lipschitz bound in Eq. 1.
  • standard math The Tucker decomposition of channelwise covariance gives a projection that preserves the relevant feature geometry.
    Standard tensor decomposition tool; the paper verifies empirically in Table 3 for 4 configurations, but the projection is fit on training data.
  • domain assumption KFAC Laplace approximation gives a calibrated posterior for the linear head on the projected features.
    Borrowed from prior work (Daxberger et al., Ritter et al.); the paper relies on it for NLL/ECE results in Table 4 without validating the approximation on these features.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks." pith.science (2026). https://pith.science/paper/UQG4W7EZ

@misc{pith2026250104816,
  author       = {Pith},
  title        = {Pith review of: Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UQG4W7EZ}},
  note         = {Machine review of arXiv:2501.04816}
}
read the original abstract

Deterministic uncertainty quantification (UQ) in deep learning aims to estimate uncertainty with a single pass through a network by leveraging outputs from the network's feature extractor. Existing methods require that the feature extractor be both sensitive and smooth, ensuring meaningful input changes produce meaningful changes in feature vectors. Smoothness enables generalization, while sensitivity prevents feature collapse, where distinct inputs are mapped to identical feature vectors. To meet these requirements, current deterministic methods often retrain networks with spectral normalization. Instead of modifying training, we propose using measures of neural collapse to identify an existing intermediate layer that is both sensitive and smooth. We then fit a probabilistic model to the feature vector of this intermediate layer, which we call a probabilistic skip connection (PSC). Through empirical analysis, we explore the impact of spectral normalization on neural collapse and demonstrate that PSCs can effectively disentangle aleatoric and epistemic uncertainty. Additionally, we show that PSCs achieve uncertainty quantification and out-of-distribution (OOD) detection performance that matches or exceeds existing single-pass methods requiring training modifications. By retrofitting existing models, PSCs enable high-quality UQ and OOD capabilities without retraining.

Figures

Figures reproduced from arXiv: 2501.04816 by the authors.

Figure 1
Figure 1. Neural-collapse metrics help identify intermediate layers without feature collapse but high nearest-centroid accu￾racy, allowing us to place probabilistic skip connections (PSCs) that enhance uncertainty quantification. The top left shows the trade-off between collapse and accuracy across network depth. In this example, dog, mouse, and cat images are processed through a network trained only on dog and cat images, wi… view at source ↗
Figure 2
Figure 2. Adding PSCs to a pretrained model involves measuring collapse (left), projecting intermediate layers to a feature vector (center), and fitting a probabilistic model to the feature vector (right). The first step is to find a subset of the layers that tradeoff accuracy and collapse, higher is better for both accuracy and collapse. Once we have those layers we then combine them and project to a single feature vector. F… view at source ↗
Figure 3
Figure 3. SN delays neural collapse to the final layer, whereas without SN, intermediate layers behave similarly to the penul￾timate layer of a network trained with SN. (Left) SN pushes the point where N C1 drops below 0.2 to the final layer. Without SN, there is a region where N C1 remains above 0.2, but its val￾ues differ from those observed with SN. (Right) SN reduces the performance of the nearest-centroid classifier (NCC… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: (a) PSC feature density separates iD and OOD just as well as when using SN. The panels each show the probability of the feature vector under GDA trained on the embeddings of the training data, but each panel differs in how the feature vectors are computed. The base net…
Figure 5
Figure 5. Figure 5: shows the impact of varying cproj and dproj on the neural collapse metrics for ResNet-18 on MNIST, with sim￾ilar trends observed across other networks and datasets. As expected, there are some values of cproj and dproj where a previously uncollapsed layer (N C1 > ϵ) be…
Figure 6
Figure 6. Figure 6: highlights that we choose the candidate layer using measures of neural collapse. In particular we meausure N C1 and N C4 after each layer. This is different than the work of Rangamani et al. [21] where the metrics are measured before each layer. This difference arises …
Figure 7
Figure 7. Figure 7: We can handle different layer outputs in a similar way to convolutional layers. The candidate layers of a network are both fully connected layers, and after concatenating and projecting the feature vector can be handled just as before. B. Experiment details B.1. Networ…
Figure 9
Figure 9. Figure 9: In panel (b) we have the histograms for Resnet-18 [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 8
Figure 8. Figure 8: As the upper bound for the spectral norm increases unique points collapse at earlier layers, but are unique for some early layer. This figure shows the 2D intermediate representations after PCA for the sign-example from van Amersfoort et al. [25] as we change the upper…
Figure 9
Figure 9. Figure 9: Predictive entropy can be used to separate iD-Ambiguous and iD-Clean. The figure shows the predictive entropy values when the base model used was Resnet-18 (a) or VGG-16 (b). 5 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 19 canonical work pages

  1. [1]

    Rep- resenting smooth functions as compositions of near-identity functions with implications for deep network optimization

    Peter L Bartlett, Steven N Evans, and Philip M Long. Rep- resenting smooth functions as compositions of near-identity functions with implications for deep network optimization. arXiv preprint arXiv:1804.05012, 2018

    arXiv 2018

  2. [2]

    Nearest class-center sim- plification through intermediate layers

    Ido Ben-Shaul and Shai Dekel. Nearest class-center sim- plification through intermediate layers. In Topological, Al- gebraic and Geometric Learning Workshops 2022 , pages 37–47. PMLR, 2022

    work page 2022

  3. [3]

    Stochastic gra- dient hamiltonian monte carlo

    Tianqi Chen, Emily Fox, and Carlos Guestrin. Stochastic gra- dient hamiltonian monte carlo. In International conference on machine learning, pages 1683–1691. PMLR, 2014

    work page 2014

  4. [4]

    Laplace redux–effortless Bayesian deep learning

    Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux–effortless Bayesian deep learning. In NeurIPS, 2021

    work page 2021

  5. [5]

    Adam: A method for stochastic opti- mization

    P Kingma Diederik. Adam: A method for stochastic opti- mization. (No Title), 2014

    work page 2014

  6. [6]

    Dropout as a bayesian ap- proximation: Representing model uncertainty in deep learn- ing

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian ap- proximation: Representing model uncertainty in deep learn- ing. In international conference on machine learning , pages 1050–1059. PMLR, 2016

    work page 2016

  7. [7]

    On the implicit bias towards minimal depth of deep neural net- works, 2022

    Tomer Galanti, Liane Galanti, and Ido Ben-Shaul. On the implicit bias towards minimal depth of deep neural net- works, 2022. URL https://arxiv.org/abs/2202. 09028

    work page 2022

  8. [8]

    Probabilis- tic forecasting

    Tilmann Gneiting and Matthias Katzfuss. Probabilis- tic forecasting. Annual Review of Statistics and Its Application, 1(1):125–151, 1 2014. ISSN 2326-8298. doi: 10.1146/annurev-statistics-062713-085831. URL http://www.annualreviews.org/doi/abs/10. 1146/annurev-statistics-062713-085831

    doi:10.1146/annurev-statistics-062713-085831 2014

Show all 26 references
  1. [9]

    Strictly proper scoring rules, prediction, and estimation

    Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359–378, 2007

    2007

  2. [10]

    Vecchia gaussian process ensembles on internal representations of deep neural networks

    Felix Jimenez and Matthias Katzfuss. Vecchia gaussian process ensembles on internal representations of deep neural networks. arXiv preprint arXiv:2305.17063, 2023

    2023 arXiv

  3. [11]

    Tensor decomposi- tions and applications

    Tamara G Kolda and Brett W Bader. Tensor decomposi- tions and applications. SIAM review, 51(3):455–500, 2009. Publisher: SIAM

    2009

  4. [12]

    Simple and scalable predictive uncertainty estima- tion using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estima- tion using deep ensembles. Advances in neural information processing systems, 30, 2017

    2017

  5. [13]

    Gradient-based learning applied to document recog- nition

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  6. [14]

    Simple and principled uncertainty estimation with deterministic deep learning via distance awareness

    Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshminarayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Infor- mation Processing Systems, 33:7498–7512, 2020

    2020

  7. [15]

    Sgdr: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016

    2016 arXiv

  8. [16]

    A simple baseline for bayesian uncertainty in deep learning

    Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. Advances in neural information processing systems, 32, 2019

    2019

  9. [17]

    Spectral normalization for generative adver- sarial networks

    Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adver- sarial networks. arXiv preprint arXiv:1802.05957, 2018

    2018 arXiv

  10. [18]

    Deep deterministic un- certainty: A new simple baseline

    Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic un- certainty: A new simple baseline. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 24384–24394, 2023

    2023

  11. [19]

    Bayesian learning for neural networks , volume 118 of Lecture Notes in Statistics

    Radford M Neal. Bayesian learning for neural networks , volume 118 of Lecture Notes in Statistics . Springer New York, NY , 1996

    1996

  12. [20]

    Prevalence of neural collapse during the terminal phase of deep learning training

    Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences , 117(40):24652–24663, 2020

    2020

  13. [21]

    Feature learning in deep classifiers through intermediate neural collapse

    Akshay Rangamani, Marius Lindegaard, Tomer Galanti, and Tomaso A Poggio. Feature learning in deep classifiers through intermediate neural collapse. In International con- ference on machine learning , pages 28729–28745. PMLR, 2023

    2023

  14. [22]

    A scalable laplace approximation for neural networks

    Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable laplace approximation for neural networks. In 6th International Conference on Learning Representations, ICLR 2018-Conference Track Proceedings, volume 6. International Conference on Representation Learning, 2018

    2018

  15. [23]

    Very deep con- volutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep con- volutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations , pages 1–14, 2015

    2015

  16. [24]

    Implications of factor analysis of three- way matrices for measurement of change

    Ledyard R Tucker. Implications of factor analysis of three- way matrices for measurement of change. Problems in mea- suring change, 15(122-137):3, 1963. 9 Probabilistic Skip Connections

    1963

  17. [25]

    tucker" ) # fit PSC. net = fit_psc( net=net, projection=projection, psc=

    Joost van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using a single deep de- terministic neural network. In International conference on machine learning, pages 9690–9700. PMLR, 2020. 10 Probabilistic Skip Connections Supplementary Material Sup...

    2020

  18. [26]

    During training, we apply the SN procedure from Mukhoti et al

    with the learning rate adjusted from 3e-2 to 3e-4 using cosine annealing [15], additionally we set the weight-decay to 1e-5. During training, we apply the SN procedure from Mukhoti et al. [18] using two different values for the SN constant: one large and one small. After train...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.