PAC-Bayes with Backprop

Csaba Szepesvari; Omar Rivasplata; Vikram M Tankasali

REVIEW 2 major objections 4 minor 29 references

PAC-Bayes with Backprop

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that backprop-minimized PAC-Bayes bounds on MNIST yield 1.4% test error together with a 2.3% risk certificate, an order of magnitude tighter than previous non-vacuous bounds.

desk verdict The 2.3% MNIST bound is not a valid PAC-Bayes certificate because the prior scale was picked by grid search on the training data; the rest of the paper is correct but standard. read the letter →

arxiv 1908.07380 v5 pith:PQMV4KFB submitted 2019-08-19 cs.LG stat.ML

Omar Rivasplata , Vikram M Tankasali , Csaba Szepesvari This is my paper · ORCID

classification cs.LGstat.ML

keywords PAC-Bayesprobabilisticneuralnetworksriskcertificatesnon-vacuousboundsself-boundinglearningBayesbybackpropMNISTstochasticgradientdescent

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to show that a neural network can be trained by minimizing PAC-Bayes risk bounds directly, so that the output of training is not just a predictor but also a valid numerical guarantee on its misclassification risk. On MNIST, the two proposed training objectives—one built on a known PAC-Bayes-lambda bound and one on a new quadratic bound—achieve roughly 1.4% test error while producing a risk upper bound of about 2.3%, an order of magnitude tighter than the previously reported 0.21 certificate. The authors read this as evidence for self-bounding learning: the same training data yields both the model and the certificate, with no separate data split needed. They use classical fixed data-free priors centered at randomly initialized weights and ordinary SGD, in contrast to earlier work that needed data-dependent priors.

What carries the argument

The load-bearing objects are two PAC-Bayes upper bounds on the average risk $Q[L]$ of a randomized weight distribution $Q$. The PAC-Bayes-quadratic bound (Theorem 1, Eq. (6)) comes from inverting a binary-KL inequality and solving a quadratic inequality for the square root of $Q[L]$; the PAC-Bayes-$\lambda$ bound (Theorem 2, Eq. (7)) follows from the inequality $\sqrt{ab}\le(\lambda a + b/\lambda)/2$ and is minimized by alternating over $\lambda$ and $Q$. Both bounds are functions of the empirical risk, the KL divergence to a fixed data-free prior $Q_0$, the sample size $n$, and the confidence $\delta$. To make the objectives trainable by backprop, the paper uses the pathwise gradient estimator to differentiate through samples drawn from Gaussian or Laplace posteriors centered at the randomly initialized weights.

What would settle it

Fix every prior hyperparameter, including the prior scale, before seeing the MNIST labels, run the same PBB training, and recompute the risk bound; if the certificate is materially above 2.3%, the reported number depends on data-driven prior selection rather than on the training method.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that PAC-Bayes with Backprop (PBB) works: minimizing the two objectives in Eqs. (6) and (7) with SGD produces a stochastic neural network whose average risk $Q[L]$ is both small and certified. The headline numbers are a test-set error near 0.014 on MNIST and a simultaneous non-vacuous upper bound near 0.023, compared with the previous best non-vacuous bound of 0.21 from data-dependent priors; on Binary MNIST the paper reports 0.015 test error with a 0.022 bound. The authors conclude that fixed data-free priors and standard SGD may suffice for self-bounding learning on MNIST, and they report similarly close bound/error pairs on five UCI data sets, while conceding that initial CIFAR-10 experiments gave certificates about three times the test error.

Load-bearing premise

The 2.3% guarantee holds only if the prior distribution, including its scale, is fixed without using the training data; the reported run was chosen from a grid sweep that picked the best bound on that same data.

Editorial extensions

If this is right

If the central claim is right, a single training run on the available data produces both a neural network and a high-probability guarantee on its true risk, so the test set is no longer required to certify generalization.
A certificate near the test error means PAC-Bayes bounds can be used as practical model-selection and early-stopping criteria, not just as theoretical afterthoughts.
On MNIST, the data-dependent-prior machinery used in earlier work is not necessary for tight certificates; plain SGD with fixed priors reaches the same goal.
The UCI and Binary MNIST results suggest the bound/error gap stays small on small and mid-size problems, while the CIFAR-10 gap shows the method has not yet scaled to large natural-image tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A fair re-test that fixes the prior scale before seeing the training data would likely loosen the reported 2.3% value, because the paper's grid sweep selected the best bound over that same data; this is testable and should be done before treating the certificate as valid.
The KL term is the main obstacle on larger networks, so structured posteriors, weight coupling, or hierarchical priors are the natural next levers for extending PBB to CIFAR-10-class problems.
Because the same machinery emits a certificate for any posterior, PBB could be used to fine-tune or compress an already trained network while producing a fresh, valid bound.
The near-equality of bound and test error suggests the bound itself could serve as the training signal for architecture search, provided the prior is kept data-free.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The only genuinely new thing here is the empirical claim: vanilla SGD with fixed Gaussian or Laplace priors gives a 2.3% PAC-Bayes risk bound on MNIST, an order of magnitude tighter than Dziugaite and Roy. That claim does not survive contact with the paper's own experimental description. Section 6 says plainly that on every dataset they grid swept all hyperparameters and selected the run with the best risk upper bound, and the sweep includes the prior scale. So the prior Q0 is data-dependent: the chosen scale depends on the training data through the bound. Theorems 1 and 2 require Q0 fixed before seeing data. No union bound over grid configurations is applied. The 2.3% number is therefore not a valid high-probability certificate. The stress-test note is right.

Credit where due: the paper is transparent about its setup, which is why the flaw is so easy to see. Theorem 1 is correctly derived from the cited kl-inequality, though it is a few algebraic lines and not really novel. The experiments are clean and the comparison to BBB and SGD is useful. They also honestly report the CIFAR-10 failure, which limits their own self-bounding claim. The theoretical derivations are mathematically sound.

A second soft spot, minor by comparison: the bound applies to the clamped cross-entropy loss, while the reported test error is 0/1 loss. So the claimed tight gap between 1.4% test error and 2.3% bound is not on the same loss. That is a comparability issue, not the main problem.

Who is this for? People working on PAC-Bayes and self-bounding learning. It is a good cautionary example of how easy it is to void a bound via hyperparameter selection. I would not accept it as is, but I would not desk-reject it either. A revision that applies a proper union bound over the grid or uses a hold-out set for prior selection could make the empirical claim solid. As it stands, the central result is unsupported, so my vote is reject.

Referee Report

2 major / 4 minor

Summary. The paper proposes two PAC-Bayes training objectives for probabilistic neural networks: the objective flamb in Eq. (7), derived from the PAC-Bayes-lambda bound of Thiemann et al., and fquad in Eq. (6), derived from a quadratic inversion of the PAC-Bayes-kl bound. Both objectives are minimized by backpropagation-style gradient estimates. The authors report experiments on MNIST and five UCI datasets, claiming test error around 1.4% on MNIST together with non-vacuous risk upper bounds around 2.3%, which they describe as much tighter than earlier results by Dziugaite and Roy. They argue that these results suggest 'self-bounding learning' with fixed, data-free priors and classical SGD, without a data-splitting protocol.

Significance. If the reported certificates were valid, the paper would be a significant step in PAC-Bayes training of neural networks: it would show that a simple method using classical SGD and fixed priors can produce both competitive predictions and non-vacuous risk guarantees, and it would provide a useful baseline against methods that rely on data-dependent priors or specialized optimizers. The theoretical derivations of Theorems 1 and 2 appear mathematically sound under the stated assumptions, and the connection to the Bayes-by-Backprop objective is informative. However, the central empirical claim depends on the validity of the reported risk certificates, and that validity is undermined by the data-dependent selection of the prior scale and other hyperparameters. The significance is therefore conditional on repairing the experimental protocol.

major comments (2)

[Section 6, Section 6.1] The reported PAC-Bayes certificate is voided by data-dependent prior-scale selection. The paper states that 'we performed a grid sweep over all the hyper parameters and then selected the run with the best risk upper bound,' and the swept hyperparameters include the prior distribution scale parameter with standard deviation values in [0.1, 0.09, ..., 1e-3], as well as pmin, learning rate, and momentum. Theorems 1 and 2 require Q0 to be a fixed, data-free distribution. A prior scale selected by minimizing the bound on the same training data is data-dependent, so the high-probability statement no longer holds for the selected run. Section 6.1 defends only the randomness of weight initialization as external to the training process; it does not address the selection of the prior scale or the selection of the best run. No union bound over the configuration grid is given. Consequently, the 2.3% bound in Table 1 is not a valid PAC-Bayes risk certificate, and the comparisons in Tables 1 and 2 do not support the paper's headline claim of tighter valid bounds.
[Abstract, Section 7] The claim of 'self-bounding learning' is unsupported by the experimental protocol. The abstract explicitly frames the contribution as obtaining competitive test error and simultaneously computing a valid risk bound 'with no need to follow a data-splitting protocol.' That framing relies entirely on the prior being data-free. Since the prior scale and the reported run are selected using the training data, the bound is not a valid high-probability guarantee, and the conclusion in Section 7 that PBB 'output models that come with a performance guarantee' is not justified. A correct protocol would require fixing the prior before seeing the training data, using a separate validation set for hyperparameter selection, or applying a union bound over the grid; none of these is present.

minor comments (4)

[Section 3] There is a typo in the sentence 'there are many other PC-Bayes bounds available'; this should read 'PAC-Bayes bounds.'
[Section 4] The boundedness of the cross-entropy loss is not fully specified. Lower-bounding network probabilities by pmin makes the loss finite but does not by itself place it in the interval [0,1] required by Theorems 1 and 2; if the bound is evaluated for the cross-entropy loss, a normalization step or explicit statement is needed, and if the reported bounds are for the zero-one loss, this should be stated clearly.
[Section 6, Eq. (3)] The inversion step leading to Eq. (3) uses the inequality kl(p||p') >= (p'-p)^2/(2p'), which is stated to be valid for p' < p, but this side condition is not mentioned in the text surrounding Eq. (3).
[Section 6.2] The paper does not specify how the plotted and tabulated 'risk upper bound' is computed at the end of training: which loss function, which value of delta, and whether the bound is evaluated at the last iterate or at the iterate with the minimal bound. This information is needed for reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported 2.3% MNIST bound is a post-selection minimum over data-chosen prior scales, not a valid PAC-Bayes certificate.

fitted input called prediction [Section 6, 'In all the experiments...' and Section 6.1; Eqs. (4) and (5)]
"In all the experiments on various data sets we performed a grid sweep over all the hyper parameters and then selected the run with the best risk upper bound. ... We did a grid sweep over the prior distribution scale parameter with standard deviation values in [0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 4e-2, 3e-2, 2e-2, 1e-2, 5e-3, 1e-3]. The variance of the posterior distribution was initialized to the same value as the prior distribution variance."

The reported 0.023 risk upper bound is not a bound guaranteed by Theorems 1 and 2, because those theorems require a fixed, data-free prior Q0. Here the prior scale was selected by minimizing the same PAC-Bayes bound on the same training data, and the best run over the grid was then reported as the certificate. Section 6.1 defends only the randomness of weight initialization as 'data-free'; it does not address the data-dependent choice of the prior scale or the selection of the best of many runs. Without a union bound over grid configurations, the minimum of many data-dependent bounds is not a valid high-probability statement.

full rationale

The derivations of the two training objectives themselves are self-contained: Eq. (4) is obtained by solving the quadratic inequality (3), and Eq. (5) is the PAC-Bayes-lambda bound of Thiemann et al.; neither bound has fitted constants, and no self-citation chain is load-bearing. Test error values are measured on a held-out test set, so the 1.4% error estimate is not circular. The circularity is confined to the central certificate claim: the prior scale hyperparameter is chosen by sweeping over values and picking the run with the smallest bound on the training data, and that selected bound is then reported as a valid risk certificate. This makes the reported 2.3% bound a post-selection minimum rather than a certificate satisfying the data-free-prior condition of Theorems 1 and 2. Consequently, the paper's 'self-bounding learning' conclusion is partially circular in the sense that the predicted certificate is the output of an optimization over certificates. The mathematical bound derivations are not circular, so the score is not higher.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. The theoretical structure rests on standard PAC-Bayes results. The practical protocol introduces data-dependent selection of the prior scale and other hyperparameters, which is the main threat to validity and is captured in the free parameters list.

free parameters (4)

Prior scale parameter (sigma) for Gauss/Laplace prior = Selected from grid [0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 4e-2, 3e-2, 2e-2, 1e-2, 5e-3, 1e-3] by best risk upper bound
This hyperparameter controls the prior distribution's variance. It is chosen on the training data by minimizing the computed PAC-Bayes bound, making the prior data-dependent and invalidating the certificate.
Learning rate = Selected from grid [1e-3, 5e-3, 1e-2]
Grid-selected based on the risk upper bound, contributing to the data-dependent selection that breaks the bound's validity.
Momentum = Selected from grid [0.95, 0.99]
Grid-selected based on the risk upper bound, part of the data-dependent hyperparameter selection.
p_min (minimum probability clamp) = Selected from grid [1e-2, 1e-3, 1e-4, 1e-5, 1e-8, 1e-16]
Clamps network probabilities to bound the cross-entropy loss. Selection is data-dependent and affects the loss magnitude, and whether the bound is on a [0,1]-valued loss is not clarified.

assumptions (3)

standard math PAC-Bayes-kl theorem (Langford-Seeger/Maurer)
Invoked in Section 3 to derive the initial kl upper bound on Q[L] and is the foundation for both Theorem 1 and Theorem 2.
standard math Inequality kl(p_hat || p) >= (p - p_hat)^2 / (2p) for p_hat < p
Used in Section 3 to convert the kl bound into the quadratic inequality that leads to Eq. (3) and Theorem 1.
domain assumption Prior centered at randomly initialized weights satisfies the data-free requirement
Section 6.1 argues the randomness in initialization is external to training so the prior is data-free. The argument addresses the centroid but not the scale parameter, which is selected using the training data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PAC-Bayes with Backprop." pith.science (2026). https://pith.science/paper/PQMV4KFB

@misc{pith2026190807380,
  author       = {Pith},
  title        = {Pith review of: PAC-Bayes with Backprop},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQMV4KFB}},
  note         = {Machine review of arXiv:1908.07380}
}

read the original abstract

We explore the family of methods "PAC-Bayes with Backprop" (PBB) to train probabilistic neural networks by minimizing PAC-Bayes bounds. We present two training objectives, one derived from a previously known PAC-Bayes bound, and a second one derived from a novel PAC-Bayes bound. Both training objectives are evaluated on MNIST and on various UCI data sets. Our experiments show two striking observations: we obtain competitive test set error estimates (~1.4% on MNIST) and at the same time we compute non-vacuous bounds with much tighter values (~2.3% on MNIST) than previous results. These observations suggest that neural nets trained by PBB may lead to self-bounding learning, where the available data can be used to simultaneously learn a predictor and certify its risk, with no need to follow a data-splitting protocol.

Figures

Figures reproduced from arXiv: 1908.07380 by the authors.

**Figure 1.** Objectives ‘pb lambda’ and ‘pb quad’ with Gauss and Laplace distributions. Plots for risk upper bound and normalized KL divergence (KL(QkQ0 )/ntrain) vs training iterations for MNIST. The KL divergence starts at 0.0, increases quickly and then starts decreasing later in the training. training and performed a grid sweep over learning rate in [1e − 3, 5e − 3, 1e − 2] and momentum in [0.95, 0.99]. We found that learnin… view at source ↗

**Figure 2.** Plots for training objectives and test errors vs the training iterations for MNIST. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 24 canonical work pages

[1]

Weight uncertainty in neural networks

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks . In 32nd International Conference on Machine Learning, pages 1613--1622, 2015

work page 2015
[2]

Stochastic gradient descent tricks

L \'e on Bottou. Stochastic gradient descent tricks . In Neural networks: Tricks of the trade, pages 421--436. Springer, 2012

work page 2012
[3]

Concentration inequalities: A nonasymptotic theory of independence

St \'e phane Boucheron, G \'a bor Lugosi, and Pascal Massart. Concentration inequalities: A nonasymptotic theory of independence . Oxford university press, 2013

2013
[4]

Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping

Rich Caruana, Steve Lawrence, and C Lee Giles. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping . In Advances in Neural Information Processing Systems, pages 402--408, 2001

work page 2001
[5]

UCI Machine Learning Repository , 2017

Dheeru Dua and Casey Graff. UCI Machine Learning Repository , 2017. URL http://archive.ics.uci.edu/ml

work page 2017
[6]

Gintare Karolina Dziugaite and Daniel M. Roy. Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data . In UAI, 2017

work page 2017
[7]

Data-dependent PAC-Bayes priors via differential privacy

Gintare Karolina Dziugaite and Daniel M Roy. Data-dependent PAC-Bayes priors via differential privacy . In Advances in Neural Information Processing Systems, pages 8430--8441, 2018

work page 2018
[8]

Self bounding learning algorithms

Yoav Freund. Self bounding learning algorithms . In Proceedings of the eleventh annual conference on Computational Learning Theory, pages 247--258. ACM, 1998

work page 1998

Show all 29 references

[9]

Keeping neural networks simple

Geoffrey E Hinton and Drew van Camp. Keeping neural networks simple . In International Conference on Artificial Neural Networks, pages 11--18. Springer, 1993

1993
[10]

Pathwise Derivatives Beyond the Reparameterization Trick

Martin Jankowiak and Fritz Obermeyer. Pathwise Derivatives Beyond the Reparameterization Trick . arXiv:1806.01851, 2018

2018 arXiv
[11]

Microchoice bounds and self bounding learning algorithms

John Langford and Avrim Blum. Microchoice bounds and self bounding learning algorithms . Machine Learning, 51 0 (2): 0 165--179, 2003

2003
[12]

(Not) bounding the true error

John Langford and Rich Caruana. (Not) bounding the true error . In Advances in Neural Information Processing Systems, pages 809--816, 2001

2001
[13]

Bounds for averaging classifiers

John Langford and Matthias Seeger. Bounds for averaging classifiers . Technical Report CMU-CS-01-102 , Carnegie Mellon University , 2001

2001
[14]

Distribution-dependent PAC-Bayes priors

Guy Lever, Fran c ois Laviolette, and John Shawe-Taylor. Distribution-dependent PAC-Bayes priors . In International Conference on Algorithmic Learning Theory, pages 119--133. Springer, 2010

2010
[15]

Tighter PAC-Bayes bounds through distribution-dependent priors

Guy Lever, Fran c ois Laviolette, and John Shawe-Taylor. Tighter PAC-Bayes bounds through distribution-dependent priors . Theoretical Computer Science, 473: 0 4--28, 2013

2013
[16]

A PAC-Bayesian analysis of randomized learning with application to stochastic gradient descent

Ben London. A PAC-Bayesian analysis of randomized learning with application to stochastic gradient descent . In Advances in Neural Information Processing Systems, pages 2931--2940, 2017

2017
[17]

A note on the PAC Bayesian theorem

Andreas Maurer. A note on the PAC Bayesian theorem . arXiv:cs/0411099, 2004

2004 arXiv
[18]

Bayesian Learning via Stochastic Dynamics

Radford M Neal. Bayesian Learning via Stochastic Dynamics . In Advances in Neural Information Processing Systems, pages 475--482, 1993

1993
[19]

Exploring generalization in deep learning

Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning . In Advances in Neural Information Processing Systems, pages 5947--5956, 2017 a

2017
[20]

A PAC-Bayesian approach to spectrally-normalized margin bounds for neural networks

Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A PAC-Bayesian approach to spectrally-normalized margin bounds for neural networks . arXiv:1707.09564, 2017 b

2017 arXiv
[21]

Robust forward algorithms via PAC-Bayes and Laplace distributions

Asaf Noy and Koby Crammer. Robust forward algorithms via PAC-Bayes and Laplace distributions . In Artificial Intelligence and Statistics, pages 678--686, 2014

2014
[22]

PAC-Bayesian Margin Bounds for Convolutional Neural Networks

Konstantinos Pitas, Mike Davies, and Pierre Vandergheynst. PAC-Bayesian Margin Bounds for Convolutional Neural Networks . arXiv:1801.00171, 2017

2017 arXiv
[23]

A useful theorem for nonlinear devices having Gaussian inputs

Robert Price. A useful theorem for nonlinear devices having Gaussian inputs . IRE Transactions on Information Theory, 4 0 (2): 0 69--72, 1958

1958
[24]

PAC-Bayesian Generalization Error Bounds for Gaussian Process Classification

Matthias Seeger. PAC-Bayesian Generalization Error Bounds for Gaussian Process Classification . Journal of Machine Learning Research, 3: 0 233--269, 2002

2002
[25]

Understanding Machine Learning

Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning . Cambridge University Press, Cambridge, 2014

2014
[26]

Dropout: a simple way to prevent neural networks from overfitting

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting . Journal of Machine Learning Research, 15 0 (1): 0 1929--1958, 2014

1929
[27]

A strongly quasiconvex PAC-Bayesian bound

Niklas Thiemann, Christian Igel, Olivier Wintenberger, and Yevgeny Seldin. A strongly quasiconvex PAC-Bayesian bound . In International Conference on Algorithmic Learning Theory, pages 466--492, 2017

2017
[28]

Regularization of neural networks using dropconnect

Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun, and Rob Fergus. Regularization of neural networks using dropconnect . In International Conference on Machine Learning, pages 1058--1066, 2013

2013
[29]

Are All Layers Created Equal? arXiv:1902.01996, 2019

Chiyuan Zhang, Samy Bengio, and Yoram Singer. Are All Layers Created Equal? arXiv:1902.01996, 2019

1902 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Weight uncertainty in neural networks

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks . In 32nd International Conference on Machine Learning, pages 1613--1622, 2015

work page 2015

[2] [2]

Stochastic gradient descent tricks

L \'e on Bottou. Stochastic gradient descent tricks . In Neural networks: Tricks of the trade, pages 421--436. Springer, 2012

work page 2012

[3] [3]

Concentration inequalities: A nonasymptotic theory of independence

St \'e phane Boucheron, G \'a bor Lugosi, and Pascal Massart. Concentration inequalities: A nonasymptotic theory of independence . Oxford university press, 2013

2013

[4] [4]

Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping

Rich Caruana, Steve Lawrence, and C Lee Giles. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping . In Advances in Neural Information Processing Systems, pages 402--408, 2001

work page 2001

[5] [5]

UCI Machine Learning Repository , 2017

Dheeru Dua and Casey Graff. UCI Machine Learning Repository , 2017. URL http://archive.ics.uci.edu/ml

work page 2017

[6] [6]

Gintare Karolina Dziugaite and Daniel M. Roy. Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data . In UAI, 2017

work page 2017

[7] [7]

Data-dependent PAC-Bayes priors via differential privacy

Gintare Karolina Dziugaite and Daniel M Roy. Data-dependent PAC-Bayes priors via differential privacy . In Advances in Neural Information Processing Systems, pages 8430--8441, 2018

work page 2018

[8] [8]

Self bounding learning algorithms

Yoav Freund. Self bounding learning algorithms . In Proceedings of the eleventh annual conference on Computational Learning Theory, pages 247--258. ACM, 1998

work page 1998