Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes

Daniel M. Roy; Jun Yang; Shengyang Sun

REVIEW 2 major objections 3 minor 44 references

Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes

T0 review · 2 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper uses shifted Rademacher processes to prove fast-rate PAC-Bayes bounds and introduces a flatness-driven bound that converges as 1/m.

desk verdict New shifted-Rademacher PAC-Bayes framework, but Theorem 4.3's constant is wrong as stated. read the letter →

arxiv 1908.07585 v2 pith:3AOPLWK2 submitted 2019-08-20 cs.LG math.STstat.MLstat.TH

Jun Yang , Shengyang Sun , Daniel M. Roy This is my paper · ORCID

classification cs.LGmath.STstat.MLstat.TH MSC 68Q32

keywords PAC-BayesboundsRademachercomplexityshiftedprocessesfastlearningratesflatminimaGibbsclassifiersgeneralizationKLdivergence

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that Rademacher-process techniques and PAC-Bayesian theory are not as disjoint as the literature suggests, and that shifted Rademacher processes are a natural bridge between them. Its first contribution is a proof, up to constants, of the classic fast-rate PAC-Bayes bound: instead of the usual $\mathrm{O}(1/\sqrt{m})$ gap, the generalization gap shrinks as $\mathrm{KL}(Q\|P)/m$ plus $\log(1/\delta)/m$. Its second contribution is a new bound of the same fast type in which the usual inflation of empirical risk is replaced by an empirical flatness term: the average squared distance between a posterior draw's loss on a training point and the posterior's expected loss on that point. The paper shows that a posterior that concentrates in a flat region of the empirical risk surface pays only a KL penalty, which clarifies in what sense flat minima are good for generalization.

What carries the argument

The machinery is the shifted Rademacher process, an average $\frac{1}{m}\sum_i \varepsilon'_i f(z_i)$ in which each $\varepsilon'_i$ takes two values with equal probability but is not zero mean. The step that produces fast rates is the convex dual identity $\sup_{Q:\mathrm{KL}(Q\|P)\le\kappa} \frac{1}{m}\sum_i \varepsilon'_i \mathbb{E}_Q f(z_i) = \inf_{\lambda>0}\{\kappa/\lambda + \frac{1}{\lambda}\log \mathbb{E}_P \exp(\frac{\lambda}{m}\sum_i \varepsilon'_i f(z_i))\}$, combined with a shifted symmetrization-in-deviation inequality that turns a tail of the generalization gap into a tail of shifted Rademacher processes. The final piece is an elementary moment bound (Lemma A.5) that caps the exponential expectation at 1 when the loss variables are Bernoulli, which is what replaces the $\mathrm{O}(1/\sqrt{m})$ bounded-differences term with $\mathrm{O}(1/m)$.

What would settle it

Fix a single function $f$, draw $z_i$ i.i.d., and simulate the expectation in Eq. (25) with $Y_i = \mathbb{E}_{\hat Q(\epsilon,z)} f(z_i)$ under the paper's parameter choices; if any configuration yields a value greater than 1 when averaging conditionally on $f$ rather than over the prior, then the step that supports Theorem 4.3 is false.

Watch

Extended reading notes

Core claim

The main result, Theorem 4.3, states that for any $c>0$ and $h\in(0,1)$, with probability at least $1-\delta$ over $m$ i.i.d. examples, every posterior $Q$ over the loss class satisfies $$L_D(Q) \le \hat L_S(Q) + \frac{c}{m}\sum_i \mathbb{E}_Q[f(z_i)-(1+h)\mathbb{E}_Q f(z_i)]^2 + \frac{4}{Cm}\left[3\,\mathrm{KL}(Q\|P)+\log(1/\delta)+5\right],$$ where $C=2h^4c/(1+16h^2c)$. The central object on the right, the $h$-flatness, is the paper's new notion of flatness. Under zero-one loss, $h$-flatness equals $\hat L_S(Q) - \frac{1-h^2}{m}\sum_i (\mathbb{E}_Q f(z_i))^2$, so the bound improves on simply inflating the empirical risk exactly when the posterior's expected losses on the training points are nonzero. The proof proceeds by a shifted symmetrization in deviation and a convex-duality step for the KL ball, and the paper shows independently (Proposition 3.1) that the same shifted-Rademacher machinery recovers the classical fast-rate PAC-Bayes bound up to constants.

Load-bearing premise

The proof's key expectation bound assumes the loss values are independent Bernoulli random variables, whereas in the theorem they are independent only after conditioning on a function drawn from the prior, and the paper never spells out that conditional step.

Editorial extensions

If this is right

If Theorem 4.3 is correct, algorithms that explicitly minimize $h$-flatness on the training set, not just empirical risk, will enjoy a generalization gap that scales as $\mathrm{O}(1/m)$ with a KL penalty.
The shifted-Rademacher derivation gives a proof of the classical fast-rate bound that does not pass through the usual change-of-measure argument, so future work can transfer the techniques to other divergences or bounded losses.
The flatness term can vanish even when the posterior's empirical variance is large, so the bound captures a different phenomenon than empirical-Bernstein PAC-Bayes bounds and may be tighter in exactly those cases.
Because the bound holds uniformly for all posteriors with the same prior, it is directly usable as an objective for PAC-Bayes posterior optimization, for instance in neural network training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct extension, not explored in the paper, is to replace the fixed prior with a data-dependent prior built by differentially private mechanisms; the shifted-Rademacher machinery would then need a new proof, but the flatness term would likely survive.
The paper's preliminary note that optimized posteriors are not flat suggests the practical bottleneck for this bound may be optimization rather than looseness; a testable prediction is that constraining flatness during posterior optimization yields tighter nonvacuous bounds on real datasets.
The moment bound in Lemma A.5 is the step that depends on an independent-Bernoulli structure; if a conditional version cannot be supplied, the flatness bound may still hold but needs a different proof.
Because the constants are not optimized, the proof leaves room for sharper shifted symmetrization inequalities; tightening $C$ in Theorem 4.3 could make the bound competitive in finite-sample deep learning applications without changing the core ideas.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this paper because of the framework, not because of the main theorem as written.

The genuinely new thing is the proof technique: shifted Rademacher processes plus convex duality to get fast-rate PAC-Bayes bounds. Proposition 3.1 rederives Catoni's bound up to constants, which is not new as a result but is a clean proof of concept. The flatness theorem (Theorem 4.3) is the actual contribution—empirical flatness of the posterior appears in the bound, which I haven't seen before. The authors also give a good related-work section, correctly separating their flatness from empirical variance and from Audibert's distribution-dependent notion.

But the main theorem as stated has a real gap. The stress-test note is right: the constants chosen in Eq. (107), C = 2h^4c/(1+16h^2c) and c2 = h^2c/(1+16h^2c), do not satisfy the proof's own condition (77) for small c. The issue is that C is O(c) while the gap h^2c - c2 is O(c^2), so e^A + e^{-B} > 2. Numerically it's tiny—about 2.0000014—but the theorem claims all c>0. This is a localized error: the proof can be repaired by choosing a smaller C (e.g., O(h^4c^2) with the same c2), which changes the displayed bound but not the structure of the argument.

There is also a smaller clarity issue in Lemma A.5: it says 'independent Bernoulli variables X_i', but in the application X_i = f(z_i) where f is one draw from the prior. Independence holds only conditionally on f. The lemma should be stated conditionally, and then the product step is fine. The reader's concern here is real but minor.

The paper is honest about the empirical side: preliminary experiments show optimized posteriors are not flat in this sense, which the authors report themselves. That doesn't invalidate the theory, but it does limit the practical significance until someone finds posteriors that do concentrate on flat regions.

Who should read this: PAC-Bayes theorists and anyone working on fast-rate bounds. The proof framework is worth engaging, and the flatness notion may be useful even if the current constant is wrong. I'd send it to peer review, but only with the expectation that the proof of Theorem 4.3 is corrected before acceptance. It's not ready as-is.

Referee Report

2 major / 3 minor

Summary. The paper develops a connection between Rademacher-process techniques and PAC-Bayesian generalization bounds. It first shows that shifted Rademacher processes can recover Catoni's fast-rate bound up to constants, and then introduces a new fast-rate PAC-Bayes bound (Theorem 4.3) that depends on the empirical "flatness" of the posterior. The proofs use symmetrization-in-deviation, convex duality, and exponential moment bounds for shifted Rademacher variables.

Significance. If the main theorem is correct, the flatness-dependent bound is a novel contribution that gives a Rademacher-process route to fast-rate PAC-Bayes bounds and a new measure of posterior flatness. The paper includes detailed proofs and makes its derivations transparent, and the recovery of Catoni's bound via shifted Rademacher processes is a useful conceptual bridge. However, the proof of the main theorem contains a load-bearing constant error, so the main result as stated is not currently established.

major comments (2)

[Theorem 4.3, Eq. (107), Lemma A.5] The constants chosen in Eq. (107) do not satisfy the hypothesis of Lemma A.5. Lemma A.5 requires 0 < λ/m < (h^2c - c2)/(2(1+h^2c)(1+c2)) in order for e^A + e^{-B} ≤ 2 in Eq. (77). Substituting c2 = h^2c/(1+16h^2c) and C = 2h^4c/(1+16h^2c) yields, for h = 0.5 and c = 0.01, A = C(1+c2) ≈ 0.0012048, B = C(1+h^2c) ≈ 0.0012049, and e^A + e^{-B} ≈ 2.000002 > 2. Thus Lemma A.5 cannot be applied, and Theorem 4.3 as stated is unproved. The proof can be repaired by choosing a smaller C, for example C = 8h^4c^2 / ((1+h^2c)(1+17h^2c)) with the same c2, but then the explicit bound in Eq. (15) changes. This issue is load-bearing because Theorem 4.3 is the paper's main contribution.
[Eq. (100), Eq. (107)] The "sufficient if" assertion in Eq. (107) also leaves the second constraint in Eq. (100) unverified. That constraint involves κ + log(4/δ), and since the subsequent union bound runs over κ = 2^{j+1}, the proof should confirm that the fixed constant C satisfies the lower-bound condition for the smallest κ and for all δ ∈ (0,1). For the proposed constants this appears to hold, but the paper does not demonstrate it, and any repair of the first issue must recheck this condition as well.

minor comments (3)

[Lemmas A.2 and A.5] The independence of the random variables X_i = f(z_i) is conditional on the draw f ∼ P. The lemmas should be applied conditionally on f and then integrated over P; the current wording "independent Bernoulli random variables" is ambiguous. The argument is repairable, but the conditional step should be stated explicitly.
[Notation in Appendix A.2] The proof uses the symbol C both for the upper bound on λ/m in Lemma A.5 and for the constant appearing in Theorem 4.3; this clash makes the verification in Eq. (107) easy to misread and should be fixed.
[Section 4.1] The comparison with Catoni's bound introduces the symbols cc, cr, Cc, Cr informally; a small table or a more systematic notation would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the bounds are derived from external shifted-Rademacher symmetrization and convex duality, with no fitted parameter renamed as a prediction.

full rationale

The paper's derivation chain is self-contained rather than circular. Proposition 3.1 and Theorem 4.3 are obtained from external tools (Zhivotovskiy and Hanneke's shifted symmetrization [44, Cor. 1], Catoni's PAC-Bayes bound [8], and convex duality [7]) together with lemmas proved in the paper (Lemmas A.2, A.3, A.4, and A.5). The flatness quantity in Definition 4.1 is defined, not fitted, and Theorem 4.3 proves a generalization guarantee as a function of that quantity; it is not obtained by plugging in the very quantity it is supposed to predict. No parameter is estimated from data and then reported as a prediction, and no data-dependent constants are tuned against a held-out quantity. The only self-citation is the motivational reference to Dziugaite and Roy [9] for the idea of formalizing flat minima via PAC-Bayes bounds; that reference is not load-bearing in the proofs. There is no uniqueness theorem imported from the authors' prior work and no ansatz smuggled in via self-citation. The potential concerns about the proof of Theorem 4.3, such as whether Lemma A.5 is applied conditionally on the draw of f and whether the constants in Eq. (107) satisfy Eq. (77) for all c > 0, are matters of mathematical correctness or missing assumptions, not circularity. Accordingly, the appropriate finding is no significant circularity with score 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central theorem depends on user-chosen constants c and h, the 0/1 loss assumption, and external results on shifted Rademacher symmetrization and convex duality. No parameters are fitted to data, and no new physical entities are introduced. The h-flatness quantity is a new mathematical definition, but it is not an independently falsifiable entity.

free parameters (2)

c (empirical risk inflation constant)
User-chosen positive constant in Proposition 3.1 and Theorem 4.3; appears as (1+c) factor and in C. No data fitting.
h (flatness parameter)
User-chosen in (0,1) defining h-flatness in Definition 4.1. No data fitting.

assumptions (4)

domain assumption The loss class F consists of {0,1}-valued functions, so f^2=f and Eq. (14) is an equality.
Used throughout the flatness theorem; the paper claims extension to bounded loss is straightforward but does not prove it.
domain assumption The shifted symmetrization in deviation of Zhivotovskiy and Hanneke [44, Cor. 7] (Lemma A.1) is valid and applicable to the function class G_kappa.
Black-box external result used in the proofs of Proposition 3.1 and Theorem 4.3; not reproved here.
standard math Convex duality for KL divergence: sup_{Q: KL(Q||P)<=kappa} E_Q[A] <= kappa/lambda + (1/lambda) log E_P[e^{lambda A}].
Used in Eqs. (9), (27), and (93). Standard result, not proved.
domain assumption Data S is i.i.d. from D, prior P is independent of S, and f(z_i) are conditionally independent Bernoulli variables given f.
Standard PAC-Bayes setup; the conditional independence is needed for Lemma A.5 but is not stated explicitly when the lemma is applied.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes." pith.science (2026). https://pith.science/paper/3AOPLWK2

@misc{pith2026190807585,
  author       = {Pith},
  title        = {Pith review of: Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3AOPLWK2}},
  note         = {Machine review of arXiv:1908.07585}
}

read the original abstract

The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari (2008), which is established via Rademacher complexity theory by viewing Gibbs classifiers as linear operators. The goal of this paper is to extend this bridge between Rademacher complexity and state-of-the-art PAC-Bayesian theory. We first demonstrate that one can match the fast rate of Catoni's PAC-Bayes bounds (Catoni, 2007) using shifted Rademacher processes (Wegkamp, 2003; Lecu\'{e} and Mitchell, 2012; Zhivotovskiy and Hanneke, 2018). We then derive a new fast-rate PAC-Bayes bound in terms of the "flatness" of the empirical risk surface on which the posterior concentrates. Our analysis establishes a new framework for deriving fast-rate PAC-Bayes bounds and yields new insights on PAC-Bayesian theory.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 38 canonical work pages

[1]

Simpler P AC-Bayesian bounds fo r hostile data

P . Alquier and B. Guedj. “Simpler P AC-Bayesian bounds fo r hostile data”. Machine Learning 107.5 (2018), pp. 887–902

work page 2018
[2]

Fast learning rates in statistical inf erence through aggregation

J.-Y . Audibert. “Fast learning rates in statistical inf erence through aggregation”. The Annals of Statistics 37.4 (2009), pp. 1591–1646

work page 2009
[3]

Combining P AC-Bayesia n and Generic Chaining Bounds

J.-Y . Audibert and O. Bousquet. “Combining P AC-Bayesia n and Generic Chaining Bounds”. Journal of Machine Learning Research 8 (2007), pp. 863–889

work page 2007
[4]

Local Rad emacher Complexities

P . L. Bartlett, O. Bousquet, and S. Mendelson. “Local Rad emacher Complexities”. The Annals of Statistics 33.4 (2005), pp. 1497–1537

work page 2005
[5]

Rademacher and Gaussia n complexities: Risk bounds and structural results

P . L. Bartlett and S. Mendelson. “Rademacher and Gaussia n complexities: Risk bounds and structural results”. Journal of Machine Learning Research 3 (2002), pp. 463–482

work page 2002
[6]

P AC- Bayesian bounds based on the Rényi divergence

L. Bégin, P . Germain, F. Laviolette, and J.-F. Roy. “P AC- Bayesian bounds based on the Rényi divergence”. In: Artiﬁcial Intelligence and Statistics . 2016, pp. 435–444

work page 2016
[7]

Boyd and L

S. Boyd and L. V andenberghe. Convex Optimization. Cambridge University Press, 2004

2004
[8]

O. Catoni. PAC-Bayesian Supervised Classiﬁcation: The Thermodynami cs of Statistical Learning. V ol. 56. Lecture Notes – Monograph Series. Institute of Mat hematical Statistics, 2007

work page 2007

Show all 44 references

[9]

Computing Nonvacuous Gene ralization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters tha n Training Data

G. K. Dziugaite and D. M. Roy. “Computing Nonvacuous Gene ralization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters tha n Training Data”. In: Proceed- ings of the 33rd Annual Conference on Uncertainty in Artiﬁci al Intelligence (UAI). 2017

2017
[10]

Data-dependent P AC-Baye s priors via differential privacy

G. K. Dziugaite and D. M. Roy. “Data-dependent P AC-Baye s priors via differential privacy”. In: Advances in Neural Information Processing Systems . 2018, pp. 8430–8441

2018
[11]

Entropy-SGD optimizes th e prior of a P AC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priors

G. K. Dziugaite and D. M. Roy. “Entropy-SGD optimizes th e prior of a P AC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priors”. In: International Con- ference on Machine Learning . 2018, pp. 1376–1385

2018
[12]

van Erven

T. van Erven. PAC-Bayes Mini-tutorial: A Continuous Union Bound . 2014. arXiv: 1405.1580

2014 arXiv
[13]

Fast Rates in Statistical and Online Learning

T. van Erven, P . D. Grünwald, N. A. Mehta, M. D. Reid, and R . C. Williamson. “Fast Rates in Statistical and Online Learning”. Journal of Machine Learning Research16 (2015), pp. 1793– 1861

2015
[14]

P AC-Bayesian theory meets Bayesian inference

P . Germain, F. Bach, A. Lacoste, and S. Lacoste-Julien. “P AC-Bayesian theory meets Bayesian inference”. In: Advances in Neural Information Processing Systems . 2016, pp. 1884–1892

2016
[15]

Risk bounds for the ma- jority vote: From a P AC-Bayesian analysis to a learning algo rithm

P . Germain, A. Lacasse, F. Laviolette, M. Marchand, and J.-F. Roy. “Risk bounds for the ma- jority vote: From a P AC-Bayesian analysis to a learning algo rithm”. The Journal of Machine Learning Research 16.1 (2015), pp. 787–860

2015
[16]

Concentration inequalit ies and asymptotic results for ratio type empirical processes

E. Giné and V . Koltchinskii. “Concentration inequalit ies and asymptotic results for ratio type empirical processes”. The Annals of Probability 34.3 (2006), pp. 1143–1216

2006
[17]

A tight excess risk bound via a uniﬁed P AC-Bayesian– Rademacher–Shtarkov–MDL complexity

P . D. Grünwald and N. A. Mehta. “A tight excess risk bound via a uniﬁed P AC-Bayesian– Rademacher–Shtarkov–MDL complexity”. In: Algorithmic Learning Theory . 2019, pp. 433–

2019
[18]

B. Guedj. A primer on PAC-Bayesian learning. 2019. arXiv: 1901.05353

2019 arXiv
[19]

Reﬁned error bounds for several learning a lgorithms

S. Hanneke. “Reﬁned error bounds for several learning a lgorithms”. The Journal of Machine Learning Research 17.1 (2016), pp. 4667–4721

2016
[20]

Minimax analysis of active lear ning

S. Hanneke and L. Y ang. “Minimax analysis of active lear ning”. The Journal of Machine Learning Research 16.1 (2015), pp. 3487–3602. 10

2015
[21]

On the comple xity of linear prediction: risk bounds, margin bounds, and regularization

S. M. Kakade, K. Sridharan, and A. Tewari. “On the comple xity of linear prediction: risk bounds, margin bounds, and regularization”. In: Advances in Neural Information Processing Systems. 2008, pp. 793–800

2008
[22]

Local Rademacher complexities and o racle inequalities in risk minimiza- tion

V . Koltchinskii. “Local Rademacher complexities and o racle inequalities in risk minimiza- tion”. The Annals of Statistics 34.6 (2006), pp. 2593–2656

2006
[23]

Empirical margin di stributions and bounding the gener- alization error of combined classiﬁers

V . Koltchinskii and D. Panchenko. “Empirical margin di stributions and bounding the gener- alization error of combined classiﬁers”. The Annals of Statistics 30.1 (2002), pp. 1–50

2002
[24]

P AC-Bayes bounds for the risk of the majority vote and the variance of the Gibbs cla ssiﬁer

A. Lacasse, F. Laviolette, M. Marchand, P . Germain, and N. Usunier. “P AC-Bayes bounds for the risk of the majority vote and the variance of the Gibbs cla ssiﬁer”. In: Advances in Neural Information Processing Systems. 2007, pp. 769–776

2007
[25]

Tutorial on practical prediction theory for classiﬁcation

J. Langford. “Tutorial on practical prediction theory for classiﬁcation”. Journal of Machine Learning Research 6.Mar (2005), pp. 273–306

2005
[26]

Lecué and S

G. Lecué and S. Mendelson. Learning subgaussian classes: Upper and minimax bounds
[27]

Oracle inequalities for cros s-validation type procedures

G. Lecué and C. Mitchell. “Oracle inequalities for cros s-validation type procedures”. Elec- tronic Journal of Statistics 6 (2012), pp. 1803–1837

2012
[28]

Tighter P AC-Bayes bounds through distribution-dependent priors

G. Lever, F. Laviolette, and J. Shawe-Taylor. “Tighter P AC-Bayes bounds through distribution-dependent priors”. Theoretical Computer Science 473 (2013), pp. 4–28

2013
[29]

Learning with s quare loss: Localization through offset Rademacher complexity

T. Liang, A. Rakhlin, and K. Sridharan. “Learning with s quare loss: Localization through offset Rademacher complexity”. In: Conference on Learning Theory . 2015, pp. 1260–1285

2015
[30]

A P AC-Bayesian analysis of randomized lear ning with application to stochastic gradient descent

B. London. “A P AC-Bayesian analysis of randomized lear ning with application to stochastic gradient descent”. In: Advances in Neural Information Processing Systems . 2017, pp. 2931– 2940

2017
[31]

Risk bounds for statistical learning

P . Massart and É. Nédélec. “Risk bounds for statistical learning”. The Annals of Statistics 34.5 (2006), pp. 2326–2366

2006
[32]

D. A. McAllester. A PAC-Bayesian Tutorial with A Dropout Bound. 2013. arXiv: 1307.2118

2013 arXiv
[33]

P AC-Bayesian Model Averaging

D. A. McAllester. “P AC-Bayesian Model Averaging”. In: Conference on Learning Theory . 1999, pp. 164–170

1999
[34]

Learning without concentration

S. Mendelson. “Learning without concentration”. In: Conference on Learning Theory . 2014, pp. 25–39

2014
[35]

“Local” vs. “global

S. Mendelson. ““Local” vs. “global” parameters–break ing the Gaussian complexity barrier”. The Annals of Statistics 45.5 (2017), pp. 1835–1862

2017
[36]

Exploring generalization in deep learning

B. Neyshabur, S. Bhojanapalli, D. McAllester, and N. Sr ebro. “Exploring generalization in deep learning”. In: Advances in Neural Information Processing Systems . 2017, pp. 5947– 5956

2017
[37]

Neyshabur, S

B. Neyshabur, S. Bhojanapalli, and N. Srebro. A PAC-Bayesian approach to spectrally- normalized margin bounds for neural networks . 2017. arXiv: 1707.09564

2017 arXiv
[38]

A P AC Analysis of a Bayesian Estimator

J. Shawe-Taylor and R. C. Williamson. “A P AC Analysis of a Bayesian Estimator”. In: Con- ference on Learning Theory . 1997, pp. 2–9

1997
[39]

S. L. Smith and Q. V . Le. A Bayesian perspective on generalization and stochastic gr adient descent. 2017. arXiv: 1710.06451

2017 arXiv
[40]

Thiemann, C

N. Thiemann, C. Igel, O. Wintenberger, and Y . Seldin. A strongly quasiconvex PAC-Bayesian bound. 2016. arXiv: 1608.05610

2016 arXiv
[41]

P AC-Bayes-empirical- Bernstein inequality

I. O. Tolstikhin and Y . Seldin. “P AC-Bayes-empirical- Bernstein inequality”. In: Advances in Neural Information Processing Systems. 2013, pp. 109–117

2013
[42]

M. J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint . Cambridge University Press, 2019

2019
[43]

Model selection in nonparametric regress ion

M. Wegkamp. “Model selection in nonparametric regress ion”. The Annals of Statistics 31.1 (2003), pp. 252–273

2003
[44]

Localization of VC cla sses: Beyond local Rademacher complexities

N. Zhivotovskiy and S. Hanneke. “Localization of VC cla sses: Beyond local Rademacher complexities”. Theoretical Computer Science 742 (2018), pp. 27–49. 11 A Proofs A.1 Proof of Proposition 3.1 To match Catoni’s bound, we need to control sup f ∈F LD ( f ) − (1 + c) ˆLS( f ), g...

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Simpler P AC-Bayesian bounds fo r hostile data

P . Alquier and B. Guedj. “Simpler P AC-Bayesian bounds fo r hostile data”. Machine Learning 107.5 (2018), pp. 887–902

work page 2018

[2] [2]

Fast learning rates in statistical inf erence through aggregation

J.-Y . Audibert. “Fast learning rates in statistical inf erence through aggregation”. The Annals of Statistics 37.4 (2009), pp. 1591–1646

work page 2009

[3] [3]

Combining P AC-Bayesia n and Generic Chaining Bounds

J.-Y . Audibert and O. Bousquet. “Combining P AC-Bayesia n and Generic Chaining Bounds”. Journal of Machine Learning Research 8 (2007), pp. 863–889

work page 2007

[4] [4]

Local Rad emacher Complexities

P . L. Bartlett, O. Bousquet, and S. Mendelson. “Local Rad emacher Complexities”. The Annals of Statistics 33.4 (2005), pp. 1497–1537

work page 2005

[5] [5]

Rademacher and Gaussia n complexities: Risk bounds and structural results

P . L. Bartlett and S. Mendelson. “Rademacher and Gaussia n complexities: Risk bounds and structural results”. Journal of Machine Learning Research 3 (2002), pp. 463–482

work page 2002

[6] [6]

P AC- Bayesian bounds based on the Rényi divergence

L. Bégin, P . Germain, F. Laviolette, and J.-F. Roy. “P AC- Bayesian bounds based on the Rényi divergence”. In: Artiﬁcial Intelligence and Statistics . 2016, pp. 435–444

work page 2016

[7] [7]

Boyd and L

S. Boyd and L. V andenberghe. Convex Optimization. Cambridge University Press, 2004

2004

[8] [8]

O. Catoni. PAC-Bayesian Supervised Classiﬁcation: The Thermodynami cs of Statistical Learning. V ol. 56. Lecture Notes – Monograph Series. Institute of Mat hematical Statistics, 2007

work page 2007