On the Sample Complexity of One Hidden Layer Networks with Equivariance, Locality and Weight Sharing

Arash Behboodi; Gabriele Cesa

REVIEW 2 major objections 3 minor 21 references

On the Sample Complexity of One Hidden Layer Networks with Equivariance, Locality and Weight Sharing

T0 review · 2 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes dimension-free sample-complexity bounds for one-hidden-layer group-convolution networks: the generalization error is controlled by the product of filter norms, with a matching lower bound.

desk verdict Useful upper-bound paper, but the tightness proof drops a sqrt(|G|) factor in the lower bound and needs repair before the dimension-free claim is credible. read the letter →

arxiv 2411.14288 v2 pith:B4KFPYMV submitted 2024-11-21 cs.LG math.STstat.MLstat.TH

Arash Behboodi , Gabriele Cesa This is my paper · ORCID

classification cs.LGmath.STstat.MLstat.TH MSC 68Q3268T07

keywords samplecomplexitygeneralizationboundsequivariantnetworksgroupconvolutionRademacherweightsharinglocalfiltersone-hidden-layer

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which architectural choices — equivariance, weight sharing, local filters — actually reduce how many samples a one-hidden-layer network needs to generalize, and it answers with sharp uniform bounds. For group-convolution networks with average pooling (or any pooling that can be written as a pointwise function of a group average) and activations that are 1-Lipschitz and positively homogeneous, it proves that with high probability $L(h) \le \hat{L}(h) + 2 b_x M_1 M_2/\sqrt{m} + 4\sqrt{2\log(4/\delta)/m}$, with no dependence on input or channel dimensions or on the group size. The argument runs through Rademacher complexity, a standard measure of how well a hypothesis class can fit random labels, and a matching lower bound shows the rate $M_1M_2/\sqrt{m}$ is tight for ReLU average-pooling networks. The same machinery shows that non-equivariant weight sharing with an orthogonal basis matches the equivariant bound, that locality buys an extra $\sqrt{O_\Phi/|G|}$, and that the uncertainty principle forces a trade-off between spatial and frequency locality. If the bounds are right, filter norms rather than parameter counts are the quantity that controls sample efficiency in these shallow equivariant models.

What carries the argument

The central object is the norm-constrained hypothesis space $H=\{h_{u,w}:\|u\|\le M_1,\ \|w\|\le M_2\}$, and the mechanism is Rademacher complexity with a peeling argument. The proof first strips off the final linear layer, reducing the Rademacher sum to $\frac{M_1}{m}\mathbb{E}_\epsilon \sup_w \|\sum_i \epsilon_i P(\sigma(Wx_i))\|$, then uses 1-Lipschitzness of the loss and activations via the contraction lemma, and positive homogeneity to normalize each channel's weight vector and collapse the remaining supremum to $b_xM_1M_2/\sqrt{m}$. For general pooling, the representation $P(z)=\phi((1/|G|)1^\top\rho(z))$ is what lets the group average pass through the pointwise maps and preserve the same chain. Covering-number arguments (Dudley's inequality plus a norm-covering lemma for linear classes) appear only in the max-pooling and multi-channel covering proofs, which is where the mild dimension dependence enters.

What would settle it

Run the lower-bound construction of Theorem 4.4 numerically for small $m$: draw inputs supported in the positive orthant with fixed norm $b_x$, train or enumerate the norm-ball group-convolution model with ReLU and average pooling, and compute the empirical Rademacher complexity exactly. If it fails to lie between $c b_x M_1M_2/\sqrt{m}$ and $b_x M_1M_2/\sqrt{m}$ up to constants, the claimed tightness is wrong; separately, repeating the PAC experiment many times, the frequency with which $L(h)-\hat{L}(h)$ exceeds the Theorem 4.1 bound must be at most $\delta$, otherwise the inequality is violated.

Watch

Extended reading notes

Core claim

The central claim is that the empirical Rademacher complexity of the hypothesis space $H=\{h_{u,w}: \|u\|\le M_1,\ \|w\|\le M_2\}$ for a multi-channel group-convolution network $h_{u,w}(x)=u^\top P(\sigma(Wx))$ is bounded by $b_x M_1 M_2/\sqrt{m}$ when the pooling functional has the form $P(z)=\phi((1/|G|)1^\top \rho(z))$ with $\rho,\phi,\sigma$ all 1-Lipschitz and positively homogeneous. Theorem 4.1 converts this into a PAC guarantee whose leading term is $2b_x M_1 M_2/\sqrt{m}$, and Theorem 4.4 shows by a Khintchine-based construction that for average pooling and ReLU there is a data distribution on which the same quantity is at least $c b_x M_1 M_2/\sqrt{m}$. The same peeling argument applies to general equivariant networks parameterized in Fourier space (Theorem 5.1), while max-pooling and multilayer extensions retain the norm dependence but acquire mild group-size or channel factors. For weight sharing through fixed matrices $B_k$, the bound depends on a shared-norm $\|w\|_B$, so orthogonal sharing schemes reproduce the equivariant guarantee without being equivariant (Proposition 6.1). Local filters improve the leading term by $\sqrt{O_\Phi/|G|}$ (Proposition 7.1), and band-limited filters trade that gain away through the uncertainty principle (Proposition 7.2).

Load-bearing premise

The whole dimension-free result rests on every nonlinear element—activation and pooling—being 1-Lipschitz and positively homogeneous, and on general pooling decomposing as a scalar function of a group average; norm-based activations used in steerable networks fall outside this.

Editorial extensions

If this is right

For a fixed norm budget $M_1M_2$ and input radius $b_x$, increasing the group size $|G|$ or the number of channels does not worsen the leading generalization term in the average-pooling group-convolution network.
Equivariance and a suitably orthogonal weight-sharing scheme yield the same dimension-free bound, so on generic non-symmetric data the two design choices are interchangeable at the level of sample complexity.
Local filters improve the leading term by $\sqrt{O_\Phi/|G|}$; the fewer patches a given input coordinate participates in, the better the bound.
Band-limited filters cannot get the full spatial-locality gain: the uncertainty principle forces a spatial support of at least $|G|/B$ for filters with $B$ non-zero frequency entries, leaving a trade-off rather than a free lunch.
A matching lower bound shows the $\sqrt{m}$ rate and the $M_1M_2$ norm dependence are not artifacts of a loose analysis for ReLU average-pooling networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not pursue it, but the norm-only bound suggests a concrete design rule: regularize $M_1M_2$ (for instance by weight decay on both layers) rather than the number of parameters, if the goal is sample efficiency in shallow equivariant models.
The equivalence between equivariance and orthogonal weight sharing is distribution-free; on data that genuinely has the group symmetry, equivariance should add a further advantage that this bound does not quantify, since the bound never uses the symmetry of the data.
A direct test of the stated conjecture for general Lipschitz activations would be to compute Rademacher complexity of group-convolution networks with sigmoid activations; if dimension-free scaling breaks down, positive homogeneity marks the real boundary of the argument.
Applied to steerable networks, the uncertainty-principle trade-off suggests that band-limiting filters to gain frequency-domain locality costs a factor of order $1/\sqrt{B}$ in the sample-complexity gain, giving a quantitative knob for choosing filter bandwidth.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Worth a look: the paper gives genuinely new norm-based Rademacher bounds for one-hidden-layer group-convolution and equivariant networks, with honest comparisons to VSS22 and GRS18. The max-pooling bounds, the weight-sharing norm characterization, the locality factor, and the frequency-domain no-gain remark are the real additions; the average-pooling theorem is, as the authors say, essentially a reworked special case of VSS22. The paper is well written, the appendix proofs are mostly coherent, and limitations (positive homogeneity, pooling representation, no norm nonlinearities) are stated without hand-waving. No code, but that is normal for this line of work.

The soft spot is in Appendix E, and it is real. Theorem 4.4's lower bound uses constant filters w_(1,k) = a_k 1. The constraint ||w|| <= M2 then reads sqrt(|G|) ||a|| <= M2, so the supremum over a is (M2/sqrt(|G|)) ||V_+||, not M2 ||V_+||. The proof drops that sqrt(|G|). The positive-orthant data assumption does not make ||V|| larger by sqrt(|G|); the final lower bound should scale as c M1 M2 B / sqrt(m |G|), not c M1 M2 B / sqrt(m). So the claimed dimension-free tightness is not proven. The upper-bound theorems may still be correct; the issue is localized to the tightness claim, which is a key selling point in the abstract and introduction. This needs a fix, or the claim should be downgraded to a dimension-dependent lower bound.

Other concerns are minor: the M_max spectral term in the max-pooling bound is data dependent and the authors already note it can be loose for large m; the positive-homogeneity restriction is acknowledged. The citation pattern is fine; self-citation to BCC22 appears only in comparison remarks.

This paper deserves a serious referee, but the referee should be told to check Appendix E carefully. I would accept it only after the lower-bound factor is corrected or the claim is weakened. For readers focused on the upper bounds, the paper is useful and citable.

Referee Report

2 major / 3 minor

Summary. The paper studies the sample complexity of one-hidden-layer group-equivariant networks through Rademacher complexity. It proves norm-based generalization bounds for group-convolution networks with average, max, and a class of general pooling operations, for general equivariant networks parameterized in the frequency domain, for non-equivariant weight-sharing architectures, and for local receptive fields. The headline result is a dimension-free upper bound of order O(||u|| ||w|| / sqrt(m)) for positively homogeneous 1-Lipschitz activations, together with an extension to multilayer networks with a |G|^{(L-1)/2} dependence, a lower bound intended to show tightness, and experiments on rotated MNIST and CIFAR10.

Significance. If the upper-bound theorems are correct, the paper is a useful contribution: it gives clean, dimension-free generalization bounds for a natural class of equivariant one-hidden-layer networks, and it isolates the separate roles of equivariance, weight sharing, and locality. The proofs are based on standard tools, the comparisons with prior work are informative, and the numerical study supports the qualitative scaling of the bound. The main weakness is the lower-bound proof in Appendix E, which contains a concrete error in the norm constraint and therefore does not establish the claimed tightness; the error appears repairable, but the tightness claim is not proven as written.

major comments (2)

[Appendix E, Theorem 4.4] The lower-bound proof drops a sqrt(|G|) factor in the norm constraint. For w in W_hat^+_1 with w_(1,k) = a_k 1, we have ||w||^2 = |G| sum_k a_k^2, so the constraint ||w|| <= M2 gives sum_k a_k^2 <= M2^2/|G|. Consequently the displayed equality E sup_{w in W_hat^+_1} |sum_i eps_i sum_k w_(1,k) 1^T x_i(k)| = M2 E ||sum_i eps_i (1^T x_i(1), ..., 1^T x_i(c0))|| is missing a factor 1/sqrt(|G|), and it also ignores the nonnegativity restriction on the a_k. With the subsequent bound ||x_i^(k)||_1 >= ||x_i^(k)||_2, the argument as written yields at best c b_x M1 M2 / sqrt(m |G|), not the claimed c b_x M1 M2 / sqrt(m). The theorem can likely be repaired by specializing the data, e.g., c0 = 1 and x_i = (B/sqrt(|G|)) 1 so that 1^T x_i = sqrt(|G|) B, and by handling the positive-part/sign issue carefully, but as written the tightness claim is not established.
[Appendix C.2, C.5 and Lemma B.4] The proofs apply the contraction lemma to squared Rademacher sums, for example after Eq. (21): E sup_w (sum_i eps_i sigma(...))^2 <= E sup_w (sum_i eps_i (linear))^2. Lemma B.4 is stated for convex increasing G, while G(x) = x^2 is not increasing on R. The argument needs the standard two-sided or positive/negative-part version of the contraction inequality, or an explicit justification that the structure of the set T allows applying the lemma to squares. This is a genuine gap in the written proof, although the dimension-free rate is likely preserved after a constant-factor repair.

minor comments (3)

[Theorem 4.1 and Appendix C.5] The theorem assumes pooling of the form P(z) = phi((1/|G|) 1^T rho(z)) with a non-identity rho, but the proof in C.5 applies phi directly to (1/|G|) 1^T sigma(...) and never uses rho. Since rho and sigma are both 1-Lipschitz positively homogeneous, one can absorb rho into sigma, but the statement and proof should be aligned.
[Theorem B.3 and Section 4] The confidence term 4 sqrt(2 log(4/delta)/m) implicitly assumes the loss is bounded by 1. The paper states only that the loss is 1-Lipschitz; the boundedness normalization should be stated explicitly.
[Appendix C.4] In the proof of the multi-channel max-pooling covering bound, the set H1 is defined with ||w|| <= M1 in one place and with M2 in the covering-number calculation below it. This appears to be a notational inconsistency, not a mathematical error, but it should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main bounds are derived from external Rademacher, contraction, peeling, and covering-number results; the self-citations are not load-bearing.

full rationale

The upper-bound proofs (Theorems 4.1, 5.1, Propositions 6.1 and 7.1, with Appendices C.2, C.5, D, G and H) reduce the Rademacher complexity of the network to a linear Rademacher term by peeling the last layer, applying standard contraction lemmas ([LT11], Lemma B.4) and the positive-homogeneity peeling technique of [GRS18], then bounding the remaining term by b_x. The one structural input, the representation P(z)=phi((1/|G|)1^T rho(z)) for permutation-invariant pooling, is imported from the external Deep Sets theorem [Zah+17]; the paper then restricts to positively homogeneous rho and phi as an explicit assumption rather than deriving it. The covering-number arguments cite [Zha02], and the average-pooling comparison explicitly reuses [VSS22]; all of these are external results with stated assumptions that do not include the target bounds. There is no fitted parameter that is later renamed a prediction: the experiments only measure the trained norms and compare the bound with observed generalization error. The self-citations to [BCC22] (same first two authors) appear in Related Works, in Remark 4.2 as a comparison ('In our Rademacher analysis, such a shift would not bring any additional gain'), and in Appendix A.1 as background ('see [BCC22]; [CLW22] for more details'); none of these is a premise in any proof, so the self-citation is not load-bearing. The paper also explicitly scopes its theorems to 1-Lipschitz positively homogeneous activations and states that the current proof techniques do not extend to norm-based nonlinearities, which is a limitation statement rather than a circular step. A possible factor issue in the Appendix E lower bound (the supremum over constant filters appears to yield ||V||/sqrt(|G|) rather than ||V||) would be a proof-correctness concern, not a circularity concern, and does not affect the upper-bound derivations.

Assumptions & free parameters 0 free parameters · 9 assumptions · 0 invented entities

No free parameters are fitted to data; M1, M2, b_x, and O_Phi are structural constants or norm bounds. The paper relies on standard learning-theoretic theorems. The most restrictive assumption is the positively homogeneous, scalar-sum pooling and activation class, which the authors flag. No new physical or mathematical entities are postulated.

assumptions (9)

standard math Rademacher generalization bound relating test error to training error plus 2 R_S(loss composed with H) plus a log(1/delta) term.
Used in every upper bound as the base generalization guarantee; cited as Theorem 26.5 in Shalev-Shwartz and Ben-David.
standard math Contraction lemma for Rademacher complexity of 1-Lipschitz functions.
Invoked to remove 1-Lipschitz loss functions and activations in the Rademacher sums; standard result from Ledoux and Talagrand.
standard math Dudley's entropy integral bound on Rademacher complexity via covering numbers.
Used for the max-pooling bounds in Appendix C.3 and C.4.
standard math Zhang's covering-number lemma for linear function classes.
Used to bound covering numbers of the linear maps arising from convolution and max-pooling in Appendix C.3.
standard math Khintchine inequality for Rademacher sums.
Used in the lower bound proof of Theorem 4.4 in Appendix E.
standard math Uncertainty principle for finite Abelian groups: |supp(f)| |supp(hat f)| >= |G|.
Used in Proposition 7.2 and Appendix I to derive the spatial size lower bound for band-limited filters.
domain assumption Deep Sets representation of permutation-invariant pooling as phi((1/|G|) 1^T rho(z)) with additional positive homogeneity.
Restricts the class of pooling operations covered by Theorem 4.1; average pooling satisfies the representation, but max pooling is handled separately.
domain assumption Lower-bound data distribution: inputs supported in a single orthant with maximum norm B.
Required for the lower bound construction in Theorem 4.4; the authors explicitly construct this distribution.
domain assumption Hypothesis class norm constraints ||u|| <= M1, ||w|| <= M2 and 1-Lipschitz loss.
These are the explicit working assumptions of the paper stated in Section 4 and the notation section.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Sample Complexity of One Hidden Layer Networks with Equivariance, Locality and Weight Sharing." pith.science (2026). https://pith.science/paper/B4KFPYMV

@misc{pith2026241114288,
  author       = {Pith},
  title        = {Pith review of: On the Sample Complexity of One Hidden Layer Networks with Equivariance, Locality and Weight Sharing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B4KFPYMV}},
  note         = {Machine review of arXiv:2411.14288}
}

read the original abstract

Weight sharing, equivariance, and local filters, as in convolutional neural networks, are believed to contribute to the sample efficiency of neural networks. However, it is not clear how each one of these design choices contributes to the generalization error. Through the lens of statistical learning theory, we aim to provide insight into this question by characterizing the relative impact of each choice on the sample complexity. We obtain lower and upper sample complexity bounds for a class of single hidden layer networks. For a large class of activation functions, the bounds depend merely on the norm of filters and are dimension-independent. We also provide bounds for max-pooling and an extension to multi-layer networks, both with mild dimension dependence. We provide a few takeaways from the theoretical results. It can be shown that depending on the weight-sharing mechanism, the non-equivariant weight-sharing can yield a similar generalization bound as the equivariant one. We show that locality has generalization benefits, however the uncertainty principle implies a trade-off between locality and expressivity. We conduct extensive experiments and highlight some consistent trends for these models.

Figures

Figures reproduced from arXiv: 2411.14288 by the authors.

**Figure 2.** Numerical results for the generalization error on the rotated MNIST and CIFAR10 datasets. The [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** Our bound vs. the generalization error on CIFAR10 (training set size m = 3200) when varying the maximum frequency used to parameterize the filters, which is the locality effect in the frequency domain. Each dot and its error bars represent the mean and standard deviation over at least 3 runs with the same configuration. As expected, architectures leveraging a lower frequency design achieve lower generalization … view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Graphical representation of the equivariant linear projection used to preprocess the image [PITH_FULL_IMAGE:figures/full_fig_p047_5.png]

**Figure 6.** Figure 6: Subset of Fig. 2c focusing on the models training with the largest dataset size m = 25600. The models using the smallest equivariance groups show increased norms. optimizer with a learning rate of 1e − 2 for 30 epochs, while the CIFAR10 models are trained using with a …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 13 canonical work pages

[1]

Stronger Generalization Bounds for Deep Nets via a Compression Approach

[Aro+18] Sanjeev Arora et al. “Stronger Generalization Bounds for Deep Nets via a Compression Approach”. In:International Conference on Machine Learning. 2018, pp. 254–263. [BCC22] Arash Behboodi, Gabriele Cesa, and Taco S. Cohen. “A pac-bayesian generalization bound for equivariant networks”. In:Advances in Neural Information Processing Systems35 (2022),...

work page 2022
[3]

2c focusing on the models training with the largest dataset sizem = 25600

All MNIST models are trained using the Adam 47 0.010 0.005 0.000 0.005 0.010 0.015 Generalization Error 0.010 0.012 0.014 0.016 0.018 0.020 M1M2 m log|G| 0 1 2 3 4 5 6 log m 14 G CN DN Figure 6: Subset of Fig. 2c focusing on the models training with the largest dataset sizem = 25600. The models using the smallest equivariance groups show increased norms. ...

work page
[4]

Symmetries and discriminability in feedforward network architectures

Morgan-Kaufmann, 1991.url: https://proceedings.neurips.cc/ paper_files/paper/1991/file/087408522c31eeb1f982bc0eaf81d35f-Paper.pdf. [Sha93] J Shawe-Taylor. “Symmetries and discriminability in feedforward network architectures”. In: IEEE Trans. Neural Netw.(1993), pp. 1–25. [SIK21] Akiyoshi Sannai, Masaaki Imaizumi, and Makoto Kawano. “Improved generalizati...

work page arXiv 1993
[9]

Provably strict generalisation benefit for equivariant models

[EZ21] Bryn Elesedy and Sheheryar Zaidi. “Provably strict generalisation benefit for equivariant models”. In:International Conference on Machine Learning. PMLR, 2021, pp. 2959–2969. [Fin+20] Marc Finzi et al. “Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data”. In:International Conference on Machine Lea...

2021
[11]

Group invariant scattering

[Mal12] Stéphane Mallat. “Group invariant scattering”. In:Communications on Pure and Applied Mathematics 65.10 (2012), pp. 1331–1398. [MRT18] MehryarMohri,AfshinRostamizadeh,andAmeetTalwalkar. Foundations of machine learning. 2nd ed. Adaptive Computation and Machine Learning series. London, England: MIT Press, Dec

work page 2012
[12]

In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictors

[NDR21] Jeffrey Negrea, Gintare Karolina Dziugaite, and Daniel M. Roy.In Defense of Uniform Con- vergence: Generalization via derandomization with an application to interpolating predictors. Tech. rep. arXiv:1912.04265. arXiv, Sept. 2021.url: http://arxiv.org/abs/1912.04265 (visited on 05/19/2022). [NK19] Vaishnavh Nagarajan and J. Zico Kolter. “Uniform c...

work page Pith review arXiv 1912
[17]

Coordinate Independent Convolutional Networks–Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds

[Wei+21] Maurice Weiler et al. “Coordinate Independent Convolutional Networks–Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds”. In:arXiv preprint arXiv:2106.06020 (2021). [WHS18] Maurice Weiler, Fred A. Hamprecht, and Martin Storath. “Learning Steerable Filters for Rotation Equivariant CNNs”. In:Conference on Computer Vision and Patter...

arXiv 2021
[18]

Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin

[WM19] Colin Wei and Tengyu Ma. “Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin”. In:arXiv:1910.04284 [cs, stat](Oct. 2019). arXiv: 1910.04284. [Wor+17] Daniel E. Worrall et al. “Harmonic Networks: Deep Translation and Rotation Equivariance”. In: Conference on Computer Vision and Pattern Recognition (CVPR)

arXiv 1910

Show all 21 references

[20]

Covering number bounds of certain regularized linear function classes

(Visited on 07/24/2018). [Zha02] Tong Zhang. “Covering number bounds of certain regularized linear function classes”. In: Journal of Machine Learning Research2.Mar (2002), pp. 527–550. A An Overview of Representation Theory and Equivariant Net- works We provide a brief overvie...

2002
[30]

Non-Vacuous Generalisation Bounds for Shallow Neural Networks

Ed. by I. Guyon et al. Curran Associates, Inc., 2017, pp. 6240–6249. [BG22] Felix Biggs and Benjamin Guedj. “Non-Vacuous Generalisation Bounds for Shallow Neural Networks”. In: arXiv:2202.01627 [cs, stat] (Feb. 2022). arXiv: 2202.01627. url: http : //arxiv.org/abs/2202.01627 (...

2002 arXiv
[32]

A functional approach to rotation equivariant non-linearities for Tensor Field Networks

Ed. by H. Wallach et al. Curran Associates, Inc., 2019, pp. 11615–11626. [PG21] Adrien Poulenard and Leonidas J. Guibas. “A functional approach to rotation equivariant non-linearities for Tensor Field Networks”. In:2021 IEEE/CVF Conference on Computer Vision and Pattern Recogn...

1980
[33]

A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups

Curran Associates, Inc., 2020, pp. 1970–1981.url: https://proceedings.neurips. cc/paper_files/paper/2020/file/15231a7ce4ba789d13b722cc5c955834-Paper.pdf. [FWW21] Marc Finzi, Max Welling, and Andrew Gordon Wilson. “A practical method for constructing equivariant multilayer perc...

2022
[1977]

Dynamic routing between capsules

[SFH17] Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. “Dynamic routing between capsules”. In: Advances in neural information processing systems30 (2017). 15 [Sha89] J. Shawe-Taylor. “Building symmetries into feedforward networks”. In:1989 First IEE International Confer...

2017
[1989]

Some limitations of norm based generalization bounds in deep neural networks

[Pit+19] Konstantinos Pitas et al. “Some limitations of norm based generalization bounds in deep neural networks”. In:arXiv:1905.09677 [cs, stat](May 2019). arXiv: 1905.09677. [PT23] Mircea Petrache and Shubhendu Trivedi. “Approximation-Generalization Trade-offs under (Approxi...

2023 arXiv
[2011]

On the Benefits of Invariance in Neural Networks

[Lyl+20] Clare Lyle et al. “On the Benefits of Invariance in Neural Networks”. In:arXiv preprint arXiv:2005.00178 (2020). [LZA21] Zhiyuan Li, Yi Zhang, and Sanjeev Arora. “Why Are Convolutional Nets More Sample- Efficient than Fully-Connected Nets?” In:International Conference...

2020 arXiv
[2017]

Deep Sets

[WW23] Zihao Wang and Lei Wu.Theoretical Analysis of Inductive Biases in Deep Convolutional Networks. arXiv:2305.08404 [cs, stat]. May 2023.url: http://arxiv.org/abs/2305.08404. [Zah+17] Manzil Zaheer et al. “Deep Sets”. In:Advances in Neural Information Processing Systems30 (...

2017 arXiv
[2018]

Implicit Bias of Linear Equivariant Networks

[Law+22] Hannah Lawrence et al. “Implicit Bias of Linear Equivariant Networks”. In:International Conference on Machine Learning. PMLR. 2022, pp. 12096–12125. [Led+21] Antoine Ledent et al. “Norm-based generalisation bounds for deep multi-class convolutional neural networks”. I...

2022 arXiv
[2019]

Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data

[DR17] Gintare Karolina Dziugaite and Daniel M. Roy. “Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data”. In:arXiv preprint arXiv:1703.11008(2017). 13 [DR18a] Gintare Karolina Dziugaite and Daniel Roy....

2017 arXiv
[2021]

On the generalization of equivariance and convolution in neural networks to the action of compact groups

url: https://openreview.net/forum?id=FyOhThdDBM (visited on 05/19/2022). [KT18] Risi Kondor and Shubhendu Trivedi. “On the generalization of equivariance and convolution in neural networks to the action of compact groups”. In:International Conference on Machine Learning (ICML)

2022
[2022]

Gauge equivariant convolutional networks and the icosahedral CNN

url: https://openreview.net/forum?id=WE4qe9xlnQw. [Coh+19] Taco Cohen et al. “Gauge equivariant convolutional networks and the icosahedral CNN”. In: International conference on Machine learning. PMLR. 2019, pp. 1321–1330. [CW16a] TacoS.CohenandMaxWelling.“GroupEquivariantConvo...

2019 arXiv
[8441]

Uncertainty Principles and Signal Recovery

[DS89] David L. Donoho and Philip B. Stark. “Uncertainty Principles and Signal Recovery”. en. In: SIAM Journal on Applied Mathematics 49.3 (June 1989), pp. 906–931. (Visited on 10/16/2023). [Dzi+20] Gintare Karolina Dziugaite et al. “In search of robust measures of generalizat...

2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Stronger Generalization Bounds for Deep Nets via a Compression Approach

[Aro+18] Sanjeev Arora et al. “Stronger Generalization Bounds for Deep Nets via a Compression Approach”. In:International Conference on Machine Learning. 2018, pp. 254–263. [BCC22] Arash Behboodi, Gabriele Cesa, and Taco S. Cohen. “A pac-bayesian generalization bound for equivariant networks”. In:Advances in Neural Information Processing Systems35 (2022),...

work page 2022

[2] [3]

2c focusing on the models training with the largest dataset sizem = 25600

All MNIST models are trained using the Adam 47 0.010 0.005 0.000 0.005 0.010 0.015 Generalization Error 0.010 0.012 0.014 0.016 0.018 0.020 M1M2 m log|G| 0 1 2 3 4 5 6 log m 14 G CN DN Figure 6: Subset of Fig. 2c focusing on the models training with the largest dataset sizem = 25600. The models using the smallest equivariance groups show increased norms. ...

work page

[3] [4]

Symmetries and discriminability in feedforward network architectures

Morgan-Kaufmann, 1991.url: https://proceedings.neurips.cc/ paper_files/paper/1991/file/087408522c31eeb1f982bc0eaf81d35f-Paper.pdf. [Sha93] J Shawe-Taylor. “Symmetries and discriminability in feedforward network architectures”. In: IEEE Trans. Neural Netw.(1993), pp. 1–25. [SIK21] Akiyoshi Sannai, Masaaki Imaizumi, and Makoto Kawano. “Improved generalizati...

work page arXiv 1993

[4] [9]

Provably strict generalisation benefit for equivariant models

[EZ21] Bryn Elesedy and Sheheryar Zaidi. “Provably strict generalisation benefit for equivariant models”. In:International Conference on Machine Learning. PMLR, 2021, pp. 2959–2969. [Fin+20] Marc Finzi et al. “Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data”. In:International Conference on Machine Lea...

2021

[5] [11]

Group invariant scattering

[Mal12] Stéphane Mallat. “Group invariant scattering”. In:Communications on Pure and Applied Mathematics 65.10 (2012), pp. 1331–1398. [MRT18] MehryarMohri,AfshinRostamizadeh,andAmeetTalwalkar. Foundations of machine learning. 2nd ed. Adaptive Computation and Machine Learning series. London, England: MIT Press, Dec

work page 2012

[6] [12]

In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictors

[NDR21] Jeffrey Negrea, Gintare Karolina Dziugaite, and Daniel M. Roy.In Defense of Uniform Con- vergence: Generalization via derandomization with an application to interpolating predictors. Tech. rep. arXiv:1912.04265. arXiv, Sept. 2021.url: http://arxiv.org/abs/1912.04265 (visited on 05/19/2022). [NK19] Vaishnavh Nagarajan and J. Zico Kolter. “Uniform c...

work page Pith review arXiv 1912

[7] [17]

Coordinate Independent Convolutional Networks–Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds

[Wei+21] Maurice Weiler et al. “Coordinate Independent Convolutional Networks–Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds”. In:arXiv preprint arXiv:2106.06020 (2021). [WHS18] Maurice Weiler, Fred A. Hamprecht, and Martin Storath. “Learning Steerable Filters for Rotation Equivariant CNNs”. In:Conference on Computer Vision and Patter...

arXiv 2021

[8] [18]

Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin

[WM19] Colin Wei and Tengyu Ma. “Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin”. In:arXiv:1910.04284 [cs, stat](Oct. 2019). arXiv: 1910.04284. [Wor+17] Daniel E. Worrall et al. “Harmonic Networks: Deep Translation and Rotation Equivariance”. In: Conference on Computer Vision and Pattern Recognition (CVPR)

arXiv 1910