Impact of Batch Normalization on Convolutional Network Representations

Coenraad Mouton; Hermanus L. Potgieter; Marelie H. Davel

REVIEW 4 major objections 4 minor 36 references

Impact of Batch Normalization on Convolutional Network Representations

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read BatchNorm-trained convolutional networks generalize better not because their internal activations are sparser, but because those activations form purer, class-consistent clusters.

desk verdict Learning-rate confound in the VGG-16 comparison leaves the causal claims about BatchNorm unsecured, but the sparsity result is a useful empirical addition. read the letter →

arxiv 2501.14441 v2 pith:5MDC3PIX submitted 2025-01-24 cs.LG

Hermanus L. Potgieter , Coenraad Mouton , Marelie H. Davel This is my paper · ORCID

classification cs.LG

keywords batchnormalizationrepresentationalsparsityclusterpurityDavies-BouldinindexconvolutionalneuralnetworksgeneralizationCIFAR-10MNIST

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Batch normalization is widely used because it speeds training and improves generalization, but the reason is still debated. This paper probes the internal activations of convolutional networks trained with and without BatchNorm, comparing two candidate explanations: representational sparsity and the purity of the clusters these activations form. Across a small CNN on MNIST and a VGG-16 on CIFAR-10, sparsity differences between BatchNorm and non-BatchNorm models are inconsistent across datasets and do not track generalization. In contrast, BatchNorm models consistently form purer class-based clusters, and class-agnostic clusters appear earlier and track the number of classes more closely. The paper concludes that sparsity is not the mechanism behind BatchNorm's generalization benefit, and that clustering quality is the more informative correlate.

What carries the argument

The analysis turns hidden-layer activations into comparable objects: after each ReLU, the 4D activation tensor is flattened or spatially averaged per channel, sparsity is measured as the fraction of exactly zero elements per layer or per channel, and cluster quality is assessed with the Davies-Bouldin index (DBI), which compares mean intra-cluster distances to centroid distances, with lower values indicating purer clusters. Representations are clustered either by true class label or by k-means with a purity-selected number of clusters. BatchNorm itself, the per-channel normalization of activations by batch mean and variance followed by a learned affine transform, is the treatment whose representational consequences are compared against non-normalized training.

What would settle it

Train VGG-16 on CIFAR-10 without BatchNorm at the same learning rate used for the BatchNorm models, then measure the Davies-Bouldin index of its class-based clusters; if cluster purity matches or exceeds that of the BatchNorm models, the paper's attribution of purer clusters to BatchNorm would be undermined.

Watch

Extended reading notes

Core claim

The paper's central claim is that BatchNorm's well-known generalization advantage is not explained by sparser internal representations, but is accompanied by better clustering structure in learned representations. On CIFAR-10, BatchNorm models are more sparse than their non-BatchNorm counterparts, while on MNIST they are less sparse, so there is no consistent sparsity signature; yet BatchNorm models generalize better in both settings. By contrast, measuring cluster purity with the Davies-Bouldin index, both when clusters are defined by class labels and when they are found by k-means, BatchNorm models show purer clusters, particularly in deeper layers, and the number of clusters selected by purity converges to the number of classes earlier and more consistently. The paper states the finding directly: BatchNorm's effect on representational sparsity is not a significant factor affecting generalization, while BatchNorm representations tend to show more advantageous clustering characteristics.

Load-bearing premise

The load-bearing premise is that the clustering and sparsity differences between the CIFAR-10 models are caused by BatchNorm itself, not by the different learning rates used for the BatchNorm models (0.001) and the non-BatchNorm models (0.0001).

Editorial extensions

If this is right

If sparsity is not the mechanism, explanations that tie BatchNorm's benefit to sparser, more selective activations should be set aside for CNNs, and the search should focus on representational geometry.
Cluster purity becomes a candidate diagnostic for generalization, so models with purer class-based clusters may be expected to generalize better.
BatchNorm's earlier, more class-consistent clustering suggests normalization shapes representational geometry during training, not only the optimization landscape.
The purer deep-layer clusters under BatchNorm connect its generalization benefit to the layers where class-specific information is concentrated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

I infer a testable consequence: if cluster purity is the causal channel, then explicitly penalizing the Davies-Bouldin index during training should reproduce part of BatchNorm's accuracy gain even without normalization layers.
A threshold sweep on sparsity, counting near-zero activations rather than exact zeros, could show whether the paper's sparsity conclusion is an artifact of its zero-only threshold.
Because the CIFAR-10 models were trained at different learning rates, checking whether a non-BatchNorm model trained at 0.001 also develops purer clusters would isolate the effect of BatchNorm from the effect of optimization speed.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper reports that BatchNorm's effect on hidden representation sparsity is not a consistent predictor of generalization in CNNs, while trained models show purer class-based clusters. The sparsity observation is genuinely new for CNNs (prior work covered MLPs), the clustering comparison is a reasonable addition, and the authors include multiple seeds and both class-based and class-agnostic analyses. The writing is clear and the limitations are at least partially acknowledged.

That said, the central weakness is in the VGG-16/CIFAR10 setup: BatchNorm models are trained with learning rate 0.001 and non-BatchNorm models with 0.0001. So any BN-vs-NBN difference on that architecture, whether sparsity direction, cluster purity, or stability across seeds, is confounded with optimization dynamics. The MNIST experiment holds hyperparameters fixed and shows BN leading to less sparse representations, while CIFAR10 shows the reverse. The paper interprets this as evidence that sparsity isn't a reliable explanatory factor, but a simpler reading is that the opposite CIFAR10 trend may be an artifact of the learning-rate difference. I don't think this destroys the negative claim about sparsity, because across both datasets the BN models generalize better despite sparsity moving in opposite directions, so sparsity isn't a consistent marker. Still, the specific CIFAR10 numbers used to support the claim are weaker than they appear.

The clustering claim rests more heavily on CIFAR10, since MNIST class-agnostic results show BatchNorm only clearly better at the last layer. Because the deeper VGG-16 clusters are substantially purer for BatchNorm models that also achieve about 6.5 points higher test accuracy, one cannot separate "BatchNorm causes clustering" from "models that generalize better form purer clusters" or "a larger learning rate helps clustering." The class-based cluster analysis avoids the k-selection circularity (k chosen by minimizing DBI, then the same DBI reported), but it still suffers from the same LR confound.

Minor issues: no statistical significance tests, only standard deviations across four seeds; sparsity threshold fixed at zero; no code or data released. The paper itself flags the threshold limitation.

Who is it for? People doing empirical analysis of BatchNorm's internal representations will find this a useful, clearly-presented starting point. It deserves a serious referee, but the referee should insist on either matching learning rates or an ablation that separates LR from BatchNorm, and ideally significance testing. As is, the causal language in the abstract is stronger than the evidence supports.

Recommendation: accept for peer review, but with substantial revision requested.

Referee Report

4 major / 4 minor

Summary. The paper empirically compares internal representations of convolutional networks trained with and without batch normalization on two settings: a standard CNN on MNIST and VGG-16 on CIFAR10. It measures representational sparsity (fraction of exactly zero activations per channel and per layer) and clustering quality, using both a class-based DBI score and a class-agnostic approach in which k-means clusters are selected and then scored with DBI. The main claims are that BatchNorm does not consistently induce or prevent sparsity, that sparsity does not correlate with generalization, and that BatchNorm models tend to form purer representational clusters. The authors acknowledge several limitations, including the fixed zero threshold for sparsity and the different learning rates used in the VGG-16 comparison.

Significance. If the central empirical claims held, the paper would be a useful caution against sparsity-based explanations of BatchNorm's benefit and would add evidence for cluster-purity as a more relevant representational correlate. The study has notable strengths: the MNIST comparison holds hyperparameters fixed, the architectures and datasets are standard, multiple random seeds are used, and the metric definitions are explicit. It is also honest about several limitations. However, the main causal attribution is weakened by the learning-rate confound in the CIFAR10/VGG-16 experiment, by the selection-and-evaluation-on-the-same-metric issue in the class-agnostic clustering analysis, and by the absence of statistical inference. These issues are load-bearing because the sparsity finding reverses between the two settings and the positive clustering claim rests partly on the confounded deeper-network experiment.

major comments (4)

[§3.1.2, Table 1] The VGG-16/CIFAR10 comparison confounds BatchNorm status with optimizer learning rate: BatchNorm models use Adam with learning rate 0.001, while non-BatchNorm models use 0.0001. Since this is the only architecture in which the sparsity ordering reverses (Fig. 3b) and a main source of the clustering claims (Figs. 4b and 6b), the observed representational differences cannot be cleanly attributed to BatchNorm; they could be caused by the learning-rate difference or by the resulting large accuracy gap (85.99% vs. 79.44%). Please add controlled runs with matched learning rates, ideally with at least two learning-rate values per condition, or otherwise demonstrate that the learning-rate change alone does not produce the same sparsity and clustering differences.
[§3.3, §5.2.1, §5.2.2] The class-agnostic analysis selects the number of clusters by minimizing DBI and then reports the DBI of the selected clusters as the purity result. Selecting and evaluating on the same metric introduces an optimistic bias, and the bias can differ between BatchNorm and non-BatchNorm models because their optimal-k trajectories differ (Fig. 5). Please evaluate cluster purity at fixed values of k, use a separate metric or a cross-validation scheme for selection, or use a stability-based cluster-selection criterion.
[§4.1, §4.2, §5.2.2] No statistical significance tests or effect sizes are reported, and with only four seeds the error bars in Figures 2, 3b, 4b and 6b overlap for several layers despite the text using wording such as 'significantly more sparse' (§4.1) and 'pronounced difference' (§4.2). Please provide per-layer confidence intervals, bootstrap or permutation tests, and effect sizes so the reliability of the claimed differences can be assessed.
[§3.2 and Discussion] Sparsity is defined only as exactly zero activations, and the discussion acknowledges that no threshold sweep was performed. Because the central claim is that BatchNorm's effect on representational sparsity is not a significant factor for generalization, the conclusion should be tested against the threshold choice; a single fixed threshold leaves open the possibility that near-zero activations, which are common under BatchNorm, change or even reverse the observed ordering.

minor comments (4)

[§2.3.4, §2.3.5, §3.1.1] There are several typographical errors, including 'purity of the of representational clusters' (§2.3.4), 'optinization' (§2.3.5), and 'architecutre' (§3.1.1).
[Figures 4 and 6] The captions describe DBI scores, but the axis labels and units should be stated explicitly in each figure; currently the reader cannot tell which axis is the layer index and which is the DBI value.
[§3.2] The sparsity and clustering analyses use only training-set representations, while the claims concern generalization; please justify this choice explicitly or repeat the main analysis on held-out data.
[§5.2.2 and Discussion] For the MNIST class-agnostic analysis, Figure 6a shows that BatchNorm models often have worse DBI than non-BatchNorm models except at the final layer; the discussion does acknowledge this, but the abstract's broad statement that BatchNorm models 'tend to show more advantageous clustering characteristics' should be qualified to match this mixed result.

Circularity Check

1 steps flagged · score 3.0 of 10

Class-agnostic cluster purity is selected by the same DBI that is then reported as the result; the central BatchNorm comparison is otherwise empirically independent.

fitted input called prediction [Section 3.3 (class-agnostic clustering) and Section 5.2.2]
"To select the optimal number of clusters we first use 2 to 15 clusters during an initial clustering step. We then measure the purity of these representation clusters using the DBI metric, as discussed in Section 5.2. We then select the optimal number of clusters as the clusters that produced the lowest DBI score. ... We now consider the DBI score of each optimal cluster found using k-means."

The number of clusters k is chosen as the minimizer of DBI(k), and the DBI at that selected k is then reported as the cluster-purity score (Figure 6). Thus the reported purity is, by construction, the minimized selection objective: DBI_reported = min_{k=2..15} DBI(k). It is not an independent estimate of cluster quality and is statistically forced to look better than non-optimal k values. This inflates the apparent purity of the class-agnostic clusters. It does not force the BatchNorm/non-BatchNorm comparison because both model types use the same selection rule, and the class-based DBI analysis in Figure 4 is unaffected.

full rationale

This is an empirical study rather than a derivation, so most of its claims are not equivalent to their inputs by construction. The central sparsity and clustering comparisons come from fresh MNIST and CIFAR10 experiments and do not depend on the authors' prior results; self-citations such as Pretorius et al. [15] are contextual and are also independently reproduced in this paper. The one genuine self-referential step is the class-agnostic k-means pipeline: DBI is used to select k and the same DBI is then reported as cluster purity, making the Figure 6 values minima over the searched k by construction. This is a partial circularity in one sub-analysis. The class-based DBI analysis and the relative BN-vs-NBN comparisons remain independent, so the main claim is not forced by this selection rule. The different learning rates for BatchNorm and non-BatchNorm VGG-16 models (0.001 vs 0.0001, Section 3.1.2) are a real confound for causal attribution, but a confound is not a circularity and was therefore not counted in the score.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims are empirical and rest on experimental design choices rather than on a formal derivation. The main risks are the learning rate confound in the VGG-16 comparison, the DBI-based cluster number selection, and the reliance on a single sparsity threshold.

free parameters (3)

Sparsity threshold = 0
Sparsity is defined as exactly zero activation. The authors note this threshold was chosen without experimenting with alternatives (Section 6).
Number of clusters k (class-agnostic) = 2 to 15, chosen per layer by lowest DBI
The optimal number of clusters is selected by minimizing DBI, which may bias the subsequent DBI comparison (Sections 3.3 and 5.2).
VGG-16 learning rate = 0.001 (BN) and 0.0001 (NBN)
Different learning rates for BN and non-BN VGG-16 models introduce a potential confound, as differences attributed to BatchNorm could reflect optimization settings (Section 3.1.2).

assumptions (3)

domain assumption The Davies-Bouldin index is a valid measure of cluster purity for comparing representations.
The paper uses DBI to define cluster purity and draws conclusions about generalization from it (Section 2.3.4).
domain assumption Spatial average pooling and L2 normalization preserve the clustering structure relevant to generalization.
Representations are reduced from NxCxHxW to NxC by averaging over spatial dimensions and normalized per sample (Section 3.3).
domain assumption Four random initializations provide a reliable estimate of model behavior.
All comparisons are based on four seeds, and no statistical significance testing is reported (Section 3.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Impact of Batch Normalization on Convolutional Network Representations." pith.science (2026). https://pith.science/paper/5MDC3PIX

@misc{pith2026250114441,
  author       = {Pith},
  title        = {Pith review of: Impact of Batch Normalization on Convolutional Network Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5MDC3PIX}},
  note         = {Machine review of arXiv:2501.14441}
}

read the original abstract

Batch normalization (BatchNorm) is a popular layer normalization technique used when training deep neural networks. It has been shown to enhance the training speed and accuracy of deep learning models. However, the mechanics by which BatchNorm achieves these benefits is an active area of research, and different perspectives have been proposed. In this paper, we investigate the effect of BatchNorm on the resulting hidden representations, that is, the vectors of activation values formed as samples are processed at each hidden layer. Specifically, we consider the sparsity of these representations, as well as their implicit clustering -- the creation of groups of representations that are similar to some extent. We contrast image classification models trained with and without batch normalization and highlight consistent differences observed. These findings highlight that BatchNorm's effect on representational sparsity is not a significant factor affecting generalization, while the representations of models trained with BatchNorm tend to show more advantageous clustering characteristics.

Figures

Figures reproduced from arXiv: 2501.14441 by the authors.

**Figure 2.** Average channel sparsity for BatchNorm (blue) and Non-BatchNorm (orange) models in selected VGG-16 [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Average layer sparsity for models trained with (blue) and without (orange) BatchNorm. Error bars indicate [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Average class-based DBI cluster purity score per layer for BatchNorm (blue) and Non-BatchNorm (orange) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Average optimal number of class-agnostic clusters per layer for BatchNorm (blue) and Non-BatchNorm [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Average class-agnostic DBI cluster purity score for k-means per layer for BatchNorm (blue) and Non [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 35 canonical work pages

[1]

Impact of batch normalization on convolutional network representations

Hermanus L Potgieter, Coenraad Mouton, and Marelie H Davel. Impact of batch normalization on convolutional network representations. In Southern African Conference for Artificial Intelligence Research SACAIR , pages 235–252. Springer, 2024

work page 2024
[2]

Understanding deep learning (still) requires rethinking generalization

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 2021

work page 2021
[3]

Sharpness-aware minimization for efficiently improving generalization

Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In 9th International Conference on Learning Representations, ICLR, 2021

work page 2021
[4]

Algorithmic stability and generalization performance

Olivier Bousquet and André Elisseeff. Algorithmic stability and generalization performance. Advances in Neural Information Processing Systems NeurIPS, 13, 2000

work page 2000
[5]

Input margins can predict generaliza- tion too

Coenraad Mouton, Marthinus Wilhelmus Theunissen, and Marelie H Davel. Input margins can predict generaliza- tion too. Proceedings of the AAAI Conference on Artificial Intelligence, 38(13), 2024

work page 2024
[6]

How does information bottleneck help deep learning? In Proceedings of the 40th International Conference on Machine Learning ICML, volume 202, 2023

Kenji Kawaguchi, Zhun Deng, Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In Proceedings of the 40th International Conference on Machine Learning ICML, volume 202, 2023

work page 2023
[7]

Measuring generalization with optimal transport

Ching-Yao Chuang, Youssef Mroueh, Kristjan Greenewald, Antonio Torralba, and Stefanie Jegelka. Measuring generalization with optimal transport. In Advances in Neural Information Processing Systems NeurIPS, volume 34, 2021

work page 2021
[8]

Predicting the generalization gap in deep networks with margin distributions

Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions. In 7th International Conference on Learning Representations, ICLR, 2018

work page 2018

Show all 36 references

[9]

Representation based complexity measures for predicting generalization in deep learning

Parth Natekar and Manik Sharma. Representation based complexity measures for predicting generalization in deep learning. arXiv preprint arXiv:2012.02775, 2020

2012 arXiv
[10]

Glorot, A

X. Glorot, A. Bordes, and Y . Bengio. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, 2011

2011
[11]

Batch normalization: Accelerating deep network training by reducing internal covariate shift

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning ICML, volume 37, 2015

2015
[12]

An exponential learning rate schedule for deep learning

Zhiyuan Li and Sanjeev Arora. An exponential learning rate schedule for deep learning. In 8th International Conference on Learning Representations, ICLR, 2019

2019
[13]

How does batch normalization help optimization? Advances in Neural Information Processing Systems NeurIPS, 31, 2018

Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. How does batch normalization help optimization? Advances in Neural Information Processing Systems NeurIPS, 31, 2018

2018
[14]

Deep learning

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016

2016
[15]

Relu and sigmoidal activation functions

Arnold M Pretorius, Etienne Barnard, and Marelie H Davel. Relu and sigmoidal activation functions. In Proceedings of South African Forum for Artificial Intelligence Research FAIR, 2019

2019
[16]

Learning deep parsimonious representations

Renjie Liao, Alex Schwing, Richard Zemel, and Raquel Urtasun. Learning deep parsimonious representations. Advances in Neural Information Processing Systems NeurIPS, 29, 2016

2016
[17]

Understanding batch normalization

Nils Bjorck, Carla P Gomes, Bart Selman, and Kilian Q Weinberger. Understanding batch normalization. In Advances in Neural Information Processing Systems NeurIPS, volume 31, 2018

2018
[18]

Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective

Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021

2021
[19]

Batch normalization explained

Randall Balestriero and Richard G Baraniuk. Batch normalization explained. arXiv preprint arXiv:2209.14778, 2022

2022 arXiv
[20]

Batch normalization orthogonalizes representations in deep random networks

Hadi Daneshmand, Amir Joudaki, and Francis Bach. Batch normalization orthogonalizes representations in deep random networks. Advances in Neural Information Processing Systems NeurIPS, 34, 2021

2021
[21]

Bengio, Aaron Courville, and Pascal Vincent

Y . Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives.IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 2013

2013
[22]

Some methods for classification and analysis of multivariate observations

James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, 1967

1967
[23]

Hierarchical clustering, pages 195–211

Frank Nielsen. Hierarchical clustering, pages 195–211. 2016. 12 A PREPRINT - F EBRUARY 14, 2025

2016
[24]

Normalized cuts and image segmentation

Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 2000

2000
[25]

Exploring layerwise decision making in dnns.Communications in Computer and Information Science CCIS, 1551, 2022

Coenraad Mouton and Marelie Davel. Exploring layerwise decision making in dnns.Communications in Computer and Information Science CCIS, 1551, 2022

2022
[26]

Deep clustering for unsupervised learning of visual features

Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), 2018

2018
[27]

K-means++: the advantages of careful seeding

David Arthur and Sergei Vassilvitskii. K-means++: the advantages of careful seeding. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, volume 8, 2007

2007
[28]

A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters

Joseph C Dunn. A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics, 3(3), 1973

1973
[29]

Algorithms for clustering data

Anil K Jain and Richard C Dubes. Algorithms for clustering data. Prentice-Hall, Inc., 1988

1988
[30]

A cluster separation measure

David L Davies and Donald W Bouldin. A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2), 1979

1979
[31]

Online deep clustering for unsupervised representation learning

Xiaohang Zhan, Jiahao Xie, Ziwei Liu, Yew-Soon Ong, and Chen Change Loy. Online deep clustering for unsupervised representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

2020
[32]

Prototypical contrastive learning of unsupervised representations

Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. Prototypical contrastive learning of unsupervised representations. In 9th International Conference on Learning Representations, ICLR, 2021

2021
[33]

The MNIST database of handwritten digits

Yann LeCun, Corinna Cortes, and Christopher Burges. The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 2, 1998

1998
[34]

Learning multiple layers of features from tiny images

Alex Krizhevsky. Learning multiple layers of features from tiny images. 05 2012

2012
[35]

Deep double descent: where bigger models and more data hurt

Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment , 2021, 2019

2021
[36]

Very deep convolutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations, ICLR, 2015. 13

2015

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Impact of batch normalization on convolutional network representations

Hermanus L Potgieter, Coenraad Mouton, and Marelie H Davel. Impact of batch normalization on convolutional network representations. In Southern African Conference for Artificial Intelligence Research SACAIR , pages 235–252. Springer, 2024

work page 2024

[2] [2]

Understanding deep learning (still) requires rethinking generalization

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 2021

work page 2021

[3] [3]

Sharpness-aware minimization for efficiently improving generalization

Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In 9th International Conference on Learning Representations, ICLR, 2021

work page 2021

[4] [4]

Algorithmic stability and generalization performance

Olivier Bousquet and André Elisseeff. Algorithmic stability and generalization performance. Advances in Neural Information Processing Systems NeurIPS, 13, 2000

work page 2000

[5] [5]

Input margins can predict generaliza- tion too

Coenraad Mouton, Marthinus Wilhelmus Theunissen, and Marelie H Davel. Input margins can predict generaliza- tion too. Proceedings of the AAAI Conference on Artificial Intelligence, 38(13), 2024

work page 2024

[6] [6]

How does information bottleneck help deep learning? In Proceedings of the 40th International Conference on Machine Learning ICML, volume 202, 2023

Kenji Kawaguchi, Zhun Deng, Xu Ji, and Jiaoyang Huang. How does information bottleneck help deep learning? In Proceedings of the 40th International Conference on Machine Learning ICML, volume 202, 2023

work page 2023

[7] [7]

Measuring generalization with optimal transport

Ching-Yao Chuang, Youssef Mroueh, Kristjan Greenewald, Antonio Torralba, and Stefanie Jegelka. Measuring generalization with optimal transport. In Advances in Neural Information Processing Systems NeurIPS, volume 34, 2021

work page 2021

[8] [8]

Predicting the generalization gap in deep networks with margin distributions

Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions. In 7th International Conference on Learning Representations, ICLR, 2018

work page 2018