REVIEW 1 cited by
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This paper follows up on a recent work of Neu et al. (2021) and presents some new information-theoretic upper bounds for the generalization error of machine learning models, such as neural networks, trained with SGD. We apply these bounds to analyzing the generalization behaviour of linear and two-layer ReLU networks. Experimental study of these bounds provide some insights on the SGD training of neural networks. They also point to a new and simple regularization scheme which we show performs comparably to the current state of the art.
Forward citations
Cited by 1 Pith paper
-
Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective
Claims computable MI/CMI bounds on fairness generalization error, but the core derivation uses an invalid variance-based Hoeffding step.
Discussion (0). Continue with ORCID to comment.