REVIEW 2 cited by
Identifying Generalization Properties in Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We connect model generalization with the local property of a solution under the PAC-Bayes paradigm. In particular, we prove that model generalization ability is related to the Hessian, the higher-order "smoothness" terms characterized by the Lipschitz constant of the Hessian, and the scales of the parameters. Guided by the proof, we propose a metric to score the generalization capability of the model, as well as an algorithm that optimizes the perturbed model accordingly.
Forward citations
Cited by 2 Pith papers
-
IKUN: Initialization to Keep snn training and generalization great with sUrrogate-stable variaNce
IKUN sets initial SNN weights with a surrogate-gradient variance correction and reaches accuracy thresholds in fewer epochs on FashionMNIST, but final accuracy is close to standard initializations.
-
A Method for Enhancing Generalization of Adam by Multiple Integrations
MIAdam adds an n-th order integral term to Adam's gradient update for the first ζ steps, then switches to Adam, and is reported to improve test accuracy and label-noise robustness.
Discussion (0). Continue with ORCID to comment.