REVIEW 2 cited by
Six Lectures on Linearized Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In these six lectures, we examine what can be learnt about the behavior of multi-layer neural networks from the analysis of linear models. We first recall the correspondence between neural networks and linear models via the so-called lazy regime. We then review four models for linearized neural networks: linear regression with concentrated features, kernel ridge regression, random feature model and neural tangent model. Finally, we highlight the limitations of the linear theory and discuss how other approaches can overcome them.
Forward citations
Cited by 2 Pith papers
-
ExpTest: Automating Learning Rate Searching and Tuning with Insights from Linearized Neural Networks
ExpTest auto-selects and tunes the learning rate by testing whether the training loss decays exponentially, without needing an initial learning rate choice.
-
Tracing sources of epistemic uncertainty in deep learning predictions: homo- and hetero-scedastic linearized estimators
A deep-learning adaptation of homoscedastic and heteroscedastic linear-regression variance estimators, scaled with EKFAC, to attribute prediction uncertainty to label noise versus scarce data.
Discussion (0). Continue with ORCID to comment.