REVIEW 1 cited by
A mathematical model for automatic differentiation in machine learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Automatic differentiation, as implemented today, does not have a simple mathematical model adapted to the needs of modern machine learning. In this work we articulate the relationships between differentiation of programs as implemented in practice and differentiation of nonsmooth functions. To this end we provide a simple class of functions, a nonsmooth calculus, and show how they apply to stochastic approximation methods. We also evidence the issue of artificial critical points created by algorithmic differentiation and show how usual methods avoid these points with probability one.
Forward citations
Cited by 1 Pith paper
-
Mathematical analysis of the gradients in deep learning
For deep feedforward networks with piecewise-smooth activations, the autodiff gradient is shown to be the unique limit of gradients of smoothed activations, a limiting Frechet subgradient, and equal to the true gradie...
Discussion (0). Continue with ORCID to comment.