REVIEW 5 cited by
Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This book aims to provide an introduction to the topic of deep learning algorithms. We review essential components of deep learning algorithms in full mathematical detail including different artificial neural network (ANN) architectures (such as fully-connected feedforward ANNs, convolutional ANNs, recurrent ANNs, residual ANNs, and ANNs with batch normalization) and different optimization algorithms (such as the basic stochastic gradient descent (SGD) method, accelerated methods, and adaptive methods). We also cover several theoretical aspects of deep learning algorithms such as approximation capacities of ANNs (including a calculus for ANNs), optimization theory (including Kurdyka-{\L}ojasiewicz inequalities), and generalization errors. In the last part of the book some deep learning approximation methods for PDEs are reviewed including physics-informed neural networks (PINNs) and deep Galerkin methods. We hope that this book will be useful for students and scientists who do not yet have any background in deep learning at all and would like to gain a solid foundation as well as for practitioners who would like to obtain a firmer mathematical understanding of the objects and methods considered in deep learning.
Forward citations
Cited by 5 Pith papers
-
On MUON optimization: From non-convergence to an error analysis with Polar Express and the Newton-Schulz polynomial from implementations
For a one-dimensional quadratic stochastic optimization problem, MUON with Newton-Schulz steps provably fails to converge to the minimizer for all sufficiently large mini-batch sizes when the data is skewed, while a n...
-
Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks
Bounded trajectories of a broad class of GD optimizers (Adam, RMSprop, NAG, Adan, etc.) converge with polynomial rates to critical points of KL objectives with locally Lipschitz gradients, covering analytic-activation...
-
Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time
Discrete-time LIF spiking networks realize piecewise constant functions on polyhedral regions, and each first-layer neuron generates only O(T^2) parallel hyperplanes over T time steps, not exponentially many.
-
PADAM: Parallel averaged Adam reduces the error for stochastic optimization in scientific machine learning
PADAM runs K differently averaged Adam trajectories in parallel, selects the one with the smallest test error, and achieves the best optimization error in nearly all of 13 tested scientific machine learning problems w...
-
Survey on Methods for Detection, Classification and Location of Faults in Power Systems Using Artificial Intelligence
A broad, non-systematic review of AI-based fault detection, classification, and location methods for transmission, distribution, and DC power lines and cables.
Discussion (0). Continue with ORCID to comment.