DP-MacAdam combines adaptive clipping and Adam-style momentum in DP-SGD by sharing bias-free empirical gradient statistics, achieving higher utility than DP-SGD, AdaClip, and DP-Adam without manual clipping threshold tuning.
An improvement of the convergence proof of the ADAM-Optimizer
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
A common way to train neural networks is the Backpropagation. This algorithm includes a gradient descent method, which needs an adaptive step size. In the area of neural networks, the ADAM-Optimizer is one of the most popular adaptive step size methods. It was invented in \cite{Kingma.2015} by Kingma and Ba. The $5865$ citations in only three years shows additionally the importance of the given paper. We discovered that the given convergence proof of the optimizer contains some mistakes, so that the proof will be wrong. In this paper we give an improvement to the convergence proof of the ADAM-Optimizer.
fields
cs.LG 2verdicts
UNVERDICTED 2representative citing papers
A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.
citing papers explorer
-
DP-MacAdam: Differentially Private Mechanism with Adaptive Clipping and Adaptive Momentum
DP-MacAdam combines adaptive clipping and Adam-style momentum in DP-SGD by sharing bias-free empirical gradient statistics, achieving higher utility than DP-SGD, AdaClip, and DP-Adam without manual clipping threshold tuning.
-
Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics
A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.