The authors introduce softplus-calibrated adaptive learning rates (Sadam and SAMSGrad) and argue, with flawed proofs, that these converge faster and generalize better than Adam.
Besides the figures in main text, we have repeated experiments and show results as follows
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2019 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Calibrating the Adaptive Learning Rate to Improve Convergence of ADAM
The authors introduce softplus-calibrated adaptive learning rates (Sadam and SAMSGrad) and argue, with flawed proofs, that these converge faster and generalize better than Adam.