Under a new smoothness assumption linking local curvature to the loss gap, increasing learning rates provably accelerate GD and SGD convergence, with up to Theta(T) speedup in special cases.
Why Gradient Clipping Accelerates Training : A Theoretical Justification for Adaptivity
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence
Under a new smoothness assumption linking local curvature to the loss gap, increasing learning rates provably accelerate GD and SGD convergence, with up to Theta(T) speedup in special cases.