Schedule-Free SGD and AdEMAMix reduce to accelerated SGD variants, and a simplified one-momentum AdEMAMix matches the original on a 150m transformer.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants
Schedule-Free SGD and AdEMAMix reduce to accelerated SGD variants, and a simplified one-momentum AdEMAMix matches the original on a 150m transformer.