OptMuon combines orthogonalized momentum with trajectory-dependent AdaGrad-Norm adaptation to obtain expected-stationarity rates of order T^{-1/2} + sigma^{1/2}T^{-1/4} or T^{-1/2} + sigma^{1/3}T^{-1/3} that reduce to near-optimal deterministic first-order rates in the zero-noise regime.
Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
math.OC 2years
2026 2verdicts
UNVERDICTED 2representative citing papers
PS-Clip-SGD achieves optimal in-expectation convergence rates for non-convex optimization under heavy-tailed gradient noise, with matching high-probability guarantees, and outperforms standard methods on AlexNet trained on CIFAR-100.
citing papers explorer
-
OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality
OptMuon combines orthogonalized momentum with trajectory-dependent AdaGrad-Norm adaptation to obtain expected-stationarity rates of order T^{-1/2} + sigma^{1/2}T^{-1/4} or T^{-1/2} + sigma^{1/3}T^{-1/3} that reduce to near-optimal deterministic first-order rates in the zero-noise regime.
-
Robust and Fast Training via Per-Sample Clipping
PS-Clip-SGD achieves optimal in-expectation convergence rates for non-convex optimization under heavy-tailed gradient noise, with matching high-probability guarantees, and outperforms standard methods on AlexNet trained on CIFAR-100.