Mirror flow reaches max-margin solutions in homogeneous neural networks where the mirror map choice controls whether learned features are sparse or dense while convergence can be exponentially slow.
Scalable optimization in the modular norm
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
fields
cs.LG 4verdicts
UNVERDICTED 4representative citing papers
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
Optimizers like Adam reduce to steepest descent under particular norms, opening a design space of norm assignments tailored to layer roles.
LoRA-Muon applies Muon's spectral steepest descent to low-rank factors with split weight decay, acting as a transferable proxy for full-rank Muon and Shampoo optimizers.
citing papers explorer
-
Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
Mirror flow reaches max-margin solutions in homogeneous neural networks where the mirror map choice controls whether learned features are sparse or dense while convergence can be exponentially slow.
-
Training Deep Learning Models with Norm-Constrained LMOs
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
-
Old Optimizer, New Norm: An Anthology
Optimizers like Adam reduce to steepest descent under particular norms, opening a design space of norm assignments tailored to layer roles.
-
LoRA-Muon: Spectral Steepest Descent on the Low-Rank Manifold
LoRA-Muon applies Muon's spectral steepest descent to low-rank factors with split weight decay, acting as a transferable proxy for full-rank Muon and Shampoo optimizers.