Mirror flow reaches max-margin solutions in homogeneous neural networks where the mirror map choice controls whether learned features are sparse or dense while convergence can be exponentially slow.
Scalable optimization in the modular norm
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
fields
cs.LG 3verdicts
UNVERDICTED 3representative citing papers
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
Optimizers like Adam reduce to steepest descent under particular norms, opening a design space of norm assignments tailored to layer roles.
citing papers explorer
-
Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
Mirror flow reaches max-margin solutions in homogeneous neural networks where the mirror map choice controls whether learned features are sparse or dense while convergence can be exponentially slow.
-
Training Deep Learning Models with Norm-Constrained LMOs
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
-
Old Optimizer, New Norm: An Anthology
Optimizers like Adam reduce to steepest descent under particular norms, opening a design space of norm assignments tailored to layer roles.