Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.
Title resolution pending
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
years
2026 3roles
background 1polarities
background 1representative citing papers
Proposes equivariant optimizer updates matched to layer symmetries for embeddings, SwiGLU MLPs, and MoE routers, with reported gains in validation loss and training stability on several language model architectures.
A 4-bit quantization scheme for the Muon optimizer — combining pre-quantization normalization, singular-structure-aligned decomposition, and µ-law companding — recovers most full-precision training quality with up to 7.3x optimizer-state memory savings.
citing papers explorer
-
Why Muon Outperforms Adam: A Curvature Perspective
Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.
-
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
Proposes equivariant optimizer updates matched to layer symmetries for embeddings, SwiGLU MLPs, and MoE routers, with reported gains in validation loss and training stability on several language model architectures.
-
MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
A 4-bit quantization scheme for the Muon optimizer — combining pre-quantization normalization, singular-structure-aligned decomposition, and µ-law companding — recovers most full-precision training quality with up to 7.3x optimizer-state memory savings.