Intrinsic Muon provides closed-form linear maximization oracles on multiple Riemannian matrix manifolds for unitarily invariant norms, with convergence rates depending only on manifold dimension or rank.
arXiv preprint arXiv:2601.21487 , year=
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
citation-role summary
background 2
method 1
citation-polarity summary
years
2026 6verdicts
UNVERDICTED 6representative citing papers
MAPL learns task-specific orthogonal compression subspaces per pipeline stage via manifold-constrained optimization and recovers signals with low-overhead anchors, yielding better compression-performance tradeoffs than fixed projections on LLaMA models up to 1B parameters.
Proposes equivariant optimizer updates matched to layer symmetries for embeddings, SwiGLU MLPs, and MoE routers, with reported gains in validation loss and training stability on several language model architectures.
Manifold constraints via the new MACRO optimizer independently bound activation scales and enforce rotational equilibrium in LLM pre-training, subsuming RMS normalization and decoupled weight decay while delivering competitive performance with convergence guarantees.
Stiefel on attention and DGram on MLP layers outperforms uniform or inverted manifold assignments in transformer pretraining by avoiding attention logit amplification.
Proves linear convergence of Spectral Descent (SD) and Truncated SD for non-smooth convex problems under stated conditions, sublinear rates for regularized versions via Frank-Wolfe, and recovery guarantees for robust low-rank matrix recovery.
citing papers explorer
-
Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
Intrinsic Muon provides closed-form linear maximization oracles on multiple Riemannian matrix manifolds for unitarily invariant norms, with convergence rates depending only on manifold dimension or rank.