Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.
A scalable measure of loss landscape curvature for analyzing the training dynamics of LLMs.arXiv preprint arXiv:2601.16979
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
years
2026 5representative citing papers
Derives mechanism-based monitors from module functional roles and validates them via fault-injection experiments showing early detection of LLM training instability.
Edge of stability acts as a selective mechanism that amplifies learning on data groups with aligned persistent gradients while suppressing others.
CrispEdit edits LLMs via low-curvature projections using Bregman divergence and K-FAC approximations, achieving high edit success with under 1% average capability degradation.
Self-generated replay from language models nearly eliminates catastrophic forgetting during finetuning except when models are pretrained close to saturation.
citing papers explorer
-
Why Muon Outperforms Adam: A Curvature Perspective
Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.
-
Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability
Derives mechanism-based monitors from module functional roles and validates them via fault-injection experiments showing early detection of LLM training instability.
-
Edge of Stability Selectively Shapes Learning Across the Data Distribution
Edge of stability acts as a selective mechanism that amplifies learning on data groups with aligned persistent gradients while suppressing others.
-
CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
CrispEdit edits LLMs via low-curvature projections using Bregman divergence and K-FAC approximations, achieving high edit success with under 1% average capability degradation.
-
Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
Self-generated replay from language models nearly eliminates catastrophic forgetting during finetuning except when models are pretrained close to saturation.