Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature

Tong Che

REVIEW 4 major objections 4 minor 35 references

Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A matched spectral oracle makes Muon's update exact for any supplied geometry.

desk verdict Solid theory and honest scoping, but the headline grokking speedup is a single-seed exploratory screen; peer review should demand real replication before anyone cites the empirical claims. read the letter →

arxiv 2608.09763 v1 pith:PTPSOSYF submitted 2026-08-10 cs.AI

Tong Che This is my paper · ORCID

classification cs.AI MSC 68T0765K10

keywords GO-MUONspectralpreconditioningMuonoptimizerK-FACFisherinformationgeneralizedGauss-NewtonOASshrinkagegrokking

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GO-MUON's central claim is that Muon's polar update, exact for an unweighted operator-norm ball, becomes exact for any weighted spectral geometry when the polar map is wrapped with matched positive-definite maps. The paper proves that the direction $D_0 = P_B Q(P_B M P_A) P_A$ solves $\max_{\lVert P_B^{-1} D P_A^{-1}\rVert_{\mathrm{op}}\le 1}\langle M, D\rangle$ with value $\lVert P_B M P_A\rVert_*$, independently of how $P_A$ and $P_B$ are estimated or refreshed. It then instantiates the maps from OAS-shrunk second moments, interprets the backward factor as an approximate softmax Fisher/GGN factor under predictive consistency, and shows that refreshing the geometry every four steps is a compute-statistics tradeoff rather than a denoising mechanism. If the claim holds, GO-MUON's empirical gains on character language modeling and modular arithmetic are attributable to the curvature geometry, not to approximation error in the update.

What carries the argument

The load-bearing object is the matched spectral oracle: the sandwich $C=P_B M P_A$, the support polar $Q(C)$, and the map-back $D_0=P_B Q(C) P_A$. This is a matrix instance of a preconditioned-norm pullback; the change of variables $U=P_B^{-1} D P_A^{-1}$ converts the weighted ball $\lVert P_B^{-1} D P_A^{-1}\rVert_{\mathrm{op}}\le 1$ into the unit operator ball, the polar maximizes the trace pairing there, and the map-back returns to the original tangent coordinates. The implementation uses inverse fourth powers of OAS-shrunk empirical second moments as the SPD maps, which gives the operator-norm analogue of square-root K-FAC geometry, and an equal-Frobenius-norm graft $D_0\sqrt{k}/\lVert D_0\rVert_F$ that preserves the oracle ray and restores Muon's update energy.

What would settle it

Measure the factor mismatch $\lVert \hat B_B - B_F\rVert$ on a real training trajectory of a softmax network where the model is far from the data conditional; if GO-MUON still speeds up while the mismatch is large, curvature matching is not the cause. Alternatively, replace the estimated factors with random fixed SPD maps: the oracle guarantee still holds, so any loss of the reported speedups would show that the geometry values, not the exact update, drive the results.

Watch

Extended reading notes

Core claim

The paper's central discovery is a factor-agnostic identity. For any symmetric positive-definite $P_A$, $P_B$ and momentum source $M$, define $C=P_B M P_A$ and let $Q(C)$ be a support polar, a maximizer of $\langle C, U\rangle$ over $\lVert U\rVert_{\mathrm{op}}\le 1$. Then $D_0=P_B Q(C) P_A$ is an exact maximizer of the matched spectral support problem, with $\langle M, D_0\rangle=\lVert C\rVert_*$. The oracle is exact for the supplied geometry by construction; choosing and estimating $P_A$ and $P_B$ decides whether the geometry is useful. GO-MUON sets $P_A=A^{-1/4}$ and $P_B=\bar B^{-1/4}$ from OAS-regularized activation and backward-gradient second moments with a Frobenius graft, and reuses the cached factors for four updates. The paper further proves that under softmax cross-entropy the observed-label backward factor approaches the model Fisher/GGN factor as the predictive distribution approaches the data conditional, and that sparse refresh preserves slow-signal lag while increasing stationary factor variance.

Load-bearing premise

The practical value rests on the assumption that, during training, the OAS-shrunk observed-label backward second moments approximate the model Fisher/GGN factors closely enough to be a useful preconditioner; Corollary 4 needs predictive consistency $p_\theta \to q$ and bounded Jacobians, which the paper notes is not true for arbitrary trajectories.

Editorial extensions

If this is right

GO-MUON's raw update is an exact oracle for the weighted spectral ball it declares, so its behavior on a given loss landscape is determined by the factor geometry rather than by approximation in the polarity step.
Four-step refresh changes the optimizer's statistical properties in a precise way: it nearly preserves average lag on a slowly varying geometry while raising stationary factor noise and cutting per-step compute by roughly twenty percent in the reported setting.
For softmax cross-entropy, the backward factor converges to the Fisher/GGN factor only under predictive consistency, and absolute factor convergence does not guarantee relative inverse-root accuracy near interpolation.
In the reported paired comparisons, GO-MUON lowers the geometric-mean training-loss trajectory against Muon on Tiny Shakespeare and a hash-matched Penn Treebank run, and reaches sustained ninety-nine percent held-out grokking accuracy far earlier on the corrected modular-addition screen.
Because the oracle identity is independent of factor freshness, the exactness claim survives even when the geometry is stale; only closeness to the current ball requires controlling the coordinate-map drift quantity $\Theta_t$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: because Theorem 1 is factor-agnostic, the GO-MUON template could be applied to any SPD factor estimator, including gradient-Gram moments or other curvature surrogates, and still carry the same exact-oracle guarantee; the paper does not itself claim this generalization beyond the OAS K-FAC-root instantiation.
Editorial inference: the blockwise coherence created by holding geometry for four steps is an untested mechanism; one could isolate it by comparing $K=4$ refresh against a control that changes the preconditioner every step under matched per-step compute, separating the variance-lag tradeoff from trajectory effects.
Editorial inference: Proposition 3 suggests a direct diagnostic for curvature-aware optimizers: track $\lVert \hat B_B - B_F\rVert$ during early training and check whether GO-MUON's gains appear only after this mismatch shrinks; if the gains appear earlier, curvature matching is not the whole story.
Editorial inference: the modular-addition grokking speedup may be reproducible with a cheaper diagonal curvature approximation; the paper's full K-FAC-root machinery would then be sufficient but not necessary for the reported effect.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful content here is the theory, and it is genuinely good. Theorem 1 cleanly extends Muon's polar update to a matched spectral oracle with arbitrary SPD maps, and the paper honestly attributes the general pullback to Veprikov et al. Proposition 6 is the real gem: the sparse-EMA variance–lag tradeoff is derived correctly, and the observation that lazy refresh is a compute–statistics tradeoff, not a denoising mechanism, is a genuinely nice clarification. Proposition 3's mismatch bound and the careful caveats around when observed-label factors approach the Fisher/GGN are also well done. The paper is honest about what it does not prove—Corollary 4's predictive-consistency caveat, the K-FAC approximation error, and the fact that absolute factor convergence does not imply relative inverse-root accuracy. That intellectual honesty deserves credit.

The empirical claims, however, are not at the same standard. The PTB 0.38% result is selected post hoc by ranking the same three-window summary used as the endpoint; the paper itself labels it exploratory, and the middle window is tied. The Tiny Shakespeare result is better but still small-scale. The real problem is the grokking table. One seed per modulus, learning rates selected on the same thresholded grokking criterion, and a discontinuous endpoint that the mod-107 trajectory itself violates—the abstract's 'sustained 99%' is contradicted by the admitted drop below threshold as late as update 4470. A different seed or a slightly different LR could shrink or reverse the 8x/20.5x speedups. No code is released, and there is no comparison against Mousse or FISMO, which are the closest structural variants. The stress-test note is right: this is not yet a load-bearing empirical result, even though Theorem 1 remains true.

Who benefits: researchers working on structured optimizers will get real value from the oracle formulation and the sparse-EMA analysis. The paper deserves a serious referee because the theory is substantive and the empirical methodology is fixable—but the referee should be told to focus on the grokking screen and demand multi-seed, LR-matched, endpoint-stable replication. As-is, the empirical conclusions should be treated as preliminary, not as established.

Referee Report

4 major / 4 minor

Summary. The paper introduces GO-MUON, an optimizer that marries Muon's polar update with a data-dependent spectral geometry. The central theoretical object is Theorem 1: for any SPD maps P_A, P_B, the direction D = P_B Q(P_B M P_A) P_A is an exact maximizer for the weighted spectral support problem, with value ||P_B M P_A||_*. The paper also proves a quantitative bound on the mismatch between observed-label backward factors and the softmax Fisher/GGN factor (Proposition 3), a worst-case conditioning and alignment certificate for the quarter-power graft (Proposition 5), and an exact variance–lag characterization of sparse EMA refresh (Proposition 6). Experiments cover PTB and Tiny Shakespeare character language modeling, a modular-addition grokking screen, and a lazy-refresh comparison. The abstract claims that GO-MUON reaches sustained 99% held-out accuracy at updates 290 and 220 on moduli 103 and 107, versus 2320 and 4520 for Muon.

Significance. The theory in Section 2 is clean and correctly scoped: Theorem 1 is a direct duality argument, the paper explicitly credits the general pullback oracle to Veprikov et al., and Proposition 6 gives an exact variance–lag characterization of sparse refresh. Proposition 3 quantifies the observed-label/Fisher mismatch, and Proposition 5 provides a worst-case conditioning certificate. These are useful and, as far as I can tell, sound. The empirical portion is considerably weaker: the headline grokking speedups rest on one seed per modulus with learning rates selected on the same thresholded endpoint, the PTB 0.38% result is explicitly exploratory, and the paper does not measure the geometry-alignment quantities that would connect the theory to the gains. With the theoretical core intact, the paper is a plausible contribution to optimizer design, but the current abstract overstates the empirical support.

major comments (4)

[§5.2, Table 3] The modular-addition speedup claim is not yet supported. The table reports one deterministic split and initialization seed per modulus, and the learning-rate screen over {0.08,0.16} for GO-MUON and {0.16,0.32} for Muon selects rates using the same grokking criterion that defines the endpoint (first of five consecutive evaluations at ≥99% test accuracy, sampled every ten updates). This discontinuous, thresholded statistic is highly seed- and learning-rate-sensitive, so the reported 8.0× and 20.5× speedups could be artifacts of learning-rate selection or endpoint definition. The paper should provide multi-seed replication, a pre-specified learning-rate selection rule, and a stable endpoint or full retention curves before the abstract's speedup claim can stand.
[§5.1, Table 2] The PTB result is weakened by post-hoc learning-rate selection. The paper states that ranking the recorded GO-MUON sweep by the three-window geometric mean selects learning rate 0.080, that this ranking was introduced after the original late-window rule (which selected 0.16), and that the hash-matched comparison is exploratory. A 0.38% trajectory reduction selected in this way is not a confirmatory result; the paper should either pre-register the selection rule or report the full sweep and all paired-seed summaries for every learning rate.
[§3.1, Corollary 4 and §2.1, Proposition 7] The attribution of empirical gains to curvature matching requires a direct check of the weakest assumption: that OAS-shrunken observed-label backward factors approximate Fisher/GGN factors closely enough during training to be a useful preconditioner. Corollary 4 requires realizability or predictive consistency, and the paper itself notes in Section 3.1 that absolute factor convergence does not imply relative inverse-root accuracy near interpolation, while Section 2.1 states that the coordinate-drift quantity Θ_t is not measured in the experiments. Without measuring factor alignment or Θ_t, the grokking and loss differences could stem from learning-rate or other side effects rather than from the matched geometry.
[§5.2, Table 3 and Abstract] The abstract's 'sustained 99%' overstates the mod-107 result. The paper records that the mod-107 GO-MUON trajectory later leaves the 99% region, with below-threshold evaluations as late as update 4470 before returning at update 4480. 'First of five consecutive evaluations' is not a description of sustained accuracy; the text should be revised to state the onset of a 99% threshold condition and to report the subsequent retention profile.

minor comments (4)

[Abstract and §5.2] The phrase 'formal one-seed-per-modulus screen' is not formal in a statistical sense; a screening run with one seed is better described as a pilot or exploratory result.
[§5.2] The grokking-delay diagnostic subtracts 'the first update with 99% training accuracy'; please clarify whether training accuracy is evaluated on the same ten-update schedule and whether it is sensitive to the same thresholding as the test endpoint.
[Table 2] The Tiny Shakespeare control is described as 'Muon+Q/K control' while the PTB control is 'Muon'; a sentence explaining the difference and why the same optimizer variant was not used for both corpora would improve comparability.
[§5.4] The sentence 'Both arms record zero negative momentum-source and current-gradient alignments' should define the alignment statistic or point to where it is defined, since the statement is otherwise unverifiable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 1 is a self-contained duality proof, and the disclosed learning-rate screens are selection-bias risks rather than circular reductions.

full rationale

The claimed first-principles result is Theorem 1: D0 = P_B Q(P_B M P_A) P_A solves the weighted operator-norm support problem. The proof is self-contained: it substitutes U = P_B^{-1} D P_A^{-1}, uses trace cyclicity, and invokes operator/nuclear duality; no fitted quantity enters, and the accompanying cached-geometry bound (Prop. 7) derives from norm inclusions rather than from the experimental setup. The conditional-Fisher statement (Prop. 3/Cor. 4) is an explicit identity with an explicit realizability caveat, and the sparse-refresh variance/lag result (Prop. 6) follows from the recursions. Citations to prior work, including Veprikov et al., are credit for a more general framework and are not load-bearing: Theorem 1 is proved in the text. The only circularity-adjacent issue is experimental: PTB and grokking learning rates were selected on the same trajectory/grokking metrics later reported, and the paper itself labels the PTB comparison exploratory and the grokking result a one-seed screen; this is a selection-bias/validity risk, not a reduction of the derivation to its inputs. The abstract's 'sustained 99%' wording also slightly overstates the mod-107 trajectory's later retention event, but that is a reporting-accuracy issue rather than circularity. Accordingly, no circular step is established.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central theoretical claims rest on standard matrix duality and polar decomposition, plus domain assumptions of weak stationarity and predictive consistency. No invented entities are introduced. The free parameters are optimizer hyperparameters, several of which were selected on the evaluation metrics used for the headline results.

free parameters (5)

GO-MUON peak learning rate, PTB run = 0.080
Selected by ranking the recorded GO-MUON sweep on the three-window trajectory geometric mean (Section 5.1); the comparison is labeled exploratory.
GO-MUON peak learning rate, modular addition = 0.080
Selected from {0.08, 0.16} under the grokking criterion on one seed per modulus (Section 5.2).
Muon learning rate, modular addition = 0.160
Selected from {0.16, 0.32} under the same grokking criterion (Section 5.2).
Backward EMA coefficient beta_B = 0.97
Hand-chosen; used in Eqs. (21)-(22).
Refresh interval K = 4
Hand-chosen; the experiments compare K=4 against refresh 1 (Section 5.4).

assumptions (5)

standard math Operator/nuclear norm duality: the dual norm of the operator norm is the nuclear norm
Invoked in the proof of Theorem 1 to evaluate the support function.
standard math Existence and properties of the polar decomposition / support polar
Used to define Q(C) in Eq. (4) and Section 2.1.
domain assumption Weak stationarity of the centered post-OAS factor observations
Assumed in Proposition 6 to derive covariance and lag formulas; standard for EMA analysis.
domain assumption Predictive consistency / realizability: p_theta -> q as training progresses
Required by Corollary 4 for observed-label backward factors to approach Fisher/GGN factors; the paper itself states this is not a property of arbitrary trajectories.
standard math OAS shrinkage and the numerical eigenvalue floor yield positive-definite effective factors
Invoked in Section 3.3 to place Theorem 1 on the effective factors.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature." pith.science (2026). https://pith.science/paper/PTPSOSYF

@misc{pith2026260809763,
  author       = {Pith},
  title        = {Pith review of: Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PTPSOSYF}},
  note         = {Machine review of arXiv:2608.09763}
}

read the original abstract

Muon's polar update is exact for an unweighted spectral geometry. We introduce GO-MUON, which uses a matched data-dependent geometry and reuses it across several optimization steps. Conditioned on any positive-definite left and right maps, its raw update exactly solves the corresponding weighted spectral oracle; this statement is independent of how the maps are estimated or how recently they were refreshed. For softmax cross-entropy, we quantify when the observed-label backward factor approaches the model Fisher and generalized Gauss--Newton factor. We also show that four-step refresh nearly preserves the tracking delay of slowly changing geometry while increasing stationary factor noise, making lazy geometry a compute--statistics tradeoff rather than a denoising mechanism.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 17 canonical work pages

[1]

2024 , howpublished =

Keller Jordan and Yuchen Jin and Vlado Boza and Jiacheng You and Franz Cesista and Laker Newhouse and Jeremy Bernstein , title =. 2024 , howpublished =

2024
[2]

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods , journal =

Andrey Veprikov and Arman Bolatov and Samuel Horv. Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods , journal =. 2025 , url =

work page 2025
[3]

arXiv preprint arXiv:2604.09967 , year =

Ziyue Liu and Ruijie Zhang and Zhengyang Wang and Yequan Zhao and Yupeng Su and Zi Yang and Zheng Zhang , title =. arXiv preprint arXiv:2604.09967 , year =

arXiv
[4]

MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning

Tongle Wu and Huanyu Dong and Ying Sun and Ziye Ma , title =. arXiv preprint arXiv:2608.05088 , year =

work page Pith review arXiv
[5]

arXiv preprint arXiv:2601.21750 , year =

Chenrui Xu and Wenjing Yan and Ying-Jun Angela Zhang , title =. arXiv preprint arXiv:2601.21750 , year =

arXiv
[6]

arXiv preprint arXiv:2604.01472 , year =

Zhehang Du and Weijie Su , title =. arXiv preprint arXiv:2604.01472 , year =

arXiv
[7]

arXiv preprint arXiv:2510.09378 , year =

Natalie Abreu and Nikhil Vyas and Sham Kakade and Depen Morwani , title =. arXiv preprint arXiv:2510.09378 , year =

arXiv
[8]

Proceedings of the 35th International Conference on Machine Learning , year =

Noam Shazeer and Mitchell Stern , title =. Proceedings of the 35th International Conference on Machine Learning , year =

work page

Show all 35 references

[9]

Proceedings of the 35th International Conference on Machine Learning , year =

Vineet Gupta and Tomer Koren and Yoram Singer , title =. Proceedings of the 35th International Conference on Machine Learning , year =
[10]

Kingma and Jimmy Ba , title =

Diederik P. Kingma and Jimmy Ba , title =. International Conference on Learning Representations , year =
[11]

Proceedings of the 32nd International Conference on Machine Learning , year =

James Martens and Roger Grosse , title =. Proceedings of the 32nd International Conference on Machine Learning , year =
[12]

Advances in Neural Information Processing Systems , volume =

Frederik Kunstner and Lukas Balles and Philipp Hennig , title =. Advances in Neural Information Processing Systems , volume =. 2019 , url =

2019
[13]

Journal of Machine Learning Research , volume =

James Martens , title =. Journal of Machine Learning Research , volume =. 2020 , url =

2020
[14]

Proceedings of the 35th International Conference on Machine Learning , series =

Kevin Luk and Roger Grosse , title =. Proceedings of the 35th International Conference on Machine Learning , series =. 2018 , url =

2018
[15]

Eldar and Alfred O

Yilun Chen and Ami Wiesel and Yonina C. Eldar and Alfred O. Hero , title =. IEEE Transactions on Signal Processing , volume =. 2010 , url =

2010
[16]

Proceedings of the 39th International Conference on Machine Learning , year =

Frederik Benzing , title =. Proceedings of the 39th International Conference on Machine Learning , year =
[17]

arXiv preprint arXiv:2409.11321 , year =

Nikhil Vyas and Depen Morwani and Rosie Zhao and Mujin Kwun and Itai Shapira and David Brandfonbrener and Lucas Janson and Sham Kakade , title =. arXiv preprint arXiv:2409.11321 , year =

arXiv
[18]

arXiv preprint arXiv:2603.09697 , year =

Yechen Zhang and Shuhao Xing and Junhao Huang and Kai Lv and Yunhua Zhou and Xipeng Qiu and Qipeng Guo and Kai Chen , title =. arXiv preprint arXiv:2603.09697 , year =
[19]

arXiv preprint arXiv:2409.20325 , year =

Jeremy Bernstein and Laker Newhouse , title =. arXiv preprint arXiv:2409.20325 , year =

arXiv
[20]

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods , journal =

Andrey Veprikov and Arman Bolatov and Aleksandr Bogdanov and Samuel Horv. Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods , journal =. 2025 , url =

2025
[21]

Advances in Neural Information Processing Systems , year =

Tim Large and Yang Liu and Minyoung Huh and Hyojin Bahng and Phillip Isola and Jeremy Bernstein , title =. Advances in Neural Information Processing Systems , year =
[22]

arXiv preprint arXiv:2502.07529 , year =

Thomas Pethick and Wanyun Xie and Kimon Antonakopoulos and Zhenyu Zhu and Antonio Silveti-Falls and Volkan Cevher , title =. arXiv preprint arXiv:2502.07529 , year =

arXiv
[23]

Gluon: Making Muon & Scion Great Again! Bridging Theory and Practice of

Artem Riabinin and Egor Shulgin and Kaja Gruntkowska and Peter Richt. Gluon: Making Muon & Scion Great Again! Bridging Theory and Practice of. arXiv preprint arXiv:2505.13416 , year =

arXiv
[24]

arXiv preprint arXiv:2510.25000 , year =

Kevin Frans and Pieter Abbeel and Sergey Levine , title =. arXiv preprint arXiv:2510.25000 , year =
[25]

arXiv preprint arXiv:2603.28254 , year =

Da Chang and Qiankun Shi and Lvgang Zhang and Yu Li and Ruijie Zhang and Yao Lu and Yongxiang Liu and Ganzhao Yuan , title =. arXiv preprint arXiv:2603.28254 , year =

arXiv
[26]

arXiv preprint arXiv:2602.21545 , year =

Ruijie Zhang and Yequan Zhao and Ziyue Liu and Zhengyang Wang and Zheng Zhang , title =. arXiv preprint arXiv:2602.21545 , year =

arXiv
[27]

arXiv preprint arXiv:2502.16982 , year =

Jingyuan Liu and Jianlin Su and Xingcheng Yao and Zhejun Jiang and Guokun Lai and Yulun Du and Yidao Qin and Weixin Xu and Enzhe Lu and Junjie Yan and others , title =. arXiv preprint arXiv:2502.16982 , year =

arXiv
[28]

arXiv preprint arXiv:2002.09018 , year =

Rohan Anil and Vineet Gupta and Tomer Koren and Kevin Regan and Yoram Singer , title =. arXiv preprint arXiv:2002.09018 , year =

2002 arXiv
[29]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

Kazuki Osawa and Yohei Tsuji and Yuichiro Ueno and Akira Naruse and Rio Yokota and Satoshi Matsuoka , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2019 , url =

2019
[30]

Gregory Pauloski and Zhao Zhang and Lei Huang and Weijia Xu and Ian T

J. Gregory Pauloski and Zhao Zhang and Lei Huang and Weijia Xu and Ian T. Foster , title =. 2020 , doi =

2020
[31]

Turner and Alireza Makhzani , title =

Wu Lin and Felix Dangel and Runa Eschenhagen and Kirill Neklyudov and Agustinus Kristiadi and Richard E. Turner and Alireza Makhzani , title =. Proceedings of the 41st International Conference on Machine Learning , series =. 2024 , url =

2024
[32]

Turner and Hao-Jun Michael Shi , title =

Runa Eschenhagen and Aaron Defazio and Tsung-Hsien Lee and Richard E. Turner and Hao-Jun Michael Shi , title =. Advances in Neural Information Processing Systems , volume =. 2025 , url =

2025
[33]

International Conference on Learning Representations , year =

Jimmy Ba and Roger Grosse and James Martens , title =. International Conference on Learning Representations , year =
[34]

Gregory Pauloski and Qi Huang and Lei Huang and Shivaram Venkataraman and Kyle Chard and Ian Foster and Zhao Zhang , title =

J. Gregory Pauloski and Qi Huang and Lei Huang and Shivaram Venkataraman and Kyle Chard and Ian Foster and Zhao Zhang , title =. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis , articleno =. 2021 , publisher =. doi:...

2021
[35]

arXiv preprint arXiv:2309.06497 , year =

Hao-Jun Michael Shi and Tsung-Hsien Lee and Shintaro Iwasaki and Jose Gallego-Posada and Zhijing Li and Kaushik Rangadurai and Dheevatsa Mudigere and Michael Rabbat , title =. arXiv preprint arXiv:2309.06497 , year =

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

2024 , howpublished =

Keller Jordan and Yuchen Jin and Vlado Boza and Jiacheng You and Franz Cesista and Laker Newhouse and Jeremy Bernstein , title =. 2024 , howpublished =

2024

[2] [2]

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods , journal =

Andrey Veprikov and Arman Bolatov and Samuel Horv. Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods , journal =. 2025 , url =

work page 2025

[3] [3]

arXiv preprint arXiv:2604.09967 , year =

Ziyue Liu and Ruijie Zhang and Zhengyang Wang and Yequan Zhao and Yupeng Su and Zi Yang and Zheng Zhang , title =. arXiv preprint arXiv:2604.09967 , year =

arXiv

[4] [4]

MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning

Tongle Wu and Huanyu Dong and Ying Sun and Ziye Ma , title =. arXiv preprint arXiv:2608.05088 , year =

work page Pith review arXiv

[5] [5]

arXiv preprint arXiv:2601.21750 , year =

Chenrui Xu and Wenjing Yan and Ying-Jun Angela Zhang , title =. arXiv preprint arXiv:2601.21750 , year =

arXiv

[6] [6]

arXiv preprint arXiv:2604.01472 , year =

Zhehang Du and Weijie Su , title =. arXiv preprint arXiv:2604.01472 , year =

arXiv

[7] [7]

arXiv preprint arXiv:2510.09378 , year =

Natalie Abreu and Nikhil Vyas and Sham Kakade and Depen Morwani , title =. arXiv preprint arXiv:2510.09378 , year =

arXiv

[8] [8]

Proceedings of the 35th International Conference on Machine Learning , year =

Noam Shazeer and Mitchell Stern , title =. Proceedings of the 35th International Conference on Machine Learning , year =

work page