{"id":"608504e2-b96c-4631-928a-efdd07bca3b7","arxiv_id":"2608.06218","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"The Stiefel Muon linear minimization oracle has an exact closed-form solution, obtainable from the polar factor of a skew-symmetric matrix.","lead":"This paper finds a closed-form solution for the Muon optimizer update on the Stiefel manifold, the set of matrices with orthonormal columns used across machine learning and scientific computing. The result replaces an iterative inner loop with a direct algebraic formula, which can speed up orthogonality-constrained optimization.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The exact closed-form and equivalence results are sound, but Theorem 3's convergence guarantee assumes exact SVD and M_t = ∇f(X_t), while Algorithms 1–2 use Newton–Schulz and are intended for momentum; the practical algorithm lacks a proven convergence rate.","rationale":"The paper's main mathematical contributions—the direct-sum decomposition (Theorem 1), the equivalence of optima (Theorem 2), and the closed-form of the Skewon problem (Proposition 1)—survive scrutiny. The proof of Theorem 2 correctly constructs a skew-symmetric lifting with the right spectral norm, and Proposition 4 plus a short computation (⟨N, −skew(UVᵀ)⟩ = −‖N‖_∗) would establish that Algorithm 1's direction is exactly optimal in the rank-deficient case, though the paper does not spell this out. The genuinely limiting step is the convergence analysis: it assumes settings that the algorithms in Section 4 do not use, and the momentum extension is not addressed. This matches the reader's weakest_assumption; hence we agree and preserve the CONDITIONAL verdict. The missing lemma about the skew-symmetrized polar factor is a minor expository issue that can be fixed without changing the theory.","tokens_in":12166,"tokens_out":28996,"duration_ms":322047,"concrete_test":"Implement Algorithm 2 with exact SVD (as in the experiments) but with momentum M_t = β M_{t-1} + (1−β)∇f(X_t) on a small nonconvex Stiefel problem such as f(X) = −tr(Xᵀ A X) with random symmetric A, and record min_{t≤T} ‖grad f(X_t)‖_F for β = 0, 0.9, 0.99; if the momentum variants do not satisfy the O(1/√T) bound of Theorem 3, the convergence guarantee does not apply to the algorithm as intended.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mathematical result—the equivalence of the SMP (2) and the Skewon problem (6), and the closed-form Proposition 1—is internally consistent. The load-bearing weakness is in Section 5. Theorem 3 is proven only when the polar factor is computed by an exact thin SVD and the direction is the Euclidean gradient M_t = ∇f(X_t). The descent proof in Appendix A.2 uses ⟨∇f(X_t), B_t⟩ = −‖skew(∇f(X_t)X_tᵀ)‖_∗, an identity that fails when M_t is a momentum accumulator because ⟨M_t, B_t⟩ then does not equal the directional derivative of f. Similarly, replacing the exact SVD with a finite Newton–Schulz iteration introduces an optimality gap δ_t in the Skewon objective; the resulting inequality f(X_{t+1}) ≤ f(X_t) − η_t(‖N_t‖_∗ − δ_t) + η_t² pL/2 has no control on δ_t. Algorithms 1 and 2 as stated in Section 4 explicitly use Newton–Schulz and permit momentum directions, so Theorem 3 does not cover the practical method. The abstract's claim of 'first-order convergence guarantees for Skewon' is therefore broader than the formal result. This is a gap in the guarantee, not a flaw in the exactness argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the Muon linear minimization oracle on the Stiefel manifold. It introduces an auxiliary optimization problem over skew-symmetric matrices, called the Skewon problem, proves that it is equivalent to the Stiefel Muon problem (SMP), and derives a closed-form solution from the SVD of skew(MX^T). Based on this solution, it proposes two practical algorithms: a general one using Newton-Schulz iterations and a low-rank QR-based variant for p << n. The paper also states first-order convergence guarantees for smooth non-convex objectives and reports numerical comparisons with recent heuristic solvers. The central mathematical claims are the equivalence theorem and the closed-form update; the convergence section is where the claims and the implemented algorithms diverge.","tokens_in":12436,"tokens_out":10999,"duration_ms":135999,"significance":"If the results are taken as stated, the paper resolves a previously open subproblem in Muon-type optimization: the Stiefel tangent-space spectral LMO has an exact closed-form solution, and the solution can be evaluated with the same asymptotic complexity as Muon itself. The equivalence proof via the Davis-Kahan-Weinberger theorem is elegant and self-contained, with no fitted constants or normalization conventions forcing the conclusion. The numerical experiments are a useful sanity check against existing iterative heuristics. The main caveat is that the advertised first-order convergence guarantees apply only to an exact-SVD, Euclidean-gradient variant of the algorithm, not to the Newton-Schulz-based implementation with momentum that the paper actually presents as Skewon.","major_comments":[{"comment":"The convergence guarantee is proved only for the variant in which M_t = ∇f(X_t) and the polar factor is computed by an exact thin SVD. The descent proof in Appendix A.2 uses the identity ⟨∇f(X_t), B_t⟩ = -‖skew(∇f(X_t)X_t^T)‖_*, which holds for the exact solution of the Skewon problem with direction equal to the Euclidean gradient. Algorithms 1 and 2 as stated in Section 4 compute Y_t = -skew(NS(N_t)) with a Newton-Schulz approximation and allow M_t to be a momentum direction. For an approximate polar factor, the descent inequality becomes f(X_{t+1}) ≤ f(X_t) - η_t(‖N_t‖_* - δ_t) + η_t^2 p L/2 with no control on the approximation gap δ_t, and for a momentum accumulator ⟨M_t, B_t⟩ is no longer the directional derivative of f. Thus the abstract's claim of 'first-order convergence guarantees for Skewon' is broader than the formal result. I recommend either proving a convergence theorem for the exact-SVD gradient variant and explicitly labeling the Newton-Schulz/momentum version as heuristic, or extending the analysis to control the approximation error and the momentum direction.","section":"Section 5, Theorem 3; Appendix A.2; Algorithms 1 and 2"}],"minor_comments":[{"comment":"The sentence 'skew-symmetry does not increase the spectral norm' is used repeatedly; making explicit that this refers to the inequality ‖(Y - Y^T)/2‖₂ ≤ ‖Y‖₂ would improve readability.","section":"Section 2, after Eq. (6)"},{"comment":"The assertion that the map C ↦ X_⊥^T C X_⊥ is surjective onto A_{n-p} is stated without proof; the construction C = X_⊥ D X_⊥^T for D ∈ A_{n-p} makes this immediate and should be included.","section":"Theorem 2, proof"},{"comment":"The phrase 'for any realization of Algorithm 1 and Algorithm 2, in which the polar factor is evaluated using a thin SVD' is ambiguous, because Algorithms 1 and 2 as written in Section 4 use Newton-Schulz iterations. I suggest naming the analyzed variant explicitly, e.g. 'Skewon-SVD', and distinguishing it from the default Newton-Schulz implementation.","section":"Section 5, first paragraph"},{"comment":"The multiset notation σ(S) = {1, ..., 1, μ_1, ..., μ_{n-2k}} is clear, but it would be helpful to state explicitly that the μ_j are allowed to be zero and that the 2k singular values equal to 1 come in pairs from the J blocks.","section":"Appendix A.3, Proposition 4"},{"comment":"The figure captions list ten methods; checking that the legend order matches the plotting order in each figure would remove potential ambiguity.","section":"Section 6, Figures 1-3"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the core mathematical contribution—the equivalence of the SMP and the Skewon problem and the closed-form solution—appears sound and is likely publishable after revision. The main issue is the mismatch between the convergence theorem and the implemented algorithms; this is fixable by rescoping the claims or adding an approximation-error analysis. The heavy reliance on blog posts and arXiv preprints in the related-work section is consistent with the fast-moving Muon literature and should not, by itself, block publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe one thing to know: the exact closed-form result is real. The paper shows the Stiefel Muon LMO can be reformulated as a spectral-norm-constrained linear problem over skew-symmetric matrices, and that the two problems are exactly equivalent, not just a relaxation. Proposition 1 gives the closed-form solution via SVD of skew(MX^T), and Theorem 2 closes the loop with a construction that uses the Davis–Kahan–Weinberger completion result. I checked the block norm computations; they are correct. This genuinely resolves the open problem Bernstein flagged and is a real step beyond the iterative heuristics from Cesista and Su.\n\nWhat it does well: the direct-sum decomposition of so(n) into the tangent-space part and the stabilizer (Theorem 1) is a clean way to see why the kernel component doesn't affect the objective but does fix the spectral norm. The proof of Proposition 2 (boundary attainment) and the careful handling of the rank-deficient SVD (Proposition 4) are solid. The complexity claims for the p≪n variant are sensible.\n\nThe soft spot is the convergence section, and the stress-test note is right. Theorem 3 assumes M_t = ∇f(X_t) and an exact thin SVD. But Algorithm 1 as written uses Newton–Schulz and says M_t can be a momentum direction. The descent inequality ⟨∇f, B_t⟩ = −‖skew(∇f X_t^T)‖_* stops holding for momentum, and the Newton–Schulz gap introduces an uncontrolled δ_t. So the stated O(1/√T) guarantee does not cover the algorithm implemented. The abstract's \"first-order convergence guarantees for Skewon\" stretches further than the formal result. This is a gap, not a death blow—the exactness result stands alone. But the authors should either extend the analysis to Newton–Schulz and momentum or scope the theorem explicitly.\n\nMinor: no code is provided for the numerical experiments, so the runtime comparisons are hard to reproduce. That's a legitimate-but-fixable issue.\n\nCitations look fine; the self-citations are relevant.\n\nWho's it for: anyone working on matrix-aware optimizers, orthogonality-constrained deep learning, or low-rank adapters. It deserves serious refereeing; the core theorem is novel and sound, and the convergence gap is exactly what a referee should push on.\n\nRecommendation: send to peer review with a request for revision on the convergence claims and a code release.\n\nBest,\n\n[Your name]","headline":"Solid exact closed-form result for the Stiefel Muon LMO, with a clean equivalence proof; the convergence theorem doesn't cover the implemented NS/momentum algorithm, but that gap is fixable.","tokens_in":12984,"tokens_out":6722,"would_cite":true,"duration_ms":71649,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"The Stiefel Muon linear minimization oracle has an exact closed-form solution, given by the SVD of a skew-symmetric matrix.","keywords":["constrained optimization","Muon optimizer","Stiefel manifold","Riemannian optimization","skew-symmetric matrices","linear minimization oracle","closed-form solution","first-order convergence"],"falsifier":"Choose $n > p$, a random $X \\in \\mathrm{St}(n,p)$ and $M \\in \\mathbb{R}^{n \\times p}$; compute $N = \\mathrm{skew}(M X^\\top)$, its SVD, and the closed-form $B = Y_* X$ from Proposition 1. Verify that $B \\in T_X \\mathrm{St}(n,p)$, $\\|B\\|_2 = 1$ unless $P_X M = 0$, and $\\langle M, B\\rangle = -\\|N\\|_*$. If a high-accuracy convex solver finds a feasible tangent vector with strictly smaller objective value, the claimed equivalence is false; likewise, the characterization in the paper's Remark 4 would be falsified by any $B$ with $\\|B\\|_2 \\le 1$ that cannot be written as $YX$ for skew $Y$ with $\\|Y\\|_2 \\le 1$.","tokens_in":11966,"feed_emoji":"🧮","tokens_out":11695,"duration_ms":113660,"temperature":0.7,"pith_summary":"The paper shows that the linear minimization oracle at the heart of the Muon optimizer, when constrained to the Stiefel manifold of matrices with orthonormal columns, is not an iterative or heuristic subproblem: it has an exact closed-form solution. The solution is obtained by reformulating the tangent-space problem as an equivalent problem over skew-symmetric matrices, the Skewon problem, whose optimum is read off from the singular value decomposition of $\\mathrm{skew}(M X^\\top)$. The authors prove the two problems have identical optimal values and that every optimal tangent vector lifts to an optimal skew-symmetric matrix with the same spectral norm. This makes the Stiefel Muon update as cheap as the unconstrained one and yields a practical algorithm, Skewon, with first-order convergence guarantees for smooth non-convex objectives.","feed_headline":"Muon's Stiefel update has an exact closed form","feed_subtitle":"A skew-symmetric SVD reformulation makes the spectral-constrained optimizer exact and cheap.","key_machinery":"The load-bearing construction is the direct sum decomposition of the space of skew-symmetric matrices $A_n = L_X \\oplus K_X$, where $L_X$ is isomorphic to the Stiefel tangent space (infinitesimal rotations that move the frame) and $K_X$ consists of rotations in the complementary subspace that leave $X$ fixed. This decomposition lets the authors lift a tangent-space candidate $B$ to a skew-symmetric $Y$ with $YX = B$ and control the spectral norm via a norm-preserving dilation lemma. The SVD of $N = \\mathrm{skew}(M X^\\top)$ then supplies the closed-form optimum of the lifted problem, and the equivalence theorem shows no gap is introduced.","core_discovery":"The central claim is that the Stiefel Muon problem $\\min \\langle M, B\\rangle$ subject to $\\|B\\|_2 \\le 1$ and $B \\in T_X \\mathrm{St}(n,p)$ is equivalent to the Skewon problem $\\min \\langle \\mathrm{skew}(M X^\\top), Y \\rangle$ subject to $\\|Y\\|_2 \\le 1$ and $Y \\in A_n$. Proposition 1 gives the closed form $Y_* = -U_1 V_1^\\top - U_2 W V_2^\\top$ from an SVD of $N = \\mathrm{skew}(M X^\\top)$, with optimal value $-\\|N\\|_*$. Theorem 2 proves that for any optimal $B_*$ of the SMP there exists an optimal $Y_*$ of the Skewon problem with $Y_* X = B_*$, $\\|Y_*\\|_2 = \\|B_*\\|_2 = 1$, so the Stiefel Muon update is exactly $B = Y_* X$.","pith_inferences":["The same lift-and-solve strategy likely extends to other matrix manifolds whose tangent spaces are parameterized by a Lie algebra, such as Grassmannian or symplectic Stiefel manifolds, giving closed-form spectral linear minimization oracles there as well.","Because the convergence theorem assumes an exact thin SVD and the Euclidean gradient, the practical iterative version should be paired with a convergence check or an adaptive step-size rule; a direct empirical test would be to track the Riemannian gradient norm under the algorithm as written.","The singular-value structure proved in the appendix for the skew-symmetrized polar factor suggests that rank-deficient cases can be handled robustly by thresholding singular values below machine precision, which may be important for numerical stability in low-rank settings.","The equivalence's characterization of the tangent-space ball could inform analyses of Muon-style optimizers beyond the Stiefel case, such as bounds on the norm of the update direction in terms of $\\mathrm{skew}(M X^\\top)$."],"forward_implications":["The Stiefel Muon update becomes a direct SVD-based computation instead of an iterative fixed-point or search procedure, eliminating per-step approximation overhead.","Skewon has the same asymptotic complexity as unconstrained Muon ($O(n^3)$ in general, $O(np^2)$ when $p \\ll n$), so orthogonality-constrained layers can be trained with Muon-style updates at comparable cost.","The equivalence gives a clean characterization of the spectral-norm ball of the Stiefel tangent space as the image of the spectral-norm ball of skew-symmetric matrices under $Y \\mapsto YX$.","A first-order convergence rate of $O(\\sqrt{pL/T})$ holds for smooth non-convex objectives under the standard Riemannian Lipschitz-gradient assumption, when the polar factor is computed exactly.","Prior heuristic or relaxed SMP solvers can be replaced by Skewon in Riemannian optimizers, with accuracy limited only by the chosen SVD or iterative polar-factor implementation."],"supporting_citations":[{"why":"Defines the Muon optimizer and its linear minimization oracle, the object this paper generalizes to the Stiefel manifold.","marker":"[1]"},{"why":"Posed the Stiefel Muon problem as open and proposed a Lagrangian-based solver that the closed form supersedes.","marker":"[3]"},{"why":"Supplies heuristic alternating-projection and ternary-search baselines compared in the numerical experiments.","marker":"[4]"},{"why":"Provides a fixed-point solver for the SMP used as a baseline in the comparisons.","marker":"[5]"},{"why":"Establishes the Riemannian optimization framework and Assumption 1 that underpin the convergence guarantee.","marker":"[8]"},{"why":"Supplies the norm-preserving dilation lemma used to control the spectral norm in the equivalence proof.","marker":"[18]"},{"why":"Provides the conic solver used to compute high-accuracy reference solutions in the numerical validation.","marker":"[21]"}],"fun_headline_variants":["Muon on Stiefel: exact update found","Skewon: Muon's exact Stiefel update","Stiefel Muon update solved exactly","Closed-form Muon step for Stiefel manifold"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The stated first-order convergence guarantee holds only when the update direction is the exact Euclidean gradient and the polar factor is computed by an exact thin SVD; the practical Skewon algorithms use an iterative polar-factor computation and momentum updates, so the $O(1/\\sqrt{T})$ rate does not directly cover the algorithm as implemented.","fun_headline_variants_meta":{"raw":{"variants":["Muon on Stiefel: exact update found","Skewon: Muon's exact Stiefel update","Stiefel Muon update solved exactly","Closed-form Muon step for Stiefel manifold"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000634,"raw_usage":{"total_tokens":2893,"prompt_tokens":878,"completion_tokens":2015,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":1952}},"tokens_in":494,"tokens_out":2015,"duration_ms":19073,"temperature":1.0,"reasoning_tokens":1952,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:17:14.577576+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Choose $n > p$, a random $X \\in \\mathrm{St}(n,p)$ and $M \\in \\mathbb{R}^{n \\times p}$; compute $N = \\mathrm{skew}(M X^\\top)$, its SVD, and the closed-form $B = Y_* X$ from Proposition 1. Verify that $B \\in T_X \\mathrm{St}(n,p)$, $\\|B\\|_2 = 1$ unless $P_X M = 0$, and $\\langle M, B\\rangle = -\\|N\\|_*$. If a high-accuracy convex solver finds a feasible tangent vector with strictly smaller objective value, the claimed equivalence is false; likewise, the characterization in the paper's Remark 4 would be falsified by any $B$ with $\\|B\\|_2 \\le 1$ that cannot be written as $YX$ for skew $Y$ with $\\|Y\\|_2 \\le 1$.","supporting_citations":[{"cited_title":"https:// kellerjordan.github.io/posts/muon/","cited_arxiv_id":null,"evidence_quote":"Defines the Muon optimizer and its linear minimization oracle, the object this paper generalizes to the Stiefel manifold."},{"cited_title":"https://thinkingmachines.ai/blog/modular- manifolds (2025)","cited_arxiv_id":null,"evidence_quote":"Posed the Stiefel Muon problem as open and proposed a Lagrangian-based solver that the closed form supersedes."},{"cited_title":"https://leloykun.github.io/ponder/steepest-descent-stiefel/","cited_arxiv_id":null,"evidence_quote":"Supplies heuristic alternating-projection and ternary-search baselines compared in the numerical experiments."},{"cited_title":"Muon + Stiefel (2025)","cited_arxiv_id":null,"evidence_quote":"Provides a fixed-point solver for the SMP used as a baseline in the comparisons."},{"cited_title":"SIAM Journal on Numerical Analysis19(3), 445–469 (1982)","cited_arxiv_id":null,"evidence_quote":"Supplies the norm-preserving dilation lemma used to control the spectral norm in the equivalence proof."},{"cited_title":"Journal of Optimization Theory and Applications169(3), 1042–1068 (2016)","cited_arxiv_id":null,"evidence_quote":"Provides the conic solver used to compute high-accuracy reference solutions in the numerical validation."}],"review_version":1}