{"id":"99ba9ff3-83d1-4dec-b1bc-bb2533f908b1","arxiv_id":"2608.01911","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"A smoothed matrix-polar spectral gradient flow for Muon-type optimization is globally convergent with O(1/T), O(1/t), and exponential rates, and its local advantage over the Frobenius direction is characterized by the product of gradient effective rank and covariance similarity.","lead":"This paper studies a continuous-time version of Muon, a neural-network optimizer that normalizes gradient matrices by their singular values, but with a smoothing term that avoids mathematical singularities. It proves the smoothed flow converges under standard assumptions and states exactly when this spectral update direction beats the ordinary gradient direction.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified: the closest risk is the rank-safe regime of Prop. 2, which is explicitly scoped and does not affect the core convergence theorems.","rationale":"I read the full text and checked the main proof chains. The central claims are supported. For the direct flow, h_ε∘∇f is locally Lipschitz, ∥Ẇ∥≤√d gives global existence, the dissipation identity (11) plus coercivity gives boundedness, and Barbalat gives ∇f(W(t))→0; the rate theorems are straightforward Gronwall/convexity/PL arguments. For the momentum system, the exact Lyapunov cancellation (14) is correct, ∫∥M∥² and ∫∥Ẇ∥² bounds follow, and the stationarity-rate transfer in Theorem 6 is valid with the stated constants. The local comparison is self-consistent: under Assumption 3 the calibrated rates R_F and R_ε are exactly (23)–(24), and in the square quadratic case the limit Γ0=r_eff(G)simHS(ρG,ρA) follows from the SVD and trace identities; the numerical crossings in §VI-B match that threshold. The closest thing to a load-bearing concern is the finite-horizon approximation result, Proposition 2. It genuinely depends on a uniform lower bound on the relevant singular values, and the non-uniformity of h_ε→Polar near zero is real (Remark 2). However, the paper flags this limitation explicitly in Remark 2 and the conclusion, restricts Proposition 2 to the rank-safe regime, and separates the global convergence theorems (which are about the smoothed flow, not about exact Muon) from the Muon connection (which is only claimed as a finite-horizon model). Thus the concern does not falsify any of the paper's formal claims; it only narrows the interpretation of the Muon motivation. No internal inconsistency or missing proof was found in the parts I checked.","tokens_in":21181,"tokens_out":19578,"duration_ms":246873,"concrete_test":"For f(W)=½||W||_F², W0=U diag(4.13,3.07,2.03,1.237) V^T as in §VI-A, set T=1.2 and σ=0.01 so inf_{0≤t≤T} σ_min(∇f(W_P(t)))≥0.037≥2σ. Integrate the scalar ODE s'(t)=-s/√(s²+ε) for each singular value with ε=10^{-3},10^{-4},10^{-5} and compare sup_{t∈[0,T]} |s_ε(t)-(σ_i-t)|. If the maximum error does not scale roughly linearly in ε, Proposition 2's bound (19) would be suspect; if it does, the only remaining failure is the already-acknowledged rank-crossing regime.","verdict_should_be":"UNCHANGED","load_bearing_attack":"No significant objection identified. The paper's core theorems (well-posedness of (10)/(12), dissipation, stationary-set convergence, and the O(1/T), O(1/t), O(e^{-ct}) rates) are internally consistent with the stated assumptions: h_ε is globally Lipschitz for fixed ε>0, the Lyapunov identities (11) and (14) are exact, and the rate proofs follow from the sector estimate (9). The local comparison in Theorem 8 is algebraically correct under Assumption 3, and Corollary 1 follows by taking the ε→0 limit in the square quadratic model. The only substantive limitation is Proposition 2's premise that the ideal flow's gradient or momentum singular values remain bounded below by 2σ on [0,T]. Remark 2 already demonstrates that h_ε does not converge uniformly to Polar near rank deficiency (scalar example h_ε(ε)→0 while Polar(ε)=1), and §VI-A shows the expected breakdown near crossing. Since the paper explicitly presents Proposition 2 as a finite-horizon, rank-safe approximation result and does not claim global equivalence with discrete Muon (Remarks 1–3, conclusion), this limitation does not undermine the stated theorems.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a smoothed matrix-polar feedback law h_epsilon(G)=G(G^T G+epsilon I)^{-1/2} for matrix-valued optimization, motivated by Muon-type spectral normalization. It proves basic analytic properties of this map (Lipschitz regularity, monotonicity, boundedness, sector dissipation), then studies the direct flow dot W=-h_epsilon(grad f(W)) and a momentum-augmented system dot W=-h_epsilon(M), dot M=a grad f(W)-b M. For both systems it establishes global well-posedness, boundedness, dissipation identities, and convergence to the stationary set. It derives quantitative rates: O(1/T) minimum-gradient nonconvex stationarity, O(1/t) convex objective-gap decay, exponential decay under a Polyak-Lojasiewicz condition for the direct flow, and an O(T^{-1/2}) averaged convex rate for the momentum system. Proposition 2 gives an O(epsilon) finite-horizon approximation of the ideal matrix-polar flow under a uniform lower singular-value bound, and Remark 2 explicitly scopes this result away from rank degeneracy. The final part develops a local descent-rate comparison between the smoothed spectral direction and the Frobenius direction under a calibrated quadratic model, yielding the criterion r_eff(G) sim_HS(rho_G,rho_A)>1 for the limiting matrix-polar advantage in the quadratic-layer case. Numerical experiments illustrate the approximation, the local threshold, and the convergence bounds.","tokens_in":21422,"tokens_out":12422,"duration_ms":146962,"significance":"If the result stands, this is a useful rigorous continuous-time foundation for spectral normalization methods of Muon type. The paper's main theorems are not empirically fitted: the rates and the local criterion are derived from the definitions and standard ODE/Lyapunov arguments, and the numerical section is a controlled verification of the algebra rather than a benchmark. The authors are unusually careful about the scope of their results: Proposition 2 is explicitly presented as a finite-horizon, rank-safe approximation statement, and Remark 2 demonstrates the nonuniformity near rank deficiency with a scalar example. The local advantage criterion is a concrete, falsifiable prediction that connects spectral uniformity of the gradient, covariance alignment, and Hessian curvature. The paper is modest in scope but internally consistent and technically sound.","major_comments":[],"minor_comments":[{"comment":"The Lipschitz estimate (5) is stated with constant 1/sqrt(epsilon). The proof via the symmetric dilation is correct, but it relies on an implicit Frobenius-norm Lipschitz property for scalar functions applied to Hermitian matrices. A one-sentence justification or reference would improve readability, especially because this constant is used for well-posedness.","section":"Section II, Lemma 1"},{"comment":"The calibrated rate R_W(D) is introduced as a comparison device, but the actual flows in Theorem 1 do not carry the factor alpha_D^*. It would help to state explicitly that alpha_D^* is the optimal step of the local quadratic model and that the comparison is intrinsic to the direction, not to the uncalibrated trajectories.","section":"Section V-A, Eqs. (23)-(24)"},{"comment":"The quantity r_eff(G)=||G||_*^2/||G||_F^2 is called the 'effective rank'. This is nonstandard terminology; the usual effective rank is (sum sigma_i^2)^2/(sum sigma_i^4). Since r_eff(G) is always at least 1, a footnote clarifying the definition would avoid confusion.","section":"Section V-B, Eq. (36)"},{"comment":"The discrete momentum recursion uses the coefficient (1-beta)nabla f(W_k), while the continuous model uses lambda(nabla f(W)-M). The matching beta=e^{-lambda eta} is only first-order in eta. Stating this approximation order explicitly would make the comparison between discrete and continuous trajectories more precise.","section":"Section VI-A, Eq. (45)-(46)"},{"comment":"The scalar example contains a rendering typo: h_epsilon(z_epsilon)=epsilon/sqrt(epsilon^2+epsilon) should equal sqrt(epsilon/(1+epsilon)), not 'r epsilon/(1+epsilon)'. The text as displayed is missing a square-root sign.","section":"Remark 2"}],"recommendation":"accept","confidential_remarks":"The paper is technically sound and carefully scoped. The self-citations ([19], [21]) and the Muon motivation are used only as context and do not affect the mathematical claims. The numerical experiments are illustrative but not benchmarks; this is appropriate to the paper's purpose. The only substantive limitation, the rank-safe finite-horizon approximation, is already acknowledged in Remark 2 and does not undermine the core convergence theorems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper does what it says. It introduces a smoothed matrix-polar feedback h_eps, proves well-posedness, dissipation, convergence, and rates for the direct and momentum-augmented flows, and gives a verifiable local condition (r_eff times simHS > 1) for when the spectral direction beats Frobenius in a quadratic model. The continuous-time Lyapunov analysis of a Muon-type update is genuinely new relative to the cited discrete analyses, and the local criterion is a clean, checkable result that goes beyond norm-based bounds.\n\nWhat is solid: the main proofs check out. Lemma 1's Lipschitz, monotonicity, and sector estimates are standard but correct. The dissipation identity for the direct flow and the Lyapunov identity for the momentum system are exact. The convergence and rate theorems (nonconvex O(1/T), convex O(1/t), PL exponential) follow from the stated inequalities. Proposition 2's finite-horizon approximation is a textbook Gronwall argument, and the authors explicitly flag in Remark 2 that the approximation fails near rank deficiency — the scalar example is correct and shows they understand the scope. Section VI's numerical sweeps are controlled constructions that verify the theory rather than overclaiming empirical success. The citation pattern looks honest; self-citations are motivational and not load-bearing.\n\nSoft spots, in proportion. The local comparison (Theorem 8) relies on Assumption 3, requiring positive directional curvature for both directions — fair, but it means the criterion only applies when the model is locally convex along both directions. The momentum-augmented convex rate is only O(T^-1/2), which is weaker than one might hope, but it's derived cleanly and not oversold. The connection to discrete Muon is explicitly finite-horizon and rank-safe; that limitation is stated in the paper itself. The proofs rely on several matrix-analysis estimates (e.g., spectral operator Lipschitz bounds) that are invoked rather than fully derived, but they are standard and the appendices give enough detail to fill gaps. No formal verification, but the math is conventional and I see no circularity or parameter fitting.\n\nBottom line: this is a careful, honest paper. It won't resolve the big question of when spectral normalization helps globally, but it gives a rigorous local criterion and a stable continuous-time model that practitioners and theorists can build on. I'd bring it to a reading group and cite it. Recommend sending to peer review; it deserves serious referees, though they should push for a full proof of the remaining matrix-analysis estimates and maybe a sharper momentum convex rate.\n\nBest,\n[You]","headline":"A clean, honest continuous-time analysis of a smoothed polar spectral flow; the core theorems hold, the local advantage criterion is a real contribution, and the limitations are properly scoped — worth refereeing.","tokens_in":21936,"tokens_out":656,"would_cite":true,"duration_ms":9516,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["34D23","90C30"],"pacs":[],"model":"deepseek-v4-flash","headline":"The smoothed polar map h_epsilon(Z)=Z(Z^T Z+epsilon I)^(-1/2) makes Muon-style spectral gradient flows globally well-posed and convergent, with O(1/t), exponential, and O(1/T) rates.","keywords":["smoothed matrix-polar feedback","spectral gradient flow","Muon optimizer","continuous-time optimization","matrix optimization","Lyapunov analysis","Polyak–Lojasiewicz condition","convergence rates"],"falsifier":"Run the smoothed direct flow and the ideal polar flow on f(W)=||W||_F^2/2 from the same full-rank W0, and record sup_{t<=T} ||W_epsilon(t)-W_P(t)||_F as epsilon -> 0: for T long enough that the minimum singular value of grad f(W_P) drops below sqrt(epsilon), the error should saturate at order 1 rather than O(epsilon), directly testing the rank-safe assumption behind Proposition 2.","tokens_in":21039,"feed_emoji":"📉","tokens_out":7985,"duration_ms":83998,"temperature":0.7,"pith_summary":"This paper studies matrix-valued optimization where the update direction is a spectrally normalized gradient, as in Muon-type optimizers. Because the exact polar factor Z(Z^T Z)^(-1/2) is nonsmooth at rank-deficient matrices, the authors replace it with the smoothed map h_epsilon(Z)=Z(Z^T Z+epsilon I)^(-1/2), which preserves singular vectors but remains smooth. They prove that the resulting continuous-time flows are globally well-posed, dissipative, and converge to the stationary set, with explicit rates in nonconvex, convex, and Polyak–Lojasiewicz settings. They also compare the smoothed spectral direction with the Frobenius direction locally, deriving a simple numerical criterion for when the spectral direction descends faster. The work provides a continuous-time foundation for spectral normalization, including a finite-horizon O(epsilon) approximation to the ideal matrix-polar flow.","feed_headline":"Smoothing the polar factor turns Muon updates into convergent flows","feed_subtitle":"An epsilon-smoothed polar map yields O(1/T), O(1/t), and exponential rates, plus a local advantage criterion.","key_machinery":"The central object is the smoothed matrix-polar map h_epsilon(Z)=Z(Z^T Z+epsilon I)^(-1/2), defined for epsilon > 0. In a singular value decomposition it acts as sigma -> sigma/sqrt(sigma^2+epsilon), so it preserves singular vectors, is smooth and globally Lipschitz, and is exactly the gradient of the convex spectral potential Phi_epsilon(Z)=Tr[(Z^T Z+epsilon I)^(1/2)-sqrt(epsilon)I]. This gradient structure yields the dissipation identity d/dt f(W) = -sum_i sigma_i^2/sqrt(sigma_i^2+epsilon), which drives all the rate bounds. The local comparison uses the ratio Gamma_epsilon = R_epsilon/R_F of calibrated descent rates, which for the quadratic layerwise loss simplifies to r_eff(G) sim_HS(rho_","core_discovery":"The central claim is that the smoothed spectral feedback law h_epsilon(G)=G(G^T G+epsilon I)^(-1/2) is a regular, monotone, bounded, dissipative model of matrix-polar, Muon-type updates. For the direct flow dot W = -h_epsilon(grad f(W)), the paper establishes global existence and uniqueness, boundedness, monotone decrease of f, vanishing of the gradient, and convergence to the stationary set. In convex and PL landscapes it obtains O(1/t) and exponential objective-gap bounds, and a min-gradient O(1/T) stationarity certificate in the nonconvex case. For the momentum-augmented system dot W = -h_epsilon(M), dot M = a grad f(W) - b M, it proves a Lyapunov identity and convergence to the stationar","pith_inferences":["If the local criterion is predictive in practice, an optimizer could switch between Frobenius and spectral directions pointwise based on measured r_eff and sim_HS, avoiding a global hyperparameter search.","The failure of uniform convergence near zero singular values suggests Muon's benefits may be confined to regimes where gradient singular values stay away from rank transitions; modeling rank-crossing dynamics is a natural next step.","The smoothness and Lipschitz constants of h_epsilon could enable convergence proofs for discrete Muon-type methods by connecting them to Euler discretizations of the smoothed flow.","The effective-rank factor in the criterion yields a testable prediction for deep networks: spectral normalization should help most in layers whose gradient spectrum is relatively flat and whose activation covariance aligns with the gradient covariance."],"forward_implications":["Every solution of the smoothed direct flow converges to the stationary set {W : grad f(W)=0}, with min-norm squared gradient O(1/T) in the nonconvex case.","Under convexity the objective gap decays as O(1/t); under a Polyak–Lojasiewicz inequality it decays exponentially, with constants governed by the sector coefficient c_epsilon = 1/sqrt(K_G^2 + epsilon).","On any finite time interval where gradient or momentum singular values stay bounded below by 2 sigma, the smoothed flow shadows the ideal matrix-polar flow uniformly with O(epsilon) error.","For the quadratic layerwise loss, the matrix-polar direction has a larger local descent rate than the Frobenius direction exactly when r_eff(G) sim_HS(rho_G, rho_A) > 1.","The momentum-augmented system is dissipative with Lyapunov function V = a(f - f_inf) + Phi_epsilon(M), and every limit point lies in S times {0}."],"fun_headline_variants":["Smoothed polar flows: Muon updates converge globally","Epsilon-smoothing fixes Muon polar flows, with rates","Matrix-polar flows made smooth: convergence guarantees","Muon-type flows: spectral smoothing ensures convergence","Smoothed spectral gradients: convergent Muon-style optimization"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the relevant gradient or momentum singular values stay uniformly bounded below by a positive constant on the time horizon; near zero singular values the smoothed map and the polar map behave differently, so the O(epsilon) approximation and the Muon connection break down.","fun_headline_variants_meta":{"raw":{"variants":["Smoothed polar flows: Muon updates converge globally","Epsilon-smoothing fixes Muon polar flows, with rates","Matrix-polar flows made smooth: convergence guarantees","Muon-type flows: spectral smoothing ensures convergence","Smoothed spectral gradients: convergent Muon-style optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000596,"raw_usage":{"total_tokens":2637,"prompt_tokens":768,"completion_tokens":1869,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":1791}},"tokens_in":512,"tokens_out":1869,"duration_ms":16884,"temperature":1.0,"reasoning_tokens":1791,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:25:22.503997+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the smoothed direct flow and the ideal polar flow on f(W)=||W||_F^2/2 from the same full-rank W0, and record sup_{t<=T} ||W_epsilon(t)-W_P(t)||_F as epsilon -> 0: for T long enough that the minimum singular value of grad f(W_P) drops below sqrt(epsilon), the error should saturate at order 1 rather than O(epsilon), directly testing the rank-safe assumption behind Proposition 2.","supporting_citations":[],"review_version":1}