{"id":"017bb20f-c866-4438-88b7-d09df3b1f299","arxiv_id":"2510.10777","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Preconditioned matrix norms unify steepest descent, quasi-Newton, and adaptive optimizers, revealing SGD, Adam, Muon, KL-Shampoo, SOAP, and SPlus as special cases and enabling new methods MuAdam and MuAdam-SANIA that are competitive in experiments.","lead":"The paper introduces preconditioned matrix norms as a unifying mathematical framework that generalizes steepest descent, quasi-Newton, and adaptive optimization methods. This abstraction shows popular optimizers like SGD, Adam, Muon, and others as special cases while proposing two new hybrid methods and conditions for invariance properties.","discovery_kind":"unification","skeptic_critique":{"model":"grok-4.3","headline":"Unification may hold only by definitional construction if preconditioners are chosen to match each optimizer's existing structure rather than derived from the norm principle","rationale":"The reader's weakest_assumption matches the identified risk precisely. If the concern is valid, the strongest_claim weakens from a genuine unification to a reparameterization, justifying a shift from UNVERDICTED to CONDITIONAL pending the concrete derivation check. No other internal inconsistency (e.g., invariance conditions or experiments) appears more load-bearing from the given material.","tokens_in":1746,"tokens_out":363,"duration_ms":35058,"concrete_test":"Take the paper's definition of the preconditioned norm (likely in §3 or §4); for Adam, substitute only the general curvature term without referencing Adam's moment estimates and re-derive the update direction; check whether the standard Adam rule (including bias correction) emerges exactly or requires additional ad-hoc choices.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that minimizing a preconditioned matrix norm ||ΔW||_P (with P encoding geometry/curvature) independently recovers the exact update rules of SGD (P = I), Adam (P involving second-moment estimates), Muon (spectral norm), KL-Shampoo, SOAP, and SPlus. If the definition of the preconditioned norm in the framework is constructed by embedding each method's specific preconditioner (e.g., diagonal scaling for Adam or spectral projection for Muon) as a parameter choice, then equivalence follows tautologically from the setup rather than from a shared geometric principle. This would make the unification non-predictive and limit its ability to generate the new methods (MuAdam, MuAdam-SANIA) on non-arbitrary grounds.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces a unified optimization framework based on preconditioned matrix norms that generalizes steepest descent (via norm choice), quasi-Newton methods, and adaptive methods (via curvature incorporation). It claims that SGD (P = I), Adam (second-moment diagonal preconditioning), Muon (spectral norm), KL-Shampoo, SOAP, and SPlus all arise as special cases. The work derives necessary and sufficient conditions for affine and scale invariance under these generalized norms and proposes two new hybrids, MuAdam and MuAdam-SANIA, which combine Muon's spectral geometry with Adam-style preconditioning; experiments indicate these are competitive with or superior to existing methods on standard benchmarks. Code is provided for reproducibility.","tokens_in":1949,"tokens_out":624,"duration_ms":22196,"significance":"A non-tautological unification that independently recovers existing update rules from a shared geometric principle, together with explicit invariance conditions and new competitive hybrids, would constitute a useful organizing framework for optimizer design. The systematic invariance analysis and experimental validation of the proposed MuAdam variants are potentially valuable contributions if the derivations hold without post-hoc embedding of each method's preconditioner.","major_comments":[{"comment":"§3 (Definition of preconditioned matrix norms and recovery of existing methods): The central unification claim requires that minimizing ||ΔW||_P independently yields the exact update rules of SGD, Adam, Muon, etc. The manuscript should explicitly demonstrate that the choice of P for each optimizer is derived from geometric or curvature considerations rather than reverse-engineered to match the known update; otherwise the equivalence risks being definitional. A concrete example showing the norm minimization step for at least Adam and Muon, with the resulting closed-form update, would clarify this.","section":"§3"},{"comment":"§4 (Invariance conditions): The necessary and sufficient conditions for affine and scale invariance are presented under generalized norms. It is unclear whether these conditions are satisfied by the specific P choices that recover the listed optimizers (e.g., Adam's second-moment estimate or Muon's spectral projection), or whether additional restrictions are imposed that limit the framework's applicability. A table or proposition verifying invariance for each recovered method would strengthen the claim.","section":"§4"}],"minor_comments":[{"comment":"The abstract and introduction should more clearly distinguish the novel contribution (preconditioned norms as a unifying principle) from the known fact that many optimizers can be viewed as preconditioned gradient steps.","section":"Abstract / §1"},{"comment":"Experimental section: baseline comparisons should include recent hybrids such as SOAP and SPlus with identical hyperparameter tuning protocols to ensure the competitiveness claim for MuAdam variants is not due to tuning differences.","section":"Experiments"}],"recommendation":"major_revision","confidential_remarks":"The citation pattern leans heavily on recent optimizer papers; ensure that foundational work on matrix norms and invariance in optimization is adequately referenced to avoid the appearance of novelty inflation."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive and insightful comments on our manuscript. These have helped us clarify the presentation of the preconditioned norms framework and its connections to existing methods. We address each major comment point by point below, with revisions made to strengthen the derivations and invariance analysis.","responses":[{"response":"We agree that explicit derivation of each P from geometric principles is essential to substantiate the unification. In the revised manuscript, we have added a dedicated subsection (3.3) that derives the preconditioner choices from first principles. For Muon, the spectral norm is obtained by taking the operator norm induced by the Euclidean vector norm on the matrix space, so that minimizing ||ΔW||_P yields the update aligned with the dominant singular vector (scaled by the step size), recovering the exact Muon rule without post-hoc fitting. For Adam, the diagonal preconditioner P is motivated as a curvature approximation via the second-moment estimate of the gradient, which corresponds to a diagonal Hessian approximation; the closed-form minimizer of ||ΔW||_P is then the element-wise scaled update, matching Adam exactly. These derivations are presented with the full minimization steps and resulting closed forms for both methods, showing that the P selections follow directly from the desired geometry or curvature model rather than being reverse-engineered.","revision_made":"yes","referee_comment":"[§3] §3 (Definition of preconditioned matrix norms and recovery of existing methods): The central unification claim requires that minimizing ||ΔW||_P independently yields the exact update rules of SGD, Adam, Muon, etc. The manuscript should explicitly demonstrate that the choice of P for each optimizer is derived from geometric or curvature considerations rather than reverse-engineered to match the known update; otherwise the equivalence risks being definitional. A concrete example showing the norm minimization step for at least Adam and Muon, with the resulting closed-form update, would clarify this."},{"response":"We thank the referee for highlighting this verification gap. In the revision, we have inserted a new table (Table 1) in §4 that enumerates each recovered optimizer (SGD, Adam, Muon, KL-Shampoo, SOAP, SPlus, and the proposed MuAdam variants), specifies the corresponding P, and indicates satisfaction of the affine and scale invariance conditions from Propositions 4.1 and 4.2. We also add a short corollary proving that the listed P choices satisfy the necessary and sufficient conditions under the problem assumptions already stated in the paper (e.g., Adam satisfies scale invariance but not full affine invariance, while Muon's spectral norm satisfies both when the matrix dimensions permit). No additional restrictions beyond those in the original framework are required, confirming broad applicability.","revision_made":"yes","referee_comment":"[§4] §4 (Invariance conditions): The necessary and sufficient conditions for affine and scale invariance are presented under generalized norms. It is unclear whether these conditions are satisfied by the specific P choices that recover the listed optimizers (e.g., Adam's second-moment estimate or Muon's spectral projection), or whether additional restrictions are imposed that limit the framework's applicability. A table or proposition verifying invariance for each recovered method would strengthen the claim."}],"tokens_in":1522,"tokens_out":682,"duration_ms":59180,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this work treats steepest descent, quasi-Newton, and adaptive methods as instances of minimizing a preconditioned matrix norm ||ΔW||_P. Picking the right P recovers SGD (identity), Adam (second-moment diagonal), Muon (spectral), KL-Shampoo, SOAP, and SPlus as special cases. They also derive necessary and sufficient conditions for affine and scale invariance under these generalized norms, which appears to be the first systematic treatment for matrix parameters. From there they define MuAdam and MuAdam-SANIA, which blend Muon-style spectral geometry with Adam-style preconditioning, and report that the new variants are competitive with or beat existing methods on the tested tasks. Code is released, which is helpful for checking the claims directly. The invariance analysis is the clearest addition; it gives concrete conditions that could actually constrain future optimizer design rather than just catalog existing ones. The unification itself organizes the landscape neatly and makes the geometric trade-offs more explicit. The soft spot is that the framework can feel definitional. If the preconditioner is chosen precisely to reproduce each method's update, then the equivalence follows from the setup instead of revealing a deeper shared principle. The stress-test concern lands here: without seeing the derivations, it is not obvious whether the new methods arise from the norm principle on independent grounds or from post-hoc fitting. Experiments would also benefit from tighter controls on hyperparameter budgets and more varied architectures to show the hybrids are not just lucky on the reported runs. This is for researchers who design or analyze training algorithms, especially those interested in invariance or geometric views of curvature. A reader who wants a compact way to think about why certain preconditioners work across methods will get value. The paper shows clear thinking and honest engagement with the literature, so it deserves a serious referee even if the unification needs sharpening.","headline":"The paper frames many optimizers as special cases of minimizing a preconditioned matrix norm, which yields clean invariance conditions and two new hybrids, though the unification risks being partly by construction.","tokens_in":2454,"tokens_out":448,"would_cite":false,"duration_ms":37790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"We propose a unified framework generalizing steepest descent, quasi-Newton methods, and adaptive methods through the novel notion of preconditioned matrix norms... SGD and Adam... Muon and KL-Shampoo... SOAP and SPlus... all emerge as special cases"},{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/ArithmeticFromLogic.lean","rs_theorem":"embed_injective","paper_passage":"Theorem 1: lmoL,R,∥·∥(G) = L⁻¹ lmo∥·∥(L^{-T} G R^{-T}) R^{-1}"}],"headline":"Optimization unification via preconditioned norms lies outside RS forcing chain","alignment":"orthogonal","rationale":"The paper's central machinery (Definitions 1-2 for (L,R)- and D-preconditioned norms, Theorem 1 on LMOs, Table 1 mapping SGD/Adam/Muon/SOAP to special cases, and invariance conditions in Theorem 2) operates entirely within the domain of matrix-norm geometry for gradient updates. RS derives a unique reciprocal cost J(x) = ½(x + x⁻¹) − 1, φ-ladder constants, 8-tick periodicity and D=3 from a single distinction (reality_from_one_distinction, AbsoluteFloorClosure, Cost.FunctionalEquation). No shared structure (J-cost, ratio symmetry, cosh identities, or parameter-free constant derivation) appears; the work is a conventional unification in cs.LG.","tokens_in":55773,"confidence":"high","tokens_out":389,"duration_ms":14074,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Preconditioned matrix norms provide a single framework in which steepest descent, quasi-Newton, and adaptive optimizers all appear as special cases.","keywords":["optimization","preconditioned norms","unified framework","adaptive methods","quasi-Newton","steepest descent","invariance"],"falsifier":"An explicit derivation showing that Adam or Muon cannot be recovered from any choice of preconditioned matrix norm without extra structure that lies outside the framework.","tokens_in":2678,"feed_emoji":"🔄","tokens_out":637,"duration_ms":45345,"temperature":0.7,"pith_summary":"The paper shows that a new abstraction called preconditioned matrix norms lets optimizers adapt to problem geometry through norm choices while also incorporating curvature information. This single principle recovers SGD, Adam, Muon, KL-Shampoo, SOAP, and SPlus as instances rather than separate inventions. The framework supplies necessary and sufficient conditions for affine and scale invariance when parameters are matrices. Two new hybrids, MuAdam and MuAdam-SANIA, are derived by mixing spectral geometry with Adam-style preconditioning and are shown to compete with current methods on standard tasks.","feed_headline":"Preconditioned norms unify SGD, Adam, Muon as special cases","feed_subtitle":"One abstraction recovers steepest descent, quasi-Newton and adaptive optimizers, plus new hybrids that mix spectral geometry with Adam-style","key_machinery":"Preconditioned matrix norms, which augment standard matrix norms with a preconditioning operator to encode both geometric adaptation and curvature utilization in a single object.","core_discovery":"Preconditioned matrix norms generalize steepest descent by allowing arbitrary norm choices that adapt to different geometries, extend quasi-Newton and adaptive methods beyond the Frobenius inner product, and establish that SGD, Adam, Muon, KL-Shampoo, SOAP, and SPlus emerge directly as special cases of the same construction. Necessary and sufficient conditions for affine and scale invariance are derived under these generalized norms.","pith_inferences":["The unification may allow automatic selection or interpolation between norms based on observed curvature or architecture type.","Similar preconditioned-norm constructions could be carried over to Riemannian or manifold-constrained optimization settings.","Convergence rates for the new MuAdam variants could be derived by specializing existing analyses of steepest descent under matrix norms."],"forward_implications":["Existing optimizers can be re-derived and compared inside one formalism instead of being developed in isolation.","Hybrid methods such as MuAdam arise systematically by selecting different combinations of norm and preconditioner.","Invariance properties for matrix-valued parameters can be checked or enforced by verifying the stated necessary and sufficient conditions.","New optimizers can be constructed by exploring other preconditioned norms that have not yet been instantiated."],"fun_headline_variants":["Preconditioned norms cast SGD, Adam, Muon as special cases","Preconditioned norms generalize steepest descent and quasi-Newton","Matrix preconditioning unifies adaptive and curvature methods","Preconditioned norms derive SGD and Adam from same principle"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The chosen abstraction of preconditioned matrix norms is assumed to capture the essential geometry and curvature of the listed optimizers without the unification holding only by how the norms are defined.","fun_headline_variants_meta":{"raw":{"variants":["Preconditioned norms cast SGD, Adam, Muon as special cases","Preconditioned norms generalize steepest descent and quasi-Newton","Matrix preconditioning unifies adaptive and curvature methods","Preconditioned norms derive SGD and Adam from same principle"]},"model":"grok-4.3","cost_usd":0.0067,"raw_usage":{"total_tokens":3059,"prompt_tokens":704,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":67003000,"prompt_tokens_details":{"text_tokens":704,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2292,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":704,"tokens_out":63,"duration_ms":22884,"temperature":1.0,"reasoning_tokens":2292,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-21T21:01:43.379336+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An explicit derivation showing that Adam or Muon cannot be recovered from any choice of preconditioned matrix norm without extra structure that lies outside the framework.","supporting_citations":[],"review_version":1}