{"id":"3e69d6ef-df25-4bc0-ae8d-b1d7b53fda11","arxiv_id":"2607.17822","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"MRSNorm pairs channels into 2D phasors, normalizes by the mean of per-pair L2 norms, and ties each pair's affine weight, reporting stable ResNet training on CIFAR-100 under high learning rates.","lead":"The paper introduces Mean Root Square Normalization (MRSNorm), which groups channels into 2D pairs, scales by the L1 average of the per-pair L2 magnitudes, and keeps one affine weight per pair. It claims this prevents gradient explosion and halves parameters, but the evidence is a small ResNet/CIFAR-100 stress test rather than large-scale validation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gradient-homogeneity proof in Section 5.1 bounds only the interference term, not the full update norm; Eq. 6 does not establish the claimed unconditional uniformity.","rationale":"The load-bearing premise of the paper is that gradient homogeneity is unconditional. That premise requires the full update norm to be independent of the phasor's phase and scale. Eq. 6 establishes this only for the collective interference term; the first term in Eq. 5 is exactly the kind of input-dependent, phase-sensitive signal that the theorem must control but does not. This is not an external disagreement with a heuristic; it is an internal gap in the proof that the paper's own equations expose. The Appendix A/B RMS-vs-L1 confusion strengthens the concern: the only quantitative stability guarantee offered for attention uses a different normalization than the one defined and implemented, so even the auxiliary claims are not internally consistent. Experimental evidence is a single CIFAR-100 ResNet stress test with no attention experiments and no final accuracy table, so it does not compensate for the theoretical gap. The code in Appendix C is concrete, and the 1D-periodicity lemma in §5.2 is valid as far as it goes, but neither rescues the core claim. I found no evidence of fraud or intent; the issue is that the central mathematical claim does not follow from the presented derivation. A revised paper would need to either prove a bound on the full gradient norm (likely requiring assumptions on ∇o_i) or substantially weaken the 'unconditional' claim.","tokens_in":9589,"tokens_out":6987,"duration_ms":56103,"concrete_test":"Re-derive the total gradient norm from Eq. 5 including the γ_i∇o_i/S_MRS term for a single phasor with arbitrary upstream gradient ∇o_i=(a,0) and u_i=(cosθ, sinθ). Show ||∂L/∂z_i||² = (1/S_MRS²)(γ_i² a² - 2γ_i a Ω cosθ + Ω²), which depends on a and θ. Then run a two-phasor forward/backward in the provided Appendix C code with asymmetric upstream gradients; if the per-phasor update norms are unequal, Eq. 6's 'uniform and invariant' claim is false.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that MRSNorm 'unconditionally equalizes the local gradient norm' (abstract, §5.1). The proof in §5.1 considers the gradient ∂L/∂z_i = (1/S_MRS)[γ_i∇o_i - u_i Ω] (Eq. 5) and then computes the norm of only the second ('interference') term: ||(Ω/S_MRS)u_i|| = |Ω|/S_MRS (Eq. 6). But the full gradient norm is ||∂L/∂z_i||² = (1/S_MRS²)(γ_i²||∇o_i||² - 2γ_iΩ(∇o_i·u_i) + Ω²). This depends on the upstream gradient ∇o_i both through its magnitude and through its alignment with u_i; it is not uniform in θ_i or in the activation scale. Nothing in the paper bounds ∇o_i or shows it is phase-invariant. Therefore the advertised built-in gradient clipper and Gradient Homogeneity are unproven. A second, independent inconsistency compounds this: Appendix A (Eq. 11) defines the normalizer as the RMS energy S_q = sqrt((2/d)Σr²), whereas Eq. 1 and the Appendix C implementation use the L1 mean of L2 magnitudes S_MRS = (1/P)Σ||z_i||+ε. The attention-logit bound in Appendix B (Eqs. 17-18) relies on Σr² = (d/2)S², which holds only for the RMS definition; the L1-mean normalizer does not satisfy that identity, so the claimed |Score|≤d/2 bound does not apply to the proposed method as defined. Since both the gradient-homogeneity theorem and the attention-stability theorem rest on these unchecked terms/definitions, the paper's central theoretical claims are not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Mean Root Square Normalization (MRSNorm), which pairs the channels of an activation vector into P 2D phasors, computes the L2 norm of each phasor, and divides all phasors by the L1 mean of those norms plus epsilon, with a single scalar gain shared within each pair. The authors claim that this design preserves phase angles, halves the number of affine parameters relative to RMSNorm, and 'unconditionally equalizes the local gradient norm' so that MRSNorm acts as a built-in gradient clipper; they also claim that applying the normalization to queries and keys in attention yields an 'energy-weighted cosine similarity' operator with a strict logit bound of d/2 via a 'semantic phase veto.' Empirical support consists of a ResNet trained on CIFAR-100 under large learning rates and small batches, compared with LayerNorm and RMSNorm, where MRSNorm shows more stable training curves. The paper also derives a claimed impossibility result for 1D periodic gradient stability, discusses a 'radial gradient blockade' of instance-wise normalization, and provides a PyTorch implementation in Appendix C.","tokens_in":9945,"tokens_out":12774,"duration_ms":107416,"significance":"MRSNorm is genuinely simple and parameter-efficient, and the paper does ship a self-contained implementation (Appendix C) together with an explicit analytical framework, which is commendable. The polar-coordinate decomposition in Appendix A (Eq. 14) is mathematically correct as a rewriting of the dot product. If the gradient-homogeneity theorem were valid, the method would be an attractive drop-in replacement for RMSNorm. However, the two central theoretical results are not supported by the manuscript as written: the gradient-homogeneity proof bounds only one of two terms in the gradient, and the attention-logit bound is derived for an RMS normalizer that is inconsistent with the L1-mean normalizer defined in Eq. (1) and implemented in Appendix C. The empirical evidence is narrow (one architecture, one dataset, no attention experiments despite the paper's title and Section 5.3), so the advertised guarantees of built-in gradient clipping and logit stability do not currently carry the weight of the paper's claims.","major_comments":[{"comment":"The central claim that MRSNorm 'unconditionally equalizes the local gradient norm' is not established: Eq. (6) computes the norm of only the second term (Omega/S_MRS)u_i of the gradient in Eq. (5), while the full norm is ||∂L/∂z_i||^2 = (γ_i^2||∇o_i||^2 − 2γ_iΩ(∇o_i·u_i) + Ω^2)/S_MRS^2. This quantity depends on the upstream gradient ∇o_i, both through its magnitude and through its alignment with u_i, and the paper provides no bound or phase-invariance property for ∇o_i; indeed, for ∇o_i = λu_i with large λ the norm is |γ_iλ − Ω|/S_MRS, which varies with i and θ_i. Consequently the abstract's claim of a built-in gradient clipper and the 'Gradient Homogeneity' guarantee are unsupported, and the 'Conformal Gradient Collinearity' discussion in the same section is also overstated because the γ_i∇o_i term in Eq. (5) is not in general collinear with z_i.","section":"§5.1, Eqs. (5)–(6); abstract; §1"},{"comment":"The attention-stability theorem applies to a different normalizer than the proposed method: Eq. (1) defines S as the L1 mean of per-phasor L2 norms plus epsilon, and Appendix C implements that definition, but Appendix A Eq. (11) defines the attention normalizer as the RMS energy S_q = sqrt((2/d)Σ r_{q,i}^2 + epsilon). The bound |Score| ≤ d/2 in Eq. (18) uses the identity Σ r^2 = (d/2)S^2, which holds only for the RMS definition; under the L1-mean normalizer a single phasor's normalized magnitude r_i/S can be as large as P (with P = d/2), and the Cauchy-Schwarz chain in Eq. (17) yields no d/2 bound, since the maximal value of Σ w_i v_i with nonnegative weights satisfying Σ w_i = Σ v_i = P is P^2. The proof must be redone consistently with Eq. (1), and the related claim that a 'Semantic Phase Veto' completely neutralizes magnitude spikes overstates the geometry, because the magnitude weight r_{q,i}r_{k,i}/(S_q S_k) is only multiplied by cos(Δθ_i), not thresholded by it.","section":"Appendix A Eq. (11); Appendix B Eqs. (15)–(18); Eq. (1); Appendix C"},{"comment":"The experimental section does not evaluate the attention claims that motivate the title: all experiments are ResNet training on CIFAR-100 with LayerNorm and RMSNorm baselines, and no Transformer or attention task is reported, even though Section 5.3 and Appendix B make quantitative claims about attention logits and QK-MRSNorm. In addition, the reported results are coarse: Figure 1 shows shaded regions but no final accuracies or seed-wise tables, and the text concedes that at lr = 0.4 MRSNorm diverges for batch sizes 32 and 64 within 30 epochs, which is weaker than the claim that MRSNorm prevents immediate divergence. Because the main theoretical and empirical promises concern attention, the missing attention experiments leave the title claim unverified, and the ResNet experiments alone cannot carry the claim of 'proving' that unconstrained spatial scaling in standard norms is harmful redundancy.","section":"§4; §5.3; §6; title"}],"minor_comments":[{"comment":"An unresolved cross-reference 'Section??' appears in the paragraph on Capsule Networks, so the promised analysis of the Radial Gradient Blockade is not pointed to anywhere in the text.","section":"§2.2"},{"comment":"The 1D impossibility argument assumes a periodic activation with constant gradient norm, a premise satisfied by no standard activation function, so the conclusion that 1D scalar channels cannot have gradient stability in actual networks does not follow from the stated theorem.","section":"§5.2"},{"comment":"The epsilon term is placed inside the square root in Appendix A Eq. (11) but added outside the L1 mean in Eq. (1); the two conventions should be aligned, and the paper should state whether epsilon is included in the Appendix B identities.","section":"Eq. (11) vs. Eq. (1)"},{"comment":"The implementation is a grouped variant (GroupMRSNorm with num_groups), but the main text defines the ungroupped MRSNorm of Eq. (1), and Section 4 does not state which variant and group setting were used in the experiments, so the exact evaluated method is ambiguous.","section":"Appendix C; §4"},{"comment":"The 'energy-weighted cosine similarity' of Eq. (14) is a direct polar-coordinate rewriting of the dot product, and the assertion that the network 'learns to dynamically allocate magnitude' is an interpretation of that identity rather than a consequence of MRSNorm; the novelty of this derivation should be stated accordingly.","section":"§5.3; Appendix A"},{"comment":"The claim that RMSNorm's quadratic accumulation Σx^2 'triggers catastrophic numerical explosions' is presented as a general fact, but no mixed-precision experiment is reported, so the claim should be attributed to the extreme-hyperparameter regime tested here and to the cited massive-activations literature rather than stated unconditionally.","section":"§1; §2.1"}],"recommendation":"reject","confidential_remarks":"The paper's central theorems do not hold as written, and the fixes are not local: the gradient-homogeneity claim is false in general because the upstream gradient term is unaccounted for, and the attention-logit bound requires either changing the method to an RMS normalizer or redoing the entire bound for the L1-mean normalizer. The mismatch between the title and the experimental content (ResNet/CIFAR-100 only, no attention experiments) goes beyond presentation, and the claim of 'proving' that unconstrained spatial scaling is redundant is far stronger than the evidence. I see no scope-limited revision that would preserve the abstract's guarantees, so I recommend rejection despite the paper's readable exposition and the presence of a code appendix."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: MRSNorm is a new normalization variant — pair channels into 2D phasors, compute per-pair L2 norms, average them with an L1 mean, and share one affine weight per pair — and the paper's small geometric observations (why 1D scalars can't have both periodicity and constant gradient norm; why strict unit-norm projection causes a radial gradient blockade) are correct and clearly explained. That's the real value here. But the advertised stability theorems don't hold as written.\n\nThe gradient-homogeneity proof in Section 5.1 considers the gradient ∂L/∂z_i = (1/S)[γ_i∇o_i − u_i Ω] and then computes the norm of only the second term: ||u_i Ω/S|| = |Ω|/S. The first term γ_i∇o_i/S and its cross-term with u_i are ignored. The full gradient norm depends on the upstream gradient ∇o_i, both its magnitude and its alignment with u_i. Nothing in the paper bounds that or shows it is phase-invariant, so the abstract's claim of \"unconditionally equalizing the local gradient norm\" is unsupported. This is a load-bearing gap, not a minor omission.\n\nSecond, Appendix A defines the normalizer as an RMS energy S_q = sqrt((2/d)Σr²), while Eq. 1 and Appendix C use the L1 mean S_MRS = (1/P)Σ||z_i||. The logit bound |Score| ≤ d/2 in Appendix B relies on the identity Σr² = (d/2)S², which only holds for the RMS definition. So the attention-stability theorem does not apply to the proposed method as defined.\n\nWhat is genuinely new is narrow but real: the specific L1-mean-of-L2-norms formula with tied affine parameters is not in the cited literature. The polar decomposition of dot products is textbook, so the attention analysis is a restatement. The experiments are thin — one CIFAR-100 ResNet stress test, no final accuracy table, no attention experiments, and the \"halved parameters\" claim is never quantified against baselines.\n\nThe paper reads like a promising workshop draft. The references are appropriate and the limitations are acknowledged. But the internal inconsistencies and the proof gap would need to be fixed before the conclusions are trustworthy.\n\nI'd send it to peer review — a serious referee can identify these issues and the underlying normalization idea deserves discussion. I wouldn't cite it in its current form.","headline":"New normalization variant with a plausible geometric idea, but the gradient-homogeneity theorem ignores the upstream gradient term and the attention appendix defines a different normalizer, so the central claims are not established.","tokens_in":10478,"tokens_out":6565,"would_cite":false,"duration_ms":52661,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MRSNorm pairs channels into phasors so one shared scale preserves phase and equalizes gradient-update norms, giving built-in gradient clipping that survives extreme learning rates.","keywords":["MRSNorm","phasor","normalization layer","gradient homogeneity","conformal invariance","gradient clipping","attention","numerical stability"],"falsifier":"Record the per-phasor Euclidean norm of the full backward update in a trained MRSNorm layer while feeding inputs whose phasors have identical lengths but different phases; if the norms vary with $\\theta_i$ (or with the magnitude of the upstream gradient) beyond numerical rounding, the unconditional equalization claim is false. A second check is to evaluate the attention bound under the main-text mean-of-lengths scale, which is not the root-mean-square scale used in Appendix B.","tokens_in":9371,"feed_emoji":"📐","tokens_out":11590,"duration_ms":94041,"temperature":0.7,"pith_summary":"MRSNorm is a normalization layer that treats every pair of channels as a two-dimensional phasor, computes each pair's length, and divides by the average of those lengths. Because the scale is global, the phase of each phasor is untouched and a single learned gain serves both channels, halving the affine parameters of RMSNorm and LayerNorm. The paper's central claim is that this geometry equalizes the local gradient-update magnitude across all phasors: the identity $\\cos^2\\theta+\\sin^2\\theta=1$ removes the phase angle from the update norm, so no channel is starved and no outlier can monopolize the gradient. The authors call this built-in Gradient Homogeneity and position it as a structural alternative to gradient clipping. On CIFAR-100 with a ResNet, under learning rates up to 0.4 and batch sizes down to 32, MRSNorm diverges later or not at all where LayerNorm and RMSNorm collapse.","feed_headline":"Phasor pairing gives normalization built-in gradient equality","feed_subtitle":"MRSNorm swaps sum-of-squares scaling for paired phasor lengths, keeping updates uniform under extreme learning rates.","key_machinery":"The central object is the 2D phasor pair $z_i=[x_i,y_i]$ with unit direction $u_i=[\\cos\\theta_i,\\sin\\theta_i]$, normalized by the global mean-root-square scale $S_{\\text{MRS}}$. The load-bearing identity is $\\cos^2\\theta_i+\\sin^2\\theta_i\\equiv 1$, which makes $\\lVert u_i\\rVert_2$ exactly 1 and therefore converts the gradient interference term $(\\Omega/S_{\\text{MRS}})u_i$ into a vector of fixed norm $|\\Omega|/S_{\\text{MRS}}$ with no dependence on the phase. The same unit-norm factor, scaled by $1/P$ in the Jacobian, is what keeps radial gradients unblocked while instance-wise projection would annihilate them; in the attention setting the same identity produces the $\\cos(\\Delta\\theta_i)$ factor that gates magnitude by semantic alignment.","core_discovery":"The paper claims that the usual normalization denominator---the quadratic accumulation $\\sum x^2$---is not merely expensive but geometrically harmful, and that inverting the computation order removes the harm. In MRSNorm, channels $z_i=(x_i,y_i)$ are paired, each pair contributes its Euclidean length $\\lVert z_i\\rVert_2$, and the layer scales by $S_{\\text{MRS}}=\\frac{1}{P}\\sum_i \\lVert z_i\\rVert_2 + \\epsilon$. The normalized phasor keeps its phase angle, and the gradient decomposes into a term along the unit direction $u_i=[\\cos\\theta_i,\\sin\\theta_i]$ whose norm is $|\\Omega|/S_{\\text{MRS}}$, independent of $\\theta_i$. This is the paper's Gradient Homogeneity: every phasor receives the same update magnitude, which it describes as a built-in trigonometric gradient clipper that prevents starvation and explosion. Beyond normalization, the same phasor decomposition rewrites dot-product attention as a sum of energy weights times $\\cos(\\Delta\\theta_i)$, and the paper bounds the resulting logits by $d/2$, arguing that magnitude spikes cannot dominate unless phases align.","pith_inferences":["Because the proof isolates the interference term, the decisive experiment the paper does not run is a direct histogram of per-phasor gradient norms across phases; if the upstream-gradient term varies with $\\theta$, the unconditional wording would need softening.","The main-text scale $S_{\\text{MRS}}$ is the mean of phasor lengths, while Appendix A defines the query scale as a root-mean-square of lengths; reconciling these definitions is a natural first check before porting the $d/2$ attention bound to the actual implementation.","The same pairing idea suggests a family of grouped norms in which phasor bundles rather than single pairs share a scale, tuning the trade-off between parameter savings and phase granularity."],"forward_implications":["If the gradient-homogeneity argument transfers from the derived formula to full training, normalization layers can be built with half the affine parameters of RMSNorm while gaining stability rather than losing it.","Under extreme optimization settings (learning rates of 0.3\\u20130.4 with small batches), MRSNorm should remain trainable longer than LayerNorm and RMSNorm, matching the CIFAR-100 stress-test observations.","For attention, applying MRSNorm to queries and keys makes scores energy-weighted cosine similarities bounded by $d/2$, so a single magnitude spike cannot hijack the softmax without phase alignment.","The framework predicts that unconstrained per-channel scaling in standard norms is a redundant source of phase distortion; removing it should flatten the optimization landscape and reduce gradient starvation."],"supporting_citations":[{"why":"RMSNorm is the baseline whose summation-over-squares denominator MRSNorm inverts, and it supplies the gradient formula being contrasted.","marker":"[1]"},{"why":"LayerNorm is the mean-centering predecessor and serves as an experimental baseline in the stress tests.","marker":"[2]"},{"why":"The massive-activations phenomenon motivates the claim that quadratic accumulation causes outliers and starvation.","marker":"[3]"},{"why":"Capsule squashing is the instance-wise normalization whose radial gradient blockade MRSNorm is designed to avoid.","marker":"[5]"},{"why":"RoPE is the prior phase-based method that MRSNorm extends from positional encoding to normalization geometry.","marker":"[6]"},{"why":"The geometric account of gradient clipping frames why a built-in equalizer is preferable to truncating update vectors.","marker":"[8]"},{"why":"CIFAR-100 is the dataset on which the ResNet stress tests are run.","marker":"[11]"}],"fun_headline_variants":["Inverting normalization order yields phasor-stable gradients","Phasor-aware norm halves parameters, equalizes updates","MRSNorm: phase-preserving scaling with uniform gradients","Swap sum-of-squares for phasor means to stabilize training","Pairing channels into phasors fixes gradient starvation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The gradient-homogeneity proof assumes that the upstream-gradient term in the total update is negligible or phase-invariant; only the interference term $(\\Omega/S_{\\text{MRS}})u_i$ is shown to have uniform norm, so a large upstream gradient could break the equality.","fun_headline_variants_meta":{"raw":{"variants":["Inverting normalization order yields phasor-stable gradients","Phasor-aware norm halves parameters, equalizes updates","MRSNorm: phase-preserving scaling with uniform gradients","Swap sum-of-squares for phasor means to stabilize training","Pairing channels into phasors fixes gradient starvation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1345,"prompt_tokens":1062,"completion_tokens":283,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":215}},"tokens_in":678,"tokens_out":283,"duration_ms":3494,"temperature":1.0,"reasoning_tokens":215,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:33:39.512021+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record the per-phasor Euclidean norm of the full backward update in a trained MRSNorm layer while feeding inputs whose phasors have identical lengths but different phases; if the norms vary with $\\theta_i$ (or with the magnitude of the upstream gradient) beyond numerical rounding, the unconditional equalization claim is false. A second check is to evaluate the attention bound under the main-text mean-of-lengths scale, which is not the root-mean-square scale used in Appendix B.","supporting_citations":[{"cited_title":"Understanding gradient clipping in private sgd: A geometric perspective.Advances in neural information processing systems, 33:13773– 13782, 2020","cited_arxiv_id":null,"evidence_quote":"The geometric account of gradient clipping frames why a built-in equalizer is preferable to truncating update vectors."},{"cited_title":"Learning multiple layers of features from tiny images","cited_arxiv_id":null,"evidence_quote":"CIFAR-100 is the dataset on which the ResNet stress tests are run."}],"review_version":2}