{"id":"470a67db-9872-4bc5-9ab0-687fd64c02bf","arxiv_id":"2508.20036","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The limiting eigenvalue distribution of the two-layer NTK in the quadratic scaling n/(dp) tends to a Marchenko-Pastur map applied to a deterministic measure depending on the activation and output weights.","lead":"This paper computes the exact eigenvalue distribution of the neural tangent kernel of a two-layer network in a high-dimensional 'quadratic' scaling, expressing it as a free multiplicative convolution of the Marchenko-Pastur law with a deterministic measure. The result targets the interpolation threshold where the number of parameters and data points are comparable, the regime associated with double descent.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proof's nonlinearity-removal step relies on Assumption 2.2's concentration inequality, which is much stronger than the stated fourth-moment condition; this makes the theorem narrower than the abstract's 'i.i.d random matrix' claim.","rationale":"The reader's weakest_assumption identifies the same concentration inequality (2.1) as the main gap. After reading the full proof, I agree: this inequality is used essentially in Section 3 to justify replacing the nonlinear function of XW by a function of an independent Gaussian matrix. The paper states the fourth-moment hypothesis but then adds a much stronger concentration condition without highlighting that it is not implied by finite fourth moments. This is a real limitation on the theorem's scope, though not an internal contradiction: the theorem is valid under Assumption 2.2 as written. The other gaps mentioned by the reader (the T-to-Q perturbation and the Stieltjes inversion) are also present, but the concentration assumption is the most load-bearing because it affects the central reduction step. My proposed simulation directly tests whether Proposition 3.1 survives for fourth-moment data; if it fails, the theorem cannot be extended to the class of data the introduction appears to promise. Since the reader already issued a CONDITIONAL verdict, my concern does not change the verdict; it reinforces the need for the authors to either replace (2.1) with a weaker condition or clearly state the tail assumption in the abstract and main theorem.","tokens_in":44266,"tokens_out":19273,"duration_ms":215963,"concrete_test":"Set d=200, p=γ2 d with γ2=1, n=γ1 d p with γ1=1. Draw X with iid entries from a centered distribution with finite fourth moment but no exponential tails (e.g., symmetrized Pareto density c/(1+|x|^5) or Student t_5 rescaled to variance 1). Take W iid N(0,1), D=I, φ(x)=min{x,0}. Compute the empirical spectral distributions of K and of the surrogate eK from (3.1) with independent Gaussian eX. Evaluate the Kolmogorov distance between the two ESDs for d=200,400,800. If the distance does not go to 0 (or does not go to 0 at the rate predicted under (2.1)), Proposition 3.1 fails and Assumption 2.2 is truly load-bearing; if it does go to 0, the concentration assumption is not needed for this replacement step.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 2.7 is conditional on Assumption 2.2, but the abstract and introduction present X as an i.i.d random matrix without flagging the strong tail condition. The crucial Proposition 3.1 (replacing ψ(XW/√d) with ψ(eX)) uses Lemma 3.4, whose proof is Lemma 3.5: it controls E[ω_i⊗ω_i − eω_i⊗eω_i] by Taylor-expanding pseudo-Lipschitz functions around ||x_i||²/d and applying (2.1). Inequality (2.1) demands P(| ||x_i||²/d − 1 | ≥ d^ε/√d) ≤ N^{-θ} for every ε,θ>0. For iid entries with only finite fourth moment, the row-sum concentration is at best polynomial (Chebyshev gives roughly d^{-1/2−2ε}), so (2.1) fails. Consequently, the theorem as proven does not cover general fourth-moment data; a sub-Gaussian or bounded-entry hypothesis is effectively needed. This is the weakest step in the proof: if (2.1) is removed, the estimates in Lemma 3.5 and Proposition 3.1 break, and the reduction to eK is unjustified. The paper does not acknowledge that the concentration inequality is an extra assumption beyond the 'i.i.d. centered with variance 1 and finite fourth moment' description, so a reader could over-apply the theorem.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper computes the asymptotic empirical spectral distribution of the neural tangent kernel matrix K = (1/d) X X^T ⊙ (1/p) φ(XW/√d) D^2 φ(XW/√d)^T in the quadratic scaling n/(dp)→γ1, p/d→γ2, with X i.i.d., W standard Gaussian, and D diagonal with bounded i.i.d. entries. The main result, Theorem 2.7, states that the empirical spectral distributions of K and of the full two-layer NTK converge weakly in probability to μ_{MP}^{γ1} ⊠ μ_{ν,φ}, where μ_{ν,φ} is characterized by a free-probability formula in Section 5.2. In the special cases D^2 = I or φ linear, the limit is explicitly μ_{MP}^{γ1} ⊠ Law(α_φ^2 χ + β_ψ^2), where Law(χ) is a classical convolution of Marchenko–Pastur-type measures. The proof strategy is: (1) replace the nonlinearity φ by a linear part plus independent Gaussian noise (Proposition 3.1); (2) rewrite the model as a Gram matrix of random tensors and apply the Bai–Zhou sample-covariance theorem (Section 4); (3) compute the limiting spectrum of the covariance tensor Q via exact finite-rank structures and moment/free-probability calculations (Section 5).","tokens_in":44641,"tokens_out":7431,"duration_ms":89825,"significance":"If the result is correct, it is a significant contribution to nonlinear random matrix theory and to the spectral analysis of neural tangent kernels in the interpolation regime. The paper contains no fitted parameters: γ1 and γ2 are dimension ratios and α_φ, β_ψ are fixed Gaussian inner products; the special cases in Corollary 2.8 are explicit and match the four simulation configurations in Figure 2. The general free-probability formula in Proposition 5.12 is a genuinely new structural description, and the observation that the first-order chaos produces a convolution structure while higher chaoses act like an additive shift is illuminating. The main caveats are two unstated or under-proved steps in the general proof: the reduction from the covariance tensor T to its low-rank perturbation term, and the strength of the concentration assumption relative to the advertised i.i.d. data setting. Both are fixable, but they need to be addressed before the paper can be accepted.","major_comments":[{"comment":"The transition from the covariance tensor T to the tensor Q is asserted, not proved. After Lemma 4.1, T = Q + R^(1) with R^(1) = (α_φ^2/d) μ_{4,x} diag1(WD ⊗ WD). Section 5 begins 'From the previous section, we know that the limiting eigenvalue distribution of eK is given by μ_MP^{γ1} ⊠ π where π is the asymptotic e.e.d of tensor Q, the leading part of tensor T.' No argument is given that replacing T by Q does not change the limiting empirical eigenvalue distribution required by the Bai–Zhou theorem (Theorem 4.2). As defined, diag1 is supported on q=s and R^(1) is, as an operator, a direct sum of rank-one blocks over the first index, hence has rank at most d while the tensor space has dimension dp; this rank estimate would make rank/(dp)=O(1/p) → 0 and would justify the reduction, but this is not stated. Without this explicit rank/closeness argument, Theorem 2.7 does not follow from Theo","section":null},{"comment":"The concentration inequality (2.1) is substantially stronger than the stated 'i.i.d. centered random variables of variance 1 such that μ_{4,x} exists' hypothesis. It demands P(|‖x_i‖²/d − 1| ≥ d^ε/√d) ≤ N^{-θ} for every ε,θ > 0, which is effectively a sub-exponential/bounded-entry row concentration property. For i.i.d. entries with only four moments, Chebyshev gives only a polynomial bound of order d^{-2ε}, not N^{-θ} for all θ. Inequality (2.1) is used in the load-bearing estimates of Lemma 3.5, Lemma 3.6, Section 4, and Theorem A.1. Thus the theorem as proven is narrower than the abstract's 'X ∈ R^{n×d} is an i.i.d random matrix' and the introduction's claim that Gaussian data is not needed. The authors should either state the strong tail assumption prominently in the abstract and Section 1, or replace it with an explicit bounded/sub-Gaussian condition and adapt the proofs.","section":null}],"minor_comments":[{"comment":"The parameter N in (2.1) is never defined; it should be n, d, or another growth parameter. This makes the assumption ambiguous.","section":"Assumption 2.2"},{"comment":"There is a typo: 'Suppose that A ⪰ 0 and B ⪰ are a × a and b × b matrices' should read 'B ⪰ 0'.","section":"Section 2, Theorem 2.6"},{"comment":"The proof concludes with O(d^{4ε}p) while the statement claims C p d^ε. This is not an error, but the epsilon bookkeeping should be made uniform so the reader can verify the claimed rate for all ε in the statement.","section":"Section 3, Lemma 3.5 proof"},{"comment":"The proof sums the individual high-probability estimates of Lemma 3.4 over i without explicitly discussing the union bound over i = 1,...,n or the correlation between the summands. A sentence explaining the union bound and the deterministic factor 1/n would help.","section":"Section 3, Proposition 3.1"},{"comment":"The title contains a typo ('Eigenv alue') and Section 1 contains 'linear regine'; these should be corrected.","section":"Title and Section 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper computes the NTK spectrum in the quadratic scaling for the first time, and the special cases are convincing. But the main theorem as stated is narrower than the abstract suggests: Assumption 2.2's concentration inequality (2.1) is doing essential work and is not implied by the claimed fourth-moment condition.\n\nWhat's new: this is the first treatment of the shared randomness in the Hadamard product of the data Gram matrix and the nonlinear feature matrix. Prior Hadamard-product results handle independent factors. The Gaussian decomposition of φ into linear + chaos + constant, the reduction to a Gram matrix in R^d ⊗ R^p, and the covariance tensor analysis are technically solid. Corollary 2.8 gives clean explicit limiting laws for D^2=I or linear φ, and the simulations match those laws well. Proposition 5.10's moment formula and the free-probability characterization in Proposition 5.12 are genuinely new.\n\nThe main soft spot is Assumption 2.2. Inequality (2.1) demands row-sum concentration at scale d^ε/√d with polynomial tails for every ε, θ > 0. For iid entries with only four moments, Chebyshev gives at best d^{-1/2-2ε}, so (2.1) fails. The proof uses (2.1) in Lemma 3.5 and Proposition 3.1 to justify replacing ψ(XW/√d) by ψ(eX) with an independent Gaussian eX. Without it, the reduction to eK is not justified. The abstract and introduction describe X as an i.i.d. matrix with finite fourth moment, so a reader could over-apply the theorem. This is a genuine limitation, not a contradiction, and it is fixable by adding a sub-Gaussian or bounded-entry hypothesis or by proving (2.1) under a weaker condition.\n\nTwo smaller gaps: the reduction from T to Q in Section 5 is asserted rather than proved—the rank-O(d) perturbation R^(1) could in principle affect the limit—and the Stieltjes inversion in Proposition 5.12 is stated without full justification. Neither looks fatal, but both should be addressed in revision.\n\nWho this is for: people working on NTK spectra, double descent, and nonlinear random matrix theory. The result is important and likely correct under the right assumptions. It deserves a serious referee; I would ask for revision on the assumptions and the T-to-Q gap, but the core contribution stands. I'd cite it for the special cases and the moment method.","headline":"The quadratic-scaling NTK spectrum is a real new result, but the abstract overclaims: the proof relies on a sub-Gaussian-like concentration inequality that the 'i.i.d. with four moments' framing doesn't provide.","tokens_in":45087,"tokens_out":3410,"would_cite":true,"duration_ms":33830,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60B20","46L54"],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-layer NTK spectrum is a Marchenko–Pastur map of an explicit law.","keywords":["neural tangent kernel","eigenvalue distribution","Marchenko–Pastur map","free multiplicative convolution","quadratic scaling","Hadamard product","random matrix theory","interpolation threshold"],"falsifier":"Run the explicit case ν=δ1, φ(x)=x at a new (γ1,γ2) pair, e.g. γ1=0.7, γ2=1.4, with n around 20,000, and compare the empirical density of K to μ^{γ1}_{MP}⊠Law(χ), Law(χ)=γ2/2 μ^{γ2}_{MP}∗μ^{γ2}_{MP}+(1−γ2)μ^{γ2}_{MP}+γ2/2 δ0; any systematic deviation that persists as n grows would refute the main formula. To probe the load-bearing hypothesis instead, use centered data with a finite fourth moment but row norms that concentrate only polynomially at rate d^{-δ}, δ<1/2; the theorem's predicted MP-map limit should fail, revealing that Assumption 2.2 is doing real work.","tokens_in":44205,"feed_emoji":"📊","tokens_out":10857,"duration_ms":115733,"temperature":0.7,"pith_summary":"This paper proves that in the quadratic scaling of a two-layer neural network—where the number of samples n, input dimension d, and hidden width p grow with n/(dp)→γ1 and p/d→γ2, the regime around the interpolation threshold—the eigenvalue distribution of the neural tangent kernel is asymptotically deterministic. The limit is the Marchenko–Pastur map with shape γ1 applied to a deterministic measure μ_{ν,φ} that depends only on the law of the squared readout weights, the width-aspect γ2, and the activation derivative φ; the full NTK and its Hadamard-product part share this bulk limit because the extra conjugate-kernel term has rank O(√n). When D²=I or φ is linear, the measure is fully explicit: μ^{γ1}_{MP}⊠Law(α²_φ χ+β²_ψ), where χ is a classical convolution of two Marchenko–Pastur laws plus an atom at zero, and αφ, βψ are the Gaussian first-order correlation and residual variance of φ. The proof identifies the NTK as a Gram matrix of tensor-product rows, applies the Bai–Zhou theorem for sample covariance matrices with independent rows, and then computes the spectrum of the resulting covariance tensor via its moments and free probability.","feed_headline":"NTK spectrum pinned down at the interpolation threshold","feed_subtitle":"Two-layer network kernels converge to a Marchenko-Pastur transform fixed by the activation and width ratio.","key_machinery":"The Marchenko–Pastur map μ^{γ}_{MP}⊠ν, whose Stieltjes transform solves s(z)=∫ (t(1−γ(1+zs(z)))−z)^{-1} dν(t), is the object carrying the result: it sends the spectrum of a population covariance to the spectrum of a sample covariance with sample-to-feature ratio γ. The proof's workhorse is the reduction of the kernel to a Gram matrix of vectors ω_i=x_i⊗D(αφ/√d W^T x_i+ψ(ẽx_i)) in Rd⊗Rp, so that Bai–Zhou's theorem applies with independent rows. The limiting covariance tensor Q factors as D bQ D, and bQ's spectrum is computable exactly from the singular values of W: eigenvalues α²φ/d(σ²_i+σ²_j)+β²ψ plus a β²ψ eigenspace, which yields the explicit Law(α²φχ+β²ψ) in the commuting cases. In genera","core_discovery":"The central discovery is a limit theorem: under Assumptions 2.1–2.4, the empirical spectral distribution of K=(1/d)XX^T⊙(1/p)φ(XW/√d)D²φ(XW/√d)^T, and of the full two-layer NTK, converges weakly in probability to μ^{γ1}_{MP}⊠μ_{ν,φ}. The measure μ_{ν,φ} is defined by moment formulas (Proposition 5.10) and by the free-probabilistic Stieltjes transform in Proposition 5.12. In the two cases where the shift is trivial, D²=I or φ linear, the law collapses to μ^{γ1}_{MP}⊠Law(α²_φ χ+β²_ψ) with Law(χ)=γ2/2(μ^{γ2}_{MP}⊠ν)∗(μ^{γ2}_{MP}⊠ν)+(1−γ2)(μ^{γ2}_{MP}⊠ν)+γ2/2 δ0; here αφ is the Gaussian covariance of φ with the identity, ψ is the residual after removing the constant and linear Gaussian chaoses,","pith_inferences":["The proof's reliance on Assumption 2.2 suggests a sharp-threshold question the paper leaves open: how many moments, or what concentration rate, are actually needed for the MP-map limit; a plausible answer is sub-Gaussian or bounded rows, with heavy-tailed rows producing a different limit.","The tensor-Gram rewrite is not specific to the two-layer NTK; the same strategy should give MP-map-of-covariance-tensor limits for Hadamard products of other nonlinear feature matrices, including deep linearizations, at the interpolation threshold.","Since the noncommuting shift is the only obstruction to the simple convolution formula, choosing D with non-constant entries and a nonlinear φ should produce spectra measurably different from the naive Law(α²φχ+β²ψ), which is a direct numerical test of Proposition 5.12."],"forward_implications":["The eigenvalue bulk of the full NTK is identical to that of the Hadamard-only matrix K, so the conjugate-kernel term created by differentiating the outer weights does not shape the spectrum at this scaling.","For D²=I or linear activations, the limiting spectrum can be written down by convolving two Marchenko–Pastur densities with weights γ2/2 and 1−γ2 and then applying the Marchenko–Pastur map of shape γ1; no simulation is needed.","The limit depends on the activation only through αφ=E[zφ(z)] and the residual variance β²ψ=E[ψ(z)²], so many activations share the same bulk spectrum when these two numbers match.","In the general case the spectrum is characterized by the Stieltjes transform (5.4) in terms of the free non-commuting pair (D², (1/d)W^TW), giving a complete algorithm for the limit from moment data."],"supporting_citations":[{"why":"Supplies the theorem that a Gram matrix with independent rows has limiting e.e.d. equal to the Marchenko–Pastur map applied to the row covariance tensor; this is the step that turns the NTK into an MP-map limit.","marker":"[BZ08]"},{"why":"Provides the definition and Stieltjes-transform characterization of the Marchenko–Pastur map, plus rank-perturbation bounds used to discard low-rank terms.","marker":"[BS10]"},{"why":"Supplies the free probability framework and free Poisson convergence used to turn the moment formula for the covariance tensor into the free-pair characterization in Proposition 5.12.","marker":"[MS17]"},{"why":"Gives the almost-sure spectral norm bound for sample covariance matrices used to show the covariance tensor is bounded.","marker":"[BS98]"},{"why":"Defines the neural tangent kernel as the object whose eigenvalue distribution is being computed.","marker":"[JGH18]"},{"why":"Establishes the prior linear-scaling result for the NTK spectrum that this paper extends to the quadratic scaling.","marker":"[FW20]"}],"fun_headline_variants":["NTK spectrum: free convolution of MP and activation","Exact NTK eigenlaw for two-layer networks","Two-layer NTK: eigenvalue distribution is universal","NTK at large width and data: spectrum from free probability","Quadratic scaling NTK: MP twisted by activation"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The proof requires every data row's squared norm to be within d^{-1/2+ε} of its expectation with failure probability decaying faster than any polynomial; this concentration is much stronger than the stated fourth-moment condition and effectively assumes sub-exponential (near-bounded) data entries.","fun_headline_variants_meta":{"raw":{"variants":["NTK spectrum: free convolution of MP and activation","Exact NTK eigenlaw for two-layer networks","Two-layer NTK: eigenvalue distribution is universal","NTK at large width and data: spectrum from free probability","Quadratic scaling NTK: MP twisted by activation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1511,"prompt_tokens":866,"completion_tokens":645,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":568}},"tokens_in":610,"tokens_out":645,"duration_ms":8308,"temperature":1.0,"reasoning_tokens":568,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:14:54.634122+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the explicit case ν=δ1, φ(x)=x at a new (γ1,γ2) pair, e.g. γ1=0.7, γ2=1.4, with n around 20,000, and compare the empirical density of K to μ^{γ1}_{MP}⊠Law(χ), Law(χ)=γ2/2 μ^{γ2}_{MP}∗μ^{γ2}_{MP}+(1−γ2)μ^{γ2}_{MP}+γ2/2 δ0; any systematic deviation that persists as n grows would refute the main formula. To probe the load-bearing hypothesis instead, use centered data with a finite fourth moment but row norms that concentrate only polynomially at rate d^{-δ}, δ<1/2; the theorem's predicted MP-map limit should fail, revealing that Assumption 2.2 is doing real work.","supporting_citations":[],"review_version":1}