{"id":"3d73ff6c-8ecd-4a91-a552-a424cbe7e555","arxiv_id":"2412.06158","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"For PINNs solving general PDEs, the neural tangent kernel converges only if the network's output scaling exponent is large enough, with the threshold set by the homogeneity degree of the differential operator.","lead":"This paper studies when the neural tangent kernel (NTK) of physics-informed neural networks stays well-behaved for general partial differential equations, and finds that the usual network scaling does not always guarantee convergence. The authors derive conditions on a scaling exponent tied to the homogeneity of the PDE operator and test them on the sine-Gordon and KdV equations.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2.2's law-of-large-numbers step is applied to sums with non-independent, parameter-dependent coefficients; the derived thresholds s1 and s2 are therefore not established as written.","rationale":"The reader is right that the central theorem rests on an unjustified law-of-large-numbers step: the F_i coefficients depend on the entire parameter vector, so the summands in K_ff and K_bf are not independent, and the paper supplies no concentration bound. At s=1/2 for sine-Gordon this dependence creates an O(1) random common factor, so no deterministic limit exists; at s=1 it disappears. However, I disagree with the reader's arithmetic sub-claim about KdV: Eq. (39) defines s1 in terms of the homogeneity degree of the coefficient F_i, and for the term q q_x the coefficient F_1=q has degree 1, giving 3/4, not 5/6. The experiments on sine-Gordon and KdV support the qualitative prediction that s=0.5 can fail while s=1 converges, but they only compare these two values and cannot distinguish s1=3/4 from neighboring thresholds. Because the missing concentration argument is likely repairable via a delta-method analysis, the appropriate disposition is conditional acceptance pending a rigorous derivation of the claimed thresholds, rather than outright rejection.","tokens_in":20149,"tokens_out":14516,"duration_ms":148689,"concrete_test":"Carry out the missing concentration calculation on the minimal homogeneous case F[q]=q^2 (coefficient F_1=2q, degree t=1, predicted s1=3/4). Write K_ff = 4 q(x)q(x') sum_k dq(x)/dtheta_k dq(x')/dtheta_k and expand q = N^{-s} sum_k W_k sigma_k with i.i.d. weights. Compute the normalized mean and variance of this product at s=3/4 and s=1/2, using a delta-method expansion of the empirical average in q. If at s=3/4 the variance is O(N^{-1}) so the limit is deterministic, the homogeneous-case threshold stands; if any O(1) variance survives, Theorem 2.2's homogeneous-case claim is false and the bound must be revised. Independently re-deriving Eqs. (39) and (47) from Eq. (38) would settle the bookkeeping.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing step is the claim, used in Appendix B around Eqs. (31)-(35) and (38)-(40), that sums of the form sum_k F_i(q(x)) F_j(q_hat(x_hat)) dq_ix/dtheta_k dq_jx/dtheta_k converge by the law of large numbers to their expectation. The F_i depend on q(x,theta), which is a sum over all N hidden units, so the summands are not independent across k. The proof gives no variance or concentration bound for this dependent, nonlinear case. This is not cosmetic: for s=1/2 on F[q]=sin(q), q has a non-degenerate Gaussian limit, so cos(q(x))cos(q_hat(x_hat)) has non-zero variance and a deterministic limiting kernel cannot exist. For s=1 (case A) the common factor becomes deterministic and the step is salvageable by Slutsky; for the homogeneous case a correct proof would need a delta-method argument showing that normalized fluctuations of F_i vanish. The paper does not provide it, so the thresholds s1 and s2 are unproven. I do not treat the KdV value s1=3/4 as an arithmetic error: Eq. (39) uses the homogeneity degree of the coefficient F_i (degree 1 for the factor q in q q_x), not of the monomial q q_x, so 3/4 is consistent with the formula as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the neural tangent kernel (NTK) of physics-informed neural networks (PINNs) for general nonlinear PDEs. It introduces an output-layer scaling 1/N^s and claims that the initialized NTK converges in probability to a deterministic kernel only above thresholds that depend on the homogeneity of the differential operator (Theorem 2.2): s >= 1 for non-homogeneous operators (case A) and s >= s1 for homogeneous operators (case B), with s1 defined in Eq. (39). It further claims that the NTK stays constant during training for any s > 1/4 (Theorem 2.3). The claims are tested on the sine-Gordon and KdV equations by plotting ||K_{N+k} - K_N|| for widths up to 50000 at s = 0.5 and s = 1. The paper reports divergence at s = 0.5 and convergence at s = 1, and interprets this as confirmation of the theory.","tokens_in":20372,"tokens_out":19902,"duration_ms":176080,"significance":"If the quantitative thresholds were correct, the paper would provide useful design guidance for PINN initialization and identify a genuine limitation of standard NTK scaling for nonlinear PDEs. The basic qualitative phenomenon is plausible and the numerical experiments, while limited to two equations and a proxy quantity, show a clear difference between s = 0.5 and s = 1. The paper also has a useful negative message: the standard NTK convergence argument does not automatically extend to nonlinear PDEs. However, the theoretical derivation of the thresholds is the load-bearing part of the manuscript, and it contains substantial gaps: the law-of-large-numbers step is not justified for dependent nonlinear terms, the scaling analysis appears to ignore the N^{1/2} fluctuations of the hidden-layer sum, and the norm estimates used in the proof of Theorem 2.3 are not valid for the claimed range of s. I therefore cannot regard the main quantitative claims as established.","major_comments":[{"comment":"The proof of Theorem 2.2 invokes the law of large numbers for sums whose summands are not independent. In the expression for K_ff, the coefficients F_i[q,q_x,...] are nonlinear functions of q(x,theta), which is itself a sum over all N hidden units, so the summands indexed by k are dependent. No variance or concentration bound is supplied for these terms. This is not a cosmetic issue: for s = 1/2 and F[q] = sin(q), q converges to a nondegenerate Gaussian process, and cos(q(x))cos(q(hat x)) has positive variance, so a deterministic limiting kernel cannot exist. For s > 1/2 a Slutsky-type argument might be possible, and for the homogeneous case a delta-method argument controlling the normalized fluctuations of F_i would be needed. Since neither is provided, the thresholds s1 and s2 in Theorem 2.2 are not proved as stated.","section":"Appendix B, Eqs. (31)-(40)"},{"comment":"The scaling analysis appears to neglect the N^{1/2} fluctuation of the sum defining q. For mean-zero iid summands, q(x,theta) = N^{-s} sum_k W^{(1)}_k sigma(...) is of order O_P(N^{1/2-s}), not O_P(N^{-s}). For a homogeneous coefficient F_i of degree d = sum_l t_{il}, this gives F_i = O_P(N^{d(1/2-s)}), and the corresponding K_ff contribution has size N^{1+d-2s(d+1)}. Balancing this yields s = 1/2, not the formula s1 = (2d+1)/(2d+2) in Eq. (39). Consequently, the claimed KdV thresholds s1 = 3/4 and s2 = 2/3 do not follow from the displayed scaling; for the homogeneous terms in KdV the contribution would already vanish for every s > 1/2. The experiments only test s = 0.5 and s = 1.0, which cannot distinguish the paper's thresholds from the simpler condition s > 1/2.","section":"Appendix B, Eqs. (34), (39)-(40)"},{"comment":"The proof of Theorem 2.3 uses Frobenius-norm estimates that are not sufficient for the claimed range of s. Lemma C.1 gives a uniform bound O(N^{-s}) on each entry of the Jacobian, but the Jacobian has 3N+1 columns, so ||J(t)||_F = O(N^{1/2-s}), which diverges for s < 1/2. The step following Eq. (72), which treats ||J(t)||_F as bounded for all s > 1/4, is therefore unjustified. Similarly, the bound ||J(t)-J(0)||_F = O(N^{1/2-2s}) leads to a product in Eq. (72) that vanishes under these estimates only for s > 1/3, and a spectral-norm or concentration argument would be needed to reach s > 1/4. As written, Theorem 2.3 is not established for s in (1/4, 1/2].","section":"Appendix C, proof of Theorem 2.3"},{"comment":"The experimental validation of Theorem 2.3 is internally inconsistent: the text states that the convergence coefficient s is set to 0.3, while the caption of Fig. 5 says s = 0.2. Since 0.2 is below the claimed threshold 1/4, the figure as captioned would not validate the theorem. The authors should correct this mismatch and, more importantly, provide experiments at several values of s in the claimed critical ranges (for example s = 0.55, 0.6, 0.7 for the KdV equation) to distinguish between their thresholds and the simpler s > 1/2 condition.","section":"Sec. 3.2 and Fig. 5"}],"minor_comments":[{"comment":"The statement of Theorem 2.2 says the kernel converges in probability to a deterministic limiting kernel K*, but K* is never defined in the theorem; it appears only implicitly in the proof. The statement should be made precise.","section":"Theorem 2.2 statement"},{"comment":"The paper uses ||K_{N+k} - K_N|| as a proxy for ||K_N - K*||. Convergence of the former is necessary but not sufficient for convergence to a fixed limit, and this caveat should be stated more carefully in the interpretation of the experiments.","section":"Sec. 1, Fig. 1 and Sec. 3"},{"comment":"There are several typos and minor inconsistencies, including 'Theorem 3' in Sec. 3.2 instead of Theorem 2.3, and the LaTeX corruption in Eq. (59). The notation in Remark 1 is also difficult to parse because the case distinction between 'F0 ... != 0' and 'every monomial is homogeneous' is not presented cleanly.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The paper identifies an interesting qualitative phenomenon and the experiments are simple enough to be reproducible, but the central theoretical results are not correct as written. The law-of-large-numbers step in Theorem 2.2 and the norm estimates in Theorem 2.3 are load-bearing and would require substantial reworking; moreover, the scaling analysis suggests that the claimed thresholds may be quantitatively wrong. I recommend rejection rather than major revision because the main theorems, as stated, are not supported by the provided proofs, and a correct treatment would likely change the paper's central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a real observation hiding behind a half-finished proof. The claim that the standard 1/sqrt(N) output scaling does not always give a convergent NTK for nonlinear PDEs, and that the required scaling exponent depends on the homogeneity of the operator, is plausible and the experiments on sine-Gordon and KdV back it up. That is worth knowing.\n\nWhat is new: most prior NTK-for-PINNs theory assumed linear PDEs (Poisson, wave). This paper introduces the 1/N^s scaling and derives a condition on s from the homogeneity degree of the operator coefficients. That is a natural but useful extension, and the two experimental examples are clean and convincing as demonstrations of the qualitative phenomenon.\n\nThe soft spot is Theorem 2.2. The proof in Appendix B splits the kernel into sums over parameters and invokes a law of large numbers to replace each sum by an expectation. But the coefficients F_i depend on the full network output q(x,theta), which is itself a sum over all hidden units. The summands are not independent across k, and no concentration bound is offered. For the sine-Gordon case with s=1/2, the output has a non-degenerate Gaussian limit, so cos(q)cos(q_hat) has nonzero variance and a deterministic kernel cannot exist. That actually supports the paper's qualitative conclusion, but it also shows the LLN step cannot be taken for granted. A correct proof would need a delta-method or variance-vanishing argument. Without that, the thresholds s>=1 and s>=s1 are not established as theorems.\n\nOn the KdV threshold: the reader's report flags s1=3/4 as an arithmetic inconsistency with formula (39). I checked: formula (39) uses the homogeneity degree of the coefficient F_i, not the monomial. For the q q_x term, the coefficients are 6 q_x and 6 q, each homogeneous of degree 1, so 3/4 is consistent. That concern doesn't hold up.\n\nTheorem 2.3 is also weaker than it looks: assumption (i) requires uniform boundedness of parameters, which is not proven. The training-time result is therefore conditional, though the experiments at s=0.3 and s=0.25 are suggestive.\n\nOverall: the paper is worth a serious referee, but the theory needs repair. If the authors can produce a rigorous proof for the homogeneous case, or reframe the paper as an empirical study with a heuristic scaling rule, it could be publishable. As is, it's a useful conjecture with supporting numerics.\n\nRecommendation: send to peer review, but flag the LLN step as the main issue.","headline":"A plausible and experimentally supported scaling rule for PINN initialization, but the main theorem rests on an unproven law-of-large-numbers step; worth refereeing with a clear demand for a rigorous proof.","tokens_in":20938,"tokens_out":5030,"would_cite":false,"duration_ms":50660,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","60F05","35Q53"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that the PINN neural tangent kernel converges to a deterministic limit only when the output scaling $1/N^s$ has $s$ large enough—$s\\ge1$ for non-homogeneous operators and $s\\ge s_1$ for homogeneous ones—so the standard…","keywords":["neural tangent kernel","physics-informed neural networks","partial differential equations","kernel convergence","homogeneity","output scaling","sine-Gordon equation","KdV equation"],"falsifier":"Recompute the KdV threshold from formula (39): the monomial $q q_x$ has homogeneity degree $2$, so the formula gives $s_1 = (2\\cdot2+1)/(2\\cdot2+2) = 5/6$, whereas the paper states $3/4$. Simulating the initialized NTK of the KdV equation at a sequence of widths with scaling exponents $s=0.75$ and $s=0.83$ and checking whether $\\|K_{N+k}-K_N\\|_2$ decays in probability will show which threshold the dynamics actually follow.","tokens_in":19846,"feed_emoji":"🧮","tokens_out":10304,"duration_ms":84215,"temperature":0.7,"pith_summary":"The paper asks whether the neural tangent kernel (NTK) of a physics-informed neural network solving a general partial differential equation always converges to a deterministic limit as the network width grows. It answers no: with the standard output scaling $1/\\sqrt{N}$, the initialized NTK can diverge, and it proves sharp thresholds for the scaling exponent $s$ in $1/N^s$. For operators whose leading term is nonzero or that contain non-homogeneous monomials, as in the sine-Gordon equation with $\\sin(q)$, convergence requires $s \\ge 1$; for homogeneous operators such as KdV with $q q_x$, it requires $s \\ge s_1$, where $s_1$ is determined by the maximum homogeneity degree of the operator. The paper also shows that during training the NTK stays frozen for $s > 1/4$, extending the range in which kernel-regime analysis applies. Numerical experiments on the sine-Gordon and KdV equations are presented as confirmation.","feed_headline":"PINN kernels need sharper output scaling than 1/√N","feed_subtitle":"For general PDEs, the kernel converges only when the scaling exponent meets a homogeneity threshold; the standard s=1/2 can diverge.","key_machinery":"The central object is the PINN kernel of (15), assembled from gradients of the operator $F[q]$ and the boundary trace $q_x^m$ with respect to all network parameters, with the network output scaled by $1/N^s$. The argument decomposes $\\partial F/\\partial\\theta$ as $\\sum_i F_i[q,q_x,\\dots]\\,\\partial q_{ix}/\\partial\\theta$ and writes each kernel entry as a sum over the $N$ neurons; the homogeneity definition (33) assigns each monomial a degree $\\sum_l t_{i_l}$, and the proof balances powers of $N$—the factor $N^{1-2s}$ coming from the sum against the degree-dependent growth of the $F_i$—so that the law of large numbers yields a deterministic limit. The threshold $s_1$ in (39) is exactly the exponent at which that balance holds.","core_discovery":"Theorem 2.2 states that for the two-layer network (13) with output scaling $1/N^s$, the PINN kernel (15) converges in probability to a deterministic limiting kernel only above certain scaling thresholds. In case A—when $F_0[q,q_x,\\dots,q_{nx}]F_0[\\hat q,\\hat q_x,\\dots,\\hat q_{nx}]\\ne 0$, or when some monomial in the expansion $\\partial F/\\partial\\theta = \\sum_i F_i \\,\\partial q_{ix}/\\partial\\theta$ is non-homogeneous—the threshold is $s\\ge 1$. In case B—when that product vanishes and every monomial is homogeneous—the threshold is $s\\ge s_1$, where $s_1 = \\frac{2\\sum_l t_{i'_l}+1}{2\\sum_l t_{i'_l}+2}$ and $\\sum_l t_{i'_l}$ is the maximum homogeneity degree of the operator, with a secondary exponent $s_2$ for the cross-block $K_{fb}$. The standard choice $s=1/2$ therefore does not universally deliver convergence; the paper identifies the sine-Gordon equation, with the non-homogeneous term $\\sin(q)$, as a case-A example and the KdV equation, with the quadratic term $q q_x$, as a case-B example, reporting thresholds $s_1=3/4$, $s_2=2/3$ for KdV. Theorem 2.3 supplements this by proving that, under uniform boundedness of parameters, operator derivatives, and integrated residuals, the NTK stays asymptotically constant during training whenever $s>1/4$.","pith_inferences":["A rigorous proof of the sharp thresholds would require concentration bounds: the law-of-large-numbers step in the proof averages terms that are nonlinear, dependent functions of all $N$ random parameters, so the thresholds $s_1,s_2$ are best treated as scaling predictions until such bounds are supplied.","Formula (39) and the paper's stated KdV threshold $3/4$ are in tension if the quadratic monomial $q q_x$ is assigned homogeneity degree $2$, since the formula then gives $s_1=5/6$; a direct check of which value the KdV kernel follows would settle the discrepancy.","The same balancing argument suggests a practical design rule beyond the paper's examples: compute the maximum homogeneity degree of the PDE's nonlinear terms and set $s$ accordingly, which would also apply to other homogeneous equations such as Burgers' equation."],"forward_implications":["Under the paper's result, NTK-based convergence analyses of PINNs that silently assume $s=1/2$ do not transfer to general nonlinear PDEs; a practitioner must either raise the output scaling to $s\\ge 1$ or first verify that the operator is homogeneous and compute its maximum degree.","For homogeneous operators such as KdV, there is a window $s_1 \\le s < 1$ in which the initialized kernel converges even though the classical scaling does not, so kernel convergence at initialization and kernel frozenness during training are governed by different exponents ($s_1$ versus $1/4$).","The homogeneity degree of the differential operator becomes a computable design parameter: operators with higher-degree nonlinear terms demand larger scaling exponents, so the same architecture may need different $s$ for different PDEs.","The theorem gives a sufficient condition for the kernel gradient-flow description (5) to be well posed, which is a prerequisite for using NTK-based diagnostics to explain or fix PINN training failures."],"supporting_citations":[{"why":"Establishes the NTK limit theory for standard networks that this paper extends to PDE-constrained losses.","marker":"[35]"},{"why":"Introduces the PINN loss whose kernel evolution is analyzed here.","marker":"[8]"},{"why":"Applies the NTK perspective to PINNs, providing the baseline claim that the paper shows fails for general PDEs.","marker":"[40]"},{"why":"Supplies the exact shock-wave and soliton solutions used to set initial and boundary data in the sine-Gordon and KdV experiments.","marker":"[43]"},{"why":"Gives earlier convergence theory for PINNs that the paper's scaling thresholds restrict or refine.","marker":"[33]"},{"why":"Provides the over-parameterized two-layer convergence analysis whose assumptions are compared with the new thresholds.","marker":"[34]"}],"fun_headline_variants":["Standard PINN scaling can break NTK convergence","Homogeneity dictates PINN kernel convergence threshold","PINNs need sharper output scaling for kernel convergence","Operator homogeneity sets NTK scaling requirement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the sums over neurons making up the kernel entries obey a law of large numbers toward deterministic limits, even though the summed terms contain the operator coefficients $F_i$, which are nonlinear functions of the whole random network output; no concentration bound is given for those dependent, nonlinear terms.","fun_headline_variants_meta":{"raw":{"variants":["Standard PINN scaling can break NTK convergence","Homogeneity dictates PINN kernel convergence threshold","PINNs need sharper output scaling for kernel convergence","Operator homogeneity sets NTK scaling requirement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000758,"raw_usage":{"total_tokens":3404,"prompt_tokens":1015,"completion_tokens":2389,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":2342}},"tokens_in":631,"tokens_out":2389,"duration_ms":16392,"temperature":1.0,"reasoning_tokens":2342,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:58:01.557292+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the KdV threshold from formula (39): the monomial $q q_x$ has homogeneity degree $2$, so the formula gives $s_1 = (2\\cdot2+1)/(2\\cdot2+2) = 5/6$, whereas the paper states $3/4$. Simulating the initialized NTK of the KdV equation at a sequence of widths with scaling exponents $s=0.75$ and $s=0.83$ and checking whether $\\|K_{N+k}-K_N\\|_2$ decays in probability will show which threshold the dynamics actually follow.","supporting_citations":[{"cited_title":"Jacot, F","cited_arxiv_id":null,"evidence_quote":"Establishes the NTK limit theory for standard networks that this paper extends to PDE-constrained losses."},{"cited_title":"Raissi, P","cited_arxiv_id":null,"evidence_quote":"Introduces the PINN loss whose kernel evolution is analyzed here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Applies the NTK perspective to PINNs, providing the baseline claim that the paper shows fails for general PDEs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the exact shock-wave and soliton solutions used to set initial and boundary data in the sine-Gordon and KdV experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives earlier convergence theory for PINNs that the paper's scaling thresholds restrict or refine."}],"review_version":1}