{"id":"33fd7aec-76c7-45d9-bd10-1d1c4fce959b","arxiv_id":"2412.05545","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Over-parameterized shallow neural operators trained by gradient descent converge linearly to the global minimum of the empirical loss under mild sample conditions.","lead":"This paper proves that gradient descent on a wide shallow neural operator converges linearly to zero training error, for both operator regression and a physics-informed variant. It extends neural tangent kernel convergence analysis from finite-dimensional networks to operator networks mapping functions to functions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 1's positive-definiteness condition is incomplete: a zero sample satisfies the stated non-parallelism condition vacuously and yields λ0 = 0, so the convergence-rate theorems have a gap as stated.","rationale":"The reader correctly identifies the strict positive definiteness of the limiting Gram matrices as the load-bearing assumption. My stress-test sharpens this: the paper's Lemma 1 is not merely restrictive, it is false as stated for admissible degenerate data, because a zero input satisfies the 'no two samples are parallel' condition vacuously when n = 1 and gives λ0 = 0. This is a genuine gap in the theorem statements, but it is easily repaired by adding a nonzero-sample assumption (and, where needed, ruling out positive scalar multiples). The antipodal case also shows that the reader's phrasing 'if the data contain parallel points, the linear-rate conclusion collapses' is too strong: negative scalar multiples do not generally destroy positive definiteness of the ReLU-feature Gram matrices. Since the core NTK argument is otherwise standard and the gap is fixable without changing the proof structure, the conditional verdict remains appropriate. The paper should, however, be required to state the nonzero assumption explicitly and to correct Lemma 1, since the current statement makes the main theorem inapplicable to the zero-function input, which is a valid element of the function space considered.","tokens_in":1070,"tokens_out":1090,"duration_ms":267913,"concrete_test":"Directly evaluate the Kronecker formulas in Section 3 for the single-sample case n1 = n2 = 1, u_1 = 0, y_1 = e_1: H1∞ = E[σ(w̃ᵀ·0)^2] = 0 and H2∞ = E[e_1ᵀe_1 1{wᵀe_1≥0}] = 1/2, hence H∞ = 0 and λ0 = 0, contradicting Lemma 1. If one instead treats the zero vector as excluded, repeat the check with n1 = 2, u_1 = 0, u_2 = e_1 and verify that the paper's stated non-parallelism condition is either violated or must be supplemented by an explicit nonzero assumption; the theorem's m-scaling is only meaningful once that assumption is added.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of Theorems 1-3 is linear convergence with a rate controlled by λ0 and λ̃0, the smallest eigenvalues of the limiting Gram matrices H∞ and H̃∞. Lemma 1 asserts that these are strictly positive definite whenever no two samples in {u_i} are parallel and no two samples in {y_j} are parallel. The proof reduces this to positive definiteness of the Kronecker factors, e.g. H1∞(i,j) = E[σ(w̃ᵀu_i)σ(w̃ᵀu_j)] and H2∞(i,j) = E[y_iᵀy_j 1{wᵀy_i≥0,wᵀy_j≥0}]. The stated condition is not sufficient. If n1 = 1 and u_1 = 0, the condition 'no two samples are parallel' is vacuously true, but H1∞ = [E[σ(0)^2]] = [0], so H∞ = H1∞ ⊗ H2∞ has λ0 = 0 and the rate in Theorem 2 is undefined. The same failure occurs with y_1 = 0. For n1 > 1, a zero vector is arguably parallel to every other vector under the standard definition, so the condition may exclude it, but the paper never states the needed nonzero assumption, and the single-sample case is still a genuine counterexample to Lemma 1 as written. Moreover, the non-parallelism condition is stronger than necessary for antipodal pairs, which do not necessarily make the ReLU-feature Gram singular, so the reader's framing that any parallel pair collapses the conclusion is imprecise. The load-bearing point is that the paper's stated data condition does not by itself guarantee λ0 > 0; an explicit nonzero-sample assumption is required before the width bound and the convergence rate are meaningful.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the training dynamics of wide shallow neural operators of the form G(u)(y) = m^{-1/2} Σ_r [p^{-1/2} Σ_k a_{rk} σ(\\tilde{w}_{rk}^T u)] σ(w_r^T y), where u is identified with its evaluations at fixed sensors. Under random Gaussian initialization and over-parameterization, the authors claim that gradient flow (Theorem 1) and gradient descent (Theorem 2) converge to a globally optimal training loss at a linear rate, with rates governed by the smallest eigenvalues λ0 and \\tilde{λ}_0 of limiting NTK Gram matrices H^∞ and \\tilde{H}^∞. A third theorem extends the analysis to physics-informed neural operators for a parabolic PDE with a ReLU^3 trunk activation, asserting linear convergence of the residual. The proofs follow the standard NTK template: show the random Gram matrices concentrate around their kernel limits, prove stability of the Gram matrices under small weight movements, and then close an induction on weight deviations and residual norms.","tokens_in":45602,"tokens_out":6032,"duration_ms":66472,"significance":"If the results are correct, this is a useful first training-error guarantee for shallow neural operators in the NTK regime, complementing existing approximation and generalization analyses. The paper's explicit treatment of the Kronecker structure of the limiting Gram matrices, its discrete-time residual recursion (Lemma 4), and its extension to a physics-informed setting are concrete strengths. The claimed linear rates are falsifiable and the width scalings are explicit, which is valuable for subsequent work. However, several load-bearing statements need repair: the positive-definiteness lemma is incomplete as stated, the PINN dynamics equation is internally inconsistent, and the appendix's duplicate lemma numbering makes the proof of Theorem 1 difficult to verify as submitted.","major_comments":[{"comment":"The stated condition for positive definiteness is insufficient because the zero vector satisfies 'no two samples are parallel' vacuously. For example, with n1 = 1 and u1 = 0, the matrix H1^∞ has entry E[σ(0)^2] = 0, so H^∞ = H1^∞ ⊗ H2^∞ = 0 and λ0 = 0; the rates and width bounds in Theorems 1–3 are then undefined. The same failure occurs for y1 = 0. The paper should explicitly assume ui ≠ 0 and yj ≠ 0 (or, as Remark 1 suggests for a related issue, augment both nets with bias terms so that zero inputs become nonzero). This is a load-bearing correction, not a cosmetic one, since all subsequent theorems depend on λ0, \\tilde{λ}_0 > 0.","section":"Section 3, Lemma 1 (and Section 5, Lemma 6)"},{"comment":"The dynamics equation dGt(u)/dt = (H(t) + \\tilde{H}(t))(z − Gt(u)) is inconsistent with the definition of Gt(u) as the residual vector ((s(u1), h(u1)), …, (s(un1), h(un1))). Since s and h are defined as scaled residuals (LG − f) and (G − g), the correct continuous-time residual dynamics should be dGt/dt = −(H(t) + \\tilde{H}(t))Gt(u), with the target being zero, not z = ((f,g),…). As written, the equation has fixed point G∞ = z = (f,g), which is not the intended zero-residual solution. Theorem 3's claim of convergence to zero therefore does not follow from the stated dynamics; the notation must be fixed (either define Gt as the prediction and state the dynamics for the residual, or set the target to zero).","section":"Section 5, paragraph after the definitions of s(u_i) and h(u_i)"},{"comment":"The proof of Theorem 1 is not verifiable in its current numbering. The proof says 'Theorem 1 is a direct corollary of Lemma 6 and Lemma 9', but Lemma 6 already denotes the PINN positive-definiteness lemma in Section 5, and Lemma 9 denotes the PINN recursion in Section 5, neither of which is the stated result used here. In addition, the same number 'Lemma 8' appears twice (Sections 8.7 and 8.8), and the auxiliary references in the appendix (e.g., 'Theorem 3.1 in [17]', 'Theorem 2.1 in [19]') do not align with the main reference list entries. The mathematical arguments may be repairable, but the appendix needs a complete renumbering and cross-reference cleanup before the claims can be checked.","section":"Appendix, Section 7.4, Proof of Theorem 1"}],"minor_comments":[{"comment":"There are numerous typos and dangling phrases, including 'shallw' (Section 2), 'demenstra ting' (Conclusion), 'Condtion' (Condition 2 in Section 5), and '∈ [n1]' used where n2 or n3 is meant in the PINN Gram-matrix definitions.","section":"Throughout"},{"comment":"Lemma 2 states a width requirement without the log^3(m/δ) factor that appears in Theorem 1; the proof of Theorem 1 later combines additional constraints, but the lemma as stated is weaker than the theorem's condition and should be aligned.","section":"Section 3, Lemma 2 vs Theorem 1"},{"comment":"The remark refers to 'from (11)' when the relevant inequality is equation (9) in the proof sketch of Theorem 1; equation (11) is a later discrete-time definition, so the cross-reference is wrong.","section":"Section 3, Remark 2"},{"comment":"In the displayed requirement for m, the argument of one logarithm is written as log(n δ) without the denominator, and the line 'where (65), (59) and the condition in Lemma 2' uses numbers that do not correspond to the sequentially numbered lemmas of the paper.","section":"Section 4, Corollary 1 proof"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a natural and timely question, and the NTK framework is appropriate. However, I found the appendix's numbering and cross-references to be unreliable enough that I could not fully certify the proof of Theorem 1 without re-deriving it. The zero-sample gap in Lemma 1 and the sign/target inconsistency in Section 5 are substantive but locally fixable. I recommend major revision with careful renumbering and a corrected statement of the PINN dynamics."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Xu, Li and Huang prove NTK-style convergence for gradient descent on wide shallow neural operators in the Chen-Chen branch/trunk form, with a physics-informed variant included. The result is real and the gap it fills is real: nobody had done the NTK analysis for this architecture, and the Kronecker structure of the limiting Gram matrices is the natural way to get there. If you work on operator learning theory, this is worth your time. It is not a new framework; the proofs follow Du et al. and Gao et al. closely, but the extension to product-of-branch-trunk is non-trivial and the width bounds are explicit.\n\nThe main soft spot is not the method but a false lemma. Lemma 1 claims that “no two samples parallel” makes H∞ and H̃∞ strictly positive definite. That is false as stated: take n1=1, u1=0. Then the condition is vacuously true, but the H1∞ factor is [E[σ(0)^2]] = [0], so λ0 = 0 and the rates in Theorems 1–2 are undefined. The same collapse happens with yj=0. The fix is one sentence — require every sample to be nonzero — and the theorem statements then go through, but as written the lemma is wrong and it sits at the base of the whole convergence argument. That is a genuine bug, not a stylistic quibble.\n\nThe presentation is also rougher than it should be. The appendix has duplicate lemma numbers, mislabeled cross-references, and the proof of Theorem 1 cites “Lemma 6, 7, 8, 9” that don't match the main-text numbering. Typos like “shallw” and “Descrete” are cosmetic, but the numbering issue makes verification genuinely harder than it should be.\n\nOne more thing: the paper notes Shrimali et al. [15] already has convergence guarantees for DeepONets and FNOs under RSC. The authors don't spell out what the NTK route adds for the Chen-Chen architecture. A comparison of assumptions and what the different techniques buy would help position the contribution.\n\nNet: the core argument is plausible, the Kronecker-factor analysis is sound, and the gap is fixable. I would send it to review, with a required round of revision for Lemma 1 and the appendix. If the authors fix those, I'd cite it.","headline":"NTK convergence for shallow Chen-Chen operators fills a small gap, but a false positive-definiteness lemma and a scrambled appendix need a fix before this is citable.","tokens_in":46152,"tokens_out":4366,"would_cite":false,"duration_ms":44139,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Wide shallow neural operators are provably trainable to zero error under gradient descent.","keywords":["neural tangent kernel","neural operator","gradient descent","convergence analysis","over-parameterization","shallow neural network","physics-informed neural operator","linear convergence"],"falsifier":"Take a dataset with two identical input functions $u_i=u_j$ but different output labels, train the same shallow operator with a small step size, and track $\\|z-G_t(u)\\|_2^2$; Lemma 1 then fails because $H^\\infty_1$ is singular, and the predicted uniform linear contraction should break down, or the width required to keep it should blow up as the eigenvalue gap goes to zero.","tokens_in":45042,"feed_emoji":"📉","tokens_out":7537,"duration_ms":73445,"temperature":0.7,"pith_summary":"This paper tries to establish a missing piece in the theory of operator learning: a training-error guarantee for a wide shallow neural operator trained by gradient descent. Working in the neural tangent kernel regime, where random initialization and over-parameterization keep every weight near its starting point, the authors prove that continuous-time gradient flow drives the squared training error to zero exponentially fast, and that discrete-time gradient descent converges at a linear rate to a global minimum with high probability. The same conclusion is extended to physics-informed shallow neural operators, where the loss includes PDE residuals at interior points and boundary conditions. If correct, these results say that interpolating the training data is not an obstacle for shallow operator networks once the width is large enough, and that the speed of training is controlled by the smallest eigenvalues of the limiting neural tangent kernel matrices.","feed_headline":"Wide neural operators provably converge under gradient descent","feed_subtitle":"NTK analysis gives linear convergence to zero training error for operator learning and physics-informed variants.","key_machinery":"The load-bearing object is the neural tangent kernel Gram matrix of the model, split into two parts: $H^\\infty=H^\\infty_1\\otimes H^\\infty_2$ coming from the trunk weights and $\\tilde H^\\infty=\\tilde H^\\infty_1\\otimes\\tilde H^\\infty_2$ from the branch weights. The argument needs the smallest eigenvalues $\\lambda_0,\\tilde\\lambda_0$ to be strictly positive, which Lemma 1 obtains from the non-parallelism of the sampled input functions and output points. Around those fixed matrices, the proof controls two perturbations: the Gram matrices at initialization are close to their infinite-width limits, and they stay close to their initial values through training because the weights move at most $O(1/\\sqrt m)$; a Taylor-residual recursion captures the discrete-time update. Together these facts keep the smallest eigenvalues bounded away from zero and turn the loss dynamics into a contraction.","core_discovery":"On the paper's own terms, the central discovery is that the neural tangent kernel analysis of finite-dimensional ReLU networks transfers to shallow operator networks. For the branch–trunk model $G(u)(y)=\\frac{1}{\\sqrt m}\\sum_{r=1}^{m}[\\frac{1}{\\sqrt p}\\sum_{k=1}^{p}\\tilde a_{rk}\\sigma(\\tilde w_{rk}^{\\top}u)]\\sigma(w_r^{\\top}y)$, the infinite-width Gram matrices $H^\\infty$ and $\\tilde H^\\infty$ factor as Kronecker products, and under the condition that no two sampled input functions $u_i$ are parallel and no two sampled output points $y_j$ are parallel, both are strictly positive definite with smallest eigenvalues $\\lambda_0$ and $\\tilde\\lambda_0$. Theorem 2 states that, with high probability, discrete-time gradient descent satisfies $\\|z-G_t(u)\\|_2^2 \\le \\bigl(1-\\eta(\\lambda_0+\\tilde\\lambda_0)/2\\bigr)^{t}\\|z-G_0(u)\\|_2^2$ provided the width $m$ is of order $\\Omega\\bigl(n_1^4 n_2^4 \\log(n_1 n_2/\\delta)\\log^3(m/\\delta)/(\\min(\\lambda_0,\\tilde\\lambda_0)^2(\\lambda_0+\\tilde\\lambda_0)^2)\\bigr)$; Theorem 1 gives the continuous-time bound $\\exp(-(\\lambda_0+\\tilde\\lambda_0)t)$. The same mechanism proves the analogous linear-rate statement for physics-informed training, with width scaling polynomial in the sample count and PDE dimension $d$.","pith_inferences":["The paper leaves open the practical size of the width requirement; because it involves fourth powers of both sample counts and inverse eigenvalue gaps, the implied $m$ may be very large, so whether the NTK regime is reachable for realistic operator-learning datasets is not settled.","The same Kronecker-structure argument suggests a route for other operator architectures, such as encoders that randomly project input functions, as long as their limiting Gram matrices can be shown strictly positive definite.","A natural testable extension is to monitor $\\|H(t)-H(0)\\|_F$ during training; the proof predicts it stays $O(1/\\sqrt m)$, so a numerical check would immediately confirm or contradict the NTK regime."],"forward_implications":["With enough width, gradient descent finds an interpolating model, so the training loss can be driven to zero.","The convergence rate is linear, with contraction factor $1-\\eta(\\lambda_0+\\tilde\\lambda_0)/2$, so the speed is controlled by the smallest NTK eigenvalues, not by the data dimension.","The same guarantee holds in continuous time, with an exponential rate $\\exp(-(\\lambda_0+\\tilde\\lambda_0)t)$, and for physics-informed training of the same architecture on PDE residuals plus boundary data.","The strict positive-definiteness condition quantifies a data-dependent requirement: no two sampled input functions, and no two sampled output locations, may be parallel."],"supporting_citations":[{"why":"Introduces the shallow operator network architecture whose training is analyzed.","marker":"[7]"},{"why":"Supplies the NTK linear-convergence method and the positive-definiteness criterion for ReLU feature maps.","marker":"[17]"},{"why":"Proves that ReLU feature maps on non-parallel inputs are linearly independent, used for strict positive definiteness of the branch Gram matrix.","marker":"[19]"},{"why":"Provides the strict positive-definiteness result for the physics-informed Gram matrix used in Lemma 6.","marker":"[20]"},{"why":"Gives the prior optimization framework for neural operators that this paper contrasts with and extends.","marker":"[15]"}],"fun_headline_variants":["NTK ensures global minima for wide neural operators","Gradient descent provably trains wide operator networks","Wide shallow operators converge linearly via NTK","Global convergence for operator learning with NTK","Linear-rate training guaranteed for wide neural operators"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof hinges on the two infinite-width kernel matrices having strictly positive smallest eigenvalues, which is guaranteed only when no two sampled input functions are parallel and no two sampled output points are parallel; if the data contain parallel or identical samples, the linear-rate conclusion collapses.","fun_headline_variants_meta":{"raw":{"variants":["NTK ensures global minima for wide neural operators","Gradient descent provably trains wide operator networks","Wide shallow operators converge linearly via NTK","Global convergence for operator learning with NTK","Linear-rate training guaranteed for wide neural operators"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000177,"raw_usage":{"total_tokens":1356,"prompt_tokens":1070,"completion_tokens":286,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":217}},"tokens_in":686,"tokens_out":286,"duration_ms":3640,"temperature":1.0,"reasoning_tokens":217,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:37:23.524995+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset with two identical input functions $u_i=u_j$ but different output labels, train the same shallow operator with a small step size, and track $\\|z-G_t(u)\\|_2^2$; Lemma 1 then fails because $H^\\infty_1$ is singular, and the predicted uniform linear contraction should break down, or the width required to keep it should blow up as the eigenvalue gap goes to zero.","supporting_citations":[{"cited_title":"Universal approximation to nonlinear oper ators by neural net- works with arbitrary activation functions and its application to dyna mical systems,","cited_arxiv_id":null,"evidence_quote":"Introduces the shallow operator network architecture whose training is analyzed."},{"cited_title":"Gradient descent finds the global op tima of two-layer physics- informed neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides the strict positive-definiteness result for the physics-informed Gram matrix used in Lemma 6."},{"cited_title":"Optimizatio n for neural operator learn- ing: Wider networks are better","cited_arxiv_id":null,"evidence_quote":"Gives the prior optimization framework for neural operators that this paper contrasts with and extends."}],"review_version":1}