{"id":"9bb55ef4-592f-4981-ac5f-5159c415da16","arxiv_id":"2507.21429","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Local NTK positivity plus Lipschitz stability gives a local Polyak-Lojasiewicz constant lambda0 minus L_Theta times the region radius, yielding linear gradient descent convergence whenever the iterates stay in the LQCR; experiments track these diagnostics on MNIST and CIFAR-10.","lead":"The paper proves a sufficient condition for gradient descent on finite-width networks to converge linearly: the squared loss satisfies a local Polyak-Lojasiewicz inequality whenever the empirical neural tangent kernel is positive at initialization and Lipschitz stable over a local region. It then shows the rate applies if the iterates remain in that region, and probes the assumptions with NTK and PL-ratio diagnostics on MNIST and CIFAR-10 subsets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed-step iterate containment is assumed rather than derived, so Theorem 5.7 does not realize the advertised sublinear-to-linear transition in any proven GD setting; the theorem is a correct conditional result, not an unconditional local-linear-convergence guarantee.","rationale":"The reader's weakest assumption matches the most load-bearing concern I can identify: fixed-step iterate containment is the hinge of the advertised linear-rate result. I checked the proof chain and found no algebraic error; the local PL inequality is valid, the linear-rate derivation is standard, and the paper is admirably explicit about the conditional nature of the result. But the title and abstract promise a transition from the known sublinear LQCR guarantee to linear local convergence, and that transition requires the same fixed-step GD procedure to be provably confined to R. The imported LQCR confinement applies to a decaying step-size schedule, so the main hypothesis of Theorem 5.7 is not connected to the framework's own containment theorem. This is a scoping and applicability concern rather than an internal inconsistency. The paper deserves credit for flagging the containment assumption and for proposing a latent-variable empirical methodology, but the empirical protocol does not certify the missing hypothesis: it uses a subset NTK proxy, never measures r(R) or L_Theta, and even exhibits a run in which containment fails. Given the authors' own acknowledgment, the reader's CONDITIONAL verdict is appropriate, and my stress-test does not move it.","tokens_in":18525,"tokens_out":8971,"duration_ms":119800,"concrete_test":"On the width-128, depth-5 Softplus MLP of Section 6.1, reconstruct the LQCR radius r(R) from the corrected construction in Aich et al. (2025), estimate L_Theta on R by finite differences, and run fixed-step GD with eta = 1/L for 250 epochs. Record max_t ||theta(t) - theta0|| and check whether it is at most r(R) and whether L_Theta r(R) < lambda_min(Theta_theta0). If either check fails, Theorem 5.7 is not instantiated in the very regime used to claim the sublinear-to-linear transition; if both hold, repeat the same check at width 1024 with eta = 1/L, where the paper's own diagnostics indicate the containment hypothesis is violated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The conditional mathematics is sound: Lemma 5.3 and Theorem 5.4 follow from Weyl's inequality plus the squared-loss gradient identity, and Theorem 5.7 is the standard PL descent argument. The load-bearing gap is that Theorem 5.7's hypothesis that the iterates remain in R for every t is never derived, and the only containment result imported from Aich et al. (2025) uses a decaying step schedule eta_t = eta0/(1+t)^(1/3), not fixed-step GD. The paper therefore does not convert the prior sublinear-guarantee setting into a linear one: no fixed-step containment lemma links the LQCR radius r(R) to the iterate drift, and no example certifies that r(R), L_Theta, and lambda0 simultaneously satisfy L_Theta r(R) < lambda0 together with fixed-step containment. The experiments cannot substitute for this missing certification: they use a subset NTK, so the measured lambda0 is not the theorem's lambda_min(Theta_theta0); they never compute r(R) or L_Theta; and the width-1024, eta=10^-3 run is an explicit instance where the containment hypothesis fails. Thus the central advertised result is a correct theorem about a condition that the paper neither proves nor certifies in its own experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies local linear convergence of full-batch gradient descent for finite-width feedforward networks under squared loss. Its main theoretical result is that if the empirical NTK is positive at initialization, Lipschitz stable on a Locally Quasi-Convex Region of radius r(R), and the compatibility condition L_Theta r(R) < lambda_0 holds, then the squared loss satisfies a local Polyak-Lojasiewicz inequality on that region with constant mu = lambda_0 - L_Theta r(R). A second theorem states that if fixed-step gradient descent iterates remain inside the region and the step size satisfies eta <= 1/L, the loss decreases linearly to the regional minimum. The paper then probes the theory empirically on binary MNIST with an MLP, a width ablation, and a CNN robustness check, measuring the subset NTK eigenvalue, parameter drift, empirical PL ratio, and suboptimality decay. The authors are explicit that the fixed-step containment is an assumption rather than a derived property.","tokens_in":18765,"tokens_out":5004,"duration_ms":62792,"significance":"The conditional mathematics is sound and clearly presented: Lemma 5.3 correctly propagates pointwise NTK positivity via Weyl's inequality, Theorem 5.4 correctly converts the spectral lower bound into a PL inequality using the squared-loss gradient identity, and Theorem 5.7 is the standard PL descent argument. The paper also deserves credit for explicitly separating the roles of the LQCR, the NTK stability assumption, and the containment hypothesis, and for designing diagnostics that probe the latent variables of the theory rather than only final loss curves. If the containment hypothesis could be derived or certified in a concrete setting, the result would be a clean finite-width local linear-convergence guarantee. As it stands, however, the main advertised transition from a sublinear LQCR guarantee to a linear rate is not established in any provable setting because the fixed-step containment is assumed, and the experiments do not verify the full assumptions of the theorem. The significance is therefore conditional and the central claim is narrower than the title suggests.","major_comments":[{"comment":"The advertised transition from sublinear to linear convergence is not realized in any proven setting. Theorem 5.7 assumes the iterates remain in R for every t, but the only containment result imported from Aich et al. (2025) uses the decaying step-size schedule eta_t = eta0/(1+t)^(1/3), not fixed-step gradient descent. The paper therefore does not show that the prior sublinear-guarantee setting becomes linear under the same provable containment mechanism. This gap is load-bearing because the title and Contribution 1 frame the result as 'from sublinear to linear.' Please either prove a fixed-step containment theorem under explicit conditions, or explicitly reframe the contribution as a conditional local-PL theorem with the sublinear-to-linear transition as an open problem.","section":"Section 5.3, Theorem 5.7, and Abstract"},{"comment":"The empirical spectral quantity reported as lambda_0 is not the quantity used in Assumption 5.1. The theorem concerns lambda_min(Theta_theta0) for the full empirical NTK Theta_theta = (1/n) J_theta J_theta^T over all n training points, while the experiments report lambda_min on a fixed subset of n_sub = 100 inputs. Positivity of the subset eigenvalue does not certify positivity of the full NTK eigenvalue, and L_Theta and r(R) are never computed, so the compatibility condition L_Theta r(R) < lambda_0 is never verified. The empirical section should either compute the full NTK eigenvalue, provide a rigorous argument that the subset eigenvalue bounds the full one, or be presented as evidence of plausibility rather than confirmation of Assumption 5.1.","section":"Sections 6.1 and 6.2, Table 1"},{"comment":"The parameter-drift measurements do not establish the containment used in Theorem 5.7. A final drift of about 0.17 in a space of roughly 10^6 parameters is not informative without a value for r(R), and Theorem 5.7 requires theta(t) in R for every t, not merely a small final drift. The width-1024, eta = 10^-3 run is an explicit violation of the containment hypothesis (drift 1.870), so its use as a 'failure mode' is honest but should be described as an assumption violation rather than as an empirical test of the theorem. I recommend either measuring or estimating r(R) and L_Theta along the trajectory, or weakening the empirical claims to say that the diagnostics are consistent with the assumptions on the runs where the assumptions appear to hold.","section":"Section 6.1 and Section 6.2"}],"minor_comments":[{"comment":"The symbol L is used both for the number of layers in the width condition m = Omega(L^3) in Section 2.3 and for the smoothness constant in Theorem 5.7, while D denotes depth in Section 3. Please disambiguate these uses.","section":"Section 3 and Related Work"},{"comment":"Remark 4.3 refers to 'the submitted corrigendum' to Aich et al. (2025) without a citation or bibliographic entry; please either provide the reference or remove the pointer.","section":"Remark 4.3"},{"comment":"The Lipschitz condition on Theta_theta uses a matrix norm, but the norm is not specified. Since the proof of Lemma 5.3 uses Weyl's inequality for the spectral norm, please state explicitly that the operator/spectral norm is used throughout.","section":"Assumption 5.1 and Lemma 5.3"},{"comment":"The 'linear-rate fit' reports a contraction factor of 0.9851 on [20, 200], which is very close to 1 and therefore represents a shallow exponential rate. Please report the fit uncertainty and show how sensitive the rate is to the choice of the window endpoints.","section":"Figure 1d"},{"comment":"The step-size condition eta <= 1/L is never related to the LQCR curvature constant alpha or to the LQCR step-size restriction eta0 <= 1/alpha, so it is unclear whether the fixed-step and decaying-step regimes can overlap for the same region. A short remark connecting these constants would help the reader assess the practical plausibility of the fixed-step containment hypothesis.","section":"Theorem 5.7"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is unusually transparent about its conditional status, which is to its credit. The core issue is that the title and abstract promise a sublinear-to-linear transition, while the main theorem assumes the very containment that would make the transition meaningful. I would be willing to accept a revised version that either supplies a fixed-step containment result in a nontrivial setting or honestly reframes the contribution as a conditional local-PL analysis with an empirical diagnostic methodology. As written, the central claim overreaches the proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper gives a clean, short proof that pointwise NTK positivity at initialization plus Lipschitz stability on an LQCR yields a local PL inequality with explicit constant λ0 − LΘ r(R), and that fixed-step GD converges linearly on that region if the iterates stay inside. The math is correct, and the exposition is unusually candid about what is assumed. But the advertised “from sublinear to linear” transition is not actually realized in any proven setting: fixed-step containment is never derived, and the only available containment result from the LQCR prior work uses a decaying step schedule.\n\nWhat is genuinely new: the explicit coupling of the PL constant to the LQCR radius r(R) is not present in the cited NTK or PL papers, and the empirical diagnostic style — tracking the NTK spectral gap, parameter drift, and PL ratio rather than fitting loss-curve slopes — is a real improvement in how local-condition theory gets tested. Credit where earned: Lemma 5.3 and Theorems 5.4 and 5.7 are correct, and the paper repeatedly flags the terminal-loss proxy, the subset-NTK caveat, and its own scope limitations. That is more honest than the field standard.\n\nThe soft spot is structural, not a proof error. Theorem 5.7’s containment hypothesis does all the work. To claim a sublinear-to-linear transition you would need to derive containment for fixed-step GD, or at least certify LΘ r(R) < λ0 and drift bounds in the experiments. The experiments do neither: they use a subset NTK, never compute LΘ or r(R), and the width-1024 failure is explained post hoc. The stable-regime endpoints [20,200] and the reduced step size are adaptive choices, so the empirical confirmation is weaker than the prose suggests. Still, all of this is disclosed rather than buried, which counts for something.\n\nWho this is for: researchers working on finite-width optimization theory who want a crisp example of local NTK conditioning implying a PL inequality, and empiricists who want a template for testing such conditions directly. It is not a major breakthrough, but it is a solid conditional result plus a useful methodological contribution.\n\nRecommendation: send it to peer review, expecting the authors to either prove a fixed-step containment lemma (even under stronger assumptions) or reframe the paper as a conditional PL lemma with diagnostic tools, dropping the “sublinear-to-linear” framing. As is, it is correct but over-promised; with that reframing it would be a clean, publishable paper.","headline":"Correct and admirably honest, but the headline claim is a conditional statement whose key hypothesis (fixed-step containment) is assumed, not proven, so the advertised sublinear-to-linear transition is not actually demonstrated.","tokens_in":19378,"tokens_out":2429,"would_cite":false,"duration_ms":31947,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","90C26","65K05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Local NTK conditioning makes finite-width GD converge linearly on a local region","keywords":["local Polyak-Lojasiewicz inequality","neural tangent kernel","finite-width neural networks","linear convergence","gradient descent","locally quasi-convex region","squared empirical loss","NTK spectral gap"],"falsifier":"Run full-batch GD on a smooth finite-width network with $\\lambda_{\\min}(\\Theta_{\\theta_0})>0$ and $L_\\Theta r(R)<\\lambda_0$ verified, and measure the empirical PL ratio $\\|\\nabla L(\\theta)\\|^2 / [2(L(\\theta)-L_R^*)]$ at iterates inside $R$; a single interior iterate where the ratio falls to zero would contradict Lemma 5.3. Alternatively, a trajectory that starts inside the region, keeps a positive NTK gap, and shows non-geometric loss decay under $\\eta \\le 1/L$ would falsify Theorem 5.7.","tokens_in":18232,"feed_emoji":"📉","tokens_out":7408,"duration_ms":68728,"temperature":0.7,"pith_summary":"This paper proves that, inside a bounded local region around initialization, a finite-width feedforward network trained by gradient descent on squared loss converges linearly, provided the empirical neural tangent kernel starts with a positive smallest eigenvalue, does not drift too much across the region, and the region is small enough relative to that spectral gap. The region is the Locally Quasi-Convex Region (LQCR) from prior work, and the result upgrades the sublinear guarantee that came with it. The paper shows that the squared loss satisfies a local Polyak-Łojasiewicz inequality with constant $\\mu = \\lambda_0 - L_\\Theta r(R) > 0$ on that region, and that fixed-step GD then converges at rate $(1-\\eta\\lambda_R)^t$ whenever the iterates stay inside. A sympathetic reading would emphasize that this is an explicit sufficient local condition, not a claim that the mechanism is necessary or unique.","feed_headline":"Local NTK gap shifts GD from sublinear to linear rates","feed_subtitle":"A pointwise kernel condition yields a local PL inequality that upgrades the sublinear LQCR guarantee.","key_machinery":"The carrying object is the empirical neural tangent kernel matrix $\\Theta_\\theta = \\frac{1}{n} J_\\theta J_\\theta^\\top$ on the training inputs. Its smallest eigenvalue at initialization, $\\lambda_0$, is the spectral budget; its Lipschitz constant $L_\\Theta$ over the LQCR sets the erosion rate; the LQCR radius $r(R)$ is the maximum distance over which the budget must hold; and the compatibility condition $L_\\Theta r(R) < \\lambda_0$ keeps the budget positive. The squared-loss identity relating gradient norm to the kernel-weighted residual turns that spectral lower bound into the local PL inequality.","core_discovery":"The central claim is that local NTK conditioning yields a local PL inequality on the LQCR: pointwise positivity at the initialization point, Lipschitz stability of the empirical kernel over the region, and compatibility $L_\\Theta r(R) < \\lambda_0$ together imply that $R$ is a $(\\mu,R)$-Locally Polyak-Łojasiewicz Region with $\\mu = \\lambda_0 - L_\\Theta r(R)$ for the squared empirical loss. The proof runs through the identity $\\|\\nabla L(\\theta)\\|^2 = \\frac{1}{n}(f_\\theta-y)^\\top \\Theta_\\theta (f_\\theta-y)$ and Weyl's inequality, which propagates the initial spectral gap across the region. With $L$-smoothness, step size $\\eta \\le 1/L$, and the hypothesis that fixed-step iterates stay in $R$, the loss approaches the region minimum $L_R^*$ at the linear rate $(1-\\eta\\lambda_R)^t$. The paper is explicit that this is a sufficient-condition result, scoped to squared loss and to the local regime where iterates remain inside the region.","pith_inferences":["Because fixed-step containment is assumed rather than derived, the paper does not yet close the gap to the original LQCR setting, whose provable containment uses a decaying step size; a direct linear-rate proof under that decaying schedule would complete the sublinear-to-linear transition in the same provable setting.","The empirical PL-ratio lower envelope could serve as a practical monitor: tracking whether the ratio stays bounded away from zero, along with parameter drift and NTK eigenvalue, tells a practitioner whether a run is inside a locally PL regime.","The squared-loss bridge suggests a direct extension target: any loss whose gradient admits a similar Jacobian-residual factorization, for example losses with a linear output layer, may inherit a local PL inequality from the same spectral argument.","The width-1024 boundary case hints that step size, not width alone, controls whether the local regime is entered; a testable prediction is that adaptive schedules chosen to keep NTK drift below the initial gap should preserve linear-rate decay at larger widths."],"forward_implications":["On any LQCR satisfying the NTK conditioning assumption, fixed-step GD with $\\eta \\le 1/L$ converges linearly to $L_R^*$ at rate $(1-\\eta\\lambda_R)^t$ while iterates remain in the region.","The PL constant is explicit and measurable: $\\lambda_0$ and $L_\\Theta$ can be estimated, and $r(R)$ is given by the LQCR construction.","The result transfers the classical NTK linear-convergence mechanism from overparameterized or infinite-width settings to a finite-width region with explicit radius.","When iterates leave the region, the linear-rate guarantee lapses; the width ablation shows this empirically at width 1024 with $\\eta=10^{-3}$."],"supporting_citations":[{"why":"Supplies the LQCR definition, its explicit radius $r(R)$, and the containment guarantee under a decaying step-size schedule; this is the region on which the PL inequality is built.","marker":"(Aich et al., 2025)"},{"why":"Defines the neural tangent kernel and its link between kernel conditioning and squared-loss training dynamics.","marker":"(Jacot et al., 2018)"},{"why":"Establishes linear convergence of GD for overparameterized two-layer ReLU networks via Gram-matrix eigenvalue conditioning, the mechanism this paper localizes.","marker":"(Du et al., 2019b)"},{"why":"Provides the overparameterized multi-layer convergence theory whose global conditioning the paper replaces by a local pointwise-plus-Lipschitz condition.","marker":"(Allen-Zhu et al., 2019)"},{"why":"Gives geometric-rate convergence for moderately overparameterized shallow networks, a benchmark for the local finite-width statement.","marker":"(Oymak & Soltanolkotabi, 2020)"},{"why":"Shows that PL-type inequalities imply linear convergence of gradient methods, the template for Theorem 5.7.","marker":"(Karimi et al., 2016)"},{"why":"Supports the Lipschitz-stability and linearized-dynamics regularity used to propagate NTK positivity across the region.","marker":"(Lee et al., 2019)"},{"why":"Relates tangent-kernel spectrum to PL conditions in overparameterized nonlinear systems, the closest existing PL-derivation result.","marker":"(Liu et al., 2022)"}],"fun_headline_variants":["NTK gap flips GD from sublinear to linear","Local PL inequality from NTK yields linear GD","Finite-width GD goes linear via local NTK","Kernel condition upgrades GD rate to linear","Local NTK positivity gives linear convergence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fixed-step gradient descent iterates stay inside the local region $R$ for every step — the paper assumes this containment rather than proving it — and the region's radius and width condition are imported from prior work without being re-derived.","fun_headline_variants_meta":{"raw":{"variants":["NTK gap flips GD from sublinear to linear","Local PL inequality from NTK yields linear GD","Finite-width GD goes linear via local NTK","Kernel condition upgrades GD rate to linear","Local NTK positivity gives linear convergence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000218,"raw_usage":{"total_tokens":1530,"prompt_tokens":1129,"completion_tokens":401,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":745,"completion_tokens_details":{"reasoning_tokens":330}},"tokens_in":745,"tokens_out":401,"duration_ms":4993,"temperature":1.0,"reasoning_tokens":330,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:48:04.220202+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run full-batch GD on a smooth finite-width network with $\\lambda_{\\min}(\\Theta_{\\theta_0})>0$ and $L_\\Theta r(R)<\\lambda_0$ verified, and measure the empirical PL ratio $\\|\\nabla L(\\theta)\\|^2 / [2(L(\\theta)-L_R^*)]$ at iterates inside $R$; a single interior iterate where the ratio falls to zero would contradict Lemma 5.3. Alternatively, a trajectory that starts inside the region, keeps a positive NTK gap, and shows non-geometric loss decay under $\\eta \\le 1/L$ would falsify Theorem 5.7.","supporting_citations":[],"review_version":1}