{"id":"5a93b204-26e8-44ac-9cab-e1922f3c21ae","arxiv_id":"2607.10869","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Constant and linear Hermite components of a misspecified single-index target are recovered at the conjectured singular-perturbation timescales; quadratic learning remains coupled to them via an auxiliary constrained flow.","lead":"The paper proves that constant and linear parts of a hidden link function are recovered at sharp, explicit timescales in the singularly perturbed population gradient flow of an infinite-width two-layer network. It further shows that those earlier components continue to shape the onset of quadratic learning, with singular rearrangement of the weight measure.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The reader correctly isolates the two modelling hypotheses that delimit the scope of the results. Those hypotheses are not concealed; once they are accepted, the singular-perturbation machinery (tubes around the constant/linear idealized systems, then the constrained flow near the quadratic manifold) is carefully executed and yields the claimed sharp timescales and the non-negligibility of previously learned components. The absence of code and the lack of uniform-in-time propagation-of-chaos bounds are limitations of presentation and of the mean-field reduction, respectively, but they do not undermine the internal correctness of the ODE analysis that constitutes the paper’s contribution. Consequently the ACCEPT verdict with low correctness risk stands.","tokens_in":40336,"tokens_out":459,"duration_ms":6352,"concrete_test":"Independently re-derive the differential inequality for Z = ∥a⊥-b∥ + ε^{-1/2}∥s⊥-g∥ on [0,t_ε] (the argument leading to (7.45)–(7.56) in the proof of Thm. 7.1) without using the a-priori bound β(t) ≤ D ε^{-1/4}; if the same ε^{1/4-cD} factor is recovered, the tube radius remains valid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claims (Props. 3.1–3.2, Thm. 3.3, Prop. 3.6) are proved under the stated assumptions and the mean-field reduction taken from Berthier–Montanari–Zhou. The positivity of all Hermite coefficients and the validity of the autonomous ODEs (2.8)–(2.9) on the relevant timescales are genuine modelling hypotheses, but they are declared explicitly (Assumption 1 and §2) and do not create an internal inconsistency or circularity in the singular-perturbation analysis. The tube constructions, the constrained auxiliary flow that preserves the already-learned integral constraints, and the quantitative error bounds are self-contained once those hypotheses are granted. No hidden gap that would invalidate the sharp thresholds or the quadratic-onset description was found.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper rigorously analyzes the population gradient flow of an infinitely wide two-layer network learning a misspecified single-index model, with a small parameter ε controlling the relative speed of the second layer. Building on the mean-field reduction of Berthier–Montanari–Zhou, it proves that the constant and linear Hermite components of the link are recovered on the predicted timescales (Propositions 3.1–3.2, Theorem 3.3) via quantitative tubes around explicitly integrable idealized ODEs. For the quadratic component it constructs an auxiliary constrained flow that exactly preserves the already-learned integral constraints (Proposition 3.6, Appendix B) and shows that earlier components continue to influence the dynamics; simulations illustrate the associated singular rearrangement of the empirical measure of the weights.","tokens_in":40624,"tokens_out":933,"duration_ms":10550,"significance":"The work supplies the first fully rigorous confirmation of the hierarchical timescales conjectured in Berthier–Montanari–Zhou for the constant and linear components, together with a general approximation lemma for singularly perturbed flows near integral-constraint manifolds that is of independent interest. The demonstration that previously learned coefficients cannot be neglected at the quadratic stage, and the accompanying phenomenological description of singular weight-measure behaviour, clarify the structure of joint two-layer training beyond the kernel regime. The proofs are self-contained once the mean-field ODEs and positivity of Hermite coefficients are granted, and the tube constructions and Gronwall estimates are sharp enough to recover the leading constants.","major_comments":[{"comment":"The quadratic analysis (Section 3.2, equations (3.11)–(3.12) and Proposition 3.6) is performed on a truncated system that retains only the first three Hermite terms. While the truncation is declared, the paper does not quantify the error incurred by discarding higher-order terms on the ε^{1/4} timescale; a short a-priori bound showing that those terms remain negligible under the same tube radii would make the claim that earlier components continue to influence the dynamics fully rigorous for the original infinite series.","section":null},{"comment":"The reduction of the high-dimensional finite-width dynamics to the autonomous mean-field ODEs (2.8)–(2.9) is taken as given from Berthier–Montanari–Zhou. The related-work discussion (Section 4) correctly notes that existing propagation-of-chaos bounds are insufficient on the relevant timescales; a brief remark on the precise regime (m,d versus 1/ε) in which the present conclusions transfer to the original particle system would strengthen the modelling claim.","section":null}],"minor_comments":[{"comment":"Assumption 1 requires σ_k > 0 and φ_k > 0 for every k; a short discussion of what happens when a low-order coefficient vanishes (or a pointer to the corresponding open question) would help the reader assess robustness.","section":null},{"comment":"Figures 1–6 are informative but the captions could more explicitly state the precise truncation used in each panel and the numerical values of the constants C, δ appearing in the theorems.","section":null},{"comment":"The notation for the idealized linear system (β,γ versus b,g) is introduced in two places (Section 3 and Appendix A); a single consistent definition would improve readability.","section":null},{"comment":"Typographical inconsistencies appear in the arXiv header (date July 14, 2026) and in a few places where “ε” is written as “e”; these should be cleaned before final publication.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is a solid, technically careful contribution that largely settles the constant/linear part of the BMZ conjecture and introduces a useful general tool (Appendix B). The two major comments are genuine but local; either can be addressed by a short additional paragraph or lemma without altering the main theorems. I see no reason for a more severe recommendation."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This paper does what it claims: it turns the Berthier–Montanari–Zhou matched-asymptotics conjectures for the constant and linear Hermite components into theorems with sharp thresholds (Props. 3.1–3.2, Thm. 3.3), then shows that the quadratic onset cannot ignore those earlier components. The auxiliary flow (z1,z2) with time-dependent multipliers that exactly preserve the already-learned integral constraints is the right object, and Prop. B.1 / Cor. B.2 give a clean, reusable approximation lemma for singularly perturbed mean-field flows near manifolds defined by integral constraints.\n\nWhat works well is the technical core. The tube constructions around the idealized constant/linear solutions, the Gronwall estimates, and the explicit error radii that adapt to the hierarchical timescales are careful and self-contained once you grant the mean-field ODEs (2.8)–(2.9). The simulations (Figs. 1–6) line up with the predicted separation and with the singular rearrangement of the empirical measure. No free parameters are fitted; the idealized systems are obtained by retaining leading terms or enforcing constraints.\n\nSoft spots are real but declared. Everything rests on Assumption 1 (all Hermite coefficients strictly positive) and on the high-dimensional finite-width reduction taken from BMZ. If a low-order coefficient vanishes or the mean-field limit fails on the √ε log(1/ε) and ε1/4 windows, the sharp thresholds and the constrained auxiliary dynamics stop applying. The quadratic analysis also truncates the Hermite series. These are modelling hypotheses, not circularities or internal contradictions; the stress-test is right that no hidden gap invalidates the claims under the stated assumptions. Absence of code is a minor practical inconvenience, not a soundness issue.\n\nThis is for people working on high-dimensional mean-field learning theory and singular-perturbation methods for neural nets. It is not a broad empirical paper. The math is solid enough that a serious editor should send it to referees; I would bring it to reading group and expect to cite the constant/linear thresholds and Prop. B.1.","headline":"Solid rigorous proofs of the BMZ constant/linear timescales plus a reusable constrained-flow approximation for the quadratic onset; soft spots are declared modelling assumptions, not hidden gaps.","tokens_in":41169,"tokens_out":542,"would_cite":true,"duration_ms":7564,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","34E15","37N40"],"pacs":[],"model":"grok-4.5","headline":"Joint two-layer training recovers link-function components on sharp, separated timescales set by a speed-separation parameter.","keywords":["hierarchical learning","two-layer neural networks","singular perturbation","mean-field limit","single-index model","gradient flow","Hermite expansion"],"falsifier":"Numerically integrate the mean-field ODEs for small ε and check whether the hitting times for |φ_{0}-σ_{0}a|~ε and for as reaching (1-α)φ_{1}/σ_{1} match the explicit constants 1/σ_{0}^{2} and 1/(4σ_{1}φ_{1}) predicted by Propositions 3.1–3.2; a systematic mismatch for positive coefficients would refute the claimed thresholds.","tokens_in":41232,"feed_emoji":"⏱️","tokens_out":753,"duration_ms":8155,"temperature":0.7,"pith_summary":"When an infinitely wide two-layer network is trained jointly on a high-dimensional single-index target whose activation is unknown, and the outer layer is forced to move much faster than the inner layer, the network recovers the constant, linear, and then quadratic pieces of the hidden link on successively longer timescales. The paper proves the first two of those timescales exactly, matching earlier non-rigorous predictions, and shows that once those pieces are learned they keep shaping every later stage through integral constraints that the dynamics must preserve. The same analysis reveals that the weight measure becomes singular: a few neurons grow rapidly while the rest rearrange to keep the already-learned coefficients intact. A sympathetic reader cares because the result supplies the first rigorous, quantitative description of hierarchical feature learning under ordinary joint gradient flow, without layer-wise oracles.","feed_headline":"Neural nets learn link pieces on sharp, separated timescales","feed_subtitle":"Joint training recovers constant then linear terms exactly as predicted; earlier pieces keep shaping later stages","key_machinery":"Quantitative tube approximations for singularly perturbed infinite-dimensional ODEs that remain near a manifold defined by integral constraints (the already-learned Hermite coefficients); the radii of the tubes adapt to the hierarchical timescales and yield the sharp thresholds.","core_discovery":"Under a small speed-separation parameter ε the population gradient flow of the infinite-width two-layer network recovers the constant Hermite coefficient of the link to precision nearly ε on a timescale of order ε log(1/ε), reaches a non-trivial fraction of the linear coefficient by time (1/(4σ_{1}φ_{1}))√(ε log(1/ε)), and thereafter stays close to an auxiliary constrained flow that exactly preserves the already-learned integral constraints while the quadratic component begins to grow.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Two-layer nets recover constant then linear link terms on sharp ε-scales","Hierarchical learning: constants first, linears next, with lingering influence","Singularly perturbed flows hit Hermite pieces at explicit separated times","Earlier link components reshape dynamics as quadratic onset begins","Infinite-width nets learn constant and linear coeffs at predicted thresholds"],"cache_read_input_tokens":32896,"weakest_assumption_plain":"Every Hermite coefficient of both the activation and the link is strictly positive, and the high-dimensional finite-width dynamics has already been reduced to the autonomous mean-field ODEs that the paper studies.","fun_headline_variants_meta":{"raw":{"variants":["Two-layer nets recover constant then linear link terms on sharp ε-scales","Hierarchical learning: constants first, linears next, with lingering influence","Singularly perturbed flows hit Hermite pieces at explicit separated times","Earlier link components reshape dynamics as quadratic onset begins","Infinite-width nets learn constant and linear coeffs at predicted thresholds"]},"model":"grok-4.5","effort":"low","cost_usd":0.005346,"raw_usage":{"total_tokens":1441,"prompt_tokens":777,"num_sources_used":0,"completion_tokens":89,"cost_in_usd_ticks":53460000,"prompt_tokens_details":{"text_tokens":777,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":575,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":777,"tokens_out":89,"duration_ms":6379,"temperature":1.0,"reasoning_tokens":575,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T08:39:36.464402+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Numerically integrate the mean-field ODEs for small ε and check whether the hitting times for |φ_{0}-σ_{0}a|~ε and for as reaching (1-α)φ_{1}/σ_{1} match the explicit constants 1/σ_{0}^{2} and 1/(4σ_{1}φ_{1}) predicted by Propositions 3.1–3.2; a systematic mismatch for positive coefficients would refute the claimed thresholds.","supporting_citations":[],"review_version":1}