{"id":"275fe337-9516-4c3f-a376-cae570755a88","arxiv_id":"2505.21336","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In Gaussian single-index models, joint gradient flow over direction and link function converges to the true regression function from either sign of initial alignment, with rate governed by the information exponent.","lead":"The paper proves that a joint gradient flow over both the projection direction and a nonparametric function can recover a Gaussian single-index model even when the initial direction points the wrong way, with convergence time that grows polynomially in dimension. This matters because it shows joint learning can escape bad basins that defeat fixed-function ('planted') models, and it gives a practical Hermite-kernel algorithm.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proof of Theorem 1 relies on unquantified 'choose k* large' tail bounds and explicitly defers the negative-initialization half of Lemma 4, leaving a genuine rigor gap in the central claim, although the missing pieces appear fillable from the H^1 assumptions and a sign-flip symmetry.","rationale":"The reader identified the same weakest spot: the proof's reliance on uniform tail smallness and the deferred negative-m0 case. My read of the full appendix confirms that these are the places where the central theorem could fail if the asserted uniformity is false. I checked whether the tail estimate is actually derivable from the paper's own assumptions: Fact 1 bounds each |a_{k,t}| by max(|a_{k,0}|, |a*_k|), and the assumed phi* in H^1 together with the stated initialization (which yields sum k^2 a_{k,0}^2 < infinity) make the tail sum sum_{k>k*} k |a_{k,t} a*_k| uniformly small in t. So the concern is not a demonstrated counterexample but a proof-completeness gap. Similarly, the negative-m0 branch appears to follow by the exact symmetry of the ODE system, which maps m0<0 to m0>0 while flipping every coefficient by (-1)^k; this is a short argument but it is absent from the manuscript. Given that the central theorem is the paper's main contribution and its proof contains these omitted justifications, a conditional verdict is appropriate until the authors supply the quantitative tail bound and the missing branch. I do not see evidence that the central mathematical claim is false, and the paper has independent support in the careful slow-fast decomposition and the numerical agreement with the predicted phases, so a full rejection is not warranted.","tokens_in":20503,"tokens_out":25696,"duration_ms":285530,"concrete_test":"Complete the proof of Theorem 1 by (i) writing the explicit uniform tail bound: for every delta > 0 there exists k*(delta), independent of t and d, such that sup_{t>=0} sum_{k>k*} k |a_{k,t} a*_k| <= delta, using Fact 1 and the H^1 assumptions on phi* and the initialization; and (ii) deriving the negative-m0 case of Lemma 4 from the positive-m0 case via the sign-flip symmetry (m, a_k) -> (-m, (-1)^k a_k), checking that the initialization bounds are preserved. If either step produces a condition beyond H^1 (for example, a need for sum k^2 |a*_k| < infinity or an unsymmetrized initialization), Theorem 1's proof needs substantial revision.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim rests on two under-supported steps in the appendix. First, Lemma 4 states its negative-m0 case is 'very similar and is left to the reader,' yet the m0<0 branch is exactly one of the two main conclusions of Theorem 1. Second, in Lemmas 4-7 and in the contraction proof of Theorem 2, the argument repeatedly asserts that tail sums over k>k*, such as sum_{k>k*} k |a_{k,t} a*_k m_t^{k-1}|, can be 'made arbitrarily small by choosing a large k*,' and that the first k* terms give a uniform positive lower bound 2c. No quantitative uniform-in-time tail estimate is displayed, and the lower bound must hold for all t >= tau_c, not merely at one time. These assertions are load-bearing because they drive the positivity principle (m and the relevant a_k never change sign after the transient) and the exponential contraction toward +/-w*. The gap is likely repairable: Fact 1 gives |a_{k,t}| <= max(|a_{k,0}|, |a*_k|), and phi* in H^1 plus the stated initialization imply sum_{k>k*} k (|a*_k|^2 + |a_{k,0}|^2) is uniformly small for large k*, so the tail is controlled uniformly in t. The omitted m0<0 case can be closed by the symmetry (m, a_k) -> (-m, (-1)^k a_k), which leaves the ODE system (8)-(9) invariant. But as written, the proof does not supply these bounds or this argument, so Theorem 1 is not yet fully rigorous.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes continuous-time gradient flow for jointly learning a direction w on the sphere and a univariate link function f in the Gaussian single-index model y = φ*(<w*,x>) + ε, with population squared loss. Working in the Hermite basis, the authors reduce the infinite-dimensional flow to the coupled ODEs (8)-(9) for the correlation m_t = <w_t,w*> and the coefficients a_{k,t}. The main theorem (Theorem 1) claims that after a transient of order d^{s-1} for s ≥ 2, where s is the information exponent, the flow converges exponentially to (w*, φ*) when m_0 > 0 and to (-w*, φ*(-·)) when m_0 < 0, so that the learned predictor always recovers φ*(<w*,x>); for s = 1 the convergence is claimed to hold immediately. The proof strategy is a fast-slow analysis: coefficients a_{k,t} first track a*_k m_t^k, a positivity principle freezes the sign of m_t and of the relevant coefficients, and then exponential contraction follows. The paper also proposes an RKHS implementation via truncated Hermite expansions and reports numerical experiments.","tokens_in":20867,"tokens_out":6741,"duration_ms":74416,"significance":"If the main theorem is correct, the result is significant: it identifies a mechanism by which joint learning of the direction and the link function escapes the spurious fixed points that plague the planted model with negative initial correlation, and it gives a dimension-dependent time scale governed by the Hermite information exponent. The derivation of the reduced ODE system is clean and the Hermite diagonalization is standard, and the numerical experiments are consistent with the claimed phases. However, as written, the proof of Theorem 1 contains load-bearing gaps: the negative-initialization branch of Lemma 4 is explicitly omitted, and the uniform-in-time control of the high-degree tail of the Hermite expansion is asserted rather than proved. These gaps are likely repairable from the stated regularity assumptions, but they are currently essential to the main claim.","major_comments":[{"comment":"The proof of Lemma 4 states that the case m0 < 0 is \"very similar and is left to the reader.\" This is not a minor omission: the m0 < 0 branch is exactly one of the two central conclusions of Theorem 1 for s ≥ 2, and the subsequent Lemmas 5, 6, and 7 as well as Theorem 2 rely on the sign conclusions of Lemma 4 in both cases. The symmetry (m, a_k) -> (-m, (-1)^k a_k) plausibly supplies the missing argument, but as written the proof of the main theorem is incomplete for negative initial correlation.","section":"Appendix C.1.1, Proof of Lemma 4"},{"comment":"Several load-bearing steps assert that tail sums over k > k*, such as sum_{k>k*} k |a_{k,t} a*_k m_t^{k-1}|, can be made arbitrarily small by choosing k* large, and that the first k* terms have a positive contribution bounded below by a constant independent of t. No quantitative uniform-in-time bound is displayed, even though the needed lower bound must hold for all t >= tau_c, not just at a single time. This underpins the positivity principle and the exponential contraction of m_t. The gap appears fillable using Fact 1 and the H^1 assumptions (sum_k k^2 |a*_k|^2 < infinity and the stated initialization of a_{k,0}), but the uniform bound should be written out explicitly.","section":"Appendix C.1.1, Lemma 7 and Theorem 2"},{"comment":"The s = 1 case is treated much more tersely than the main theorem requires. The proof of Lemma 9 contains unquantified assertions and apparent typos: the lower bound T1 is said to be \"of order 1\" without an explicit constant, the sentence \"a1,t must reach -delta before a1,t does\" appears to have a missing or misidentified quantity, and the bound a2,t >= -(a*_1)^2/(4 a*_2) \"in a time independent of delta\" is asserted without derivation. Since Theorem 1 for s = 1 depends on Lemma 9's conclusion that |m_t| and |a_{1,t}| remain bounded away from zero, this part needs a careful rewrite with concrete estimates.","section":"Appendix C.2, Lemmas 8-9"}],"minor_comments":[{"comment":"The informal aside \"as we say in the London Society Club of French Gentlemen\" and the sentence thanking ChatGPT, \"which arguably should earn it co-authorship,\" are stylistically inappropriate for a journal submission and should be removed or rewritten.","section":"Footnote 2 and Acknowledgments"},{"comment":"The displayed inequality \"|a_k*| >= a_{k,t}\" does not by itself imply the claimed uniform bound on |a_{k,t}|; the intended statement appears to be |a_{k,t}| <= max(|a*_k|, |a_{k,0}|). The proof should be corrected accordingly.","section":"Appendix, Proof of Fact 1"},{"comment":"In the proof of Lemma 4, the coefficient update line reads \"ca*_s m_0^s - a_{2,t} <= da_{s,t}/dt\"; the index in the negative term should be a_{s,t}, not a_{2,t}.","section":"Proof of Lemma 4"},{"comment":"Theorem 1 states only the lower bound tau_c >= C d^{s-1}, while the text and Lemma 6 claim two-sided bounds b d^{s-1} <= tau_c <= B d^{s-1}. The theorem statement and the surrounding discussion should be aligned so that the order d^{s-1} claim is stated precisely.","section":"Theorem 1 and Lemma 6"},{"comment":"In the contraction proof for m0 < 0, the displayed equation for \\dot v_t uses \"v_k\" and mixes signs; since v_t = 1 + m_t, one has \\dot v_t = (1 - m_t^2) sum ... = v_t (1 - m_t) sum ..., and the inequality \\dot v_t <= -c v_t requires the sum to be negative. The sign conventions should be checked and the display corrected.","section":"Theorem 2, m0 < 0 case"}],"recommendation":"major_revision","confidential_remarks":"The central claim is plausible and the gaps appear repairable rather than fatal: the missing negative-m0 branch can likely be closed by the sign-flip symmetry of the ODE system, and the tail estimates should follow from the stated H^1 assumption combined with Fact 1. My recommendation of major revision is based on the fact that these are load-bearing gaps in the proof of the main theorem, not on a suspicion that the result is false. Once the proof is completed and the s = 1 lemmas are made rigorous, I would support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read.\n\nThe genuinely new content is Theorem 1: joint gradient flow on (w,f) converges to the target for both signs of the initial correlation m0. That stands in contrast to the planted model (Proposition 1), which can trap at a spurious minimum when m0<0. The loss expansion and planted-model results come from the cited literature, and the paper uses them correctly; the joint-flow analysis—Lemmas 2-10 and Theorems 1-3—is the contribution.\n\nWhat the paper does well: the ODEs (8)-(9) are correctly derived from the Hermite diagonalization, and the fast-slow decomposition is convincing. The information-exponent scaling tau_c ~ d^{s-1} for s>=2, and immediate convergence for s=1, are clean. The RKHS construction in Section 5 is a sensible practical bridge, and the experiments, though small, illustrate the predicted phases. The authors also honestly flag that finite-sample and finite-truncation guarantees are not covered.\n\nSoft spots. The proof has two places where rigor is deferred rather than supplied. Lemma 4 explicitly leaves the m0<0 case to the reader, and that case is needed for the second branch of Theorem 1. It is easily closed: the map (m, a_k) -> (-m, (-1)^k a_k) leaves (8)-(9) invariant, so the positive case implies the negative case with the sign flip. The paper should state this. Second, the proof repeatedly asserts that tail sums over k>k* can be made small by choosing k* large, without displaying the uniform-in-time bound. This is also fixable: Fact 1 gives |a_{k,t}| <= max(|a*_k|, |a_{k,0}|), and phi* in H^1 plus the stated initialization imply sum_{k>k*} k(|a*_k|^2 + |a_{k,0}|^2) is small for large k*. So the gaps are expositional, not a broken argument, but a referee should ask to see the bounds.\n\nThe experiments lack error bars and code, and only use n=1e5, d=10; that is acceptable for a theory paper but should not be oversold.\n\nBottom line: this deserves a serious referee. The central claim is correct as far as I can tell, and the missing details are routine to fill. I would send it out, and I would cite it if I worked on single-index dynamics.","headline":"Joint learning genuinely fixes the negative-initialization trap of the planted model, and Theorem 1 is correct, but the proof defers the negative-m0 case and tail bounds that should be written out.","tokens_in":21417,"tokens_out":4526,"would_cite":true,"duration_ms":43544,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","62J02"],"pacs":[],"model":"deepseek-v4-flash","headline":"Jointly learning a projection and a univariate profile in a Gaussian single-index model provably recovers the target regression function, from either positive or negative alignment, after a wait time set by the information exponent.","keywords":["Gaussian single-index model","gradient flow","Hermite polynomials","information exponent","high-dimensional learning","joint representation learning","reproducing kernel Hilbert space","non-convex optimization"],"falsifier":"Numerically integrate (8)--(9) with a target in $H^1_\\gamma$ whose Hermite coefficients decay slowly but remain summable, such as $a^\\star_k=c k^{-2}$ for $k\\ge2$, initialize $m_0=-1/\\sqrt{d}$, truncate at a large $K$, and check whether $m_t$ reaches a dimension-independent level within time $O(d^{s-1})$; if the flow stays trapped near the origin or the predicted exponential phase never starts, the universal convergence claim for negative initialization would be false.","tokens_in":20301,"feed_emoji":"🎯","tokens_out":11944,"duration_ms":113977,"temperature":0.7,"pith_summary":"The paper studies gradient flow for predictors of the form $f(\\langle w, x\\rangle)$ where both $w$ and $f$ are learned from Gaussian data, in the single-index model $Y=\\varphi^\\star(\\langle w^\\star, X\\rangle)+\\varepsilon$. Its main result is that this joint flow converges to the true regression function whenever the target has finite first-order Gaussian Sobolev norm, with the waiting time controlled by the information exponent $s$: order $d^{s-1}$ for $s\\ge2$, immediate for $s=1$. In contrast to training with the link function fixed, the joint flow also converges when the initial correlation is negative, ending at $-w^\\star$ with a parity-flipped profile yet the same composed predictor. Because initialization is over the sphere, an overwhelmingly probable initial alignment of order $1/\\sqrt{d}$ is enough to seed recovery. A Hermite-based RKHS truncation makes the procedure implementable, and the paper's experiments reproduce the predicted phases.","feed_headline":"Joint flow recovers single-index targets even from negative alignment","feed_subtitle":"A direction and profile learned jointly converge to the true regression even from negative initial alignment.","key_machinery":"The machinery is the Hermite spectral decomposition of the loss, which reduces the flow on $S^{d-1}\\times L^2_\\gamma$ to the coupled ODEs $\\dot a_{k,t}=a^\\star_k m_t^k-a_{k,t}$ and $\\dot m_t=(1-m_t^2)\\sum_{k\\ge1} k a_{k,t} a^\\star_k m_t^{k-1}$. The information exponent $s$, the smallest nonzero Hermite index of $\\varphi^\\star$, controls how weak the initial gradient signal is; random initialization gives $|m_0|\\sim1/\\sqrt{d}$, so the signal starts at order $m_0^{s-1}$. The positivity principle is the mechanism that carries the argument: after a fast relaxation phase, each tracked coefficient $a_{k,t}$ keeps the sign of $a^\\star_k m_t^k$, so the direction dynamics receives a reinforcing pull instead of a cancelling one. The fast--slow separation between the coefficient equations and the $m$-equation is what converts the $1/\\sqrt{d}$ initial alignment into a $d^{s-1}$ transient.","core_discovery":"The paper's central claim is that the joint gradient flow (8)--(9) converges to a global optimum of the population squared loss for every target $\\varphi^\\star \\in H^1_\\gamma$ with Hermite expansion $\\varphi^\\star = \\sum_k a^\\star_k h_k$ and information exponent $s$. For $s\\ge2$, after a concentration time $\\tau_c$ of order $d^{s-1}$, if $m_0>0$ then $m_t\\to1$ and $a_{k,t}\\to a^\\star_k$; if $m_0<0$ then $m_t\\to-1$ and $a_{k,t}\\to(-1)^k a^\\star_k$, with exponential rates in both cases. For $s=1$ the same convergence holds from time zero, with the sign of $a^\\star_1 a_{1,0}$ selecting the attractor. Because $m_t^k\\to(\\pm1)^k$ and the coefficients flip in the matching way, the composed function $f_t(\\langle w_t,\\cdot\\rangle)$ converges to $\\varphi^\\star(\\langle w^\\star,\\cdot\\rangle)$ in both cases, so the target regression function is recovered even when the learned direction is the negative of $w^\\star$. The proof proceeds by fast--slow analysis of the coefficients and a positivity principle that prevents the coefficients from changing sign after a short transient.","pith_inferences":["A quantitative version of the assumed tail bound would turn the theorem into a fully explicit non-asymptotic rate and reveal how the required truncation degree $k^\\star$ depends on the Hermite decay of $\\varphi^\\star$.","The two-attractor conclusion implies that the predictive function is identifiable while the pair $(\\pm w^\\star, \\varphi^\\star(\\pm\\cdot))$ is identifiable only up to reflection; applications that interpret the recovered direction need an orientation convention.","For the $s=1$ case, the sign of $a^\\star_1 a_{1,0}$ decides the attractor without any transient, so mirror-image representations are settled early; perturbing a single first Hermite coefficient in experiments would test whether this early symmetry breaking is as robust as the theorem suggests."],"forward_implications":["A random initialization, whose correlation with the true direction is only of order $1/\\sqrt{d}$, is enough to seed consistent recovery; the price is a transient of order $d^{s-1}$ before exponential convergence begins.","Negative initial correlation does not doom the flow: the direction converges to $-w^\\star$, the Hermite coefficients flip to $(-1)^k a^\\star_k$, and the composed predictor still converges to $\\varphi^\\star(\\langle w^\\star, x\\rangle)$.","Joint learning removes spurious basins that the planted model with $\\varphi^\\star$ fixed can get trapped in, so learning the profile can strictly help the direction escape a bad basin.","The paper's numerical experiments indicate that the same phase structure persists for the RKHS-truncated empirical dynamics with finite samples and stochastic initialization."],"supporting_citations":[{"why":"Supplies the Hermite-coefficient expansion of the squared loss, the identity $\\sum_k a_k a^\\star_k m^k$ from which the reduced dynamics follow.","marker":"[13]"},{"why":"Gives the diagonalization of the Gaussian covariance operator in Hermite polynomials, used to compute the cross-term expectation in the loss.","marker":"[3]"},{"why":"Introduces the information exponent $s$ that sets the early gradient scale and the $d^{s-1}$ concentration time.","marker":"[5]"},{"why":"Links the information exponent to computational-statistical gaps in Gaussian single-index models, the benchmark against which the joint-flow rates are read.","marker":"[11]"},{"why":"Provides the Hermite-truncated RKHS that the paper adapts for finite-sample implementation of the joint flow.","marker":"[14]"}],"fun_headline_variants":["Joint flow learns index even from negative start","Negatively aligned? Joint flow still converges","Single-index recovery without initial alignment","Joint learning beats negative initialization","Convergence despite negative correlation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The contraction arguments assume that the tail sum $\\sum_{k>k^\\star} k |a_{k,t} a^\\star_k|$ stays uniformly small in time once a large cutoff $k^\\star$ is chosen; if that tail is not genuinely negligible, the exponential convergence after $\\tau_c$ is not established.","fun_headline_variants_meta":{"raw":{"variants":["Joint flow learns index even from negative start","Negatively aligned? Joint flow still converges","Single-index recovery without initial alignment","Joint learning beats negative initialization","Convergence despite negative correlation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000465,"raw_usage":{"total_tokens":2369,"prompt_tokens":1041,"completion_tokens":1328,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":1269}},"tokens_in":657,"tokens_out":1328,"duration_ms":11426,"temperature":1.0,"reasoning_tokens":1269,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:30:03.970190+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Numerically integrate (8)--(9) with a target in $H^1_\\gamma$ whose Hermite coefficients decay slowly but remain summable, such as $a^\\star_k=c k^{-2}$ for $k\\ge2$, initialize $m_0=-1/\\sqrt{d}$, truncate at a large $K$, and check whether $m_t$ reaches a dimension-independent level within time $O(d^{s-1})$; if the flow stays trapped near the origin or the predicted exponential phase never starts, the universal convergence claim for negative initialization would be false.","supporting_citations":[{"cited_title":"Learning single-index models in gaussian space","cited_arxiv_id":null,"evidence_quote":"Supplies the Hermite-coefficient expansion of the squared loss, the identity $\\sum_k a_k a^\\star_k m^k$ from which the reduced dynamics follow."},{"cited_title":"Nonparametric Linear Feature Learning in Regression Through Regularisation","cited_arxiv_id":"2307.12754","evidence_quote":"Provides the Hermite-truncated RKHS that the paper adapts for finite-sample implementation of the joint flow."}],"review_version":1}