Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Joint Learning in the Gaussian Single Index Model

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Jointly learning a projection and a univariate profile in a Gaussian single-index model provably recovers the target regression function, from either positive or negative alignment, after a wait time set by the information exponent.

desk verdict Joint learning genuinely fixes the negative-initialization trap of the planted model, and Theorem 1 is correct, but the proof defers the negative-m0 case and tail bounds that should be written out. read the letter →

arxiv 2505.21336 v1 pith:ZHLYFGFP submitted 2025-05-27 cs.LG math.OCstat.ML

Loucas Pillaud-Vivien , Adrien Schertzer This is my paper
classification cs.LGmath.OCstat.ML MSC 68T0762J02
keywords Gaussiansingle-indexmodelgradientflowHermitepolynomialsinformationexponenthigh-dimensionallearningjointrepresentationreproducingkernelHilbertspacenon-convexoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies gradient flow for predictors of the form $f(\langle w, x\rangle)$ where both $w$ and $f$ are learned from Gaussian data, in the single-index model $Y=\varphi^\star(\langle w^\star, X\rangle)+\varepsilon$. Its main result is that this joint flow converges to the true regression function whenever the target has finite first-order Gaussian Sobolev norm, with the waiting time controlled by the information exponent $s$: order $d^{s-1}$ for $s\ge2$, immediate for $s=1$. In contrast to training with the link function fixed, the joint flow also converges when the initial correlation is negative, ending at $-w^\star$ with a parity-flipped profile yet the same composed predictor. Because initialization is over the sphere, an overwhelmingly probable initial alignment of order $1/\sqrt{d}$ is enough to seed recovery. A Hermite-based RKHS truncation makes the procedure implementable, and the paper's experiments reproduce the predicted phases.

What carries the argument

The machinery is the Hermite spectral decomposition of the loss, which reduces the flow on $S^{d-1}\times L^2_\gamma$ to the coupled ODEs $\dot a_{k,t}=a^\star_k m_t^k-a_{k,t}$ and $\dot m_t=(1-m_t^2)\sum_{k\ge1} k a_{k,t} a^\star_k m_t^{k-1}$. The information exponent $s$, the smallest nonzero Hermite index of $\varphi^\star$, controls how weak the initial gradient signal is; random initialization gives $|m_0|\sim1/\sqrt{d}$, so the signal starts at order $m_0^{s-1}$. The positivity principle is the mechanism that carries the argument: after a fast relaxation phase, each tracked coefficient $a_{k,t}$ keeps the sign of $a^\star_k m_t^k$, so the direction dynamics receives a reinforcing pull instead of a cancelling one. The fast--slow separation between the coefficient equations and the $m$-equation is what converts the $1/\sqrt{d}$ initial alignment into a $d^{s-1}$ transient.

What would settle it

Numerically integrate (8)--(9) with a target in $H^1_\gamma$ whose Hermite coefficients decay slowly but remain summable, such as $a^\star_k=c k^{-2}$ for $k\ge2$, initialize $m_0=-1/\sqrt{d}$, truncate at a large $K$, and check whether $m_t$ reaches a dimension-independent level within time $O(d^{s-1})$; if the flow stays trapped near the origin or the predicted exponential phase never starts, the universal convergence claim for negative initialization would be false.

Watch

Extended reading notes

Core claim

The paper's central claim is that the joint gradient flow (8)--(9) converges to a global optimum of the population squared loss for every target $\varphi^\star \in H^1_\gamma$ with Hermite expansion $\varphi^\star = \sum_k a^\star_k h_k$ and information exponent $s$. For $s\ge2$, after a concentration time $\tau_c$ of order $d^{s-1}$, if $m_0>0$ then $m_t\to1$ and $a_{k,t}\to a^\star_k$; if $m_0<0$ then $m_t\to-1$ and $a_{k,t}\to(-1)^k a^\star_k$, with exponential rates in both cases. For $s=1$ the same convergence holds from time zero, with the sign of $a^\star_1 a_{1,0}$ selecting the attractor. Because $m_t^k\to(\pm1)^k$ and the coefficients flip in the matching way, the composed function $f_t(\langle w_t,\cdot\rangle)$ converges to $\varphi^\star(\langle w^\star,\cdot\rangle)$ in both cases, so the target regression function is recovered even when the learned direction is the negative of $w^\star$. The proof proceeds by fast--slow analysis of the coefficients and a positivity principle that prevents the coefficients from changing sign after a short transient.

Load-bearing premise

The contraction arguments assume that the tail sum $\sum_{k>k^\star} k |a_{k,t} a^\star_k|$ stays uniformly small in time once a large cutoff $k^\star$ is chosen; if that tail is not genuinely negligible, the exponential convergence after $\tau_c$ is not established.

Editorial extensions

If this is right

  • A random initialization, whose correlation with the true direction is only of order $1/\sqrt{d}$, is enough to seed consistent recovery; the price is a transient of order $d^{s-1}$ before exponential convergence begins.
  • Negative initial correlation does not doom the flow: the direction converges to $-w^\star$, the Hermite coefficients flip to $(-1)^k a^\star_k$, and the composed predictor still converges to $\varphi^\star(\langle w^\star, x\rangle)$.
  • Joint learning removes spurious basins that the planted model with $\varphi^\star$ fixed can get trapped in, so learning the profile can strictly help the direction escape a bad basin.
  • The paper's numerical experiments indicate that the same phase structure persists for the RKHS-truncated empirical dynamics with finite samples and stochastic initialization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A quantitative version of the assumed tail bound would turn the theorem into a fully explicit non-asymptotic rate and reveal how the required truncation degree $k^\star$ depends on the Hermite decay of $\varphi^\star$.
  • The two-attractor conclusion implies that the predictive function is identifiable while the pair $(\pm w^\star, \varphi^\star(\pm\cdot))$ is identifiable only up to reflection; applications that interpret the recovered direction need an orientation convention.
  • For the $s=1$ case, the sign of $a^\star_1 a_{1,0}$ decides the attractor without any transient, so mirror-image representations are settled early; perturbing a single first Hermite coefficient in experiments would test whether this early symmetry breaking is as robust as the theorem suggests.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper analyzes continuous-time gradient flow for jointly learning a direction w on the sphere and a univariate link function f in the Gaussian single-index model y = φ*(<w*,x>) + ε, with population squared loss. Working in the Hermite basis, the authors reduce the infinite-dimensional flow to the coupled ODEs (8)-(9) for the correlation m_t = <w_t,w*> and the coefficients a_{k,t}. The main theorem (Theorem 1) claims that after a transient of order d^{s-1} for s ≥ 2, where s is the information exponent, the flow converges exponentially to (w*, φ*) when m_0 > 0 and to (-w*, φ*(-·)) when m_0 < 0, so that the learned predictor always recovers φ*(<w*,x>); for s = 1 the convergence is claimed to hold immediately. The proof strategy is a fast-slow analysis: coefficients a_{k,t} first track a*_k m_t^k, a positivity principle freezes the sign of m_t and of the relevant coefficients, and then exponential contraction follows. The paper also proposes an RKHS implementation via truncated Hermite expansions and reports numerical experiments.

Significance. If the main theorem is correct, the result is significant: it identifies a mechanism by which joint learning of the direction and the link function escapes the spurious fixed points that plague the planted model with negative initial correlation, and it gives a dimension-dependent time scale governed by the Hermite information exponent. The derivation of the reduced ODE system is clean and the Hermite diagonalization is standard, and the numerical experiments are consistent with the claimed phases. However, as written, the proof of Theorem 1 contains load-bearing gaps: the negative-initialization branch of Lemma 4 is explicitly omitted, and the uniform-in-time control of the high-degree tail of the Hermite expansion is asserted rather than proved. These gaps are likely repairable from the stated regularity assumptions, but they are currently essential to the main claim.

major comments (3)
  1. [Appendix C.1.1, Proof of Lemma 4] The proof of Lemma 4 states that the case m0 < 0 is "very similar and is left to the reader." This is not a minor omission: the m0 < 0 branch is exactly one of the two central conclusions of Theorem 1 for s ≥ 2, and the subsequent Lemmas 5, 6, and 7 as well as Theorem 2 rely on the sign conclusions of Lemma 4 in both cases. The symmetry (m, a_k) -> (-m, (-1)^k a_k) plausibly supplies the missing argument, but as written the proof of the main theorem is incomplete for negative initial correlation.
  2. [Appendix C.1.1, Lemma 7 and Theorem 2] Several load-bearing steps assert that tail sums over k > k*, such as sum_{k>k*} k |a_{k,t} a*_k m_t^{k-1}|, can be made arbitrarily small by choosing k* large, and that the first k* terms have a positive contribution bounded below by a constant independent of t. No quantitative uniform-in-time bound is displayed, even though the needed lower bound must hold for all t >= tau_c, not just at a single time. This underpins the positivity principle and the exponential contraction of m_t. The gap appears fillable using Fact 1 and the H^1 assumptions (sum_k k^2 |a*_k|^2 < infinity and the stated initialization of a_{k,0}), but the uniform bound should be written out explicitly.
  3. [Appendix C.2, Lemmas 8-9] The s = 1 case is treated much more tersely than the main theorem requires. The proof of Lemma 9 contains unquantified assertions and apparent typos: the lower bound T1 is said to be "of order 1" without an explicit constant, the sentence "a1,t must reach -delta before a1,t does" appears to have a missing or misidentified quantity, and the bound a2,t >= -(a*_1)^2/(4 a*_2) "in a time independent of delta" is asserted without derivation. Since Theorem 1 for s = 1 depends on Lemma 9's conclusion that |m_t| and |a_{1,t}| remain bounded away from zero, this part needs a careful rewrite with concrete estimates.
minor comments (5)
  1. [Footnote 2 and Acknowledgments] The informal aside "as we say in the London Society Club of French Gentlemen" and the sentence thanking ChatGPT, "which arguably should earn it co-authorship," are stylistically inappropriate for a journal submission and should be removed or rewritten.
  2. [Appendix, Proof of Fact 1] The displayed inequality "|a_k*| >= a_{k,t}" does not by itself imply the claimed uniform bound on |a_{k,t}|; the intended statement appears to be |a_{k,t}| <= max(|a*_k|, |a_{k,0}|). The proof should be corrected accordingly.
  3. [Proof of Lemma 4] In the proof of Lemma 4, the coefficient update line reads "ca*_s m_0^s - a_{2,t} <= da_{s,t}/dt"; the index in the negative term should be a_{s,t}, not a_{2,t}.
  4. [Theorem 1 and Lemma 6] Theorem 1 states only the lower bound tau_c >= C d^{s-1}, while the text and Lemma 6 claim two-sided bounds b d^{s-1} <= tau_c <= B d^{s-1}. The theorem statement and the surrounding discussion should be aligned so that the order d^{s-1} claim is stated precisely.
  5. [Theorem 2, m0 < 0 case] In the contraction proof for m0 < 0, the displayed equation for \dot v_t uses "v_k" and mixes signs; since v_t = 1 + m_t, one has \dot v_t = (1 - m_t^2) sum ... = v_t (1 - m_t) sum ..., and the inequality \dot v_t <= -c v_t requires the sum to be negative. The sign conventions should be checked and the display corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the joint-learning convergence theorem is proven from the derived ODE system, with no fitted parameter or load-bearing self-citation.

full rationale

The paper's derivation chain is self-contained. Lemma 1's loss expansion is not assumed from the cited [13] alone; it is re-proved in Appendix A via the diagonalization of the Mehler operator T_m in the Hermite basis, citing the external text [3] for that classical fact. Lemma 2 derives the coefficient dynamics (8) and the correlation dynamics (9) directly from the L^2 and spherical gradients of the expanded loss, so the central object of Theorem 1 is obtained from first-order conditions rather than being imposed. The proof of Theorem 1 (Appendix C) then proceeds by Gronwall estimates, comparison arguments, and the positivity lemmas; no conclusion of the theorem is used as an input. The information exponent s is defined from the target Hermite coefficients and only determines rates, so the dimension-dependent transient tau_c ~ d^{s-1} is a consequence of the ODE analysis in Lemma 6, not an assumed fit. Self-citations appear in background contexts ([7], [11] for multi-index and sample-complexity framing, [24] in the conclusion's outlook) and in the folklore attribution for the planted-model warm-up, but none of these carries the proof of Theorem 1. The genuine weaknesses are rigor gaps rather than circularity: the proof repeatedly asserts that tail sums over k > k* can be made arbitrarily small by choosing a large k* (e.g., in Theorem 2), and Lemma 4's negative-m0 case is deferred with 'is very similar and is left to the reader'; these omissions are repairable from the bounds in Fact 1 and the H^1 assumption, and they do not identify any step where a prediction reduces by construction to an input. Therefore no circular step is exhibited and the score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on Gaussian single-index assumptions, Hermite spectral tools, and an H^1 regularity condition; no fitted parameters or invented entities are introduced. The proof does rely on a finite cutoff k* chosen 'large enough', but this is an existence argument rather than a fitted data-dependent parameter.

assumptions (5)
  • domain assumption Data follow Y = phi*(<w*,X>) + epsilon with X ~ N(0,I_d), w* on the sphere, epsilon independent with finite variance.
    This is the model setup stated in Section 2 and used throughout.
  • domain assumption The target phi* lies in H^1 (weighted Sobolev), and all learned functions lie in L2_gamma.
    Stated in Section 2: 'for technical reasons, we shall assume that phi* in H^1'. Used for summability of Hermite coefficients and tail estimates.
  • standard math Hermite polynomials form an orthonormal basis of L2_gamma, and the operator T_m[g](y)=E_Z[g(m y + sqrt(1-m^2) Z)] is diagonal in this basis with T_m[h_i]=m^i h_i.
    Cited to [3, Section 2.7.1] and used in the proof of Lemma 1 to expand the loss.
  • domain assumption Initialization: w0 uniform on S^{d-1} so |m0| ~ 1/sqrt(d) with overwhelming probability, and k^2 a_{k,0} ~ U([-1,1]) so f0 in H^2 almost surely.
    Specified in Section 3; the time-scale and sign arguments rely on these initialization properties.
  • standard math Standard ODE facts: Picard-Lindelof uniqueness and Gronwall inequalities.
    Used throughout the appendix to control solutions and convergence rates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Joint Learning in the Gaussian Single Index Model." pith.science (2026). https://pith.science/paper/ZHLYFGFP

@misc{pith2026250521336,
  author       = {Pith},
  title        = {Pith review of: Joint Learning in the Gaussian Single Index Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZHLYFGFP}},
  note         = {Machine review of arXiv:2505.21336}
}
abstract

We consider the problem of jointly learning a one-dimensional projection and a univariate function in high-dimensional Gaussian models. Specifically, we study predictors of the form $f(x)=\varphi^\star(\langle w^\star, x \rangle)$, where both the direction $w^\star \in \mathcal{S}_{d-1}$, the sphere of $\mathbb{R}^d$, and the function $\varphi^\star: \mathbb{R} \to \mathbb{R}$ are learned from Gaussian data. This setting captures a fundamental non-convex problem at the intersection of representation learning and nonlinear regression. We analyze the gradient flow dynamics of a natural alternating scheme and prove convergence, with a rate controlled by the information exponent reflecting the \textit{Gaussian regularity} of the function $\varphi^\star$. Strikingly, our analysis shows that convergence still occurs even when the initial direction is negatively correlated with the target. On the practical side, we demonstrate that such joint learning can be effectively implemented using a Reproducing Kernel Hilbert Space (RKHS) adapted to the structure of the problem, enabling efficient and flexible estimation of the univariate function. Our results offer both theoretical insight and practical methodology for learning low-dimensional structure in high-dimensional settings.

Figures

Figures reproduced from arXiv: 2505.21336 by the authors.

Figure 1
Figure 1. At early stages, the weak signal from m0 induces a fast decay of the coefficient a4,t, which acts as a passive mode. Then, the dynamics enters a phase of partial equilibrium (slow-fast dynamics), where ak,t ≃ a ⋆ kmk t and mt grows very slowly. Once mt crosses a critical threshold, the dynamics switch regimes and a4,t begins to grow, reinforcing the directional alignment. Empirical RKHS-based dynamics. In the second… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Singular perturbations and hierarchical learning in two-layer neural networks

    cs.LG 2026-07 accept novelty 7.0 of 10

    Constant and linear Hermite components of a misspecified single-index target are recovered at the conjectured singular-perturbation timescales; quadratic learning remains coupled to them via an auxiliary constrained flow.

Reference graph

Works this paper leans on

34 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Sgd learning on neural networks: leap complexity and saddle-to-saddle dynamics.Proceedings of Machine Learning Research, 195:1–72, 2023

    Emmanuel Abbe, Enric Boix-Adsera, and Theodor Misiakiewicz. Sgd learning on neural networks: leap complexity and saddle-to-saddle dynamics.Proceedings of Machine Learning Research, 195:1–72, 2023

    work page 2023

  2. [2]

    MIT press, 2024

    Francis Bach.Learning theory from first principles. MIT press, 2024

    2024

  3. [3]

    Springer Science & Business Media, 2013

    Dominique Bakry, Ivan Gentil, and Michel Ledoux.Analysis and geometry of Markov diffusion operators, volume 348. Springer Science & Business Media, 2013

    2013

  4. [4]

    Online stochastic gradient descent on non-convex losses from high-dimensional inference.J

    Gerard Ben Arous, Reza Gheissari, and Aukosh Jagannath. Online stochastic gradient descent on non-convex losses from high-dimensional inference.J. Mach. Learn. Res., 22:106–1, 2021

    work page 2021

  5. [5]

    High-dimensional limit theorems for sgd: Effective dynamics and critical scaling.Advances in neural information processing systems, 35:25349–25362, 2022

    Gerard Ben Arous, Reza Gheissari, and Aukosh Jagannath. High-dimensional limit theorems for sgd: Effective dynamics and critical scaling.Advances in neural information processing systems, 35:25349–25362, 2022

    2022

  6. [6]

    Learning time-scales in two-layers neural networks.Found Comput Math, 2024

    Raphaël Berthier, Andrea Montanari, and Kangjie Zhou. Learning time-scales in two-layers neural networks.Found Comput Math, 2024

    work page 2024

  7. [7]

    On learning gaussian multi-index models with gradient flow.arXiv preprint arXiv:2310.19793, 2023

    Alberto Bietti, Joan Bruna, and Loucas Pillaud-Vivien. On learning gaussian multi-index models with gradient flow.arXiv preprint arXiv:2310.19793, 2023

    arXiv 2023

  8. [8]

    Gaussian measures.American Mathematical Soc., 36(62), 1998

    Vladimir Igorevich Bogachev. Gaussian measures.American Mathematical Soc., 36(62), 1998

    work page 1998

Show all 34 references
  1. [9]

    Survey on algorithms for multi-index models.arXiv preprint arXiv:2504.05426, 2025

    Joan Bruna and Daniel Hsu. Survey on algorithms for multi-index models.arXiv preprint arXiv:2504.05426, 2025. 11

    2025 arXiv

  2. [10]

    Alex Damian, Eshaan Nichani, Rong Ge, and Jason D Lee. Smoothing the landscape boosts the signal for sgd: Optimal sample complexity for learning single index models.Proceedings of the 37th International Conference on Neural Information Processing Systems, 2023

    2023

  3. [11]

    Computational-statistical gaps in gaussian single-index models.arXiv preprint arXiv:2403.05529, 2024

    Alex Damian, Loucas Pillaud-Vivien, Jason D Lee, and Joan Bruna. Computational-statistical gaps in gaussian single-index models.arXiv preprint arXiv:2403.05529, 2024

    2024 arXiv

  4. [12]

    Learning two-layer neural networks, one (giant) step at a time.Journal of Machine Learning Research, 2024

    Yatin Dandi, Florent Krzakala, Bruno Loureiro, Luca Pesce, and Ludovic Stephan. Learning two-layer neural networks, one (giant) step at a time.Journal of Machine Learning Research, 2024

    2024

  5. [13]

    Learning single-index models in gaussian space

    Rishabh Dudeja and Daniel Hsu. Learning single-index models in gaussian space. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors,Proceedings of the 31st Conference On Learning Theory, volume 75 ofProceedings of Machine Learning Research, pages 1887–1930. PML...

    1930

  6. [14]

    Nonparametric linear feature learning in regression through regularisation.arXiv preprint arXiv:2307.12754, 2023

    Bertille Follain, Umut Simsekli, and Francis Bach. Nonparametric linear feature learning in regression through regularisation.arXiv preprint arXiv:2307.12754, 2023

    2023 arXiv

  7. [15]

    Agnostic learning of a single neuron with gradient descent

    Spencer Frei, Yuan Cao, and Quanquan Gu. Agnostic learning of a single neuron with gradient descent. Advances in Neural Information Processing Systems, 33:5417–5428, 2020

    2020

  8. [16]

    High-dimensional integration on rd, weighted hermite spaces, and orthogonal transforms.Journal of Complexity, 31(2):174–205, 2015

    Christian Irrgeher and Gunther Leobacher. High-dimensional integration on rd, weighted hermite spaces, and orthogonal transforms.Journal of Complexity, 31(2):174–205, 2015

    2015

  9. [17]

    Efficient learning of generalized linear and single index models with isotonic regression

    Sham M Kakade, Varun Kanade, Ohad Shamir, and Adam Kalai. Efficient learning of generalized linear and single index models with isotonic regression. Advances in Neural Information Processing Systems, 24, 2011

    2011

  10. [18]

    The isotron algorithm: High-dimensional isotonic regression

    Adam Tauman Kalai and Ravi Sastry. The isotron algorithm: High-dimensional isotonic regression. In COLT, 2009

    2009

  11. [19]

    Phase retrieval in high dimensions: Statistical and computational phase transitions

    Antoine Maillard, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová. Phase retrieval in high dimensions: Statistical and computational phase transitions. Advances in Neural Information Processing Systems, 33:11071–11082, 2020

    2020

  12. [20]

    The landscape of empirical risk for nonconvex losses

    Song Mei, Yu Bai, and Andrea Montanari. The landscape of empirical risk for nonconvex losses. The Annals of Statistics, 46(6A):2747–2774, 2018

    2018

  13. [21]

    Random features for large-scale kernel machines.Advances in neural information processing systems, 20, 2007

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines.Advances in neural information processing systems, 20, 2007

    2007

  14. [22]

    Exact solution for on-line learning in multilayer neural networks

    David Saad and Sara A Solla. Exact solution for on-line learning in multilayer neural networks. Physical Review Letters, 74(21):4337, 1995

    1995

  15. [23]

    Kernel techniques: from machine learning to meshless methods

    Robert Schaback and Holger Wendland. Kernel techniques: from machine learning to meshless methods. Acta numerica, 15:543–639, 2006

    2006

  16. [24]

    Stochastic differential equations models for least- squares stochastic gradient descent.arXiv:2407.02322, 2024

    Adrien Schertzer and Loucas Pillaud-Vivien. Stochastic differential equations models for least- squares stochastic gradient descent.arXiv:2407.02322, 2024

    2024 arXiv

  17. [25]

    MIT press, 2002

    Bernhard Schölkopf and Alexander Smola.Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002

    2002

  18. [26]

    Learning kernel-based halfspaces with the zero-one loss.SIAM J

    Shai Shalev-Shwartz, Ohad Shamir, and Karthik Sridharan. Learning kernel-based halfspaces with the zero-one loss.SIAM J. COMPUT., 2010. 12

    2010

  19. [27]

    Learning relus via gradient descent.Advances in neural information processing systems, 2017

    Mahdi Soltanolkotabi. Learning relus via gradient descent.Advances in neural information processing systems, 2017

    2017

  20. [28]

    Phase diagram of stochastic gradient descent in high-dimensional two-layer neural networks

    Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová. Phase diagram of stochastic gradient descent in high-dimensional two-layer neural networks. arXiv preprint arXiv:2202.00293, 2022

    2022 arXiv

  21. [29]

    Learning a single neuron for non-monotonic activation functions

    Lei Wu. Learning a single neuron for non-monotonic activation functions. InInternational Conference on Artificial Intelligence and Statistics, pages 4178–4197. PMLR, 2022

    2022

  22. [30]

    Learning a single neuron with gradient methods

    Gilad Yehudai and Ohad Shamir. Learning a single neuron with gradient methods. InConference on Learning Theory, pages 3756–3786. PMLR, 2020

    2020

  23. [31]

    Symmetric single index learning.arXiv preprint arXiv:2310.02117, 2023

    Aaron Zweig and Joan Bruna. Symmetric single index learning.arXiv preprint arXiv:2310.02117, 2023

    2023 arXiv

  24. [32]

    On single-index models beyond gaussian data

    Aaron Zweig, Loucas Pillaud-Vivien, and Joan Bruna. On single-index models beyond gaussian data. Advances in Neural Information Processing Systems, 36:10210–10222, 2023. 13 Appendix The appendix is organized as follows: • We first present the proof of Lemma 1 in Section A, whi...

    2023

  25. [33]

    We now prove that, after timeT, both mt and a1,t remain uniformly bounded away from zero

    At timeT, mt and a1,t are of order 1 and both have the sign ofa1,0. We now prove that, after timeT, both mt and a1,t remain uniformly bounded away from zero. Lemma 9. There exists constantsc, C >0 so that after timet ≥ C, |mt|, |a1,t| ≥c. Moreover, • if a∗ 1a1,0 > 0, then mt >...

  26. [34]

    We now prove that an arbitrary number of coefficientsak,t adopt the sign of their target values (the sign ofa∗ kmk t)

    in a time independent ofδ (see da2,t dt ≤ −a2,t). We now prove that an arbitrary number of coefficientsak,t adopt the sign of their target values (the sign ofa∗ kmk t). 24 Lemma 10. For k∗ > 0, it existstk∗ ≥ C, for all k ≤ k∗, we haveak,tk∗ a∗ kmk tk∗ > 0. Proof. The proof is...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.