{"id":"884ff178-61e7-478c-888e-dd341831012b","arxiv_id":"2504.12519","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Corner Gradient Descent achieves O(t^{-θζ}) loss rates for SGD on power-law spectral problems, with optimal θ = min(2, ν, 2/(ζ+1/ν)).","lead":"Corner Gradient Descent treats optimization algorithms as shapes, or contours, in the complex plane. A contour with a sharp corner accelerates stochastic gradient descent on hard infinite-dimensional quadratic problems from rate O(t^{-ζ}) to O(t^{-θζ}), giving a new theoretical path to faster learning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4 is proved only under the SE approximation with τ2=0, while the distributions where SE is exact have τ2≠0; the τ2-independence claim rests on an unproved sketch in Appendix H.","rationale":"The reader's CONDITIONAL verdict is appropriate. The paper's internal τ2=0 analysis is substantial: Theorem 3 is proved with detailed Mittag-Leffler estimates and supporting lemmas, and Sections 5–6 provide a concrete rational-approximation construction plus experiments, which is real evidence for the proposed mechanism. The weakest point is the transfer from this idealized τ2=0 model to actual SGD dynamics. The SE approximation is exact for Gaussian and translation-invariant distributions, and in both cases τ2≠0; the paper itself says τ2=0 does not match any natural distribution. Therefore Theorem 4 as stated is not proved for the standard linear-regression problems where the moment closure is exact. Appendix H is explicitly a sketch and does not supply the uniform error bounds needed to promote the claim to a theorem. This matches the reader's weakest_assumption, so no verdict change is needed. The infinite-memory idealization is also present, but it is disclosed and paired with finite-memory rational approximations, so I would not weight it as the primary load-bearing concern. The concrete test of completing Appendix H for τ2=−1 would either elevate the result or force a narrower statement of the abstract and Theorem 4.","tokens_in":33486,"tokens_out":9014,"duration_ms":98489,"concrete_test":"Independently complete the Appendix H perturbation analysis for the Gaussian closure (τ1=1, τ2=−1): derive closed-form bounds for the m-th correction in Eqs. (247)–(249), verify that the first correction is O(t^{θ/ν−3}) and that the series in Eq. (261) converges whenever |B| > |τ2| λ_max^{1/θ} ∫_0^∞ F_U^2(r) dr (Eq. 262) for every θ < θmax in the phase diagram. If the bounds close, the τ2-independence of Theorem 4 is established; if not, the theorem must be restated with τ2=0 as an explicit hypothesis.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central rate formula θmax = min(2, ν, 2/(ζ+1/ν)) is derived from Theorem 3, whose propagator asymptotics are proved under the τ2=0 simplification of Eq. (8) (Sections 2–4). But the SE closure (6) is exact for Gaussian data with (τ1,τ2)=(1,−1) and for translation-invariant data with (1,1), as the paper itself notes in Section 7; it also states that τ2=0 'does not seem to correspond to any specific natural data distribution'. Thus the exact model problems to which the theory would most naturally apply are outside the proved statement. The only bridge to τ2≠0 is Appendix H, which is explicitly a sketch 'without much rigor': it expands the propagator in a binomial series and bounds corrections, but the conclusion that all τ2 terms are subleading is not a theorem. Since the abstract and Theorem 4 state the acceleration result without this qualification, the scope of the central claim is materially narrower than advertised. This is not an objection to the internal mathematics of the τ2=0 case, which appears coherent; it is a warning that the headline result has not been shown for the standard SGD problems where the SE closure is exact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes \"Corner Gradient Descent\", a family of stationary SGD algorithms with memory for infinite-dimensional quadratic problems with power-law spectra (capacity exponent ν, source exponent ζ). The author identifies stationary finite-memory (S)GD with rational maps Ψ=P/Q (Section 3) and shows that algorithms whose contour γ=Ψ({|μ|=1}) has a corner of external angle θπ at the origin, i.e. Ψ(μ)∼−cΨ(μ−1)^θ, have propagators Ut=O(t^{θ/ν−2}) and Vt=O(t^{−θζ}) (Theorem 3, proved under the Spectrally Expressible closure with τ2=0). Balancing signal and noise exponents gives Theorem 4: in the signal-dominated phase, the supremum of achievable acceleration factors is θmax=min(2,ν,2/(ζ+1/ν)), summarized in a three-region phase diagram (Figure 1 right). Corner maps are approximated by memory-M rational algorithms via midpoint quadrature of an integral representation (Propositions 1–2) and implemented with M=5 on a synthetic indicator problem (predicted/observed exponent 0.45/0.41) and on MNIST (acceleration factor ≈1.5). The paper ends with an explicit discussion of the limitations of the τ2=0 assumption and of finite-memory approximation (Section 7).","tokens_in":33752,"tokens_out":32218,"duration_ms":313145,"significance":"The conceptual claim is substantial: it offers a path beyond the barrier O(t^{−ζ}) that Yarotsky & Velikanov (2024) proved for stationary finite-memory SGD, and it produces a clean, falsifiable prediction (the θmax formula) with a three-region phase diagram that subsumes the earlier phase picture. The contour/Riemann-map viewpoint is elegant and likely to be reused. The proof of Theorem 3 is detailed and self-contained for the τ2=0 case (Appendix C), including explicit coefficient formulas in terms of Mittag-Leffler functions; the stability and separation arguments for the concrete corner map (Proposition 1) are rigorous; and the experiments ship reproducible code and agree qualitatively with the predicted exponents. The paper is also commendably explicit about what it does not prove (Section 7, Appendix H). The significance is real but conditional: the proven rate claims live in the τ2=0 SE propagator model, the natural exact-SE distributions have τ2≠0, and the bridge in Appendix H is a sketch. If either a rigorous τ2≠0 extension or an honest scoping of the claims is provided in revision, this would be a valuable contribution to the optimization literature.","major_comments":[{"comment":"The scope of the headline claim is narrower than what is proved, and the paper's own text confirms this. Section 2 fixes τ2=0 for the main results (Eq. (8)), and Section 7 states that τ2=0 \"does not seem to correspond to any specific natural data distribution\", whereas the SE closure is exact for translation-invariant (τ1=τ2=1) and Gaussian (τ1=1,τ2=−1) data. Appendix H is explicitly introduced as a sketch \"without much rigor\"; its conclusion is an upper bound |Ut−U(0)t|≲C t^{θ/ν−3} under the batch-size condition (Eq. (262)) and a still sketchier treatment for Vt. Since Theorem 4's θmax=min(2,ν,2/(ζ+1/ν)) is derived directly from Theorem 3's propagator exponents, the central claim is proved only for a regime containing no natural exact-SE distribution. Within the τ2=0 model the derivation of Theorem 4 from Theorems 1 and 3 is internally coherent; the problem is that the abstract and Theorem 4 assert the result without the τ2 qualification. The paper should either prove the general-τ2 claim or restate Theorem 4 and the abstract with the τ2=0 hypothesis explicit, presenting τ2-independence as a conjecture supported by Appendix H.","section":"Section 7 / Appendix H / Theorem 4"},{"comment":"The object carrying the rates is a limiting propagator system, not a concrete stochastic iteration of the form (1). The propagators Ut,Vt in Eqs. (20)–(22) are defined by contour integrals involving an irrational map Ψ, and the corner maps of Section 4 (e.g. Eq. (29)) are not rational, so there is no matrix Sλ and no memory-M system realizing them; moreover, by Eqs. (24) and (26) the effective learning rate αeff diverges when θ>1. Consequently, Theorem 4's \"there exists a corner algorithm and batch size B such that Lt=O(t^{−θζ})\" is a statement about the SE propagator expansion (7) and the associated infinite-memory limit, not about a runnable finite-dimensional algorithm. The body of the paper is reasonably transparent about this, but the abstract's phrasing \"rates up to O(t^{−2ζ}) can be achieved by a generalized stationary SGD with infinite memory\" overstates what is constructed. Please state precisely in what sense the ideal corner algorithm exists and which objects achieve the rates.","section":"Sections 3–4; Eqs. (20)–(26)"},{"comment":"The bridge from the ideal corner map to the implemented finite-memory algorithm has two unquantified links. First, the midpoint rule (31) is a one-sided quadrature, and the cited O(e^{−c√M}) rational-approximation bounds of Gopal & Trefethen are not shown to apply to this discretization as presented; no theorem bounds the approximation error of Ψ(M) to Ψ in a relevant norm, nor translates such an error into a propagator or loss error over finite horizons. Second, the stability of the Sλ matrices associated with Ψ(M) is asserted by construction but not verified. Section 7 concedes that the finite-memory algorithm does not accelerate the asymptotic exponent, so the experimental exponents in Section 6 — fitted over finite iteration ranges, on a nonlinear finite-dimensional model in the MNIST case — are evidence for, but not a proof of, the claimed acceleration. This does not affect Theorem 4, but the abstract's claim that ideal corner algorithms \"can be efficiently approximated\" should be softened or substantiated with approximation-error results.","section":"Sections 5–6; Propositions 1–2"}],"minor_comments":[{"comment":"The pointwise loss is written ℓ(x,w)=½|ŷ(x,w)−y(x)|; the square appears to be missing, unlike the quadratic loss defined in Section 2.","section":"Section 6, Eq. (39)"},{"comment":"The annotations \"MNIST+MLP\" and \"CIFAR10+ResNet\" in the left phase diagram are never explained in the text or caption.","section":"Figure 1"},{"comment":"The Lebesgue–Stieltjes notation ∫_0^∞ r²F_U²(r) dr^{−θ/ν} (and similarly for F_V) is nonstandard; one sentence of explanation in the main text would help readers who do not consult Appendix C.","section":"Theorem 3, Eq. (20)"},{"comment":"The symbol G0(t,λ)≡U2(t,λ) introduces U2 without prior definition; the definition U2(t,λ)=|(1 0^T)Sλ^{t−1}(−α c)|² should be stated explicitly.","section":"Appendix H, Eq. (237)"},{"comment":"The quadrature constant l is fixed to l=5 in the experiments without comment or sensitivity analysis; a brief justification or a robustness check would be useful.","section":"Section 5"},{"comment":"For θ=1.3 the measured acceleration factor (≈1.5) exceeds the infinite-dimensional prediction θmax=ν≈1.3 from Theorem 4; the text explains this via finite dimensionality, but spelling out the finite-dimensional expectation would make the comparison easier to assess.","section":"Section 6, MNIST"}],"recommendation":"major_revision","confidential_remarks":"I am persuaded that the τ2=0 core (Theorem 3, Appendix C) is coherent and that the flagged scope gap in Theorem 4 is fixable by either a rigorous τ2≠0 argument or a scoped restatement; the paper should not be rejected on these grounds. The relationship to Yarotsky & Velikanov (2024) is properly credited, and the contour viewpoint is a genuine new element. Editorially, I would ask the author to make the abstract match the proved statements, in particular the phrases \"show that rates up to O(t^{−2ζ}) can be achieved\" and \"can be efficiently approximated\"."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this paper: it gives a new geometric way to think about stationary (S)GD with memory, by identifying algorithms with contours in the complex plane via the map Psi = P/Q. The key theoretical move is that contours with a corner of exterior angle theta*pi accelerate the signal propagator from O(t^{-zeta}) to O(t^{-theta*zeta}), while worsening the noise propagator. Balancing those two gives the phase diagram theta_max = min(2, nu, 2/(zeta + 1/nu)). That formula is new, and it is derived, not fitted. The proof of Theorem 3 for the tau2=0 case is detailed and looks coherent; the Mittag-Leffler asymptotics and the finite-memory rational approximation construction are solid pieces of work.\n\nThe soft spot is real and is exactly where the stress-test note lands. The SE closure with tau2=0 is the assumption under which the propagators have the simple contour representation that the whole analysis uses. But the paper itself notes that tau2=0 does not correspond to any natural data distribution, while the cases where SE is exact (Gaussian, translation-invariant) have tau2 = -1 or +1. Appendix H tries to bridge this by perturbation theory, but it is explicitly a sketch, and the claim that all tau2 corrections are subleading, uniformly in t, is not proved. So Theorem 4 as stated overstates the scope: the rigorous result is for the tau2=0 model problem, and the transfer to the standard SE-exact cases rests on an unverified conjecture. That is not a reason to dismiss the paper, but it is a reason to demand a revision that either proves the extension under explicit conditions or clearly reframes the theorem as a statement about the tau2=0 model.\n\nTwo smaller points. The ideal corner algorithm is defined through its contour map rather than an explicit finite-dimensional dynamical system, so the rates are for limiting propagators; the paper is honest that the finite-memory approximation does not accelerate asymptotically. The experiments on the synthetic problem and MNIST are suggestive but are not the core evidence. Self-citation is not a problem here; the new derivation is not a restatement of the earlier finite-memory no-go results.\n\nWho gets value: researchers working on stochastic optimization, learning rates in kernel/linear models, and the SE approximation. It deserves a serious referee. I would send it to peer review, with the expectation of major revision: either close the tau2 gap or narrow the claims to match the proof.","headline":"A genuinely new contour-based framework for accelerating stationary SGD, with a clean phase diagram—but the headline rate formula is proved only under the tau2=0 simplification, and the advertised scope is wider than the proof.","tokens_in":34257,"tokens_out":1461,"would_cite":true,"duration_ms":17952,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68W40","90C25","65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Corner-shaped contours in the complex plane let SGD accelerate beyond plain gradient rates.","keywords":["stochastic gradient descent","optimization acceleration","heavy ball","momentum methods","ill-conditioned quadratic problems","power-law spectra","complex contours","rational approximation"],"falsifier":"Run memory-M corner SGD on a quadratic least-squares problem with power-law spectrum and non-Gaussian features whose fourth moments violate the $\\tau_2=0$ identity; fit the empirical loss exponent at the theoretically optimal $\\theta_{\\max}$. If the exponent remains at the plain SGD value $\\zeta$ instead of approaching $\\theta_{\\max}\\zeta$ even with large batch sizes, the $\\tau_2=0$ simplification is doing the work.","tokens_in":33251,"feed_emoji":"⚡","tokens_out":5672,"duration_ms":56084,"temperature":0.7,"pith_summary":"This paper claims that stationary stochastic gradient algorithms with long memory can be encoded as contours in the complex plane, and that a contour with a corner of exterior angle $\\theta\\pi$ accelerates the loss decay from $O(t^{-\\zeta})$ to $O(t^{-\\theta\\zeta})$. If true, mini-batch SGD can reach rates approaching $O(t^{-2\\zeta})$ in the signal-dominated regime, a regime where no stationary finite-memory algorithm is known to accelerate. The paper derives the optimal acceleration factor $\\theta_{\\max}=\\min(2,\\nu,2/(\\zeta+1/\\nu))$, balances signal decay against sampling-noise amplification, and constructs finite-memory rational approximations that reproduce the acceleration numerically on a synthetic problem and on MNIST. The result matters because it gives a geometric design principle for accelerated stochastic optimization on ill-conditioned quadratic problems.","feed_headline":"Corner contours accelerate SGD beyond plain gradient rates","feed_subtitle":"A corner of exterior angle θπ lifts loss decay to O(t^{-θζ}), approaching O(t^{-2ζ}) on power-law spectra.","key_machinery":"The central object is the map $\\Psi=P/Q$ built from the characteristic polynomial of a memory-$M$ gradient algorithm, together with the contour $\\gamma=\\Psi(\\{|\\mu|=1\\})$ it traces in the complex plane. A corner map obeys $\\Psi(\\mu)=-c_\\Psi(\\mu-1)^\\theta(1+o(1))$ near $\\mu=1$, producing a contour with external angle $\\theta\\pi$. This corner condition determines the leading decay of the propagators through Mittag-Leffler functions: signal propagators decay as $t^{-\\theta\\zeta}$ and noise propagators as $t^{\\theta/\\nu-2}$, and Theorem 4 balances them to yield $\\theta_{\\max}$.","core_discovery":"On infinite-dimensional quadratic problems with power-law spectra, any stationary SGD with finite linear memory is confined to the phase diagram of plain SGD, with loss $O(t^{-\\zeta})$. This paper identifies generalized SGD algorithms with rational maps $\\Psi=P/Q$ and their contour images, and shows that a contour with a corner of external angle $\\theta\\pi$ produces signal propagators $O(t^{-\\theta\\zeta})$ and noise propagators $O(t^{\\theta/\\nu-2})$. Balancing the two exponents gives the maximal achievable acceleration $\\theta_{\\max}=\\min(2,\\nu,2/(\\zeta+1/\\nu))$ in the signal-dominated phase, with rates arbitrarily close to $O(t^{-2\\zeta})$ when $\\theta_{\\max}=2$. The ideal infinite-memory corner algorithm is approximated by memory-$M$ algorithms via rational approximations of power functions with error $O(e^{-c\\sqrt{M}})$, and a memory-5 implementation shows acceleration on a synthetic indicator problem and on MNIST.","pith_inferences":["A testable extension: if the sketch in Appendix H holds, the acceleration diagram is independent of the fourth-moment constant $\\tau_2$, so corner SGD should accelerate on non-Gaussian data too, not only on translation-invariant or Gaussian data.","The contour viewpoint suggests a design route for non-stationary schedules, where approximating an ideal corner with time-changing finite memory might restore asymptotic acceleration even where stationary finite memory is provably incapable of it.","Because the deterministic rate is governed only by the source exponent $\\zeta$, corner algorithms may transfer directly to kernel-ridge and overparameterized linear-model settings with power-law spectra.","The paper establishes rates for the limiting propagators of an ideal infinite-memory algorithm; turning this into a concrete infinite-dimensional dynamical system would be a next step, though finite-memory rational approximations may already supply the practical algorithm."],"forward_implications":["Corner SGD attains loss $O(t^{-\\theta\\zeta})$ with $\\theta$ up to 2 in the signal-dominated phase, improving on the plain SGD rate $O(t^{-\\zeta})$.","In the fully accelerated region $\\nu>2,\\ 0<\\zeta<1-1/\\nu$, rates arbitrarily close to $O(t^{-2\\zeta})$ become feasible for mini-batch SGD.","In the noise-limited region, the finite-noise condition forces $\\theta_{\\max}=\\nu<2$, so the noise propagator boundary, not the signal exponent, caps the achievable acceleration.","Finite-memory approximations converge to the ideal corner algorithm with error $O(e^{-c\\sqrt{M}})$, making the acceleration practically accessible with small memory.","The phase diagram predicts concrete exponent transitions that can be checked on synthetic least-squares problems with known spectral exponents."],"supporting_citations":[{"why":"Supplies the memory-M SGD framework, the propagator expansion, and the finite-memory phase diagram that corner algorithms are designed to go beyond.","marker":"Yarotsky & Velikanov (2024)"},{"why":"Introduces the Spectrally Expressible approximation used to write the loss evolution in terms of signal and noise propagators.","marker":"Velikanov et al. (2023)"},{"why":"Establishes the signal-dominated and noise-dominated phases of plain SGD that define the setting of this paper.","marker":"Berthier et al. (2020)"},{"why":"Provides the Heavy Ball momentum algorithm that is recovered as the memory-1 special case of the contour view.","marker":"Polyak (1964)"},{"why":"Establishes the O(t^{-2ζ}) deterministic accelerated rate that the stochastic corner algorithms aim to approach.","marker":"Nemirovskiy & Polyak (1984)"},{"why":"Supplies the rational approximation of power functions used to build finite-memory approximations of corner maps.","marker":"Gopal & Trefethen (2019)"},{"why":"Origin of the fast rational approximation quality for nonsmooth functions that underlies the finite-memory construction.","marker":"Newman (1964)"},{"why":"Formalizes the capacity and source conditions that become the exponents ν and ζ in the spectral power laws.","marker":"Caponnetto & De Vito (2007)"},{"why":"Provides the Mittag-Leffler integral representations used to express the propagator asymptotics.","marker":"Haubold et al. (2011)"},{"why":"Supplies the Riemann mapping and Carathéodory extension facts used to reconstruct a map from a contour.","marker":"Garnett & Marshall (2005)"}],"fun_headline_variants":[],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fourth-order moment evolution is exactly described by the Spectrally Expressible approximation with $\\tau_2=0$, which is true only for translation-invariant or Gaussian data; for general distributions, the main theorem is conditional on this approximation, and the extension to $\\tau_2\\neq 0$ is only sketched.","fun_headline_variants_meta":{"error":"DeepSeek 429: {\"error\":{\"message\":\"Too many requests. Your current concurrency is 172, which exceeds your concurrency limit of 117 based on your remaining balance. Please top up your balance to restore your concurrency.\",\"type\":\"rate_limit_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:29:52.044867+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run memory-M corner SGD on a quadratic least-squares problem with power-law spectrum and non-Gaussian features whose fourth moments violate the $\\tau_2=0$ identity; fit the empirical loss exponent at the theoretically optimal $\\theta_{\\max}$. If the exponent remains at the plain SGD value $\\zeta$ instead of approaching $\\theta_{\\max}\\zeta$ even with large batch sizes, the $\\tau_2=0$ simplification is doing the work.","supporting_citations":[],"review_version":1}