{"id":"73a35f87-bf90-4f05-88aa-04e752aca062","arxiv_id":"2507.14122","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":1,"one_line_summary":"SGD's last iterate reaches an O(log T / sqrt(T)) expected optimality gap for convex smooth stochastic problems under only convexity, smoothness, and finite gradient variance at a minimizer.","lead":"Stochastic gradient descent's last iterate, the point practitioners actually use, converges to the optimum at a near-optimal rate of roughly log(T)/sqrt(T) for convex smooth problems, even when gradient noise has no uniformly bounded variance. This removes a long-standing mismatch between SGD theory and practice and settles an open question in optimization.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 4.3's stated hypothesis (-a < b) excludes the parameters used in Theorem 3.1; the proof needs the opposite inequality, so the central rate is likely correct but the printed proof is formally invalid.","rationale":"The reader's weakest-assumption pick, Assumption 2.2 (GV*), is reasonable but is not where the proof is most fragile: if sigma_*^2 is infinite, the theorem is vacuous by design, and the paper gives practical sufficient conditions for the assumption. The genuinely load-bearing gap is Lemma 4.3. Its statement and the application in Section 4.3 are incompatible: the theorem's a < 1, b = -1 fails the lemma's -a < b, and the condition that actually makes the weighting scheme work is the reverse inequality 2 + a/b > 0. I checked the summation algebra: with the corrected hypothesis, p_t = alpha_{t-1}/alpha_t is in (0,1], the coefficients of r_0,...,r_{T-1} vanish, and the r_T coefficient is a * alpha_{T-1} > 0, so the claimed bound follows. The independence remark in Lemma 4.2 and the t = 1 endpoint in Lemma A.1 are presentation issues, not threats to the central rate. Thus the paper's main result is very likely correct conditional on a sign/condition repair in Lemma 4.3; the reader's CONDITIONAL verdict remains appropriate, and no stronger objection is warranted.","tokens_in":934,"tokens_out":1356,"duration_ms":686728,"concrete_test":"Re-derive Lemma 4.3 with its hypothesis replaced by b < 0 < a and 2 + a/b > 0, and verify that p_t = (T - t + 1 + a/b)/(T - t + 1) is in (0,1] for t = 0,...,T, that alpha_t is positive and nondecreasing, and that the coefficient cancellation in the displayed summation leaves only the r_T term with coefficient a * alpha_{T-1}. Then substitute a = (1 - gamma L)/(1 + gamma L), b = -1, with gamma L in (0,1), and confirm that Theorem 3.1 follows exactly as claimed. If the corrected lemma holds, the central rate is valid; if not, the proof has an unfixable gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is the formal state of Lemma 4.3, the black box that converts the one-step inequality into a last-iterate bound. As printed, Lemma 4.3 assumes -a < b <= 0, while Theorem 3.1 applies it with a = (1 - gamma L)/(1 + gamma L) in (0,1) and b = -1. This gives -a in (-1,0), so -a < b is false. The defect is not a harmless weakening: with b = -1, the condition -a < b means a > 1, i.e. a/b < -1, whereas the proof's construction p_t = alpha_{t-1}/alpha_t = (T - t + 1 + a/b)/(T - t + 1) needs p_t in [0,1]. For t = T - 1, this requires 2 + a/b > 0 (roughly a < 2). Thus the printed hypothesis describes the wrong side of the boundary. The proof's algebra goes through for the theorem's values, where a/b in (-1,0), so the central claim is very likely correct, but as written Lemma 4.3 cannot be invoked. A corrected hypothesis (e.g. b < 0 < a and 2 + a/b > 0) and a consistency check of the alpha_t recurrence are needed before the proof is formally valid.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies constant-step-size SGD for minimizing f(x)=E[f_i(x)] where each f_i is convex and L-smooth. The main result (Theorem 3.1) states that, under Assumptions 2.1 and 2.2, E[f(x_T)-inf f] ≤ T^φ [2D^2/(γ(1-γL)T) + 8γ ln(T+1)/(1-γL)^2 σ_*^2], where φ=2γL/(1+γL) and γL∈(0,1); with γ≍1/√T this yields O(ln T/√T) last-iterate complexity. The proof combines a variance-transfer inequality at a minimizer (Lemma 4.1), a one-step recursion (Lemma 4.2), and a weighted telescoping argument (Lemma 4.3) adapted from Zamani-Glineur and Liu-Zhou. The paper also derives corollaries for polynomial step sizes and sample complexity, and acknowledges the concurrent work of Attia et al.","tokens_in":14319,"tokens_out":16766,"duration_ms":176228,"significance":"If the proof is made formally correct, this is a significant result: it removes the uniform gradient/variance assumption that previous last-iterate SGD bounds relied on and reduces the noise condition to finiteness of E||∇f_i(x*)||^2 at a minimizer. The paper proves its key technical lemmas in the text, gives explicit constants, and honestly describes the relation to parallel work; self-citations are used only for standard background inequalities. The O(ln T/√T) rate matches known ergodic rates up to a logarithm, and the authors correctly flag whether the logarithm is removable as open.","major_comments":[{"comment":"The stated hypothesis -a < b ≤ 0 is incompatible with the application in Theorem 3.1. There a=(1-γL)/(1+γL)∈(0,1) and b=-1, so -a∈(-1,0) and the inequality -a < b becomes -a < -1, equivalently a>1, which is false for every γL∈(0,1). The proof's own construction p_t = α_{t-1}/α_t = (T-t+1+a/b)/(T-t+1) requires p_t∈[0,1]; with b<0 and a>0 this holds when 2+a/b>0, i.e. for b=-1 when 0<a<2. Thus the printed hypothesis describes the wrong side of the boundary and Lemma 4.3 cannot be invoked as stated. The algebra in the proof goes through under the corrected hypothesis b<0<a and 2+a/b>0; the authors should restate the lemma accordingly and re-verify the recurrence for α_t in that regime.","section":"Lemma 4.3 (Section 4.2)"},{"comment":"The index ranges in the telescoping argument need to be harmonized. Lemma 4.3 defines α_t for t=1,...,T-1 via the recurrence, while the proof later sets α_T=α_{T-1} and sums the weighted inequality from 0 to T; Lemma A.3, invoked in Theorem 3.1, defines α_t for t=0,...,T with a recurrence that includes t=0. The identity α_T/α_{T-1}=1, which is used to obtain 4T^φ ln(T+1) from Lemma A.1, is not stated as part of the lemmas. Please specify the recurrence for α_0 and α_T and the exact summation range used to derive the final bound in Lemma 4.3.","section":"Lemma 4.3 and Lemma A.3 (Sections 4.2 and A)"}],"minor_comments":[{"comment":"The phrase 'for all zt ∈ H contained in F (x0, . . . , xt)' should be replaced by a clear measurability condition, e.g. 'for every z_t that is F(x0,...,xt)-measurable'.","section":"Lemma 4.3 (Section 4.2)"},{"comment":"The sentence 'we take γ = 1/(2L√T), for some , and T (1+ln(T+1))^2 ≥ K^2/ε^2' contains a missing placeholder; it should read 'for some T' or similar.","section":"Corollary 3.5"},{"comment":"The second displayed inequality is written with a parameter θ that appears to be a typo for φ; additionally, the definition 'for t=0,...,T' conflicts with the recurrence range in Lemma 4.3.","section":"Lemma A.3"},{"comment":"The constant D^2 is not explicitly defined as an infimum over minimizers; since σ_*^2 is defined via an infimum, the bound should clarify that it holds for any x*∈argmin f with D^2=E||x0-x*||^2, or that D^2 is taken as the infimum over minimizers.","section":"Theorem 3.1"},{"comment":"The statement that z_t is 'independent from x_t' is imprecise; what is used is that z_t is measurable with respect to F(x0,...,xt), so that the conditional expectation E_t[⟨∇f_i(x_t), z_t⟩] = ⟨E_t[∇f_i(x_t)], z_t⟩ is valid.","section":"Proof of Lemma 4.2"}],"recommendation":"major_revision","confidential_remarks":"The mathematical contribution is solid, but the formal gap in Lemma 4.3 must be fixed before the proof of Theorem 3.1 is valid. Please also consider the priority situation with Attia et al. (arXiv:2507.11274), which was made public earlier and covers the boundary case γL=1; the current paper acknowledges this. I do not see grounds for rejection on priority grounds, but the novelty discussion should be handled carefully by the editor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Main theorem is new and important: plain SGD last iterate O(ln T/sqrt T) under just convexity, smoothness, and finite variance at the minimizer. That is a real result. The paper is mostly a clean combination of the variance-transfer inequality and the Zamani-Glineur weighted-averaging trick, and it cites the concurrent Attia et al. preprint honestly.\n\nThe proof of Lemma 4.1 is standard and correct. Lemma 4.2 is fine except for one misleading sentence about independence: z_t is in the sigma-algebra, not independent, but the conditional expectation step holds because it is measurable. That is a wording fix.\n\nThe real soft spot is Lemma 4.3. The stress-test note is correct: the stated assumption -a < b <= 0 does not hold for the parameters used in Theorem 3.1, where a = (1 - gamma L)/(1 + gamma L) is in (0,1) and b = -1. For those values, -a > b (e.g., -0.5 > -1). The proof inside Lemma 4.3 needs p_t = alpha_{t-1}/alpha_t = (T - t + 1 + a/b)/(T - t + 1) to lie in [0,1]; the paper's hypothesis points the wrong way. The algebra goes through for the theorem's regime, so the central rate is very likely correct, but as printed Lemma 4.3 cannot be invoked. The lemma statement needs a corrected condition (something like b < 0 < a and 2 + a/b > 0, or a/b in (-1,0)) and a check of the alpha_t recurrence.\n\nOther soft spots: the paper excludes gamma L = 1, which the earlier Attia et al. covers; the logarithmic factor may not be optimal; the results are not adaptive to interpolation. Those are minor, and the authors flag them.\n\nBottom line: if the lemma statement gets fixed, this is a solid contribution that the community will use. The reader's conditional verdict is right. I would send it to a serious referee; the fix is mechanical.","headline":"Solid last-iterate result that settles an open question; printed proof has a fixable sign-error in Lemma 4.3, so the paper deserves peer review after a minor correction.","tokens_in":14958,"tokens_out":2505,"would_cite":true,"duration_ms":25217,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C25","90C15"],"pacs":[],"model":"deepseek-v4-flash","headline":"SGD's last iterate reaches O(ln T/√T) for convex smooth problems, with no uniform variance assumption.","keywords":["stochastic gradient descent","last-iterate convergence","convex smooth optimization","variance-free stochastic analysis","interpolation constant","iterate complexity","minimax rates"],"falsifier":"Run SGD with γ = 1/(2L√T) on any convex, L-smooth finite-sum or expectation problem satisfying σ_*² < ∞ and measure E[f(x_T)-inf f] over many seeds; if for some such problem the decay is slower than C ln(T)/√T for all constants C, the bound is false. A more targeted falsifier is an explicit, convex, L-smooth, non-interpolating example where σ_*² is finite but the last-iterate gap provably violates the stated upper bound, or a performance-estimation computation of the worst-case last-iterate gap exceeding the theorem's constant.","tokens_in":13828,"feed_emoji":"📉","tokens_out":6616,"duration_ms":68788,"temperature":0.7,"pith_summary":"Most guarantees for stochastic gradient descent in convex, smooth problems track the average of the iterates, not the point practitioners actually use, the last iterate. This paper closes that gap for plain SGD: under convexity, L-smoothness, and the single mild condition that the stochastic gradient has finite expected squared norm at some minimizer, the last-iterate expected function gap is O(ln T/√T) with the standard step size γ ≈ 1/(L√T). Earlier last-iterate bounds at this rate required uniform variance or bounded-gradient assumptions, which are hard to verify in practice. The result shows that no modification of SGD, no projection, and no uniform noise control is needed for a near-optimal last-iterate rate.","feed_headline":"Plain SGD's last iterate hits the usual O(ln T/√T) rate","feed_subtitle":"Only a single-point noise condition is needed for the final iterate to match averaged-iterate guarantees.","key_machinery":"The engine is a variance-transfer inequality: because the gradients ∇f_i are L-Lipschitz and the f_i are convex, E[‖∇f_i(x)‖²] ≤ 2L(1+ε)(f(x)-inf f) + (1+1/ε) σ_*² for every ε>0. This replaces a global uniform variance bound by the single constant σ_*² plus a function-gap term, which is exactly the quantity being bounded. The proof then feeds this inequality into a weighted telescoping lemma that converts pointwise descent inequalities into a last-iterate bound: it chooses convex weights α_t, telescopes the squared-distance terms in the SGD recursion, and isolates f(x_T)-inf f. The parameter ε is tuned as (1-γL)/(1+γL), making the coefficient of the function gap positive exactly when γL<1.","core_discovery":"The paper's central claim is that for a convex and L-smooth expected-risk problem with nonempty minimizer set, SGD's last iterate converges at the same near-optimal rate as its averaged iterates. If x* is a minimizer and σ_*² = E[‖∇f_i(x*)‖²] < ∞, then for any fixed horizon T ≥ 3 and step size with γL ∈ (0,1), E[f(x_T)-inf f] ≤ T^φ [2D²/(γ(1-γL)T) + 8γ ln(T+1) σ_*²/(1-γL)²], with φ = 2γL/(1+γL) and D² = E[‖x0-x*‖²]. Choosing γ proportional to 1/(L√T) makes T^φ a constant and yields E[f(x_T)-inf f] = O(ln T/√T). The only noise information needed is the variance at a minimizer; no uniform bound on the variance or norm of the stochastic gradients over the whole space is assumed. The paper presents these as the first such bounds while acknowledging a concurrent independent result with similar conclusions.","pith_inferences":["A natural next test is whether the logarithmic factor is removable: by analogy with the convex Lipschitz case, a nonstandard step-size schedule might give the exact O(1/√T) last-iterate rate; the paper leaves this as a conjecture.","The bound is not interpolation-adaptive: when σ_*² = 0 it remains O(1/√T) rather than O(1/T), so an open extension is a step-size rule that automatically switches to the fast rate when interpolation holds, using only quantities the algorithm can estimate.","It would be worth testing whether existing adaptive step-size rules inherit last-iterate guarantees under the same single-point variance condition; the paper notes this as an open direction.","A high-probability version of the last-iterate guarantee without uniform noise bounds does not follow from the expectation analysis and is not claimed by the paper; that is a concrete open problem."],"forward_implications":["With step size γ = 1/(C L √T), C ≥ 2, the explicit corollary gives E[f(x_T)-inf f] ≤ 9 C L D²/√T + 67 ln(T+1) σ_*²/(C L √T), matching the classical ergodic SGD rate up to a logarithm.","For any polynomial step size γ = 1/(C L T^β), β ∈ (0,1), the bound is O(D²/T^{1-β} + ln T σ_*²/T^β); β = 1/2 is the best exponent in this family.","The same framework extends to mini-batch SGD, because minibatch sampling is SGD on a modified problem whose smoothness constant and σ_*² are explicit functions of batch size.","A consequence is an iterate-complexity guarantee: E[f(x_T)-inf f] ≤ ε once T is of order (K/ε)^β for any β > 2, ignoring logarithms.","The bound is adaptive to the noise constant σ_*² in the sense that the algorithm need not know it to obtain the stated rate."],"supporting_citations":[{"why":"Supplies the foundational non-asymptotic SGD analysis and the first last-iterate bounds in this setting.","marker":"Bach and Moulines (2011)"},{"why":"Provides the variance-transfer lemma and expected smoothness inequality that the proof's Lemma 4.1 is built on.","marker":"Garrigos and Gower (2024)"},{"why":"Contributes the weighted telescoping last-iterate technique that Lemma 4.3 generalizes.","marker":"Zamani and Glineur (2023)"},{"why":"Prior last-iterate SGD result relying on the same technique under a uniform variance assumption, which this paper removes.","marker":"Liu and Zhou (2023)"},{"why":"Establishes the variance-transfer approach for ergodic bounds and supplies the mini-batch reduction used in Remark 3.9.","marker":"Gower et al. (2019)"},{"why":"Provides optimal ergodic bounds without variance assumptions, and a lemma used to verify the single-point variance condition.","marker":"Cortild et al. (2025)"},{"why":"Concurrent independent work with analogous results, additionally covering the boundary γL = 1.","marker":"Attia et al. (2025)"},{"why":"Earlier sharp last-iterate bounds under uniform variance assumptions, providing the comparison baseline.","marker":"Taylor and Bach (2019)"}],"fun_headline_variants":["Last-iterate SGD matches averaged rates without uniform variance bounds","Only minimizer variance needed for SGD last-iterate rate","SGD last iterate: no uniform noise bound needed","SGD's final iterate reaches O(ln T/√T) under weak noise"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole proof is carried by the assumption that σ_*², the expected squared norm of the stochastic gradient at some minimizer, is finite — if that single-point variance is infinite, the stated bound is vacuous — and by the step-size restriction γL < 1, so the endpoint γL = 1, which a concurrent work covers, is outside the theorem.","fun_headline_variants_meta":{"raw":{"variants":["Last-iterate SGD matches averaged rates without uniform variance bounds","Only minimizer variance needed for SGD last-iterate rate","SGD last iterate: no uniform noise bound needed","SGD's final iterate reaches O(ln T/√T) under weak noise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001016,"raw_usage":{"total_tokens":4267,"prompt_tokens":902,"completion_tokens":3365,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":3293}},"tokens_in":518,"tokens_out":3365,"duration_ms":28267,"temperature":1.0,"reasoning_tokens":3293,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:01:22.653670+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SGD with γ = 1/(2L√T) on any convex, L-smooth finite-sum or expectation problem satisfying σ_*² < ∞ and measure E[f(x_T)-inf f] over many seeds; if for some such problem the decay is slower than C ln(T)/√T for all constants C, the bound is false. A more targeted falsifier is an explicit, convex, L-smooth, non-interpolating example where σ_*² is finite but the last-iterate gap provably violates the stated upper bound, or a performance-estimation computation of the worst-case last-iterate gap exceeding the theorem's constant.","supporting_citations":[{"cited_title":"and Moulines, E","cited_arxiv_id":null,"evidence_quote":"Supplies the foundational non-asymptotic SGD analysis and the first last-iterate bounds in this setting."},{"cited_title":"M., Loizou, N., Qian, X., Sailanbayev, A., Shulgin, E., and Richtárik, P","cited_arxiv_id":null,"evidence_quote":"Establishes the variance-transfer approach for ergodic bounds and supplies the mini-batch reduction used in Remark 3.9."},{"cited_title":"and Bach, F","cited_arxiv_id":null,"evidence_quote":"Earlier sharp last-iterate bounds under uniform variance assumptions, providing the comparison baseline."}],"review_version":1}