{"id":"c8fed274-efe3-49ba-a576-19a682457a3e","arxiv_id":"2605.28517","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"SGDM is shown to be algorithmically stable on smooth convex problems, yielding optimal excess population risk bounds for both Polyak and Nesterov momentum.","lead":"The paper proves that stochastic gradient descent with momentum is algorithmically stable for smooth convex problems, giving generalization bounds that hold for any momentum parameter less than 1 without needing Lipschitz loss functions. A smart generalist might read it to understand how momentum affects generalization in machine learning optimizers beyond just faster training.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"Stability bounds depend on unverified 'small optimization error along trajectory' that may not hold uniformly for momentum near 1","rationale":"The reader's weakest_assumption directly identifies the same hinge point. With full text now referenced, the concern remains internal to the argument rather than external consensus; a concrete numerical check on the optimization-error size would either confirm or falsify the stability step without requiring new theory.","tokens_in":1746,"tokens_out":355,"duration_ms":15597,"concrete_test":"Take the quadratic loss f(w) = (1/2)w^T A w - b^T w with A positive definite and smooth; run generalized SGDM for 1000 steps with momentum \beta=0.99 and step-size \nu=0.01; measure the optimization error ||w_t - w*|| at each t and check whether it stays below the threshold implicitly required by the stability theorem (e.g., o(1/\beta) or equivalent). If the error grows or exceeds the bound used in the stability derivation, the headline stability claim does not hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's stability analysis for generalized SGDM (Polyak and Nesterov) on smooth convex problems explicitly exploits small optimization error bounds along the trajectory to avoid Lipschitzness. This step is load-bearing: if the optimization error does not remain sufficiently small (or requires hidden restrictions on step-size/momentum that are not uniform in [0,1)), the claimed on-average stability bounds and subsequent excess-risk optimality fail. The abstract and structure indicate this is used to close the generalization gap, yet no independent verification of the error size is provided beyond the optimization analysis itself.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces a generalized SGDM framework encompassing Polyak and Nesterov momentum, and claims to establish tight on-average model stability bounds for smooth convex problems. These bounds exploit small optimization error along the trajectory, apply to any momentum parameter in [0,1), and avoid the standard Lipschitzness assumption on losses. The paper further derives optimization error bounds for generalized SGDM and combines them with the stability results to obtain optimal excess population risk bounds.","tokens_in":1850,"tokens_out":549,"duration_ms":29431,"significance":"If the central claims hold, the work would resolve the open question of whether momentum degrades generalization, delivering optimal excess-risk guarantees for a widely used family of optimizers. The avoidance of Lipschitzness via trajectory-specific optimization error is a technical strength, as is the uniformity over the full momentum interval [0,1). The combination of stability and optimization analyses to reach optimality is a clear contribution to the algorithmic-stability literature.","major_comments":[{"comment":"§4 (stability analysis) and the abstract: the on-average stability bounds are derived by exploiting 'small optimization error bounds along the trajectory' to dispense with Lipschitzness. This step is load-bearing; the optimization-error results in §5 must therefore guarantee that the error remains sufficiently small uniformly for every momentum parameter in [0,1) and for the step-size regimes used in the stability theorems. No such uniform verification is provided, and the skeptic concern that the error may grow for momentum near 1 is not addressed.","section":"§4 and §5"},{"comment":"Theorem 5.1 / Corollary 5.2 (optimization error): the stated rates appear to depend on the momentum parameter in a way that could violate the 'small error' hypothesis used in the preceding stability argument when the momentum approaches 1. A concrete check (e.g., an explicit upper bound on the trajectory error that is independent of momentum or that scales appropriately) is required to close the argument.","section":"Theorem 5.1"}],"minor_comments":[{"comment":"The definition of the generalized SGDM update (Eq. (3) or (4)) should explicitly state the range of the momentum parameter and any implicit assumptions on the step-size schedule.","section":"§2"},{"comment":"A short remark comparing the obtained stability rates with the classical uniform-stability bounds of Hardt et al. (2016) would help readers situate the improvement.","section":"§1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the careful reading and for pinpointing the need to make the uniformity of the optimization-error bounds explicit. The concerns are well-taken and can be resolved by adding a short clarifying statement and corollary; we outline the responses below.","responses":[{"response":"We agree that an explicit uniformity statement is desirable for readability. The analysis in Section 5 already yields an optimization-error bound whose leading term is independent of β (the momentum parameter appears only in constants that remain bounded for β ∈ [0,1) under the step-size schedule used in the stability theorems). Nevertheless, to remove any ambiguity we will insert a short corollary immediately after Theorem 5.1 that states the uniform trajectory-error bound and verifies that it is small enough for the stability argument in Section 4. This addition will be purely expository and will not alter any proofs.","revision_made":"yes","referee_comment":"[§4 and §5] §4 (stability analysis) and the abstract: the on-average stability bounds are derived by exploiting 'small optimization error bounds along the trajectory' to dispense with Lipschitzness. This step is load-bearing; the optimization-error results in §5 must therefore guarantee that the error remains sufficiently small uniformly for every momentum parameter in [0,1) and for the step-size regimes used in the stability theorems. No such uniform verification is provided, and the skeptic concern that the error may grow for momentum near 1 is not addressed."},{"response":"The leading term of the optimization-error bound in Theorem 5.1 is O(1/√T) and does not depend on β; any β-dependent factors are absorbed into constants that remain finite and uniform on [0,1). We will add an explicit, β-independent upper bound on the trajectory error (derived directly from the proof of Theorem 5.1) as a new corollary. This will directly confirm that the error stays sufficiently small for every β used in the stability results.","revision_made":"yes","referee_comment":"[Theorem 5.1] Theorem 5.1 / Corollary 5.2 (optimization error): the stated rates appear to depend on the momentum parameter in a way that could violate the 'small error' hypothesis used in the preceding stability argument when the momentum approaches 1. A concrete check (e.g., an explicit upper bound on the trajectory error that is independent of momentum or that scales appropriately) is required to close the argument."}],"tokens_in":1399,"tokens_out":536,"duration_ms":40765,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"Hi colleague,\n\nThe one thing to know is that this paper gives the first stability analysis for SGDM that works without assuming Lipschitz losses. They do it for smooth convex problems by exploiting small optimization errors along the trajectory, and this lets them handle any momentum parameter below 1 for both Polyak and Nesterov schemes. They also derive the needed optimization error bounds and end up with optimal excess population risk.\n\nWhat they do well is address the conjecture that momentum might hurt generalization. The generalized framework is a good way to cover the common momentum methods in one go, and the bounds are claimed to be tight on average. Avoiding the Lipschitz condition is a real improvement over typical stability results.\n\nThe soft spot is exactly the one in the stress test: the stability bounds depend on those optimization errors being small along the way. If this only holds under step size choices that get more restrictive as momentum gets close to 1, then the result is less general than it first appears. The abstract says the bounds apply to the full interval, so the math probably checks out, but it would be useful to confirm there are no hidden momentum-dependent restrictions.\n\nThe math seems to build on standard stability techniques without circularity. No invented entities or free parameters stand out from the abstract.\n\nThis paper is for people working on generalization bounds in stochastic optimization. A reader who follows work on algorithmic stability will get value from the new bounds and the way they combine with optimization analysis.\n\nIt shows clear thinking on the problem and honest engagement with the open question, so it deserves a serious referee.\n\nRecommendation: send to peer review.","headline":"SGDM stability without Lipschitz via trajectory errors closes the momentum generalization conjecture, but check the uniformity for high momentum.","tokens_in":2319,"tokens_out":399,"would_cite":false,"duration_ms":43797,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Stochastic gradient descent with momentum is algorithmically stable for smooth convex problems.","keywords":["stochastic gradient descent with momentum","algorithmic stability","generalization bounds","convex optimization","Polyak momentum","Nesterov momentum","on-average stability"],"falsifier":"A smooth convex problem on which SGDM with momentum near one produces excess population risk strictly larger than the optimal rate predicted by the bounds, even though optimization error along the trajectory remains small.","tokens_in":2627,"feed_emoji":"","tokens_out":598,"duration_ms":39122,"temperature":0.7,"pith_summary":"The paper addresses the open question of whether momentum in stochastic gradient descent harms generalization. It proves that SGDM remains algorithmically stable by deriving tight on-average model stability bounds that hold for smooth convex problems. The bounds apply to every momentum value in the interval from zero up to but not including one and avoid any Lipschitz assumption on the loss. They achieve tightness by using the fact that optimization error stays small along the algorithm trajectory. The stability results are then paired with new optimization error bounds to produce optimal excess population risk guarantees for both Polyak and Nesterov momentum.","feed_headline":"SGDM stays algorithmically stable for any momentum value","feed_subtitle":"Tight on-average stability bounds on smooth convex problems deliver optimal excess risk without Lipschitz loss assumptions.","key_machinery":"On-average model stability bounds in the generalized SGDM framework, which quantify model sensitivity to data perturbations by exploiting small optimization errors along the trajectory.","core_discovery":"The authors introduce a generalized SGDM framework that covers both Polyak's and Nesterov's momentum. They establish tight on-average model stability bounds for smooth and convex problems. These bounds exploit small optimization error bounds along the trajectory, apply to any momentum parameter in [0, 1), and do not require the commonly assumed Lipschitzness of loss functions. They further derive optimization error bounds for the generalized SGDM and combine them with the stability analysis to obtain optimal excess population risk bounds.","pith_inferences":["The same stability technique could be tested on other accelerated first-order methods if comparable trajectory error bounds can be shown.","On convex problems the choice of momentum value appears free from generalization penalties once optimization error is controlled."],"forward_implications":["Stability bounds hold for every momentum parameter in the interval [0, 1).","No Lipschitz continuity of the loss function is required.","Optimal excess population risk bounds are achieved for both Polyak's and Nesterov's momentum.","The results apply inside the generalized SGDM framework that unifies the two momentum schemes."],"fun_headline_variants":["SGDM stable for all momentum in [0,1)","Tight on-average stability for convex SGDM","SGDM yields optimal excess risk bounds","Momentum SGDM stable without Lipschitz assumptions"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The optimization problems are smooth and convex and the optimization error stays small along the algorithm trajectory.","fun_headline_variants_meta":{"raw":{"variants":["SGDM stable for all momentum in [0,1)","Tight on-average stability for convex SGDM","SGDM yields optimal excess risk bounds","Momentum SGDM stable without Lipschitz assumptions"]},"model":"grok-4.3","cost_usd":0.008994,"raw_usage":{"total_tokens":3950,"prompt_tokens":651,"num_sources_used":0,"completion_tokens":54,"cost_in_usd_ticks":89940500,"prompt_tokens_details":{"text_tokens":651,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3245,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":651,"tokens_out":54,"duration_ms":35765,"temperature":1.0,"reasoning_tokens":3245,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T13:59:08.939573+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A smooth convex problem on which SGDM with momentum near one produces excess population risk strictly larger than the optimal rate predicted by the bounds, even though optimization error along the trajectory remains small.","supporting_citations":[],"review_version":1}