{"id":"a43ad50b-ebe5-4a5d-8c98-2426a6b1fdb1","arxiv_id":"2412.00640","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"For locally Lipschitz tame functions, stable points of first-order methods are exactly local minima, strict local minima are stable, and the critical-point set is globally stable for coercive functions.","lead":"This doctoral thesis proves that, for nonsmooth tame optimization problems, constant-step-size first-order methods keep iterates near a point exactly when that point is a local minimum: stable points are local minima and strict local minima are stable. It also shows that the set of critical points is globally attracting for coercive tame functions and gives conditions under which spurious local minima are actually unstable.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Approximation property for Algorithms 2–4 is asserted, not proved; since Definition 2.1 is stronger than [83, Def. 3], the stability theorems for momentum, reshuffling, and coordinate descent lack support.","rationale":"The reader's weakest-assumption identification is accurate: the local and global stability theorems are conditional on Definition 2.1, and for Algorithms 2–4 that condition is delegated to a self-cited prior work rather than established in the thesis. My reading confirms that this is the most load-bearing gap. The subgradient-method portion, Theorem 2.2 and the resulting stability theorems, appears internally coherent and is supported by direct arguments. The missing piece is scope: the abstract and Table 2.1 advertise momentum, random reshuffling, and coordinate descent, but their approximation proofs are not present. Because Definition 2.1 is stronger than the approximation notion in [83], the adaptation is not automatic. Thus the conditional verdict is appropriate; no change is needed.","tokens_in":51702,"tokens_out":14553,"duration_ms":139581,"concrete_test":"Write out a complete proof of Definition 2.1 for Algorithm 2 with scaling c = 1/(1-β), starting from the two-step state space and verifying the finite-horizon uniform approximation estimate for arbitrary k0 and compact X1. In particular, derive the uniform bound ||x_k - x_{k-1}|| = O(α) from the initialization condition and β < 1, and check that the error due to evaluating ∂f at y_k rather than x_k is uniform over X1. If the derivation requires restricting β, γ, or X1 beyond what Definition 3.1 imposes, then Table 2.1 overclaims and the stability theorems should be scoped to Algorithm 1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Theorems 3.5 and 3.6, together with Theorem 4.2, apply to any iterative method satisfying Definition 2.1. For the subgradient method, Theorem 2.2 supplies the approximation property. For Algorithms 2–4, however, the document only states that 'using the techniques in [83], the same proof can be adapted' (paragraph after Table 2.1) and Remark 2.4 delegates the argument to [83, Section 4]. This is a proof gap rather than a proof: Definition 2.1 requires finite-horizon uniform approximation for every compact X0, X1 and every starting index k0, and the text explicitly notes that Definition 2.1 differs from [83, Definition 3], which was sufficient for the global-stability setting. Furthermore, Algorithm 2 is a two-step method: Definition 3.1 constrains only x0, while the algorithm's state includes x_{-1} with ||x_{-1}-x0|| ≤ δα; Definition 2.1's shifted quantifier over k0 must be shown compatible with this state condition. None of the required estimates involving β, γ, boundedness of iterates, or the O(α) momentum state appear in the thesis. If any of these methods fails to satisfy Definition 2.1, Theorems 3.5, 3.6, and 4.2 do not apply to it, so the advertised scope over 'common first-order methods' is unsupported. This is an unsupported-scope concern, not an internal-inconsistency claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript studies constant-step-size first-order methods for minimizing locally Lipschitz tame functions. It introduces Definition 2.1, an approximation property requiring that discrete iterates be uniformly close, over finite horizons, to some subgradient trajectory of the objective. Theorem 2.2 proves this property for the vanilla subgradient method. The thesis then defines local M-stability (Definition 3.1) and global stability (Definition 4.1), and proves that stable points are necessarily local minima (Theorem 3.5), that strict local minima are stable for every method satisfying Definition 2.1 (Theorem 3.6), and that the set of critical points of a coercive locally Lipschitz tame function is globally stable (Theorem 4.2). Chapter 5 gives sufficient conditions for strong instability of the subgradient method, using Chetaev functions, metric theta-subregularity, and the Verdier condition, with applications to a ReLU-network loss and robust PCA. The central results are conditional on Definition 2.1; Table 2.1 asserts this property for momentum, random reshuffling with momentum, and random-permutations coordinate descent, but the derivations are delegated to the author's earlier work rather than reproduced.","tokens_in":52010,"tokens_out":6201,"duration_ms":59431,"significance":"If the approximation property is established for all four methods, the paper provides a coherent discrete Lyapunov theory for nonsmooth tame optimization: local stability coincides with local minimality modulo strictness, and the critical set is globally attracting in the coercive case. The proofs of Theorems 3.5, 3.6, and 4.2 use the Kurdyka-Lojasiewicz inequality and the chain rule for subgradient trajectories carefully, and Theorem 2.2 for the subgradient method is fully proved in the text. The instability criteria in Chapter 5 are concrete and are verified on two nontrivial applications, which is a strength. However, the advertised scope over 'common first-order methods' is not supported by the manuscript itself: Definition 2.1 is stronger than the approximation notion used in the cited earlier work, and for Algorithms 2-4 the paper only states that the proof 'can be adapted' without giving the required estimates. The stability theorems are therefore rigorously established only for the subgradient method within this document.","major_comments":[{"comment":"Definition 2.1 requires uniform finite-horizon approximation for every pair of compact sets X0, X1 and every starting index k0, and the text explicitly notes that this is stronger than [83, Definition 3]. For Algorithms 2-4 the manuscript only says that 'using the techniques in [83], the same proof can be adapted' and Remark 2.4 delegates the argument to [83, Section 4]. No explicit estimates involving the momentum parameters beta, gamma, the boundedness of iterates, or the O(alpha) momentum state are provided. Since Theorems 3.5, 3.6, and 4.2 are all formulated for an arbitrary method satisfying Definition 2.1, the local and global stability claims for Algorithms 2-4 are not supported by the arguments given in this manuscript.","section":"Chapter 2, Table 2.1 and Remark 2.4"},{"comment":"Algorithm 2 is a two-step method whose state includes x_{k-1}, and its initialization requires ||x_{-1}-x0|| <= delta*alpha. Definition 3.1, however, constrains only x0 when testing local stability. Definition 2.1 quantifies over all k0 and over initial segments x0,...,x_{k0} in X1, but the stability condition gives no information about x_{-1}. The manuscript does not show that a uniform approximation property holds when the momentum state is initialized only through the condition ||x_{-1}-x0|| <= delta*alpha, nor does it explain how Definition 2.1's shifted quantifier over k0 is compatible with this state condition. This gap must be resolved before Theorem 3.6 can be applied to Algorithm 2.","section":"Definitions 2.1 and 3.1 with Algorithm 2"},{"comment":"For the random reshuffling and random-permutations coordinate descent methods, Definition 2.1 is written deterministically: it requires the approximation for every sequence in M(f, alpha, X0, k0). The manuscript does not clarify whether this property is meant to hold for every random permutation realization or only with high probability. If [83, Section 4] provides only a probabilistic or in-expectation approximation, the deterministic statements of Theorems 3.5, 3.6, and 4.2 do not follow from that reference. The author should state the exact mode of approximation and either prove the deterministic version or restrict the scope of the theorems accordingly.","section":"Definition 2.1 with Algorithms 3 and 4"}],"minor_comments":[{"comment":"The sentence 'Examples of such functions are can be found in Chapter 3' is grammatically broken and should be corrected.","section":"Introduction, page 2"},{"comment":"Theorem 1.6 defines the desingularizing function on [0, infinity), while Theorems 3.5 and 3.6 use it on [0, rho); the notation should be harmonized to avoid a mismatch of domains.","section":"Theorem 1.6 vs. Theorems 3.5 and 3.6"},{"comment":"The roles of X0 and X1 could be stated more explicitly: X0 contains the k0-th term and X1 contains the initial segment, but the condition x(0) in X0 does not require x(0) to equal x_{k0}; this is intentional for shifted trajectories but should be clarified, especially because Theorem 3.6 invokes Definition 2.1 with a new initial compact set.","section":"Definition 2.1"},{"comment":"The proof says 'Consider a sequence generated by the subgradient method' even though the theorem is stated for an arbitrary method M satisfying Definition 2.1; this should read 'by M' to avoid a logical mismatch.","section":"Proof of Theorem 3.6, around Eq. (3.7)"},{"comment":"The function sign is defined as a set-valued mapping with sign(0) = [-1,1], but the text repeatedly calls it a function; the set-valued convention should be stated consistently in the proof and in the application to robust PCA.","section":"Lemma 5.9 and Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a doctoral thesis whose main results overlap substantially with the author's earlier papers, especially [83] and the two articles cited at the starts of Chapters 3 and 5. The genuinely new material is the local stability characterization and the stronger finite-horizon approximation definition. The main revision needed is to close the gap between Definition 2.1 and the asserted behavior of Algorithms 2-4; without that, the breadth of the main theorems exceeds what is proven. I would not reject the paper, since the gap is fillable in principle by reproducing or extending the arguments from [83], but the current text does not do so."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a dissertation that compiles the author's prior peer-reviewed work on discrete Lyapunov stability for first-order methods on tame functions. The core theorems are real: for locally Lipschitz tame functions, stable points are local minima (Thm 3.5), strict local minima are stable (Thm 3.6), and there is a global stability result for coercive functions (Thm 4.2) plus an instability analysis for non-strict minima (Thm 5.10). The proofs of these for the subgradient method are careful and use the Kurdyka-Łojasiewicz inequality and the chain rule properly. The examples, especially the Chetaev-function instability for a ReLU network and robust PCA, are concrete and illuminating.\n\nThe main soft spot is a genuine proof gap in the advertised scope. Algorithms 2–4 (momentum, random reshuffling, coordinate descent) are claimed to satisfy Definition 2.1, but the thesis only says the proof in [83] can be adapted. Definition 2.1 is stronger than [83, Def. 3], requiring approximation over any finite horizon and any starting index. For momentum, the two-step state also needs a compatibility argument involving x_{-1}. Without those estimates, Theorems 3.5, 3.6, and 4.2 are rigorously established only for the subgradient method in this document. This is a serious caveat, but not a fatal one: the gap is fixable if the missing approximations are supplied or the claims are explicitly scoped to the subgradient method.\n\nNovelty is modest relative to the cited prior papers (Mathematical Programming 2023, SIAM J Optim 2024). This is a compilation with fuller exposition rather than a new result. The minor typo in Lemma 5.9 does not affect the conclusion.\n\nWho should read this? Someone who wants a single source for the stability theory of the subgradient method on tame nonsmooth functions, and who wants the Chetaev-function examples. If you work on momentum or coordinate descent in this setting, do not rely on the current version for those algorithms.\n\nMy recommendation: send it to serious referees, but with a request to either fill the approximation gap for Algorithms 2–4 or narrow the theorems to the subgradient method. As it stands, treat the subgradient-method results as solid and the broader claims as conditional.","headline":"Solid compilation of previously published stability results for the subgradient method on tame functions; the advertised extension to momentum, reshuffling, and coordinate methods is asserted rather than proved.","tokens_in":52523,"tokens_out":3845,"would_cite":false,"duration_ms":34511,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["49J52","90C30","37B25","03C64"],"pacs":[],"model":"deepseek-v4-flash","headline":"In tame nonsmooth optimization, stable points of first-order methods must be local minima, strict local minima are stable, and coercive objectives draw iterates to their critical set.","keywords":["first-order methods","subgradient method","constant step size","tame functions","o-minimal structures","Lyapunov stability","nonsmooth optimization","spurious local minima"],"falsifier":"Take a semialgebraic locally Lipschitz function with a strict local minimum $x^*$ and simulate the subgradient method for step sizes $\\alpha=2^{-j}$ from initializations in $B(x^*,2^{-j})$; if for some fixed $\\epsilon>0$ the iterates leave $B(x^*,\\epsilon)$ for arbitrarily small such pairs, the sufficiency direction of the stability theorem would be refuted, while any stable point that is not a local minimum would refute the necessity direction.","tokens_in":51458,"feed_emoji":"📉","tokens_out":10512,"duration_ms":95094,"temperature":0.7,"pith_summary":"This paper develops a discrete Lyapunov stability theory for first-order optimization methods with constant step size on tame nonsmooth objectives, meaning functions definable in an o-minimal structure, a class that covers most objectives used in statistical learning. It establishes a local dichotomy: a point is stable for any such method only if it is a local minimum, and it is stable whenever it is a strict local minimum. For coercive tame functions, it proves the set of critical points is globally stable, so iterates from any bounded initialization eventually enter every prescribed neighborhood of the critical set once the step size is small enough. It also shows that some spurious local minima are strongly unstable for the subgradient method without added noise, under geometric conditions (higher-order metric subregularity and the Verdier condition) verified on a ReLU neural-network loss and robust PCA. The practical consequence is that for highly irregular objectives, whether first-order iterates stay near a point is governed by the tame structure of the objective rather than by convexity or smoothness.","feed_headline":"Stable points of tame optimization are exactly local minima","feed_subtitle":"Discrete Lyapunov theory pins down when constant-step iterates stay near a point or escape to a critical set.","key_machinery":"The carrying mechanism is the approximation property (Definition 2.1): a method is approximated by subgradient trajectories when, for every compact initialization and finite time horizon, all sufficiently small constant step sizes produce iterates that stay within $\\epsilon$ of some absolutely continuous solution of $x'(t)\\in -c\\,\\partial f(x(t))$. This transfers the discrete iteration into the continuous subgradient flow, where the chain rule for tame functions, $(f\\circ x)'=-\\|x'\\|^2$, guarantees monotone decrease until a critical point is reached. Around a local minimum, strictness plus a Łojasiewicz-type inequality creates a connected sublevel-set barrier that keeps the shadowing trajectory inside a small ball; around a spurious critical point, a Chetaev function whose values grow along iterates, together with metric subregularity and the Verdier condition, prevents the distance to the critical manifold from shrinking, forcing escape.","core_discovery":"The central claim is that first-order methods that are approximated by subgradient trajectories behave, over finite horizons, like the continuous subgradient flow $x'\\in -c\\partial f(x)$, and that this approximation turns discrete Lyapunov stability into a local-optimality statement. The paper proves that a stable point of a locally Lipschitz tame function must be a local minimum (Theorem 3.5), that every strict local minimum is stable (Theorem 3.6), and that for coercive tame functions the set of critical points is globally stable (Theorem 4.2): for every $\\epsilon>0$ and every bounded set of initial points, a sufficiently small constant step size forces the iterates to eventually lie in $B(\\mathrm{crit}(f),\\epsilon)$. For the subgradient method, it further proves that a spurious local minimum is strongly unstable when a Chetaev function increases along the iterates and the geometry satisfies metric $\\theta_2$-subregularity of $\\partial f$ with $\\theta_2>1$ together with the Verdier condition along the manifold of critical points. Applications verify these conditions for a ReLU neural-network loss with $\\ell^1$ error and for robust principal component analysis with data matrices having zero rows or columns, so the escape is deterministic and almost sure for nearby initializations.","pith_inferences":["The approximation framework is likely portable to other first-order schemes, such as proximal or adaptive step methods, because the proof of shadowing uses only upper semicontinuity of the Clarke subdifferential, boundedness of iterates, and the tame chain rule; this extension is not carried out in the paper.","The instability criteria suggest a preprocessing certificate: on semialgebraic objectives one could algorithmically search stratifications for a Chetaev function and Verdier condition and flag spurious minima before running optimization; the paper does not propose such an algorithm.","If the approximation property extends to nonsummable diminishing step sizes, as the introduction indicates for most results, the local stability dichotomy would be schedule-independent; this would be a testable strengthening for momentum and reshuffling variants.","The verified examples hint that noise-free escape from sharp spurious minima may be common in real training objectives; a broader numerical survey across benchmark losses would test how often the Verdier condition holds in practice."],"forward_implications":["Stable non-minimizing points are impossible for tame objectives: for any method satisfying the approximation property, stability of a point forces it to be a local minimum.","Every strict local minimum of a locally Lipschitz tame function is stable, so sufficiently close initialization and sufficiently small step size confine all iterates to any prescribed neighborhood.","For coercive tame functions, the critical set is globally attracting: from any bounded initialization and with small enough constant step size, iterates eventually enter every neighborhood of the critical set.","Spurious local minima can be left without stochastic noise: under the geometric conditions of Theorem 5.10, the subgradient method escapes from almost every nearby initialization for all but finitely many step sizes.","Function values along the iterates settle near a critical value, so the global behavior is a value-level stabilization and not merely set-wise convergence."],"supporting_citations":[{"why":"Supplies the approximation-by-subgradient-trajectories proof scheme that the paper adapts to momentum, random reshuffling, and coordinate descent variants.","marker":"[83]"},{"why":"Provides the definable Morse–Sard theorem and the Kurdyka–Łojasiewicz inequality for locally Lipschitz tame functions used throughout Chapters 3 and 4.","marker":"[29]"},{"why":"Supplies the chain rule for subgradient trajectories of tame functions, $(f\\circ x)'=-\\|x'\\|^2$, which drives all descent estimates.","marker":"[33]"},{"why":"Establishes the Łojasiewicz-type inequality for tame functions that yields the strict-local-minimum stability proof.","marker":"[28]"},{"why":"Defines o-minimal structures and cell decomposition, the tameness framework that rules out oscillatory Rockafellar-type counterexamples.","marker":"[13]"},{"why":"Introduces the Verdier condition in nonsmooth optimization; the paper borrows it as a geometric hypothesis for instability.","marker":"[105]"},{"why":"Develops active-manifold and stratification tools for nonsmooth convergence; used as background for the Verdier condition and the nonconvergence phenomenon.","marker":"[106]"},{"why":"Defines metric $\\theta$-subregularity, the higher-order regularity hypothesis in the instability theorem.","marker":"[101]"},{"why":"Guarantees that the projection onto the critical manifold is single-valued and Lipschitz near the point, a step in the unstable-distance argument.","marker":"[121]"},{"why":"Shows that iterates avoid null sets for all but finitely many step sizes, making strong instability almost-sure.","marker":"[119]"}],"fun_headline_variants":["Stable points of tame functions are exactly local minima","Discrete Lyapunov stability equals local minimality","For tame functions, stable points are local minima","No stable spurious minima in tame optimization","Tame optimization: stability pins down local minima"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results rest on the approximation property for each method—the claim that over every finite horizon its iterates, with arbitrarily small constant step size, shadow a subgradient trajectory; for the subgradient method this is proved, whereas for the momentum, random-reshuffling, and coordinate-descent variants it is only asserted to follow from a cited proof.","fun_headline_variants_meta":{"raw":{"variants":["Stable points of tame functions are exactly local minima","Discrete Lyapunov stability equals local minimality","For tame functions, stable points are local minima","No stable spurious minima in tame optimization","Tame optimization: stability pins down local minima"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1709,"prompt_tokens":926,"completion_tokens":783,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":711}},"tokens_in":542,"tokens_out":783,"duration_ms":30293,"temperature":1.0,"reasoning_tokens":711,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:09:20.519272+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a semialgebraic locally Lipschitz function with a strict local minimum $x^*$ and simulate the subgradient method for step sizes $\\alpha=2^{-j}$ from initializations in $B(x^*,2^{-j})$; if for some fixed $\\epsilon>0$ the iterates leave $B(x^*,\\epsilon)$ for arbitrarily small such pairs, the sufficiency direction of the stability theorem would be refuted, while any stable point that is not a local minimum would refute the necessity direction.","supporting_citations":[{"cited_title":"Global stability of first-order methods for coercive tame functions,","cited_arxiv_id":null,"evidence_quote":"Supplies the approximation-by-subgradient-trajectories proof scheme that the paper adapts to momentum, random reshuffling, and coordinate descent variants."},{"cited_title":"Stochastic subgradient descent escapes active strict saddles on weakly convex functions,","cited_arxiv_id":null,"evidence_quote":"Introduces the Verdier condition in nonsmooth optimization; the paper borrows it as a geometric hypothesis for instability."},{"cited_title":"Holder metric subregularity with applications to proximal point method,","cited_arxiv_id":null,"evidence_quote":"Defines metric $\\theta$-subregularity, the higher-order regularity hypothesis in the instability theorem."},{"cited_title":"Local differentiability of distance functions,","cited_arxiv_id":null,"evidence_quote":"Guarantees that the projection onto the critical manifold is single-valued and Lipschitz near the point, a step in the unstable-distance argument."},{"cited_title":"A mathematical model for automatic differentiation in machine learning,","cited_arxiv_id":null,"evidence_quote":"Shows that iterates avoid null sets for all but finitely many step sizes, making strong instability almost-sure."}],"review_version":1}