{"id":"505db66d-95f3-4c79-86fb-8490f6ae46c2","arxiv_id":"2505.15114","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A framework that adaptively chooses the momentum coefficient in gradient-based optimization and guarantees a 1/k convergence rate under convexity.","lead":"This paper introduces AIM, an optimization procedure that adds an adjustable momentum term to gradient descent and sets the momentum strength automatically from the current gradient and direction. The authors prove a standard one-over-k convergence rate for the framework and show that a special variant can match a regularized Newton method under restrictive conditions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central O(1/k) theorem is false as stated: with μ_k→1, M_k degenerates and a 1D quadratic example gives Θ(k^{−0.2}) convergence, not O(1/k); the proof's telescoping over changing M_k norms also lacks a uniform bound.","rationale":"The paper's stated goal is a framework with adaptive inertial weight that nevertheless achieves global O(1/k) under only convexity and local Lipschitz differentiability. For that goal, the decisive condition is whether the per-iteration energy decrease can be converted into a uniform rate. The paper defines M_k in (2.4) and proves Lemma 2.8 in the M_k-norm. Lemma 2.9 attempts to telescope this over k. This is the point of failure: the norm is not fixed; the same M_k appears on both sides for a single step only, and no bound like μ_k≤μ_max<1 is stated. The notation in (2.18), with M_k on every term, suggests the authors treated M_k as constant. Even accepting (2.22), the RHS has ||x0−x*||^2_{M_k}, which may be as large as O(k) when μ_k→1 in one dimension, so the displayed inequality does not imply O(1/k). The counterexample shows the theorem is not merely underproved but false. It satisfies every hypothesis of Theorem 2.10 and converges as Θ(k^{−0.2}). Hence a reader cannot rely on the abstract's claim. A repair is straightforward: add μ_k≤μ_max<1 (or another uniform nondegeneracy) to the theorem; then M_k is uniformly bounded, the M_k-norms are uniformly equivalent to the Euclidean norm, and the standard summation with weighted coefficients yields O(1/k). The proof should be rewritten with explicit uniform constants. The availability of code and the numerical comparisons are real assets, but they do not repair the false theorem. Because the central theoretical claim is false as stated, the appropriate verdict changes from conditional acceptance to rejection of the current version; a revised paper with a corrected theorem and proof could be reconsidered.","tokens_in":16522,"tokens_out":12055,"duration_ms":97776,"concrete_test":"Run the 1D counterexample: f(x)=x^2/2, x0=1, β=0.1, η=0.4, m_k=x_k, μ_k=1−1/(k+2) for k up to 10^6. Verify that x_{k+1}=(1−β/(k+2))x_k, so f(x_k)=Θ(k^{−0.2}), and check whether (2.22) holds: LHS 2kβ f(x_k) grows like k^{0.8}, while RHS is O(1) since ||x0||^2_{M_k}=O(k). If the O(1/k) bound appears to hold instead, identify the hidden mechanism that prevents this construction; if not, Theorem 2.10 must be amended with a uniform lower bound on 1−μ_k and the proof rewritten with a fixed-norm estimate.","verdict_should_be":"REJECT","load_bearing_attack":"Theorem 2.10 claims a global O(1/k) rate for any nonzero inertial term m_k with μ_k∈[0,1) under convexity and local Lipschitz differentiability. The proof does not support this. M_k in (2.4) depends on m_k and μ_k, so the norms in Lemma 2.9 change every iteration; summing (2.15) over j telescopes only if M_j is constant or uniformly equivalent to a fixed norm, and no uniform bound on M_k is stated. Even if (2.22) were true, it would not imply O(1/k): ||x0−x*||^2_{M_k} can grow with k, so the RHS can be O(1). More seriously, the claim is false as stated. Take f(x)=x^2/2, x0=1, β=0.1, η=0.4, m_k=x_k, μ_k=1−1/(k+2). All hypotheses hold, and x_{k+1}=(1−β/(k+2))x_k, so f(x_k)=Θ(k^{−2β})=Θ(k^{−0.2}), which is not O(1/k). The missing assumption is a uniform bound μ_k≤μ_max<1 (or equivalently liminf(1−μ_k)>0); with it, M_k is uniformly bounded and the rate arguments can likely be repaired. As written, the central O(1/k) theorem is invalid.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Adaptive Inertial Method (AIM), an inertial first-order scheme of the form x_{k+1}=x_k-β_k∇f(x_k)+γ_k m_k with the momentum gain γ_k chosen adaptively as γ_k=μ_k (m_k^T∇f(x_k))/||m_k||^2 β_k, μ_k∈[0,1), so that each step is guaranteed to decrease the objective. The main theoretical contribution is a claimed global O(1/k) convergence rate for any nonzero inertial term m_k under only convexity and local Lipschitz differentiability of the objective, together with a derivation of particular inertial terms that turn AIM into a quasi-Newton method and, under special structural conditions, into a regularized Newton method with an O(1/k^2) rate. The paper also reports numerical comparisons on logistic regression and L2-Lp minimization problems.","tokens_in":16900,"tokens_out":14926,"duration_ms":110321,"significance":"The adaptive rule for γ_k is a novel and attractive feature, and the possibility of unifying many momentum methods by a single framework is appealing. The numerical experiments are extensive, include reproducible code, and show AIM variants (especially with Hessian-gradient inertia) performing well against several baselines. However, the central convergence-rate theorem is false as stated, and the quasi-Newton/regularized-Newton connections rely on restrictive or tautological constructions. The core idea is worth pursuing, and the rate claim appears repairable by adding a uniform bound on μ_k, but the paper as written does not deliver the advertised guarantees.","major_comments":[{"comment":"The theorem is false as stated. Let f(x)=x^2/2, x_0=1, β_k=0.1, η=0.4, m_k=x_k, and μ_k=1-1/(k+2). These choices satisfy all hypotheses (convexity, local Lipschitz differentiability, (2.1b), μ_k∈[0,1), m_k≠0), but the update (2.1) gives x_{k+1}=(1-β/(k+2))x_k, so f(x_k)=Θ(k^{-0.2}), contradicting the claimed O(1/k) rate in (2.22). A uniform bound μ_k≤μ_max<1 (or equivalently liminf(1-μ_k)>0) would exclude this counterexample and is needed for the theorem.","section":"§2.2, Theorem 2.10"},{"comment":"The summation leading to (2.18) is not a valid telescoping. Inequality (2.15) is stated in the M_j-norm, and M_k defined in (2.4) depends on m_k and therefore changes with j. Summing (2.15) over j leaves cross-terms of the form ∥x_{j+1}-x^*∥_{M_j} and ∥x_{j+1}-x^*∥_{M_{j+1}}, and the final bound (2.18) states ∥x_0-x^*∥_{M_k} with the terminal index k, which does not follow. The proof requires either a fixed norm or a uniform norm-equivalence bound on the family {M_j}, neither of which is stated.","section":"§2.2, Lemma 2.9"},{"comment":"Even if (2.18) were valid, (2.22) would not imply O(1/k) because the numerator contains ∥x_0-x^*∥^2_{M_k}, which grows with k when μ_k→1 (e.g., as (1-μ_k)^{-1}). Also, the constant C in (2.22) is defined with a sum starting at j=1, but the negative j=0 term (1-2η)∥x_0-x_1∥^2_{M_0} from (2.18) is not included, so the inequality does not follow from (2.18) for η>0.5.","section":"§2.2, Theorem 2.10"},{"comment":"Theorem 2.10 omits the constant-step-size assumption β_k≡β that is introduced immediately before Lemma 2.9. The proof of (2.18)-(2.22) uses a single β outside the sums; the stated theorem allows variable β_k, so the proof does not apply to the stated result.","section":"§2.2"},{"comment":"The O(1/k^2) claim is not a general property of AIM. Theorem 3.1 requires the restrictive condition H_k^2=a_k H_k, and Theorem 3.3 forces equality with regularized Newton by constructing m_k, r_k, and β_k from the characteristic polynomial of H_k. That construction is a parameter-matching device, not an independent convergence result, and computing q_k(H_k)g_k for a degree-(n-1) polynomial is not a practical \"no matrix inversion\" procedure. The abstract's phrasing should be qualified accordingly.","section":"§3.1-3.3 and abstract"}],"minor_comments":[{"comment":"The title contains a typo: \"ADAPTIVE INER TIAL METHOD\" should be \"ADAPTIVE INERTIAL METHOD\".","section":"Page 1 header"},{"comment":"The stated implicit form x_{k+1}=x_k-β_k M_k^{-1}∇f(x_{k+1}) is not equivalent to the explicit update (2.6), which uses ∇f(x_k); the proximal-point interpretation is therefore incorrect as written.","section":"§2, Remark 2.4"},{"comment":"The notation should make clear that the M_k-norm is iteration-dependent; otherwise equations such as (2.16)-(2.17) appear to treat M_k as constant.","section":"§2.2, around (2.15)"},{"comment":"The condition ∥s∥^2<α s^T y is a curvature condition on the iterates, not a statement that f is locally strongly convex; the sentence after (3.3) overstates the implication.","section":"§3.2, equation (3.3)"},{"comment":"The quantity r_k=(g_k^T H_k^2 g_k)/(g_k^T H_k g_k) is undefined when g_k^T H_k g_k=0; the nonzero-inertial-term requirement should be stated explicitly.","section":"§3.3"},{"comment":"The backtracking check uses ∥x_k-x_{k+1}∥^2_{M_k} as in Remark 2.7, but the theorems in Section 2 are stated under condition (2.1b); the relationship between the implemented condition and the theoretical condition should be clarified.","section":"Algorithm A.1, line 16"}],"recommendation":"major_revision","confidential_remarks":"The central convergence theorem (Theorem 2.10) is false as stated; the counterexample in my report is simple and should be reproduced by the authors. The proof of Lemma 2.9 also contains an indexing error and an unjustified telescoping over changing norms. Both issues appear repairable by adding a uniform upper bound on μ_k and reworking the norm argument, but the advertised generality of the O(1/k) result must be revised. The O(1/k^2) claims in Section 3 are either restrictive or tautological and need to be reframed. I recommend major revision rather than rejection because the adaptive momentum idea and the numerical results are of interest, and the theorem can likely be corrected within the framework."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: the adaptive momentum gain rule (2.1c) is a real contribution. It selects gamma_k automatically from the current inertial direction and gradient, guaranteeing descent, and the framework around an arbitrary nonzero m_k is more general than what's in the cited literature. The authors also ship code, and the experiments—though lacking error bars—cover standard benchmarks and show AIM is competitive, with the Hessian-gradient variant roughly matching DRSOM. That part is solid and useful.\n\nThe problem is the central guarantee. Theorem 2.10 claims global O(1/k) for any nonzero inertial term with mu_k in [0,1). As written, that's false. The proof telescopes over M_k norms that change every iteration because M_k depends on m_k and mu_k; there is no uniform bound, so (2.18) doesn't follow from the preceding display. And the stress-test counterexample is convincing: for f(x)=x^2/2, x_0=1, mu_k=1-1/(k+2), the iterates give f(x_k)=Theta(k^{-0.2}), not O(1/k). The fix is also clear: add a uniform upper bound mu_k <= mu_max < 1, or equivalently require liminf(1-mu_k)>0. With that, M_k is uniformly equivalent to the identity and the proof is likely repairable. This is a load-bearing flaw, but it's a missing assumption rather than a broken idea.\n\nThere are smaller issues worth naming. Lemma 2.9 also has an indexing slip in the sum limits around (2.19)-(2.20), and Theorem 2.10's definition of C uses q(eta) but the sign conventions in the proof are easy to misread. The O(1/k^2) claims rest on restrictive special cases: Theorem 3.1 needs an idempotent Hessian, and Theorem 3.3 is a coefficient-matching construction rather than a general result. The paper says this clearly enough, but the abstract's phrasing overstates the connection. The quasi-Newton variant is close to DRSOM, which the authors do cite.\n\nWho this is for: people building adaptive momentum methods and wanting a tunable inertial framework. The paper deserves a serious referee because the core idea is new, the counterexample tells you exactly what to fix, and the numerical evidence suggests the method works when the parameters stay bounded away from the degeneracy. I would not cite it as-is, but I'd read the revised version carefully.\n\nRecommendation: send to peer review, but ask for the proof repaired with the explicit mu_max assumption and the experiments rerun with error bars. The O(1/k^2) section should be reframed as a coincidence result, not a general acceleration claim.","headline":"Genuinely new adaptive momentum rule, but the headline O(1/k) theorem is false as stated and needs a repairable extra assumption; worth refereeing after major revision.","tokens_in":17419,"tokens_out":1408,"would_cite":false,"duration_ms":13877,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C25","90C30","90C53","65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Adaptive momentum rule yields O(1/k) for any inertial term","keywords":["adaptive inertial method","momentum acceleration","convex optimization","first-order methods","quasi-Newton method","regularized Newton","adaptive parameter selection","O(1/k) convergence rate"],"falsifier":"Run AIM on a convex quadratic with a deliberately chosen inertial direction that makes $\\|M_k\\|$ grow unboundedly (for example, $m_k$ alternates in a way that aligns with the projection), and check whether the bound $f(x_k)-f(x^*)$ actually follows $O(1/k)$ or whether the proof's inequality breaks; a failure of the bound would falsify the claimed rate.","tokens_in":16322,"feed_emoji":"⚙️","tokens_out":5827,"duration_ms":43643,"temperature":0.7,"pith_summary":"This paper introduces the Adaptive Inertial Method (AIM), a first-order optimization scheme in which the momentum gain is chosen adaptively rather than by hand. The central claim is that, for any nonzero inertial direction, AIM converges globally at rate O(1/k) under only convexity and local Lipschitz differentiability, with no strong-convexity assumption. The same framework, with a particular inertial term, is shown to coincide with the regularized Newton method and to inherit its O(1/$k^{2}$) rate, while still avoiding Hessian inversions. A positive result here would give practitioners a principled way to pick momentum without tuning.","feed_headline":"Adaptive momentum rule yields O(1/k) for any inertial term","feed_subtitle":"No manual tuning of momentum: any inertial direction works, and one choice recovers regularized Newton's faster rate.","key_machinery":"The central object is the matrix $M_k = I_n + (\\mu_k/(1-\\mu_k)) \\Pi_k$, built from the projection $\\Pi_k$ onto the inertial vector $m_k$. Rewriting AIM as a proximal-point-like update $x_{k+1} = x_k - \\beta_k M_k^{-1} \\nabla f(x_k)$, the adaptive gain $\\gamma_k = \\mu_k (m_k^\\top \\nabla f(x_k)) / (\\|m_k\\|^2 \\beta_k)$ is exactly the choice that makes the update a metric projection step with respect to the $M_k$-norm. The convergence argument telescopes the per-iteration inequality (2.15) over iterations to obtain the $O(1/k)$ bound; the same $M_k$ structure is used to derive the quasi-Newton and Hessian-gradient inertial terms that connect AIM to regularized Newton.","core_discovery":"The core discovery is a parameter selection rule that guarantees descent regardless of which nonzero inertial direction is used. Writing the iteration as $x_{k+1} = x_k - \\beta_k M_k^{-1}\\nabla f(x_k)$, with $M_k^{-1} = I - \\mu_k \\Pi_k$ where $\\Pi_k$ projects onto the inertial direction $m_k$, the choice $\\gamma_k = \\mu_k (m_k^\\top \\nabla f(x_k)) / (\\|m_k\\|^2 \\beta_k)$ makes the objective strictly decrease each step. The proof then establishes an $O(1/k)$ convergence bound by telescoping an inequality measured in the $M_k$-norm. A second result shows that when the inertial direction is the product of the Hessian and gradient, and the Hessian satisfies a low-degree polynomial condition, AIM becomes algebraically equivalent to the regularized Newton method and achieves $O(1/k^2)$ without computing matrix inverses.","pith_inferences":["The $O(1/k)$ proof appears to require the $M_k$-norm to telescope over iterations; since $M_k$ depends on $m_k$ and changes each step, a fully rigorous statement would need a uniform bound on $\\|M_k\\|$ or a corrected inequality. This is our reading, not the paper's claim.","If that gap is real, the guaranteed rate may degrade for inertial directions that make $M_k$ grow, even though the method may still perform well numerically.","The blockwise diagonal variant sketched in Remark 2.5 could be tested as a memory-efficient extension for large-scale problems.","The Hessian-gradient inertial term, which approximates the Hessian by finite differences, suggests a natural stochastic variant where the same curvature estimate is computed on minibatches."],"forward_implications":["Any nonzero inertial direction, such as velocity, acceleration, Hessian-gradient product, or a custom mapping, can be plugged into AIM and still yields the $O(1/k)$ rate.","No manual tuning of the momentum gain is required: $\\gamma_k$ is fully determined by $m_k$, $\\nabla f(x_k)$, and $\\beta_k$.","With the Hessian-gradient product as the inertial term and a rank-one (or low-polynomial) Hessian condition, AIM reproduces the regularized Newton update and achieves $O(1/k^2)$ with only one extra gradient evaluation per step.","The method retains the simple per-iteration cost of a first-order method while the adaptive gain can mimic second-order behavior in favorable regimes.","The step-size condition (2.1b) is satisfied by a backtracking rule, so the method is implementable without knowing the Lipschitz constant in advance."],"supporting_citations":[{"why":"Supplies the constant-step-size telescoping technique used to derive the O(1/k) bound.","marker":"[6]"},{"why":"Defines the regularized Newton method that AIM is shown to coincide with under the special inertial term.","marker":"[24]"},{"why":"Provides the step-size adjustment rule used to enforce condition (2.1b).","marker":"[19]"},{"why":"Supplies the heavy-ball momentum term that AIM generalizes and compares against.","marker":"[27]"},{"why":"DRSOM, the main benchmark whose momentum-direction strategy is contrasted with AIM's adaptive gain in numerical tests.","marker":"[33]"}],"fun_headline_variants":["Adaptive inertial method guarantees descent for any direction","O(1/k) rate for every inertial direction via adaptation","Adaptive inertial step gives O(1/k) with no tuning","Adaptive inertial rule gives O(1/k), one choice gives O(1/k^2)"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The convergence proof telescopes the $M_k$-norm over iterations as if $M_k$ were a constant matrix, but $M_k$ changes every iteration because it depends on the inertial term $m_k$, and the paper supplies no explicit uniform bound on $M_k$.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive inertial method guarantees descent for any direction","O(1/k) rate for every inertial direction via adaptation","Adaptive inertial step gives O(1/k) with no tuning","Adaptive inertial rule gives O(1/k), one choice gives O(1/k^2)"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001207,"raw_usage":{"total_tokens":4935,"prompt_tokens":869,"completion_tokens":4066,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":3990}},"tokens_in":485,"tokens_out":4066,"duration_ms":27031,"temperature":1.0,"reasoning_tokens":3990,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:24:07.394638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run AIM on a convex quadratic with a deliberately chosen inertial direction that makes $\\|M_k\\|$ grow unboundedly (for example, $m_k$ alternates in a way that aligns with the projection), and check whether the bound $f(x_k)-f(x^*)$ actually follows $O(1/k)$ or whether the proof's inequality breaks; a failure of the bound would falsify the claimed rate.","supporting_citations":[{"cited_title":"Beck and M","cited_arxiv_id":null,"evidence_quote":"Supplies the constant-step-size telescoping technique used to derive the O(1/k) bound."},{"cited_title":"Mishchenko , Regularized newton method with global convergence , SIAM Journal on Optimization, 33 (2023), pp","cited_arxiv_id":null,"evidence_quote":"Defines the regularized Newton method that AIM is shown to coincide with under the special inertial term."},{"cited_title":"He and L.-Z","cited_arxiv_id":null,"evidence_quote":"Provides the step-size adjustment rule used to enforce condition (2.1b)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the heavy-ball momentum term that AIM generalizes and compares against."}],"review_version":1}