{"id":"4c807778-cc81-4deb-a801-6a5a34b0edd1","arxiv_id":"2508.12327","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Lion is claimed to converge at O(d^{1/2}T^{-1/4}) on smooth nonconvex problems, with variance-reduced and distributed extensions, but the communication-efficient theorems are not fully proved.","lead":"This paper derives convergence rates for the Lion optimizer and for distributed variants with sign compression. The claimed rates are plausible for the centralized case, but several distributed proofs contain algebraic errors and an unproven boundedness condition.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorems 6 and 7 prove l2-norm rates but state l1-norm rates; converting l2 to l1 costs a sqrt(d) factor, so the claimed communication-efficient rates are unsupported.","rationale":"The reader's weakest assumption (the range condition for S_G) is a real but repairable gap: for the non-variance-reduced algorithm, v_j^t is a convex combination of gradients and is bounded by G; for the STORM variant, v_t = g_t + (1-beta_2)(m_{t-1}-g_{t-1}) and is bounded by roughly 3G, so using R=3G would fix the oracle-range issue without changing the rate. The stronger, load-bearing problem is the norm mismatch in the two unbiased-sign theorems. The paper's abstract and Theorems 6 and 7 advertise l1 rates for the communication-efficient variants, but the descent argument controls l2: the negative term is -(eta/(4G)) E||grad f||_2^2, and the final displayed bound in Appendix G is for the Euclidean norm. Since ||g||_1 <= sqrt(d) ||g||_2, the stated l1 rates would require an extra sqrt(d), changing d^{1/4}/T^{1/4} to d^{3/4}/T^{1/4} in Theorem 7 and similarly worsening Theorem 6. This is not a stylistic issue; no inequality in the proof bridges l2 control to l1 control without a dimension factor. The centralized Theorems 1 and 2 and the non-compressed distributed Theorems 3 and 4 appear more plausible, and the Appendix F sqrt(n) term is likely a typo that I would not base a rejection on. But the norm mismatch directly undermines two headline results, so the reader's REJECT verdict is justified.","tokens_in":21273,"tokens_out":18975,"duration_ms":183328,"concrete_test":"Re-derive the final step of Appendix G with explicit norms: verify whether the left side of the last inequality is (1/T) sum E||grad f(x_t)||_1 or (1/T) sum E||grad f(x_t)||_2. If it is the l2 norm, substitute ||g||_1 <= sqrt(d) ||g||_2 into the claimed rates and check whether the d exponents in Theorems 6 and 7 remain 1/4 or become 3/4. A quick secondary check: in Appendix F, recompute the term after Eq. (14); E||(1/n) sum (S_G(v_j)-v_j/G)||^2 <= d/n gives O(G d / sqrt(n)), not 4 d G sqrt(n), confirming a likely typo rather than a rate change.","verdict_should_be":"REJECT","load_bearing_attack":"Appendices G and H do not prove the l1 rates claimed in Theorems 6 and 7. In Appendix G, the descent inequality (Eq. 15) is E[f(x_{t+1})-f(x_t)] <= (eta/G) E||grad f(x_t)-bar v_t||^2 - (eta/(4G)) E||grad f(x_t)||^2 + 3 eta^2 L d. The negative term is the squared Euclidean norm. After summing, the proof's final display bounds E[(1/T) sum_t ||grad f(x_t)||] (Euclidean norm) by the square root of 4 Delta G/(eta T) + 12 eta L d G + variance terms. The theorem statements, however, concern (1/T) sum E||grad f(x_t)||_1. The inequality ||g||_1 <= sqrt(d) ||g||_2 runs in the wrong direction: converting the displayed l2 rate into an l1 rate multiplies it by sqrt(d), turning d^{1/4}/T^{1/4} into d^{3/4}/T^{1/4} and similarly inflating the n-dependent term in Theorem 6. Appendix H repeats the same norm mismatch for Theorem 7. Thus the two communication-efficient 'unbiased sign' theorems, which are among the paper's advertised contributions, are not supported by the written proofs. This is independent of the sign-oracle range issue: even if every boundedness assumption is granted, the proof concludes a bound on a different norm than the theorem states.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes the Lion optimizer in centralized and distributed settings. In the centralized case, it proves an O(d^{1/2}T^{-1/4}) rate for the standard Lion update under L-smoothness and bounded noise, and an O(d^{1/2}T^{-1/3}) rate for a STORM-based variance-reduced variant under average smoothness. These results are extended to a heterogeneous distributed setting, with rates O(d^{1/2}(nT)^{-1/4}) and O(d^{1/2}(nT)^{-1/3}). The final part of the paper proposes communication-efficient variants that compress both the uplink and downlink by sign operations, including an 'unbiased sign' operation, and claims rates of order O(max{d^{1/4}/T^{1/4}, d^{1/10}/(n^{1/5}T^{1/5})}) and O(d^{1/4}/T^{1/4}). The centralized proofs are largely self-contained, but the communication-efficient theorems contain norm mismatches and a miscomputed variance, and the unbiased sign operation is applied to vectors whose boundedness is not established.","tokens_in":21615,"tokens_out":17501,"duration_ms":172717,"significance":"If all the claimed rates were established, the paper would be a useful contribution: it would provide standard-assumption convergence guarantees for Lion without coerciveness, extend them to heterogeneous distributed systems, and give two-way sign-compressed variants with explicit dimension and node dependence. The centralized Theorems 1 and 2 appear to be the strongest and most defensible parts, and the paper is honest about its assumptions and prior work. However, the communication-efficient results (Theorems 5-7) are advertised in the abstract and introduction as main contributions, and those proofs do not, as written, establish the stated l1-norm rates. Because the main new selling points are precisely the unbiased-sign compression rates, the paper cannot be accepted in its present form.","major_comments":[{"comment":"There is a norm mismatch between the proofs and the theorem statements. The descent inequality in Appendix G, Eq. (15), has a negative term proportional to E||grad f(x_t)||^2, and the final displayed bound is on (1/T) sum E||grad f(x_t)||, i.e., the Euclidean norm. Theorems 6 and 7, however, claim bounds on (1/T) sum E||grad f(x_t)||_1. Since ||g||_1 <= sqrt(d) ||g||_2, the proved l2 bound cannot be converted into the claimed l1 rate; doing so would multiply the dimension exponent by sqrt(d), turning d^{1/4}/T^{1/4} into d^{3/4}/T^{1/4} and d^{1/10}/(n^{1/5}T^{1/5}) into d^{3/5}/(n^{1/5}T^{1/5}) in Theorem 6, and d^{1/4}/T^{1/4} into d^{3/4}/T^{1/4} in Theorem 7. The proofs therefore do not support the advertised communication-efficient rates.","section":"Appendices G and H; Theorems 6 and 7"},{"comment":"The unbiased sign operation S_R is only defined for vectors v with ||v||_inf <= R, but Theorems 6 and 7 apply S_G to v_j^t. Assumption 9 only bounds the stochastic gradients themselves, not the momentum vectors. Under the STORM update used in the v2 variants, m_j^t = (1-beta2)(m_{j,t-1} - g_{j,t-1}) + g_{j,t} and v_j^t = (1-beta1)(m_{j,t-1} - g_{j,t-1}) + g_{j,t}; since m_{j,1} is a batch average rather than g_{j,1}, the difference m_{j,t-1} - g_{j,t-1} is not zero, so ||v_j^t||_inf can exceed G (for example, it can be as large as 3G in sup norm). No lemma in the paper proves ||v_j^t||_inf <= G for all t, so S_G may be applied to vectors outside its domain of definition.","section":"Definition 1; Theorems 6 and 7"},{"comment":"The variance of the averaged unbiased sign vectors is miscalculated. After bounding E||(1/n) sum_j (S_G(v_j^t) - v_j^t/G)||, the proof upper-bounds the resulting square-root term by sqrt(d) sqrt(n). For n independent vectors whose coordinates have variance at most 1, E||(1/n) sum_j (S_G(v_j^t) - v_j^t/G)||^2 <= d/n, so the correct upper bound is sqrt(d/n), not sqrt(dn). As written, the proof yields a term of order d sqrt(n) in Theorem 5 rather than the claimed d/sqrt(n). The factor error is fixable without changing the stated rates, but the proof as written is internally inconsistent.","section":"Appendix F, Eq. (14)"},{"comment":"The momentum conditions beta2^2 <= beta1 <= sqrt(beta2) force beta2 <= 1. In Theorem 6, the stated choice beta2 = O(n^{1/3} eta^{2/3} d^{1/3}) can violate this when eta is in the second branch eta = O(n^{2/5}/(T^{3/5} d^{1/5})): for T = O(n) and d large, beta2 can be polynomially larger than 1. The theorem should state an explicit lower bound on T, or another restriction, that guarantees beta2 <= 1; as stated, the parameter regime is not fully specified.","section":"Theorem 6; parameter conditions"}],"minor_comments":[{"comment":"The pseudocode indentation places the server-side update x_{t+1} = x_t - eta(sign(v_t) + lambda x_t) inside the loop over nodes j. This update should be performed once after all v_j^t have been collected and averaged.","section":"Algorithms 2 and 3"},{"comment":"The initialization condition ||x_1||_inf <= eta is quite restrictive because eta is horizon-dependent (for example, eta = O(d^{-1/2}T^{-3/4}) in Theorem 1). This should be discussed explicitly, since it is not a standard assumption and it weakens the claim that the analysis avoids auxiliary conditions.","section":"Lemma 1 and Theorems 1-4"},{"comment":"In the sentence following Eq. (14), 'forth inequality' should be 'fourth inequality', and the notation for the norm of the averaged sign vectors is inconsistent with the rest of the appendix.","section":"Appendix F"},{"comment":"In the bound for E||(1/n) sum_j v_j^{t+1} - grad f(x_{t+1})||^2, the term 8L^2 eta^2 d/beta1 appears without the factor 1/n that appears in the analogous derivation in Appendix E. This is likely a typo, but it should be corrected for consistency.","section":"Appendix H"}],"recommendation":"reject","confidential_remarks":"The centralized portion of the paper (Theorems 1-2) appears to be a solid, self-contained analysis, and Theorems 3-4 are plausible extensions. My rejection is driven by Section 5: the two main communication-efficient contributions are not proved as stated because of the l1/l2 norm mismatch and the undefined S_G operation, and Theorem 5's proof contains a variance miscalculation. These are load-bearing issues in the paper's advertised new results, not cosmetic presentation problems. If the authors were to restate Theorems 6-7 in terms of the l2 norm (with a clear l1 conversion caveat), repair the range condition on the sign oracle, and correct Appendix F, a resubmission could be viable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the centralized half of this paper is plausible and probably right, but the two communication-efficient theorems the abstract sells are not supported by the written proofs. Appendix G and H prove bounds on the Euclidean norm of the averaged gradient while Theorems 6 and 7 claim l1-norm bounds. Since ||g||_1 <= sqrt(d)||g||_2, the conversion runs the wrong way: the claimed d^{1/4}/T^{1/4} would become d^{3/4}/T^{1/4} for l1, and the n-dependent term in Theorem 6 inflates similarly. This is a load-bearing mismatch, not a typo. Additionally, Theorem 7 applies the unbiased sign oracle S_G to v_j^t, whose coordinates are not shown to stay within [-G,G]; with the STORM momentum and beta2 = O(T^{-1/2}), the momentum invariant can grow like G/beta2, so the oracle may not even be defined on the stated inputs. Appendix F also has a variance error: the average of independent sign vectors has variance d/n, but the text as printed produces a non-vanishing d*sqrt(n) term that contradicts the claimed d/n^{1/2} rate. That one may be a fixable typo, but as written it is another unverified spot.\n\nWhat the paper does well: the centralized result is new relative to Dong et al. in that it drops coerciveness, and the proof using the l1 sign-mismatch argument is coherent under the stated assumptions. The variational Lemma 1 is a neat trick, even though the initialization ||x1||_inf <= eta with eta = O(d^{-1/2}T^{-3/4}) is quite restrictive and should be flagged as an assumption, not a free lunch. The combination of Lion with STORM and with unbiased two-sided sign compression is a reasonable and honestly motivated set of contributions, and the related-work coverage is fair. The citation pattern is fine; the authors reuse their own sign-based analysis where appropriate, which is legitimate given the technique overlap.\n\nWhere I part with the readers: I don't think the centralized theorem is in trouble, and Theorems 3 and 4 appear to mirror the centralized argument without the same norm problem. The paper's real weaknesses are concentrated in the communication-efficient section, which is also the part the abstract highlights.\n\nBottom line: I would not desk-reject this. A serious referee should see it, because the centralized part is salvageable and the errors in the distributed proofs are identifiable and probably repairable. But the version in front of us should not be accepted: the advertised convergence rates for Theorems 5, 6, and 7 are not established by the current proofs.","headline":"The centralized Lion analysis is credible, but the two communication-efficient theorems are proved in the wrong norm and contain an unbounded-sign-oracle gap; the paper needs major revision before it can be trusted.","tokens_in":800,"tokens_out":951,"would_cite":false,"duration_ms":70999,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves Lion converges at the optimal stochastic rate $O(d^{1/2}T^{-1/4})$ without coerciveness, and that variance reduction and distributed sign compression improve or match these rates.","keywords":["Lion optimizer","sign-based optimization","nonconvex smooth optimization","convergence rate","variance reduction","distributed optimization","sign compression","stochastic gradient methods"],"falsifier":"On a scalar problem satisfying Assumptions 3 and 9 with gradient bound $G$, run the Lion-VR recursion with $\\beta_2 = T^{-1/2}$ using zero true gradient and mean-zero noise $\\xi_t \\in \\{-G,G\\}$ with alternating signs. The STORM update $m_t = (1-\\beta_2)m_{t-1} + \\beta_2 \\xi_t + (1-\\beta_2)(\\xi_t - \\xi_{t-1})$ has a worst-case fixed point larger than $G$ for small $\\beta_2$, so $\\|m_t\\|_\\infty$ exceeds $G$ and $S_G(m_t)$ is undefined. Recording the maximum $\\|m_t\\|_\\infty$ over such a run settles whether the compression oracle used in Theorem 7 actually exists for the stated inputs.","tokens_in":21069,"feed_emoji":"🦁","tokens_out":13292,"duration_ms":134727,"temperature":0.7,"pith_summary":"This paper establishes provable convergence rates for the Lion optimizer, the sign-momentum method used in large-model training, and it does so without the coerciveness condition that earlier analyses imposed. Under standard smoothness, unbiased gradients, and bounded noise, vanilla Lion guarantees $\\frac{1}{T}\\sum_{t=1}^T \\mathbb{E}\\|\\nabla f(x_t)\\|_1 \\le O(d^{1/2}T^{-1/4})$, matching the known lower bound for nonconvex stochastic optimization. Replacing the second momentum with a recursive variance-reduced correction improves the rate to $O(d^{1/2}T^{-1/3})$. The same rates transfer to heterogeneous distributed settings with $n$ nodes, and a two-way 1-bit sign-compressed version converges at $O(\\max\\{d^{1/4}T^{-1/4}, d^{1/10}n^{-1/5}T^{-1/5}\\})$, or $O(d^{1/4}T^{-1/4})$ when variance reduction is added.","feed_headline":"Lion optimizer now matches the optimal stochastic rate","feed_subtitle":"Vanilla Lion reaches the optimal nonconvex rate under standard assumptions; variance reduction goes faster.","key_machinery":"The load-bearing object is the pair of momentum estimators $v_t$ and $m_t$ and the sign-implicit update $x_{t+1}=x_t-\\eta(\\operatorname{sign}(v_t)+\\lambda x_t)$. Lemma 1 shows this update is self-bounding: with $\\|x_1\\|_\\infty \\le \\eta$ and $\\lambda \\le 1/(2\\eta T)$, every iterate satisfies $\\|x_t\\|_\\infty \\le \\eta t$ and $\\|x_{t+1}-x_t\\|^2 \\le 4\\eta^2 d$, which removes the coerciveness assumption. The proof then reduces the average $\\ell^1$ gradient norm to the estimator error $\\mathbb{E}\\|v_t-\\nabla f(x_t)\\|^2$, controlled either by tuning $\\beta_1, \\beta_2$ or by a recursive variance-reduced momentum correction. In the communication-efficient theorems, the unbiased sign operation $S_R(v)$ — a coordinatewise coin flip that rounds each coordinate to $\\pm 1$ so that the expected outcome is $v/R$ — carries the two-way compression argument, at the price of requiring $\\|v\\|_\\infty \\le R$.","core_discovery":"On the paper's own terms, the central claim is that the sign-momentum structure of Lion is analyzable under standard assumptions, provided the update $x_{t+1}=x_t-\\eta(\\operatorname{sign}(v_t)+\\lambda x_t)$ is recognized as self-bounding. Theorems 1 and 2 give $O(d^{1/2}T^{-1/4})$ and $O(d^{1/2}T^{-1/3})$ for Lion and Lion-VR; Theorems 3 and 4 give $O(d^{1/2}(nT)^{-1/4})$ and $O(d^{1/2}(nT)^{-1/3})$ in heterogeneous distributed settings; Theorems 5--7 show that two-way sign compression with unbiased sign operations still drives the average $\\ell^1$ gradient norm to zero at $O(\\max\\{d^{1/4}T^{-1/4}, d^{1/10}n^{-1/5}T^{-1/5}\\})$, or $O(d^{1/4}T^{-1/4})$ with variance reduction. The object doing the work is the pair of momentum estimators $v_t, m_t$ plus the randomized sign oracle $S_R$, whose unbiasedness converts coordinatewise compression into a controllable error term.","pith_inferences":["We infer that the proof template transfers to other sign-based optimizers: any update of the form sign(momentum) can inherit these rates if the iterate bound of Lemma 1 and the estimator-error recursion hold.","We infer that the initialization condition $\\|x_1\\|_\\infty \\le \\eta$ couples the initial scale to the learning rate, which could be restrictive in practice; rescaling the initialization or adding a projection is a natural extension not tested in the paper.","We infer a concrete fix for the unbiased-sign domain issue: clip each $v_j^t$ coordinate to $[-G,G]$ before applying $S_G$, then check whether the rates survive; this is a testable modification of the two-way compressed algorithm.","We infer that the $\\ell^1$ stationarity metric used throughout is weaker than the more common $\\ell^2$ metric, so the rates do not directly imply comparable $\\ell^2$ gradient convergence."],"forward_implications":["Lion can be used in nonconvex stochastic problems under standard assumptions; the coerciveness condition in earlier analyses is unnecessary.","The variance-reduced Lion-VR converges at $O(d^{1/2}T^{-1/3})$ under average smoothness, matching the known lower bound for that setting.","Distributed Lion with $n$ nodes achieves centralized-style rates with iteration count replaced by $nT$, so adding nodes speeds convergence in the aggregate.","With unbiased sign compression in both directions, the communication-efficient variant keeps the gradient norm driving to zero as $T$ grows rather than plateauing.","The two-way compressed rates $O(\\max\\{d^{1/4}T^{-1/4}, d^{1/10}n^{-1/5}T^{-1/5}\\})$ and $O(d^{1/4}T^{-1/4})$ give concrete bandwidth-versus-iteration trade-offs for sign-based distributed training."],"supporting_citations":[{"why":"Introduces the Lion optimizer and its sign-momentum update rule, the algorithm whose convergence this paper analyzes.","marker":"Chen et al. (2023)"},{"why":"Provides the prior convergence analysis with the same rate under a coerciveness assumption that the paper removes.","marker":"Dong et al. (2024)"},{"why":"Supplies the recursive momentum variance-reduction estimator that underlies the Lion-VR rate improvement.","marker":"Cutkosky and Orabona (2019)"},{"why":"Establishes the lower bounds that justify calling the paper's rates optimal in their respective regimes.","marker":"Arjevani et al. (2023)"},{"why":"Gives the standard smoothness-based stochastic nonconvex analysis that the paper's proof skeleton builds on.","marker":"Ghadimi and Lan (2013)"},{"why":"Introduces signSGD-style sign compression and its convergence analysis, which the communication-efficient Lion variants adapt.","marker":"Bernstein et al. (2018)"},{"why":"Shows how variance reduction accelerates sign-based methods, serving as the template for the compressed distributed Lion analysis.","marker":"Jiang et al. (2024)"}],"fun_headline_variants":["Lion optimizer's convergence proven for centralized and distributed setups","Rigorous convergence rates for Lion and its variance-reduced variant","Distributed Lion with sign compression achieves proven convergence","Variance-reduced Lion hits faster O(d^(1/2)T^(-1/3)) rate","Communication-efficient Lion converges under sign compression"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof of the two-way sign-compressed theorems assumes that every coordinate of the vector fed to the unbiased sign operator lies in $[-G,G]$; for the variance-reduced momentum the paper does not establish this, and with $\\beta_2 = O(T^{-1/2})$ the momentum can in principle grow well past $G$.","fun_headline_variants_meta":{"raw":{"variants":["Lion optimizer's convergence proven for centralized and distributed setups","Rigorous convergence rates for Lion and its variance-reduced variant","Distributed Lion with sign compression achieves proven convergence","Variance-reduced Lion hits faster O(d^(1/2)T^(-1/3)) rate","Communication-efficient Lion converges under sign compression"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000276,"raw_usage":{"total_tokens":1708,"prompt_tokens":1068,"completion_tokens":640,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":684,"completion_tokens_details":{"reasoning_tokens":554}},"tokens_in":684,"tokens_out":640,"duration_ms":5682,"temperature":1.0,"reasoning_tokens":554,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:24:42.459342+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a scalar problem satisfying Assumptions 3 and 9 with gradient bound $G$, run the Lion-VR recursion with $\\beta_2 = T^{-1/2}$ using zero true gradient and mean-zero noise $\\xi_t \\in \\{-G,G\\}$ with alternating signs. The STORM update $m_t = (1-\\beta_2)m_{t-1} + \\beta_2 \\xi_t + (1-\\beta_2)(\\xi_t - \\xi_{t-1})$ has a worst-case fixed point larger than $G$ for small $\\beta_2$, so $\\|m_t\\|_\\infty$ exceeds $G$ and $S_G(m_t)$ is undefined. Recording the maximum $\\|m_t\\|_\\infty$ over such a run settles whether the compression oracle used in Theorem 7 actually exists for the stated inputs.","supporting_citations":[],"review_version":1}