{"id":"921a9bd8-b586-4837-9290-1909ece3ec14","arxiv_id":"2412.02382","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DPRSRM is a single-loop decentralized stochastic recursive momentum method with gradient tracking on compact submanifolds, claimed to reach ε-stationarity with O(ε^{-3/2}) oracle complexity.","lead":"This paper proposes a decentralized algorithm for minimizing sums of nonconvex functions on a curved space (a manifold) using streaming data across multiple nodes. It claims a faster statistical guarantee, oracle complexity O(ε^{-3/2}), than earlier decentralized manifold methods that reach O(ε^{-2}).","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.7 drops the ν²/τ term from Lemma 4.5; at α=K^{-1/3}, τ=K^{-2/3} it leaves a K^{-1/3} error, so Corollary 3.8's O(ε^{-3/2}) oracle complexity is unsupported.","rationale":"The paper's central contribution is the O(ε^{-3/2}) oracle complexity of DPRSRM in Corollary 3.8. I checked the proof chain Lemma 4.5 -> (4.7) -> Theorem 3.7 -> Corollary 3.8. Lemma 4.5 states ∑_{k=0}^K E‖d_k−∇f(x_k)‖² ≤ nν²/τ + 2ν²τK + 2ρ3 n α⁴K/τ. When inserted into the descent inequality (4.7), the nν²/τ term produces, after division by αK/4, an unavoidable contribution 6(1+ρ2τ²)ν²/(τK). The theorem as displayed has 12/n (1+ρ2τ²)(ν²τ + ρ3 n α⁴/τ), which does not contain this term. At the corollary's parameter choice α=K^{-1/3}, τ=K^{-2/3}, the missing term is Θ(ν² K^{-1/3}). To make it smaller than ε requires K=Ω(ε^{-3}), so the stated O(ε^{-3/2}) complexity is not supported. The reader's weakest-assumption concern about stochastic gradients being almost surely bounded and mean-squared Lipschitz is genuine: Lemma 4.1 uses ‖d_k‖≤√n L, Lemma 4.3 uses ‖∇f_i(x)‖≤L, and (B.24) uses a squared Lipschitz inequality not stated in Assumption 3.4. However, that concern is secondary: even granting all the stronger assumptions, the missing initial-error term alone invalidates the main rate. The issue is not a matter of tightening constants; it changes the exponent in the oracle complexity. A repair may be possible by making τ time-varying or by estimating d_0 with a large batch, but neither is present in the algorithm or its analysis. I therefore recommend rejecting the current central claim, while noting the algorithm may still be salvageable with these modifications.","tokens_in":20601,"tokens_out":27411,"duration_ms":273260,"concrete_test":"Re-derive Theorem 3.7 by substituting Lemma 4.5 into equation (4.7), keeping the full nν²/τ term. If any positive term proportional to ν²/(τK) survives after dividing by αK/4, plug in α=K^{-1/3}, τ=K^{-2/3} and verify that it is Θ(K^{-1/3}), destroying the claimed O(ε^{-3/2}) rate. As a numerical cross-check, run the n=1, W=1 specialization (no consensus, no gradient tracking) on a smooth nonconvex quadratic with bounded noise, with α=K^{-1/3}, τ=K^{-2/3}, and compare min_{k≤K} E‖∇f(x_k)‖² at K=10^3 and K=10^6: the claimed K^{-2/3} scaling predicts a 10^4-fold improvement, while the corrected K^{-1/3} floor predicts only a 10-fold improvement.","verdict_should_be":"REJECT","load_bearing_attack":"The central O(ε^{-3/2}) claim does not follow from the supplied proof. Substituting Lemma 4.5 into (4.7) and dividing by αK/4 yields an additional term 6(1+ρ2τ²)ν²/(τK), coming from the nν²/τ term in Lemma 4.5. The displayed Theorem 3.7 instead contains 12/n (1+ρ2τ²)(ν²τ + ρ3 n α⁴/τ), which has ν²τ in place of ν²/(τK) and introduces an unexplained 1/n. With the Corollary 3.8 choice α=K^{-1/3}, τ=K^{-2/3}, the omitted term is 6(1+ρ2τ²)ν² K^{-1/3}, not O(K^{-2/3}). Consequently, to make that term below ε requires K=O(ε^{-3}), not O(ε^{-3/2}). This is independent of the bounded-stochastic-gradient issue; even under a strengthened Assumption 3.4, the stated rate is not established. The missing term is not absorbed elsewhere: the recursion for E‖d_k−∇f(x_k)‖² amplifies the initial error E‖d_0−∇f(x_0)‖² by roughly 1/τ, and the final averaging divides by αK, leaving ν²/(τK). With constant τ=K^{-2/3}, this is K^{-1/3}; balancing 1/(τK)+τ with constant τ gives at best K^{-1/2}. A repair would require time-varying τ starting at O(1), or a large initial batch, neither of which is in Algorithm 3.1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DPRSRM, a decentralized projected Riemannian stochastic recursive momentum method for minimizing the average of smooth nonconvex functions over a compact submanifold in an online setting. The algorithm combines a clipped hybrid stochastic gradient estimator (3.6)-(3.8), gradient tracking (3.9), tangent-space projection, and a single consensus step. The main theoretical claims are Theorem 3.6 (consensus error), Theorem 3.7 (optimality error), and Corollary 3.8, which asserts that with alpha = K^{-1/3} and tau = K^{-2/3} the method reaches an epsilon-stationary point (Definition 2.1) with oracle complexity O(epsilon^{-3/2}) per node, improving on existing O(epsilon^{-2}) rates. Numerical experiments on decentralized PCA and low-rank matrix completion are reported.","tokens_in":21017,"tokens_out":8887,"duration_ms":87260,"significance":"If the O(epsilon^{-3/2}) rate were valid, this would be a meaningful advance for online decentralized manifold optimization: the method is single-loop, uses O(1) stochastic gradients per iteration per node, has no large-batch restart, and uses one consensus step for general compact submanifolds. The paper is clearly structured and the appendix contains a detailed proof attempt. However, the central rate is not established by the submitted proof. The proof of Theorem 3.7 drops a term of order nu^2/(tau K), which with the claimed parameter choices is K^{-1/3}, and the proof relies on bounded stochastic gradients and a second-moment Lipschitz property that are not present in Assumption 3.4. Because the main contribution is precisely the improved oracle complexity, these issues are load-bearing.","major_comments":[{"comment":"Substituting Lemma 4.5 into the inequality (4.7) and dividing by alpha K / 4 yields an additional term 6(1+rho2 tau^2) nu^2 / (tau K). The displayed Theorem 3.7 instead contains 12/n (1+rho2 tau^2)(nu^2 tau + rho3 n alpha^4/tau), which replaces nu^2/(tau K) with nu^2 tau and introduces an unexplained factor 1/n. With the Corollary 3.8 choices alpha = K^{-1/3} and tau = K^{-2/3}, the omitted term is 6(1+rho2 tau^2) nu^2 K^{-1/3}, not O(K^{-2/3}). Therefore the bound in (4.8), the O(epsilon^{-3/2}) oracle complexity in (3.18), and Corollary 3.8 are not supported by the supplied proof. Making the omitted term below epsilon would require K = O(epsilon^{-3}), not K = O(epsilon^{-3/2}). A repair would need a time-varying tau or an initial large batch, neither of which is present in Algorithm 3.1.","section":"3.3, Theorem 3.7 and Corollary 3.8; proof in Section 4.2 and Lemma 4.5"},{"comment":"The proof requires stronger stochastic assumptions than Assumption 3.4 provides. Lemma 4.1 uses ||d_0|| <= sqrt(n) L for the unclipped initial stochastic gradient; Lemma 4.3 uses deterministic bounds such as ||grad f(bar x_k)|| <= L; and Lemma 4.5, specifically equation (B.24), requires a second-moment Lipschitz condition E||grad f(x_k, xi_k) - grad f(x_{k-1}, xi_k)||^2 <= L^2 ||x_k - x_{k-1}||^2. Assumption 3.4 only contains the unbiasedness condition (3.3), the variance bound (3.4), and the first-moment condition (3.5). Moreover, (3.5) is labeled 'mean-squared' but is written with first moments. As written, the consensus and optimality bounds are unsupported. The proof also uses ||d_k|| <= sqrt(n) L for the clipped estimator, whereas Algorithm 3.1 clips at B >= L, so the bound should involve B unless B = L is imposed.","section":"3.1, Assumption 3.4 and Lemmas 4.1, 4.3, 4.5"}],"minor_comments":[{"comment":"The stated condition K >= max{4L, sqrt(n)D/delta, R sqrt(n)D / ((R(1-sigma2)-2delta)delta)} does not imply the step-size condition (3.13) when alpha = K^{-1/3}; the correct lower bounds would be of the form K >= (4L)^3, K >= (sqrt(n)D/delta)^3, and K >= (R sqrt(n)D / ((R(1-sigma2)-2delta)delta))^3.","section":"3.3, Corollary 3.8"},{"comment":"If the intended condition in (3.5) is indeed a mean-squared Lipschitz property, it should be written as E||grad f_i(x,xi) - grad f_i(y,xi)||^2 <= Lbar^2 ||x-y||^2 (or with squared norms on both sides), and this strengthening should appear in the assumptions used by Theorems 3.7 and 3.8.","section":"3.1, Assumption 3.4"},{"comment":"There are typographical errors, including the title 'OPTIMIZA TION' and 'Lispchitz' in the proof of Theorem 3.6, and the figure legends in Figures 1-3 render the step size as '=0.1' without displaying the symbol alpha.","section":"Title and throughout"},{"comment":"The section is titled 'Conclusions and Limitations' but contains no limitations statement; the paper should either add a limitations paragraph or rename the section.","section":"6, Conclusions and Limitations"}],"recommendation":"reject","confidential_remarks":"The central O(epsilon^{-3/2}) claim is not established: the substitution of Lemma 4.5 into Theorem 3.7 introduces a K^{-1/3} term, and a repair appears to require changing the algorithm or the parameter schedule. The assumption mismatch in Assumption 3.4 is a second independent obstruction. Because the main contribution is exactly this improved rate, I do not see a simple revision that would preserve the paper's central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the paper is the first to combine STORM-style momentum with projection-based gradient tracking and single-step consensus for online decentralized compact-submanifold problems. The algorithm is reasonable, and the experiments—though lacking error bars—do show it beating DRSGD and DRPGD on PCA and low-rank matrix completion. Second, the advertised O(ε^{-3/2}) rate does not follow from the proof as written. I checked the substitution from Lemma 4.5 into (4.7): after dividing by αK to get the min bound, the nν²/τ term in Lemma 4.5 becomes 6ν²/(τK), not the ν²τ shown in Theorem 3.7. With α=K^{-1/3}, τ=K^{-2/3}, that is K^{-1/3}, so you need K=O(ε^{-3}) to make it ε-small. The O(ε^{-3/2}) claim in Corollary 3.8 is therefore unsupported. This is a load-bearing error, not a cosmetic one.\n\nThere is also an assumption mismatch. Lemma 4.1 needs the initial stochastic gradient to be almost surely bounded by L, and Lemma 4.5 needs a second-moment Lipschitz property on the SFO. Assumption 3.4 only provides variance and a first-moment Lipschitz bound. Both are fixable by strengthening the assumptions, but as written the lemmas don't follow. The theorem statements also contain typos (e.g., the 12σ/n term in Corollary 3.8), suggesting the manuscript was not carefully proofread.\n\nWhat's good? The algorithmic idea is genuinely novel for this setting, the proof structure is standard, and the numerical results point in the right direction. The authors are honest about limitations and cite the relevant literature, including their own earlier work—which is fine since those lemmas are parameter-free.\n\nI'd send this to peer review: the combination is worth publishing if the rate claim can be repaired, and a good referee could help the authors find the missing term. But I would not cite the O(ε^{-3/2}) result until the proof is fixed.","headline":"DPRSRM is a sensible combination of known ingredients, but the advertised O(ε^{-3/2}) oracle complexity is unsupported: a dropped term from Lemma 4.5 leaves a K^{-1/3} error, and Assumption 3.4 is too weak for several lemmas.","tokens_in":21514,"tokens_out":4728,"would_cite":false,"duration_ms":45985,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C06","90C22","90C26","90C56"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes a decentralized single-loop stochastic recursive momentum method for smooth nonconvex optimization over compact submanifolds and proves an oracle complexity of $O(\\epsilon^{-3/2})$ per node in the online setting.","keywords":["decentralized optimization","Riemannian manifold","stochastic recursive momentum","variance reduction","gradient tracking","oracle complexity","nonconvex optimization","consensus"],"falsifier":"Run DPRSRM on a compact submanifold with a smooth nonconvex objective and a stochastic gradient oracle that is unbiased and has bounded variance but unbounded support, such as Gaussian noise, so that the printed Assumption 3.4 holds but the almost-sure bound fails; if the consensus error or the gradient-norm bound in Corollary 3.8 is violated, or if the clipping constant must grow with $1/\\epsilon$, then the claimed $O(\\epsilon^{-3/2})$ rate depends on the unstated stronger assumption.","tokens_in":20387,"feed_emoji":"📉","tokens_out":10046,"duration_ms":95116,"temperature":0.7,"pith_summary":"The paper proposes DPRSRM, a decentralized algorithm for minimizing a sum of smooth nonconvex functions over a compact submanifold in the online setting, where each node observes only stochastic gradients of streaming data. The central claim is that DPRSRM reaches an $\\epsilon$-stationary point with $O(\\epsilon^{-3/2})$ stochastic gradient oracle calls per node, improving on the $O(\\epsilon^{-2})$ complexity of earlier decentralized Riemannian stochastic methods. The method is single-loop, uses $O(1)$ stochastic gradient evaluations per iteration per node, and requires one consensus communication step per round, making it suitable for real-time streaming data. The analysis handles the nonconvex manifold constraint by projecting iterates onto the manifold and exploiting proximal smoothness, the property that the nearest-point projection is unique and Lipschitz in a tubular neighborhood.","feed_headline":"Reach ε-stationarity with O(ε^-3/2) oracle queries per node","feed_subtitle":"A single-loop momentum estimator plus gradient tracking cuts per-node queries below the O(ε^-2) of earlier methods.","key_machinery":"The central object is the DPRSRM update, which combines three components. The hybrid estimator $q_{i,k} = \\operatorname{grad} f_i(x_{i,k}, \\xi_{i,k}) + (1-\\tau)(d_{i,k-1} - \\operatorname{grad} f_i(x_{i,k-1}, \\xi_{i,k}))$ is a recursive momentum variance-reduction step; it is clipped to norm $B$ to form $d_{i,k}$. The gradient tracker $s_{i,k} = \\sum_j W_{ij} s_{j,k-1} + d_{i,k} - d_{i,k-1}$ propagates information about the global gradient through the network. The iterate is updated by tangent-space projection of $s$ onto the tangent bundle and then manifold projection: $x_{i,k+1} = P_{\\mathcal M}(\\sum_j W_{ij}x_{j,k} - \\alpha v_{i,k})$. Proximal smoothness of the compact submanifold supplies the Lipschitz property of the projection operator $P_{\\mathcal M}$ that controls the consensus error.","core_discovery":"On its own terms, the paper establishes that combining a hybrid stochastic recursive momentum estimator with gradient tracking and projection onto a compact submanifold yields an oracle complexity of $O(\\epsilon^{-3/2})$ for decentralized online nonconvex manifold optimization. Corollary 3.8 states that with step size $\\alpha = K^{-1/3}$ and momentum parameter $\\tau = K^{-2/3}$, after $K = O(\\max\\{(C+\\Gamma_1)^{3/2}\\epsilon^{-3/2},\\ \\Gamma_2\\epsilon^{-1},\\ (6\\rho_2\\nu^2)^{3/4}\\epsilon^{-3/4}\\})$ iterations the iterates are $\\epsilon$-stationary; the dominant term is the $\\epsilon^{-3/2}$ oracle complexity. The result applies to general compact submanifolds, not just the Stiefel manifold, and needs only one consensus step per iteration, whereas earlier projection and retraction methods required $O(\\log(1/\\sigma_2))$ consensus rounds. The paper further shows numerically on decentralized principal component analysis and low-rank matrix completion that DPRSRM beats the stochastic decentralized Riemannian gradient descent and projected gradient baselines considered.","pith_inferences":["The $O(\\epsilon^{-3/2})$ rate matches the known single-machine recursive-momentum rate, which suggests that the decentralized manifold constraint and one-round consensus need not add oracle overhead in this parameter regime.","Because the proof uses only projection Lipschitzness and a quadratic upper bound on the local objectives, the same estimator-and-consensus scheme should transfer to other proximally smooth constraint sets, such as sphere products or rank-constrained sets.","The clipping constant $B$ is a free user parameter; a data-dependent or adaptive choice of $B$ might preserve the rate while allowing larger practical steps, though this is not analyzed in the paper.","Since the method uses one communication round per iteration, it is a natural candidate for combining with gradient compression or quantization; whether the $O(\\epsilon^{-3/2})$ oracle complexity survives communication compression is left open."],"forward_implications":["Each node reaches an $\\epsilon$-stationary point with $O(\\epsilon^{-3/2})$ stochastic gradient oracle calls, up to the smaller $\\epsilon^{-1}$ and $\\epsilon^{-3/4}$ terms in Corollary 3.8.","Only one consensus communication round is needed per iteration, removing the $O(\\log(1/\\sigma_2))$ consensus-factor inflation of earlier projection and retraction methods on the Stiefel manifold.","The method is single-loop and online: no restarting with a large batch is required, and each node uses only $O(1)$ gradient evaluations per iteration.","The convergence guarantee covers general compact submanifolds, using projection $P_{\\mathcal M}$, not just Stiefel and orthogonality constraints.","Numerical tests on decentralized PCA and low-rank matrix completion show faster decrease of consensus error, objective gap, and gradient norm than the DRSGD and DRPGD baselines tested."],"supporting_citations":[{"why":"Supplies the projection-based decentralized Riemannian update and the proximal-smoothness projection inequalities that the consensus and optimality analysis reuses.","marker":"[12]"},{"why":"Provides the decentralized Riemannian gradient descent on the Stiefel manifold and its gradient-tracking variant, the baseline whose $O(\\epsilon^{-2})$ online oracle complexity DPRSRM improves.","marker":"[7]"},{"why":"Introduces the momentum-based variance-reduction estimator that the hybrid stochastic gradient estimator is built on.","marker":"[10]"},{"why":"Supplies the Riemannian stochastic recursive momentum method that DPRSRM adapts to the decentralized setting.","marker":"[17]"},{"why":"The existing variance-reduced stochastic gradient tracking algorithm for orthogonality constraints; it is double-loop and finite-sum, in contrast with the single-loop online DPRSRM.","marker":"[35]"},{"why":"A recent distributed Riemannian stochastic gradient tracking algorithm on the Stiefel manifold with $O(\\epsilon^{-2})$ oracle complexity, another baseline DPRSRM claims to beat.","marker":"[40]"},{"why":"Establishes that the Stiefel manifold is 1-proximally smooth, providing a concrete radius used in the compact-submanifold examples.","marker":"[2]"},{"why":"Gives the definition and properties of $R$-proximally smooth sets that underlie the projection operator's Lipschitz behavior on the tubular neighborhood.","marker":"[9]"}],"fun_headline_variants":["DPRSRM: Decentralized manifold optimization hits O(ε^-3/2) oracle cost","Single-loop momentum on manifolds: beats O(ε^-2) with ease","Decentralized nonconvex manifolds: O(ε^-3/2) with one consensus step","Projected Riemannian momentum: faster decentralized learning on curved spaces","O(ε^-3/2) oracle: new decentralized manifold method outperforms old"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every stochastic gradient is almost surely bounded by a fixed constant and that the average squared difference of two stochastic gradients grows at most linearly with the distance between their inputs, while the assumptions stated in the paper only guarantee bounded variance and a first-moment Lipschitz condition; if the stronger premise is not added, the consensus and optimality bounds do not follow.","fun_headline_variants_meta":{"raw":{"variants":["DPRSRM: Decentralized manifold optimization hits O(ε^-3/2) oracle cost","Single-loop momentum on manifolds: beats O(ε^-2) with ease","Decentralized nonconvex manifolds: O(ε^-3/2) with one consensus step","Projected Riemannian momentum: faster decentralized learning on curved spaces","O(ε^-3/2) oracle: new decentralized manifold method outperforms old"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00033,"raw_usage":{"total_tokens":1857,"prompt_tokens":982,"completion_tokens":875,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":763}},"tokens_in":598,"tokens_out":875,"duration_ms":8744,"temperature":1.0,"reasoning_tokens":763,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:33:17.487656+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DPRSRM on a compact submanifold with a smooth nonconvex objective and a stochastic gradient oracle that is unbiased and has bounded variance but unbounded support, such as Gaussian noise, so that the printed Assumption 3.4 holds but the almost-sure bound fails; if the consensus error or the gradient-norm bound in Corollary 3.8 is violated, or if the clipping constant must grow with $1/\\epsilon$, then the claimed $O(\\epsilon^{-3/2})$ rate depends on the unstated stronger assumption.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the decentralized Riemannian gradient descent on the Stiefel manifold and its gradient-tracking variant, the baseline whose $O(\\epsilon^{-2})$ online oracle complexity DPRSRM improves."},{"cited_title":"Cutkosky and F","cited_arxiv_id":null,"evidence_quote":"Introduces the momentum-based variance-reduction estimator that the hybrid stochastic gradient estimator is built on."},{"cited_title":"Han and J","cited_arxiv_id":null,"evidence_quote":"Supplies the Riemannian stochastic recursive momentum method that DPRSRM adapts to the decentralized setting."},{"cited_title":"W ang and X","cited_arxiv_id":null,"evidence_quote":"The existing variance-reduced stochastic gradient tracking algorithm for orthogonality constraints; it is double-loop and finite-sum, in contrast with the single-loop online DPRSRM."},{"cited_title":"Balashov and R","cited_arxiv_id":null,"evidence_quote":"Establishes that the Stiefel manifold is 1-proximally smooth, providing a concrete radius used in the compact-submanifold examples."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the definition and properties of $R$-proximally smooth sets that underlie the projection operator's Lipschitz behavior on the tubular neighborhood."}],"review_version":1}