Pith. sign in

REVIEW 2 major objections 5 minor 27 references

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees

T0 review · 2 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PowerSGD can fail to converge; PowerSGD+ fixes it with a periodic SVD safeguard.

desk verdict Counterexample to PowerSGD is real; PowerSGD+ proof has a repairable algebraic error in Eq. (19). read the letter →

arxiv 2509.11254 v1 pith:NT6KBH5X submitted 2025-09-14 math.OC cs.LG

Shengping Xie , Chuyan Chen , Kun Yuan This is my paper
classification math.OCcs.LG MSC 90C1590C3068W40
keywords low-rankgradientcompressionPowerSGDconvergenceguaranteeserrorfeedbackstochasticoptimizationdistributedSVDsafeguarditeration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that the popular low-rank gradient compressor PowerSGD is not guaranteed to converge in stochastic distributed optimization, and backs that claim with an explicit counterexample. To repair the gap, it introduces PowerSGD+, which periodically refreshes the projection subspace with a full SVD while keeping the cheap power-iteration updates in between. Under standard smoothness, variance, and uniform-gradient assumptions, the paper proves PowerSGD+ converges to a stationary point at rate O(1/sqrt(NT) + T^{-2/3}) with linear speedup in the number of workers. The result matters because PowerSGD is widely used in practice but had no rigorous convergence guarantee.

What carries the argument

The key machinery is the periodic SVD safeguard: every τ iterations, PowerSGD+ computes the optimal rank-r projection of the averaged error-corrected gradient via SVD, and uses it to reset the auxiliary basis Q. In between resets, it runs the same single-step power iteration as PowerSGD. The SVD acts as a contractive compressor with coefficient δ = r/n, which keeps the accumulated error feedback bounded; Lemmas 3 and 4 show that the residual and momentum norms stay finite, and the telescoping descent argument in Theorem 4 converts that into the convergence rate.

What would settle it

Run PowerSGD with rank r = 1 on the paper's 2x2 counterexample (f(X) = ψ(x11 - x12 - x21 + x22), noise σB) and check whether the empirical gradient norm ever drops below the predicted ϵ0; if it does for any initialization, step-size schedule, and base optimizer, Theorem 1 is false. Alternatively, construct a smooth stochastic problem satisfying Assumptions 1-2 but with unbounded gradients (so Assumption 3 fails) and show PowerSGD+ diverges, which would refute the claimed convergence guarantee.

Watch

Extended reading notes

Core claim

The central discovery is that single-step power iteration, which PowerSGD uses to track the gradient's leading subspace, can lock onto the wrong subspace and keep the gradient norm bounded away from zero forever. The paper constructs a 2x2 stochastic problem with separable noise where, for any initialization, step-size schedule, and base optimizer, PowerSGD's expected gradient norm remains above a positive constant at every time step. The proposed fix, PowerSGD+, periodically replaces the power-iteration projection with the optimal rank-r SVD projection, ensuring the subspace stays aligned with the true gradient. With that safeguard, the paper proves a non-asymptotic convergence bound for Po

Load-bearing premise

The load-bearing premise is Assumption 3, the uniform bound on the full gradient norm: if gradients are unbounded or heavy-tailed, the bound on accumulated error (Lemma 3) and the momentum bound (Lemma 4) fall apart, and the PowerSGD+ convergence proof no longer holds.

Editorial extensions

If this is right

  • If PowerSGD+ is correct, any implementation of PowerSGD can be made convergent at negligible extra communication cost by performing SVD every τ steps, with the overhead amortized as mn/τ per iteration.
  • The same convergence rate holds when the SVD reset is replaced by any contractive compressor with coefficient δ, so the safeguard is modular and can be swapped for cheaper or more communication-friendly resets.
  • The counterexample shows that PowerSGD's lack of alignment can be pathological, not merely a slow convergence issue, so practitioners using PowerSGD without a reset mechanism cannot rely on eventual convergence.
  • The convergence rate O(1/sqrt(NT) + T^{-2/3}) for PowerSGD+ matches the typical rate for distributed stochastic nonconvex optimization with error feedback, so the safeguard does not degrade the asymptotic efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The counterexample structure (noise with direction B, gradient with orthogonal direction A) suggests that any basis-tracking scheme that can lock onto a fixed subspace will suffer in settings where gradient directions rotate suddenly; this may apply to other single-step power-iteration compressors, not just PowerSGD.
  • The uniform gradient bound (Assumption 3) is the main fracture point: deep-learning gradients are often heavy-tailed or unbounded, so the proof's dependence on G² = σ² + ω² is a practical risk. A testable extension is to run PowerSGD+ on tasks with known heavy-tailed gradients (e.g., certain transformers) and check whether the residual bound still holds.
  • The periodic SVD reset could be viewed as a simple, generic 'recalibration' strategy for any low-rank or subspace-tracking method, and might be imported into related approaches such as lazy-SVD or subspace optimization without changing their core structure.
  • The paper's experiments focus on LLM fine-tuning and pre-training, but the theory is for smooth nonconvex objectives; a direct test on problems with nonsmooth or constrained domains (e.g., hinge loss or weight constraints) would clarify whether the safeguard is sufficient beyond smoothness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies convergence guarantees for PowerSGD, a low-rank gradient compression method for distributed optimization. It first claims that PowerSGD can fail to converge, constructing a simple stochastic problem in which, with positive probability, the iterates remain in the span of a fixed rank-one matrix and the gradient norm is bounded away from zero. It then introduces PowerSGD+, a variant that periodically replaces the power-iteration compressor with an optimal rank-r SVD compressor, and proves (Theorem 2/4) that PowerSGD+ with momentum SGD converges under smoothness, bounded variance, and a uniform gradient bound, with a rate O(1/sqrt(NT) + T^{-2/3}). Experiments on GLUE fine-tuning and LLaMA pre-training are reported to support the claims.

Significance. If the proof gaps are repaired, the paper makes a useful contribution: it provides a concrete counterexample to unconditional convergence of PowerSGD and proposes a simple, theoretically justified fix. The analysis is self-contained and does not rely on fitted constants or circular arguments; the stated convergence rate is falsifiable and explicit. The experiments are supportive, though not extensive. The main caveats are that the counterexample theorem is stated more strongly than what the proof establishes, and the central convergence proof contains a concrete algebraic error that must be corrected before Theorem 4 can be accepted as proven. These issues are local and appear repairable, so the paper is best handled as a major revision rather than a rejection.

major comments (2)
  1. [Appendix B, Eq. (19)-(20), Theorem 4] The proof defines \tilde X_t = X_t - \eta/(1-\mu)\bar e_t, so X_t - \tilde X_t = \eta/(1-\mu)\bar e_t. Therefore the first term in the bound on E||\nabla f(X_t)-\nabla f(Z_t)||^2 must carry a factor (1-\mu)^{-2}; the displayed coefficient 2L^2\eta^2 is missing this factor. Consequently the first error term in Eq. (20) should be 45L^2\eta^3\tau^2G^2/((1-\mu)^3\delta^2), not 45L^2\eta^3\tau^2G^2/((1-\mu)\delta^2). After multiplying by 4(1-\mu)/\eta, the corrected constant is 180/(1-\mu)^2 rather than 180. The final bound 548L^2\eta^2\tau^2G^2/((1-\mu)^4\delta^2) can still be recovered because 180/(1-\mu)^2 + 368/(1-\mu)^4 \le 548/(1-\mu)^4, but the derivation as displayed is not a valid proof of the claimed constants.
  2. [Theorem 1 and Appendix A, Claim 1] The statement 'any initialization Q' is not established. In the proof, P_0 = \sigma BQ is computed and it is concluded that \tilde P_0 = C for any Q. If Q is semi-orthogonal with a+b=0, e.g., Q=[1;-1]/\sqrt2, then P_0 is the zero matrix; economic QR is then undefined unless an arbitrary convention is adopted, and the induction does not guarantee that the convention returns C. The proof also writes 'arbitrary \sigma\ge0', but the argument requires \sigma>0 to make P_0 nonzero for the generic Q. The theorem should be weakened to existence of Q (e.g., Q with a+b\neq0) and \sigma>0, or the zero-input case must be handled explicitly. Since the paper's negative claim only needs existence, this is repairable.
minor comments (5)
  1. [Appendix A] In the line before Claim 1, the gradient is written as \psi'(x_{11}+x_{12}-x_{21}-x_{22})A; the sign should be x_{11}-x_{12}-x_{21}+x_{22} to match f(X)=\psi(x_{11}-x_{12}-x_{21}+x_{22}).
  2. [Theorems 1/3 and 2/4] The same theorems are numbered twice, once in the main text and once in the appendix. Please renumber or explicitly state that the appendix restates the main theorems.
  3. [Section 5, Tables 1 and 3] Table 1 numbers are run together (e.g., '62.8990.29'); add spaces or table formatting. Table 3's caption says GaLore 'diverged' at ranks r\in{4,8}, yet the table gives finite perplexity values; clarify what is meant.
  4. [Abstract and Assumption 3] The abstract says 'standard assumptions,' but Assumption 3 (uniform bound on the full gradient) is essential to Lemma 3 and Lemma 4 and is stronger than L-smoothness plus bounded variance. Please state this caveat explicitly in the introduction and when advertising the convergence guarantee.
  5. [Algorithm 2] The initialization line calls Q_{-1} a 'reflection matrix'; it should be 'auxiliary basis' or 'auxiliary matrix'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: both the non-convergence counterexample and the PowerSGD+ convergence proof are derived from stated assumptions and compressor definitions, not from their own conclusions.

full rationale

I walked the paper's derivation chain. Theorem 1 (Appendix A) is an explicit counterexample: the functions and stochastic oracle in Eq. (10) are checked against Assumptions 1–3, and the proof of Claim 1 establishes by induction, using the identities CC^T B = B, CC^T A = 0, AC = A^T C = 0, BC = B^T C = 2C, that with positive probability m_t ∈ span(B), e_t^(i) ∈ span(A), and P_t = C. Hence X_t − X_0 ∈ span(B), so ∇f(X_t) is constant and E||∇f(X_t)||^2 ≥ ε_0. This is a derivation from the algorithm's definitions, not a restatement of the theorem. Theorem 4 (Appendix B) introduces auxiliary sequences X̃_t and Z_t, applies L-smoothness, Assumptions 2–3, and the compressor properties proved in Lemmas 1 and 2. Lemma 2 is the standard rank-r SVD/Eckart–Young bound with δ = r/n and is proved from the SVD; Lemmas 3–4 bound error-feedback and momentum using elementary recursions. The final rate is obtained by a valid (if algebraically delicate) choice of η, not by fitting data or by normalizing with the target quantity. The paper's self-citations ([24], [25]) appear only in side remarks about alternative contractive compressors and communication comparisons; they are not load-bearing for Theorem 4, which relies on the in-paper proofs for C_SVD and C_SSP. I also considered the skeptical point about Eq. (19): as displayed, the bound on E||∇f(X_t) − ∇f(Z_t)||^2 drops a factor 1/(1−μ)^2 in the error-feedback term, since X_t − X̃_t = η/(1−μ) · (1/N)Σ e_t^(i). This is a proof-correctness issue that may affect the displayed constants, but it is not circularity: it does not identify the theorem's conclusion with an input or with a fitted parameter. Thus the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted constants enter the theory; the only algorithm hyperparameters are rank r, restart period tau, momentum mu, and step size eta. The standard smoothness, bounded-variance, and bounded-gradient assumptions are stated. One hidden regularity assumption about QR on zero matrices is present in the counterexample proof.

assumptions (4)
  • domain assumption The global loss f is L-smooth and lower bounded (Assumption 1).
    Used throughout the descent analysis in Appendix B.
  • domain assumption Local stochastic gradients are unbiased with bounded variance sigma^2 (Assumption 2).
    Used in the virtual-sequence descent lemma in Appendix B.
  • domain assumption Full gradients are uniformly bounded: ||grad f(X)||^2 <= omega^2 (Assumption 3).
    Load-bearing for Lemmas 3 and 4; without it the error accumulation bound and the final rate do not hold.
  • ad hoc to paper Economic QR decomposition is defined for any input matrix, including zero or rank-deficient inputs.
    The non-convergence proof claims 'any initialization Q', but QR(0) is undefined; the induction implicitly assumes P_0 is nonzero or a convention for QR(0) is fixed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees." pith.science (2026). https://pith.science/paper/NT6KBH5X

@misc{pith2026250911254,
  author       = {Pith},
  title        = {Pith review of: From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NT6KBH5X}},
  note         = {Machine review of arXiv:2509.11254}
}
read the original abstract

Low-rank gradient compression methods, such as PowerSGD, have gained attention in communication-efficient distributed optimization. However, the convergence guarantees of PowerSGD remain unclear, particularly in stochastic settings. In this paper, we show that PowerSGD does not always converge to the optimal solution and provide a clear counterexample to support this finding. To address this, we introduce PowerSGD+, which periodically updates the projection subspace via singular value decomposition, ensuring that it remains aligned with the optimal subspace. We prove that PowerSGD+ converges under standard assumptions and validate its effectiveness through empirical evaluation on large language model tasks.

Figures

Figures reproduced from arXiv: 2509.11254 by the authors.

Figure 1
Figure 1. Loss curves of PowerSGD and PowerSGD+ using MSGD and Adam. Here, Step (7a) is executed in parallel on each node i, where ∆ (i) t denotes the local stochastic gradient corrected by the previous compression residual e (i) t . In Step (7b), the function Comp(·) transforms both the local gradients ∆ (i) t and the averaged global gradient ∆t = 1 N PN i=1 ∆ (i) t into their low-rank approximations, ∆b (i) t and ∆b t using… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

27 extracted references · 6 linked inside Pith

  1. [1]

    Scaling distributed machine learning with the parameter server

    Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. Scaling distributed machine learning with the parameter server. In11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), pages 583–598, 2014

    2014

  2. [2]

    Bandwidth optimal all-reduce algorithms for clusters of workstations.Journal of Parallel and Distributed Computing, 69(2):117–124, 2009

    Pitch Patarasuk and Xin Yuan. Bandwidth optimal all-reduce algorithms for clusters of workstations.Journal of Parallel and Distributed Computing, 69(2):117–124, 2009

    2009

  3. [3]

    1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns

    Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu. 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns. InInterspeech, volume 2014, pages 1058–1062. Singapore, 2014

    2014

  4. [4]

    Project adam: Building an efficient and scalable deep learning training system

    Trishul Chilimbi, Yutaka Suzue, Johnson Apacible, and Karthik Kalyanaraman. Project adam: Building an efficient and scalable deep learning training system. In11th USENIX symposium on operating systems design and implementation (OSDI 14), pages 571–582, 2014

    2014

  5. [5]

    Qsgd: Communication-efficient sgd via gradient quantization and encoding.Advances in neural information processing systems, 30, 2017

    Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. Qsgd: Communication-efficient sgd via gradient quantization and encoding.Advances in neural information processing systems, 30, 2017

    2017

  6. [6]

    Terngrad: Ternary gradients to reduce communication in distributed deep learning.Advances in neural information processing systems, 30, 2017

    Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Terngrad: Ternary gradients to reduce communication in distributed deep learning.Advances in neural information processing systems, 30, 2017

    2017

  7. [7]

    Federated learning: Strategies for improving communication efficiency.arXiv preprint arXiv:1610.05492, 2016

    Jakub Koneˇ cn` y, H Brendan McMahan, Felix X Yu, Peter Richt´ arik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency.arXiv preprint arXiv:1610.05492, 2016. 9

    arXiv 2016

  8. [8]

    Gradient sparsification for communication-efficient distributed optimization.Advances in Neural Information Processing Systems, 31, 2018

    Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang. Gradient sparsification for communication-efficient distributed optimization.Advances in Neural Information Processing Systems, 31, 2018

    2018

Show all 27 references
  1. [9]

    Sparsified sgd with memory.Advances in neural information processing systems, 31, 2018

    Sebastian U Stich, Jean-Baptiste Cordonnier, and Martin Jaggi. Sparsified sgd with memory.Advances in neural information processing systems, 31, 2018

    2018

  2. [10]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  3. [11]

    Galore: Memory-efficient llm training by gradient low-rank projection.arXiv preprint arXiv:2403.03507, 2024

    Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. Galore: Memory-efficient llm training by gradient low-rank projection.arXiv preprint arXiv:2403.03507, 2024

    2024 arXiv

  4. [12]

    Sltrain: a sparse plus low rank approach for parameter and memory efficient pretraining.Advances in Neural Information Processing Systems, 37:118267–118295, 2024

    Andi Han, Jiaxiang Li, Wei Huang, Mingyi Hong, Akiko Takeda, Pratik Kumar Jawanpuria, and Bamdev Mishra. Sltrain: a sparse plus low rank approach for parameter and memory efficient pretraining.Advances in Neural Information Processing Systems, 37:118267–118295, 2024

    2024

  5. [13]

    Enhancing zeroth-order fine-tuning for language models with low-rank structures.arXiv preprint arXiv:2410.07698, 2024

    Yiming Chen, Yuan Zhang, Liyuan Cao, Kun Yuan, and Zaiwen Wen. Enhancing zeroth-order fine-tuning for language models with low-rank structures.arXiv preprint arXiv:2410.07698, 2024

    2024 arXiv

  6. [14]

    Powersgd: Practical low-rank gradient compression for distributed optimization.Advances in Neural Information Processing Systems, 32, 2019

    Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi. Powersgd: Practical low-rank gradient compression for distributed optimization.Advances in Neural Information Processing Systems, 32, 2019

    2019

  7. [15]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational conference on machine learning, pages 8821–

  8. [16]

    Angel- ptm: A scalable and economical large-scale pre-training system in tencent.arXiv preprint arXiv:2303.02868, 2023

    Xiaonan Nie, Yi Liu, Fangcheng Fu, Jinbao Xue, Dian Jiao, Xupeng Miao, Yangyu Tao, and Bin Cui. Angel- ptm: A scalable and economical large-scale pre-training system in tencent.arXiv preprint arXiv:2303.02868, 2023

    2023 arXiv

  9. [17]

    Error feedback fixes signsgd and other gradient compression schemes

    Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian Stich, and Martin Jaggi. Error feedback fixes signsgd and other gradient compression schemes. InInternational conference on machine learning, pages 3252–3261. PMLR, 2019

    2019

  10. [18]

    single-step power iteration

    extends this concept by maintaining a local gradient tracker for each node, reducing the impact of data heterogeneity and improving convergence rates. Building on this theoretical foundation, NEOLITHIC [19] establish lower bounds for distributed learning under communication co...

  11. [19]

    Ef21: A new, simpler, theoretically better, and practically faster error feedback.Advances in Neural Information Processing Systems, 34:4384–4396, 2021

    Peter Richt´ arik, Igor Sokolov, and Ilyas Fatkhullin. Ef21: A new, simpler, theoretically better, and practically faster error feedback.Advances in Neural Information Processing Systems, 34:4384–4396, 2021

    2021

  12. [20]

    Lower bounds and nearly optimal algorithms in distributed learning with communication compression.Advances in Neural Information Processing Systems, 35:18955–18969, 2022

    Xinmeng Huang, Yiming Chen, Wotao Yin, and Kun Yuan. Lower bounds and nearly optimal algorithms in distributed learning with communication compression.Advances in Neural Information Processing Systems, 35:18955–18969, 2022

    2022

  13. [21]

    The error-feedback framework: Better rates for sgd with delayed gradients and compressed communication.arXiv preprint arXiv:1909.05350, 2019

    Sebastian U Stich and Sai Praneeth Karimireddy. The error-feedback framework: Better rates for sgd with delayed gradients and compressed communication.arXiv preprint arXiv:1909.05350, 2019

    1909 arXiv

  14. [22]

    Momentum provably improves error feedback!Ad- vances in Neural Information Processing Systems, 36:76444–76495, 2023

    Ilyas Fatkhullin, Alexander Tyurin, and Peter Richt´ arik. Momentum provably improves error feedback!Ad- vances in Neural Information Processing Systems, 36:76444–76495, 2023

    2023

  15. [23]

    Haoran Sun and Mingyi Hong. Distributed non-convex first-order optimization and information processing: Lower complexity bounds and rate optimal algorithms.IEEE Transactions on Signal processing, 67(22):5912– 5928, 2019

    2019

  16. [24]

    Atomo: Communication-efficient learning via atomic sparsification.Advances in neural information processing systems, 31, 2018

    Hongyi Wang, Scott Sievert, Shengchao Liu, Zachary Charles, Dimitris Papailiopoulos, and Stephen Wright. Atomo: Communication-efficient learning via atomic sparsification.Advances in neural information processing systems, 31, 2018

    2018

  17. [25]

    Greedy low-rank gradient compression for distributed learning with convergence guarantees.arXiv preprint arXiv:2507.08784, 2025

    Chuyan Chen, Yutong He, Pengrui Li, Weichen Jia, and Kun Yuan. Greedy low-rank gradient compression for distributed learning with convergence guarantees.arXiv preprint arXiv:2507.08784, 2025

    2025

  18. [26]

    NX i=1 (λ(i) t +γ (i) t ) ! A+ NX i=1 β(i) t ! B # C∈span(C), ⇒ ˜Pt =C. Then: Qt = 1 N NX i=1 ∆(i)⊤ t ˜Pt = qt−1 N

    Yutong He, Pengrui Li, Yipeng Hu, Chuyan Chen, and Kun Yuan. Subspace optimization for large language models with convergence guarantees.arXiv preprint arXiv:2410.11289, 2024. 10 A Non convergence of PowerSGD Theorem 3(Non-Convergence ofPowerSGD).There exist local functionsf i...

    2024 arXiv

  19. [27]

    Bound the second term−E[⟨∇f(Z t),∇f(X t)⟩] with Cauchy’s inequality, we have: −E[⟨∇f(Z t),∇f(X t)⟩] =−E ∥∇f(X t)∥2 F +E[⟨∇f(X t)− ∇f(Zt),∇f(X t)⟩] ≤ −1 2 E ∥∇f(X t)∥2 F + 1 2 E ∥∇f(X t)− ∇f(Zt)∥2 F ,(18) And: E ∥∇f(X t)− ∇f(Zt)∥2 F ≤L 2E ∥Xt −Z t∥2 F ≤2L 2E h ∥Xt − ˜Xt∥2 F i +...

    2000

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.