Speeding up Stochastic Proximal Optimization in the High Hessian Dissimilarity Setting

Elnur Gasanov; Peter Richt\'arik

REVIEW 2 major objections 4 minor 61 references

Speeding up Stochastic Proximal Optimization in the High Hessian Dissimilarity Setting

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The loopless stochastic proximal point method L-SVRP converges in O((nδ/µ + n) log(1/ε)) iterations under Hessian similarity and strong convexity, improving the previous O(δ²/µ² + n) bound when δ/µ ≥ n and needing no smoothness assumption.

desk verdict A sound but sloppy analysis of L-SVRP: the main bound holds, but the headline corollary has a stepsize mismatch and the practical speedup claim ignores the proximal oracle cost. read the letter →

arxiv 2412.13619 v1 pith:XEMXH7IE submitted 2024-12-18 math.OC

Elnur Gasanov , Peter Richtárik This is my paper · ORCID

classification math.OC MSC 90C2590C1590C06

keywords stochasticproximalpointmethodvariancereductionHessiansimilaritylooplessSVRPstrongconvexitycommunicationcomplexityfederatedlearningconvergencerate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper revisits the Loopless Stochastic Variance Reduced Proximal Point method (L-SVRP) and gives a sharper convergence guarantee for expected risk minimization when the component losses have similar second derivatives. Under Hessian similarity (parameter δ) and strong convexity (parameter µ), the authors prove an iteration complexity of O((nδ/µ + n) log(1/ε)) when the snapshot probability is p = 1/n, improving on the prior O((δ/µ)² + n) bound in the regime δ/µ ≥ n. The analysis needs no Lipschitz smoothness assumption on the component losses, which broadens the class of problems the method is justified for. The proof works by choosing a Lyapunov function that combines the distance to the optimum with the distance between the two iterates and letting Hessian-similarity cross terms cancel exactly.

What carries the argument

The argument is carried by the Lyapunov function Λ_k = ∥x_k − x*∥² + c∥w_k − x_k∥², where x_k is the current iterate and w_k is the snapshot point refreshed to x_k with probability p. Lemma 1 bounds the one-step contraction of the first term using strong convexity, producing an error term proportional to E⟨∇ψ_i(x̄_{k+1}) − ∇ψ_i(w_k), x_{k+1} − x̄_{k+1}⟩, where ψ_i = f_i − f. Lemma 2 controls the evolution of the second term and yields the compensating negative term −cpξ∥w_k − x̄_{k+1}∥², which cancels the Hessian-dissimilarity contribution bounded via Assumption 1. With the parameter choices c = 2p/((3−p)²(1+µγ)), ξ = 1/2, and ζ = p/(4−2p), the mixed terms telescope and the Lyapunov sequence contracts by max{1/(1+µγ), 1−p/4} each iteration.

What would settle it

Run L-SVRP on a problem whose proximal steps have no closed form and must be solved by an inner iterative method, and compare total inner gradient evaluations against SGD to reach the same accuracy; if the total work does not decrease in the regime δ/µ ≥ n, the claimed communication speed-up does not translate into computational speed-up.

Watch

Extended reading notes

Core claim

The central claim is that L-SVRP, the loopless variance-reduced proximal point method, converges to the exact minimizer of a strongly convex finite sum under Assumptions 1 and 2 (Hessian similarity and strong convexity) at a linear rate whose condition number appears to the first power of δ/µ rather than its square. Specifically, with stepsize γ = Θ(√p/δ) and snapshot probability p = 1/n, the number of iterations to guarantee E∥x_K − x*∥² ≤ ε is O((nδ/µ + n) log(∥x0 − x*∥²/ε)). This matches the order of magnitude of the best known rates for variance-reduced gradient methods under smoothness, but here it holds without any smoothness assumption on the individual losses. In the regime δ/µ ≥ n this bound is asymptotically smaller than the earlier O((δ/µ)² + n) complexity established for the same algorithm, and the paper interprets the iteration count as communication complexity in a federated setting.

Load-bearing premise

The rate assumes an exact proximal operator for every f_i is evaluated at each iteration, so the iteration count is the real cost only if solving those subproblems is cheap; for general finite-sum losses a proximal step can be as expensive as the original problem.

Editorial extensions

If this is right

In the high Hessian dissimilarity regime δ/µ ≥ n, the new bound O((nδ/µ + n) log(1/ε)) is strictly better than the previous O((δ/µ)² + n) complexity for L-SVRP, so the method's advantage grows as data heterogeneity increases.
Because the proof avoids Lipschitz smoothness, the same rate applies to nonsmooth strongly convex losses whose proximal operators are available, a class for which gradient-based variance reduction would need extra assumptions.
When iterations are counted as communication rounds, for example in federated learning with n clients, the p = 1/n setting yields an O(nκ + n) communication complexity that improves on standard SGD's dependence on the smoothness constant.
The convex case (µ = 0) is covered as well, with an averaging bound showing E[f(x̄_K)] − f* ≤ ∥x0 − x*∥²/(2γK), giving a sublinear rate without smoothness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The exact-prox oracle assumption is structurally separate from the convergence theorem: if evaluating prox_{γf_i} is itself expensive for a given problem class, the counted iterations may not translate into wall-clock savings, and a fair comparison would need to add the inner cost of each prox evaluation.
The cancellation mechanism that makes the Hessian-dissimilarity term disappear is generic and could plausibly be injected into other loopless proximal or primal-dual methods, such as monotone inclusion solvers, to obtain analogous linear-in-κ rates.
The paper's own experiments show empirical rates well above the theoretical ones, so deriving a tighter bound from that gap is a natural next step.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version: the paper's main iteration bound for L-SVRP is probably right and is a real, if modest, improvement over the previous analysis of that algorithm. But the headline Corollary 3 has a stepsize mismatch, and the practical 'speedup' claim leans on an exact proximal oracle that the paper never discusses.

The genuinely new thing is the Lyapunov analysis. The cancellation of the Hessian dissimilarity term via the coefficient c is neat, and it removes the need for smoothness. Corollary 1's contraction is backed by a proof that checks out. So if you care about iteration complexity of stochastic proximal methods, this is a legitimate data point: O((nκ+n)) when p=1/n, versus O(κ²+n) from Khaled-Jin, in the regime κ≥n. The authors are honest that SVRS and SABER already achieve better rates; the contribution is to the analysis of L-SVRP specifically.

The soft spots are in the packaging. Corollary 3 states a stepsize γ=... that doesn't match the γ' used in the proof; the proof uses γ'=1/(δ√a). The stated γ is Θ(√p) or Θ(1) depending on how you read it, while the proof's γ' is Θ(p/δ). That's a concrete error in the main statement. Also, the simplification from √a to 3/p in the iteration count uses an inequality that fails for small p; the order is still O((κ/p)+(1/p)), so the asymptotic claim survives, but the constants are wrong as stated. Both are fixable in a revision.

The experiments are thin: quadratics only, no code or data link, no error bars, and Figures 3 and 4 look duplicated. They confirm linear convergence on easy instances, but they don't test the proximal oracle cost. The stress-test note about exact prox is a fair caveat: for general finite-sum losses, computing prox_{γf_i} can be as hard as solving a strongly convex problem, so the communication-round comparison with SGD may not translate to wall-clock time. The paper should either state this assumption explicitly or add experiments with nontrivial proximal evaluations.

If I were the editor, I'd send it to review: the core derivation is solid and the theorem is worth having, but it needs a corrected Corollary 3, a discussion of the oracle cost, and better artifacts. It's not a breakthrough, but it's a useful incremental step.

Referee Report

2 major / 4 minor

Summary. The paper revisits the Loopless Stochastic Variance Reduced Proximal Point method (L-SVRP) for ERM (1) under Hessian similarity (Assumption 1) and strong convexity (Assumption 2). It constructs a Lyapunov function Λ_k = ‖x_k − x*‖² + c‖w_k − x_k‖², proves a contraction in Theorem 1 under three coupled parameter conditions, and specializes the parameters in Corollary 1 to obtain a linear rate max{(1+μγ)^{-1}, 1−p/4} under the stepsize condition (17). Corollary 3 translates this into an iteration complexity O((nδ/μ + n) log(‖x0−x*‖²/ε)) when p = 1/n, claimed to improve on Khaled and Jin (2023) in the regime δ/μ ≥ n, with no smoothness assumption. The paper also gives a convex analogue (Corollary 2) and experiments on quadratic functions over 48 configurations.

Significance. If the central bound is correct, the paper yields the first analysis of L-SVRP achieving an O(nδ/μ) dependence under Hessian similarity, improving on the prior O(δ²/μ²) in the high-dissimilarity regime, and it does so without Lipschitz smoothness. The proof is concise and the stepsize is selected analytically with no fitted parameters; the contraction argument via Lemmas 1–2 appears sound. However, the current statement of Corollary 3 contains a stepsize mismatch that must be corrected, and the practical interpretation as a communication-speedup over SGD is conditional on an exact proximal oracle whose per-call cost is not analyzed. With those issues addressed, the result would be a meaningful contribution to the theory of stochastic proximal methods.

major comments (2)

[Section 3.4 / Corollary 3 and Appendix D] The stepsize displayed in Eq. (24) is not the stepsize used in the proof of Corollary 3. The proof (Appendix D) defines a = 2(3−p)²/(p²(p+1)) and verifies the condition (17) only for γ′ = 1/(δ√a) = (p/(3−p))·√((p+1)/2)/δ, which for small p is Θ(p/δ). The displayed γ in Eq. (24), with the square root over (3−p)/p, is Θ(1/(δ√p)) for small p, so the two differ by a factor Θ(√p). The advertised complexity O((nδ/μ + n) log) in Remark 2 is derived using γ′, not the displayed γ; moreover, with the displayed γ the condition (17) is generally violated when p = 1/n (its left-hand side becomes Θ(n)). Please correct Eq. (24) and ensure any experimental stepsize satisfies the condition used in the theory.
[Algorithm 3, Sections 2.4 and 3] The paper's central claim is presented as an improvement in communication complexity over SGD, but each iteration of Algorithm 3 requires the exact solution of the implicit subproblem x_{k+1} = prox_{γ f_{i_k}}(x_k + γ(∇f_{i_k}(w_k) − ∇f(w_k))). For general finite-sum losses this proximal operator has no closed form and can be as expensive as solving a strongly convex optimization problem; Section 2.4 only notes the stability of implicit methods, not their per-iteration expense. The experiments use only quadratics (Section 4), where the prox reduces to a linear solve, so they do not expose this cost. Please state explicitly that the bound is measured in exact proximal-oracle calls (equivalently communication rounds), and discuss the implications for the wall-clock speedup suggested by the title and abstract.

minor comments (4)

[Lemma 1 and Lemma 8] In the statements of Lemma 1 and Lemma 8, the term E[‖x_{k+1} − x̄_{k+1}‖²] should be written as a conditional expectation E[‖x_{k+1} − x̄_{k+1}‖² | x_k, w_k] to match the rest of the expression.
[Section 2.4, Eq. (6)] The equivalence y = prox_{γg}(x) ⇔ y + γ∇g(y) = x requires differentiability of g; for nondifferentiable convex g the condition is x − y ∈ γ∂g(y). The surrounding text already mentions this, so only a short pointer is needed.
[References] The reference list contains two entries with identical titles (Richtárik et al., 2024a and 2024b); please clarify whether these are two distinct works or a duplicated citation.
[Section 4] The checklist states that code is provided (item 3a), but no URL or supplementary link appears in the text; please add it or mark the item as not applicable.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the L-SVRP complexity bound follows from the paper's own Lyapunov analysis under Assumptions 1-2 with an analytic stepsize; self-citations are non-load-bearing baselines.

full rationale

The paper's central claim (Corollary 3 and Remark 2) is derived in Appendixes A-D from the stated Assumptions 1 and 2, not imported from prior work or fitted to data. Lemmas 1 and 2 bound the two parts of the Lyapunov function Lambda_k = ||x_k - x*||^2 + c||w_k - x_k||^2 using strong convexity and elementary algebra; Theorem 1 combines them and applies Assumption 1 (inequality (2)) verbatim to control the drift; Corollaries 1 and 3 resolve the parameter conditions (14)-(17) by explicit algebra and select the stepsize gamma = (p/(3-p))*sqrt((p+1)/2)/delta, yielding K >= (1 + 1/p)(3delta/mu + 4) log(||x0 - x*||^2/epsilon), i.e., O(n kappa + n) for p = 1/n. No parameter is tuned to observed data, no definition is chosen circularly, and no prior theorem is used as a premise; the asserted improvement over Khaled and Jin (2023) is arithmetic on the proven rate. Self-citations (Kovalev et al. 2020 for L-SVRG background; Szlendak et al. 2022 for the closed-form delta^2 formula in quadratics; Richtarik et al. 2024a as the baseline bound being compared; Richtarik et al. 2024b for the qualitative smoothness-free property of proximal methods) are not load-bearing: each is background, independently verifiable, or superseded by this paper's own proof. Two non-circularity caveats: the bibliography lists Richtarik, Sadiev and Demidovich 2024a and 2024b with identical titles (a citation-hygiene error), and the claimed communication improvement counts Khaled-Jin communication rounds, treating each exact prox_{gamma f_i} evaluation as cheap even though for general finite-sum losses that implicit subproblem can be as hard as a strongly convex optimization problem (Section 2.4 notes only the stability of implicit methods, not their per-step cost). These are correctness and applicability risks, not reductions of the conclusion to its inputs. Hence no circular step is present; the score reflects only minor, non-load-bearing self-citations.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two standard assumptions (Hessian similarity and per-component strong convexity) and on an oracle that returns exact proximal operators. No free parameters are fitted to data; the stepsize is chosen analytically. No invented entities are introduced.

assumptions (4)

domain assumption Assumption 1 (Hessian similarity): (1/n)Σᵢ∥∇fᵢ(x)−∇fᵢ(y)−[∇f(x)−∇f(y)]∥² ≤ δ²∥x−y∥² for all x,y
Used in Theorem 1 and Corollaries to bound the variance term by δ²∥w_k−x̄_{k+1}∥², enabling the cancellation via the cpξ term in the Lyapunov contraction.
domain assumption Assumption 2: each f_i is µ-strongly convex
Needed for Lemma 6 (Lipschitz prox property), for strong convexity of the auxiliary objective f^k in Lemma 7, and for the 1/(1+µγ) contraction factor.
domain assumption Exact proximal oracle for each f_i at each iteration
Algorithm 3 computes x_{k+1} = prox_{γf_{i_k}}(·) exactly; the proof assumes this and the communication complexity accounting ignores the cost of this implicit step.
standard math Standard convex analysis facts (variance decomposition, Young's inequality, Cauchy-Schwarz, prox optimality conditions)
Used in Lemmas 1, 2, 7 and in Appendix A; these are textbook results from Bauschke and Combettes (2011) and elementary inequalities.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Speeding up Stochastic Proximal Optimization in the High Hessian Dissimilarity Setting." pith.science (2026). https://pith.science/paper/XEMXH7IE

@misc{pith2026241213619,
  author       = {Pith},
  title        = {Pith review of: Speeding up Stochastic Proximal Optimization in the High Hessian Dissimilarity Setting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XEMXH7IE}},
  note         = {Machine review of arXiv:2412.13619}
}

read the original abstract

Stochastic proximal point methods have recently garnered renewed attention within the optimization community, primarily due to their desirable theoretical properties. Notably, these methods exhibit a convergence rate that is independent of the Lipschitz smoothness constants of the loss function, a feature often missing in the loss functions of modern ML applications. In this paper, we revisit the analysis of the Loopless Stochastic Variance Reduced Proximal Point Method (L-SVRP). Building on existing work, we establish a theoretical improvement in the convergence rate in scenarios characterized by high Hessian dissimilarity among the functions. Our concise analysis, which does not require smoothness assumptions, demonstrates a significant improvement in communication complexity compared to standard stochastic gradient descent.

Figures

Figures reproduced from arXiv: 2412.13619 by the authors.

**Figure 1.** Convergence of L-SVRP under one of the 48 configurations examined, where the number of functions n = 100 and the Hessian similarity parameter δ 2 = 975.44. As predicted by theoretical analysis, the convergence is linear. We aggregate the slopes from such figures to produce [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗

**Figure 2.** Comparison of real convergence rate vs. the [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Convergence of L-SVRP for the first half of the 48 configurations described in the experimental section [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Convergence of L-SVRP for the second half of the 48 configurations described in the experimental section [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 56 canonical work pages

[1]

F., Chang, Q., Gao, R

Arinez, J. F., Chang, Q., Gao, R. X., Xu, C., and Zhang, J. (2020). Artificial intelligence in advanced manufacturing: Current status and future outlook. Journal of Manufacturing Science and Engineering , 142(11):110804

work page 2020
[2]

and Duchi, J

Asi, H. and Duchi, J. C. (2019). Stochastic (approximate) proximal point methods: Convergence, optimality, and adaptivity. SIAM Journal on Optimization , 29(3):2257–2290

work page 2019
[3]

and Combettes, P

Bauschke, H. and Combettes, P. (2011). Convex Analysis and Monotone Operator Theory in Hilbert Spaces . CMS Books in Mathematics. Springer New York

work page 2011
[4]

Bhat, S. A. and Huang, N.-F. (2021). Big data and ai revolution in precision agriculture: Survey and challenges. IEEE Access , 9:110209--110222

work page 2021
[5]

Condat, L., Kitahara, D., Contreras, A., and Hirabayashi, A. (2023). Proximal splitting algorithms for convex optimization: A tour of recent advances, with new twists

work page 2023
[6]

and Richt \'a rik, P

Condat, L. and Richt \'a rik, P. (2023). Randprox: Primal-dual optimization algorithms with randomized proximal updates. In The Eleventh International Conference on Learning Representations

work page 2023
[7]

Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., and Dean, J. (2019). A guide to deep learning in healthcare. Nature Medicine , 25(1):24--29

work page 2019
[8]

W., Kumar, S., Lim, W

Goodell, J. W., Kumar, S., Lim, W. M., and Pattnaik, D. (2021). Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis. Journal of Behavioral and Experimental Finance , 32:100577

work page 2021

Show all 61 references

[9]

Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning . Adaptive Computation and Machine Learning series. MIT Press

2016
[10]

P., Li, Z., and Richt\'arik, P

Gorbunov, E., Burlachenko, K. P., Li, Z., and Richt\'arik, P. (2021). Marina: Faster non-convex distributed learning with compression. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machi...

2021
[11]

M., Loizou, N., Qian, X., Sailanbayev, A., Shulgin, E., and Richt \'a rik, P

Gower, R. M., Loizou, N., Qian, X., Sailanbayev, A., Shulgin, E., and Richt \'a rik, P. (2019). SGD : G eneral analysis and improved rates. In International Conference on Machine Learning , pages 5200--5209. PMLR

2019
[12]

and Higham, D

Griffiths, D. and Higham, D. (2010). Numerical Methods for Ordinary Differential Equations: Initial Value Problems . Springer Undergraduate Mathematics Series. Springer London

2010
[13]

Grudzie\'n, M., Malinovsky, G., and Richt\'arik, P. (2023). Can 5th generation local training methods support client sampling? yes! In Ruiz, F., Dy, J., and van de Meent, J.-W., editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics...

2023
[14]

R., Millman, K

Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del R \' i o, J. F., Wiebe, M., Peterson, P., G \' e rard...

2020
[15]

Hofmann, T., Lucchi, A., Lacoste-Julien, S., and McWilliams, B. (2015). Variance reduced stochastic gradient descent with neighbors. In Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 28. Cur...

2015
[16]

and Huang, H

Hu, Z. and Huang, H. (2023). Tighter analysis for P rox S kip. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J., editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Res...

2023
[17]

Jhunjhunwala, D., Wang, S., and Joshi, G. (2023). Fedexp: Speeding up federated averaging via extrapolation. In The Eleventh International Conference on Learning Representations

2023
[18]

Jiang, X., Rodomanov, A., and Stich, S. U. (2024a). Federated optimization with doubly regularized drift correction. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F., editors, Proceedings of the 41st International Conferenc...

2024
[19]

Jiang, X., Rodomanov, A., and Stich, S. U. (2024b). Stabilized proximal-point methods for federated optimization

2024
[20]

and Zhang, T

Johnson, R. and Zhang, T. (2013). Accelerating stochastic gradient descent using predictive variance reduction. In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K., editors, Advances in Neural Information Processing Systems , volume 26. Curran Associates, Inc

2013
[21]

P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A

Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A. T. (2020). SCAFFOLD : Stochastic controlled averaging for federated learning. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 o...

2020
[22]

and Jin, C

Khaled, A. and Jin, C. (2023). Faster federated optimization under second-order similarity. In The Eleventh International Conference on Learning Representations

2023
[23]

B., Yu, F., Richt\' a rik, P., Suresh, A

Kone c n\' y , J., McMahan, H. B., Yu, F., Richt\' a rik, P., Suresh, A. T., and Bacon, D. (2016). Federated learning: strategies for improving communication efficiency. In NIPS Private Multi-Party Machine Learning Workshop

2016
[24]

and Richtárik, P

Konečný, J. and Richtárik, P. (2015). Semi-stochastic gradient descent methods

2015
[25]

D., Gasnikov, A., and Scutari, G

Kovalev, D., Beznosikov, A., Borodich, E. D., Gasnikov, A., and Scutari, G. (2022). Optimal gradient sliding and its application to optimal distributed optimization under similarity. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K., editors, Advances in Neural Information ...

2022
[26]

Kovalev, D., Horv \'a th, S., and Richt \'a rik, P. (2020). Don’t jump through hoops and remove those loops: Svrg and katyusha are better without the outer loop. In Kontorovich, A. and Neu, G., editors, Proceedings of the 31st International Conference on Algorithmic Learning T...

2020
[27]

Li, H., Acharya, K., and Richtárik, P. (2024). The power of extrapolation in federated learning

2024
[28]

K., Talwalkar, A., and Smith, V

Li, T., Sahu, A. K., Talwalkar, A., and Smith, V. (2020a). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine , 37(3):50--60

2020
[29]

K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V

Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. (2020b). Federated optimization in heterogeneous networks

2020
[30]

Li, Z. (2022). Anita: An optimal loopless accelerated variance-reduced gradient method

2022
[31]

Li, Z., Bao, H., Zhang, X., and Richt\'arik, P. (2021). Page: A simple and optimal probabilistic gradient estimator for nonconvex optimization. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings...

2021
[32]

Li, Z., Kovalev, D., Qian, X., and Richt\'arik, P. (2020c). Acceleration for compressed gradient descent in distributed and federated optimization. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Procee...

2020
[33]

and Richt\'arik, P

Li, Z. and Richt\'arik, P. (2021). Canita: Faster rates for distributed convex optimization with communication compression. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W., editors, Advances in Neural Information Processing Systems , volume 34, page...

2021
[34]

Lin, D., Han, Y., Ye, H., and Zhang, Z. (2023). Stochastic distributed optimization under average second-order similarity: Algorithms and analysis. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S., editors, Advances in Neural Information Processing ...

2023
[35]

and Forouzanfar, M

Maleki Varnosfaderani, S. and Forouzanfar, M. (2024). The role of ai in hospitals and clinics: transforming healthcare in the 21st century. Bioengineering , 11(4):337

2024
[36]

Malinovsky, G., Yi, K., and Richt \'a rik, P. (2022). Variance reduced proxskip: Algorithm, theory and application to federated learning. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K., editors, Advances in Neural Information Processing Systems

2022
[37]

B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B

McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS)

2017
[38]

Mishchenko, K., Li, R., Fan, H., and Venieris, S. (2024). Federated learning under second-order data heterogeneity

2024
[39]

Mishchenko, K., Malinovsky, G., Stich, S., and Richt\'arik, P. (2022). P rox S kip: Yes! L ocal gradient steps provably lead to communication acceleration! F inally! In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S., editors, Proceedings of the 3...

2022
[40]

Nesterov, Y. et al. (2018). Lectures on convex optimization , volume 137. Springer

2018
[41]

M., Liu, J., Scheinberg, K., and Tak \'a c , M

Nguyen, L. M., Liu, J., Scheinberg, K., and Tak \'a c , M. (2017). SARAH : A novel method for machine learning problems using stochastic recursive gradient. In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 ...

2017
[42]

Oosthuizen, K., Botha, E., Robertson, J., and Montecchi, M. (2021). Artificial intelligence in retail: The ai-enabled value chain. Australasian Marketing Journal , 29(3):264--273

2021
[43]

and Boyd, S

Parikh, N. and Boyd, S. (2013). Proximal Algorithms . Foundations and Trends in Optimization Series. Now Publishers

2013
[44]

Richtárik, P., Sadiev, A., and Demidovich, Y. (2024a). A unified theory of stochastic proximal point methods without smoothness

2024
[45]

Richtárik, P., Sadiev, A., and Demidovich, Y. (2024b). A unified theory of stochastic proximal point methods without smoothness

2024
[46]

and Monro, S

Robbins, H. and Monro, S. (1951). A stochastic approximation method. The annals of mathematical statistics , pages 400--407

1951
[47]

and Boyd, S

Ryu, E. and Boyd, S. (2016). Stochastic proximal iteration: A non-asymptotic improvement upon stochastic gradient descent. Technical report, Stanford University

2016
[48]

Sadiev, A., Condat, L., and Richtárik, P. (2024). Stochastic proximal point methods for monotone inclusions under expected similarity

2024
[49]

Sebbouh, O., Cuturi, M., and Peyr\'e, G. (2022). Randomized stochastic gradient descent ascent. In Camps-Valls, G., Ruiz, F. J. R., and Valera, I., editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics , volume 151 of Proceedings o...

2022
[50]

and Ben-David, S

Shalev-Shwartz, S. and Ben-David, S. (2014). Understanding machine learning: From theory to algorithms . Cambridge university press

2014
[51]

Shamir, O., Srebro, N., and Zhang, T. (2014). Communication-efficient distributed optimization using an approximate newton-type method. In Xing, E. P. and Jebara, T., editors, Proceedings of the 31st International Conference on Machine Learning , volume 32 of Proceedings of Ma...

2014
[52]

Sun, Y., Daneshmand, A., and Scutari, G. (2020). Distributed optimization based on gradient-tracking revisited: Enhancing convergence rate via surrogation

2020
[53]

Szlendak, R., Tyurin, A., and Richt \'a rik, P. (2022). Permutation compressors for provably faster distributed nonconvex optimization. In International Conference on Learning Representations

2022
[54]

Dinh, C., Tran, N., and Nguyen, J

T. Dinh, C., Tran, N., and Nguyen, J. (2020). Personalized federated learning with moreau envelopes. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems , volume 33, pages 21394--21405. Curran Associates, Inc

2020
[55]

Traor \'e , C., Apidopoulos, V., Salzo, S., and Villa, S. (2024). Variance reduction techniques for stochastic proximal point algorithms. Journal of Optimization Theory and Applications , 203(2):1910--1939

2024
[56]

E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt , S

Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt , S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J...

2020
[57]

Wen, Y., Fashiar Rahman , M., Xu, H., and Tseng, T.-L. B. (2022). Recent advances and trends of predictive maintenance from data-driven machine prognostics perspective. Measurement , 187:110276

2022
[58]

Woodworth, B., Mishchenko, K., and Bach, F. (2023). Two losses are better than one: Faster optimization using a cheaper proxy. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J., editors, Proceedings of the 40th International Conference on Mach...

2023
[59]

Woodworth, B. E. and Srebro, N. (2016). Tight complexity bounds for optimizing composite objectives. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc

2016
[60]

Yan, Y., Xu, Y., Lin, Q., Liu, W., and Yang, T. (2020). Optimal epoch stochastic gradient descent ascent methods for min-max optimization. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems , volume ...

2020
[61]

Zhang, J., He, T., Sra, S., and Jadbabaie, A. (2020). Why gradient clipping accelerates training: A theoretical justification for adaptivity. In International Conference on Learning Representations

2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

F., Chang, Q., Gao, R

Arinez, J. F., Chang, Q., Gao, R. X., Xu, C., and Zhang, J. (2020). Artificial intelligence in advanced manufacturing: Current status and future outlook. Journal of Manufacturing Science and Engineering , 142(11):110804

work page 2020

[2] [2]

and Duchi, J

Asi, H. and Duchi, J. C. (2019). Stochastic (approximate) proximal point methods: Convergence, optimality, and adaptivity. SIAM Journal on Optimization , 29(3):2257–2290

work page 2019

[3] [3]

and Combettes, P

Bauschke, H. and Combettes, P. (2011). Convex Analysis and Monotone Operator Theory in Hilbert Spaces . CMS Books in Mathematics. Springer New York

work page 2011

[4] [4]

Bhat, S. A. and Huang, N.-F. (2021). Big data and ai revolution in precision agriculture: Survey and challenges. IEEE Access , 9:110209--110222

work page 2021

[5] [5]

Condat, L., Kitahara, D., Contreras, A., and Hirabayashi, A. (2023). Proximal splitting algorithms for convex optimization: A tour of recent advances, with new twists

work page 2023

[6] [6]

and Richt \'a rik, P

Condat, L. and Richt \'a rik, P. (2023). Randprox: Primal-dual optimization algorithms with randomized proximal updates. In The Eleventh International Conference on Learning Representations

work page 2023

[7] [7]

Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., and Dean, J. (2019). A guide to deep learning in healthcare. Nature Medicine , 25(1):24--29

work page 2019

[8] [8]

W., Kumar, S., Lim, W

Goodell, J. W., Kumar, S., Lim, W. M., and Pattnaik, D. (2021). Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis. Journal of Behavioral and Experimental Finance , 32:100577

work page 2021