Pith. sign in

REVIEW 2 major objections 3 minor 93 references

Federated Majorize-Minimization: Beyond Parameter Aggregation

T0 review · 2 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that federated optimization via majorize-minimization should aggregate surrogate parameters rather than model parameters, and proves a finite-time convergence bound for the resulting FedMM algorithm that requires no…

desk verdict The surrogate-aggregation idea is genuinely useful, but Theorem 1 does not cover the paper's own flagship examples, so the convergence claims overreach. read the letter →

arxiv 2507.17534 v1 pith:SS72YVRI submitted 2025-07-23 cs.LG cs.AImath.OCstat.ML

Aymeric Dieuleveut , Gersende Fort , Mahmoud Hegazy , Hoi-To Wai This is my paper
classification cs.LGcs.AImath.OCstat.ML
keywords federatedlearningmajorize-minimizationsurrogateaggregationstochasticapproximationcontrolvariatespartialparticipationcompressionoptimaltransport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The central claim is that in federated majorize-minimization problems, the correct aggregation variable is the parameter of the local surrogate function, not the model parameter. The paper introduces FedMM, a stochastic approximation algorithm in which each client sends a compressed, control-variate-corrected estimate of its surrogate parameter, and the server averages these to build a global surrogate before minimizing it. The main theorem bounds the mean-field residual in finite time without any assumption on client data heterogeneity, under conditions on the surrogate family and the projection geometry. If correct, this unifies federated EM, federated matrix factorization, and proximal-gradient-type methods under one recipe: take any centralized MM algorithm with a linearly parameterized surrogate and replace parameter averaging with surrogate averaging.

What carries the argument

The load-bearing object is the linearly parameterized surrogate family $U(\theta,s)=g(\theta)+\psi(\theta)-\langle s,\phi(\theta)\rangle$ together with the mirror map $T(s)=\arg\min_\theta U(\theta,s)$ and the mean field $h(s)=\mathbb{E}_\pi[\bar{S}(Z,T(s))]-s$. Because the surrogate parameter enters linearly, a weighted average of client surrogates is the surrogate of the mixture distribution $\sum_i\mu_i\pi_i$, which is why aggregation in $S$-space reproduces the centralized MM algorithm on the pooled data. FedMM's control variates $V_{t,i}$, compression operators $\mathrm{Quant}_{t+1,i}$, and the projection $\Pi(\hat{S}_t+\gamma_{t+1}H_{t+1},B_t)$ are the mechanisms that make this surrogate-space aggregation stable under partial participation, compression, and the constraint that $S$ need not be the whole space.

What would settle it

Compute the mean-field $h(s)=\mathbb{E}_\pi[\bar{S}(Z,T(s))]-s$ for the quadratic-surrogate example with $g=0$ and a strongly convex quadratic $f$: since $\bar{S}(Z,\tau)=\tau-\rho G(Z,\tau)$ and $T(s)=s$, one gets $h(s)=-\rho\mathbb{E}_\pi[G(Z,s)]$, which grows linearly with $\|s\|$, so $\sup_S\|h\|=\infty$ and the constant $\bar{C}$ in Theorem 1 is not finite. A reader can verify this directly, and it shows the theorem's assumptions do not cover the first example the paper presents.

Watch

Extended reading notes

Core claim

The paper's discovery is that for problems where the surrogate upper bounds have the linearly parameterized form $U(\theta,s)=g(\theta)+\psi(\theta)-\langle s,\phi(\theta)\rangle$, the fixed points of the centralized MM iteration are exactly the roots of the mean-field function $h(s)=\mathbb{E}_\pi[\bar{S}(Z,T(s))]-s$, where $T(s)=\arg\min_\theta U(\theta,s)$. FedMM lets each active client oracle its local $\bar{S}$ at the current mirror point $T(\hat{S}_t)$, subtracts a client-level control variate $V_{t,i}$ to correct drift, compresses the corrected increment, and the server aggregates these increments in the surrogate space before computing the global minimizer $T$ centrally. Theorem 1 provides a finite-time bound on $\bar{E}_t$, a normalized B-norm distance from $\hat{S}_t$ to the zero set of $h$, with explicit dependence on the compression variance $\omega$, the participation probability $p$, and the client noise variances $\sigma_i^2$; the bound is stated to be valid without any assumption on the heterogeneity of the client distributions $\pi_i$. The same surrogate-aggregation principle is then applied, beyond the strict framework, to design a pseudo-MM algorithm for federated optimal transport in which clients send best-response parameters of local convex potential surrogates.

Load-bearing premise

The proof assumes that the mean-field function $h(s)$ is bounded on the whole surrogate space and that the local geometry matrix $B(s)$ is uniformly positive definite, but these fail for the quadratic surrogate (where $h$ is unbounded) and for the dictionary-learning variational surrogate (where $B(s)$ is rank-deficient).

Editorial extensions

If this is right

  • Any MM algorithm with a linearly parameterized surrogate can be federated by averaging surrogate parameters; under full participation and no compression, the server-side update is exactly the centralized MM update on the mixture distribution.
  • With constant step sizes, Theorem 1 yields an explicit weighted bound on the mean-field residual of order $1/T$ plus a noise floor proportional to $\sigma^2/n$ and to the initial control-variate error.
  • Control variates are unnecessary in the ideal no-compression, full-participation regime, but in the presence of compression or partial participation they are required; the paper reports that setting $\alpha=0$ worsens convergence under data heterogeneity.
  • The surrogate-aggregation principle also works outside the strict linearly parameterized setting, giving a federated optimal transport algorithm that converges faster than a standard federated first-order method on the benchmark considered in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If surrogate-space aggregation is the right design principle, then the choice of parameterization $\phi$ is not neutral: different parameterizations of the same surrogate family lead to different federated algorithms, so the framework implicitly recommends representing surrogates in a way that makes the mean field well-behaved.
  • The convergence theorem's assumptions A6(ii) and A7(ii) exclude the paper's own quadratic-surrogate and dictionary-learning examples; extending the proof to unbounded mean fields and rank-deficient $B(s)$ would complete the advertised unification.
  • The pseudo-MM treatment of optimal transport suggests a broad recipe for federating min-max and bilevel problems: clients communicate parameters of local best-response models rather than gradients, a strategy that could be tested on other surrogate-based algorithms.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper studies federated Majorize-Minimization (MM) problems in which each client's surrogate is linearly parameterized by a finite-dimensional mirror parameter s in a surrogate space S. The authors propose SA-SSMM for the centralized setting and FedMM for the federated setting, where clients compute local estimates of the surrogate parameter, correct them with control variates, possibly compress them, and the server aggregates in S before applying the map T to obtain the next model parameter. The main theoretical result, Theorem 1, gives a finite-time bound on a mean-field residual Ē_t under assumptions A1–A8, and the authors emphasize that the bound requires no assumption on data heterogeneity. The paper also reports dictionary-learning experiments and a separate pseudo-MM algorithm for federated optimal transport maps.

Significance. The conceptual message is valuable: for linearly parameterized surrogates, aggregating surrogate parameters rather than model parameters can preserve the centralized MM structure and yields a unifying view of existing algorithms such as Federated EM and Federated Matrix Factorization. The paper is also honest and rigorous in much of its presentation: Proposition 1 gives a clean equivalence between roots of the mean-field h and stationary points, the appendix contains detailed proofs, and the constants in Theorem 1 are explicit. However, the advertised convergence guarantee is narrower than the paper's framing suggests, because two of the three canonical examples do not satisfy assumptions that are load-bearing for Theorem 1. If the scope is revised to state clearly which settings are covered, the contribution is a useful and non-circular algorithmic framework.

major comments (2)
  1. [Section 5, A6(ii) and Theorem 1] Theorem 1 requires A6(ii), sup_S ||h(·)|| < ∞, but this assumption fails for the quadratic-surrogate example that the paper presents as a central instance. In Example 1 with g=0, S=R^d, T(s)=s, and h(s)=Eπ[\bar S(Z,T(s))]-s = -ρ∇f(s); for a quadratic f with nonzero Hessian, h is affine and unbounded. The constant \bar C := C⋆⋆ + sup_S ||h(·)|| C⋆ in Theorem 1 is then infinite, and the step-size condition γ_{t+1} ∈ (0, v_min/(2\bar C)) has no feasible positive range. Appendix E.1 verifies only A7 for the quadratic surrogate, not A6(ii), so the theorem does not apply to the stochastic-gradient/proximal-gradient setting discussed in Section 2.3.
  2. [Section 5 and Appendix E.3] A7(ii) is not satisfied for the variational surrogate used in dictionary learning and matrix factorization. Appendix E.3 shows that for p=K=1 the required matrix B(s) has rank one, so no uniform lower bound v_min > 0 can exist on the surrogate space S = M+_K × R^{p×K}; the authors state that extending the proof to this case is out of scope. Despite this, Section 6 runs FedMM on exactly this dictionary-learning problem with B_t=I, which is not the B(s) required by A7/A8. Consequently, the numerical dictionary-learning study is not covered by Theorem 1, and the advertised convergence guarantee does not apply to the matrix-factorization example that the paper claims to unify.
minor comments (3)
  1. [Appendix D.6, Corollary 2] The displayed inequalities for ¯E_{t+1} and E_{t+1} omit the square on the norm of H_{t+1}-h(\hat S_t); as written the terms are dimensionally inconsistent with their use in Proposition 2, where the squared norm appears.
  2. [Section 6, experimental setup] The paper reports that the projection matrix is set to B_t=I and acknowledges that the theory requires another selection, but it would help the reader to state explicitly that the dictionary-learning experiments are therefore heuristic demonstrations rather than instances of the proved convergence theorem.
  3. [Section 7, FedMM-OT] Algorithm 3 is introduced as a 'pseudo-MM' method and the paper correctly notes that the server-side surrogate is not guaranteed to majorize the global objective; this should be kept in the abstract and conclusion so that the optimal-transport section is not read as a proven MM guarantee.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity: FedMM's convergence bound is a genuine stochastic-approximation proof; self-citations are contextual and not load-bearing, while the flagged assumption failures are correctness/scope gaps rather than self-referential reductions.

full rationale

The claimed derivation is not circular. Theorem 1 is proved in Appendix D as a self-contained stochastic-approximation argument: Proposition 2 establishes a Lyapunov descent for (f+g)∘T under MM-1, MM-2, and A7; Proposition 3 proves that the FedMM oracle H_{t+1} is conditionally unbiased for the mean field h(S_t) and bounds its variance using A3-A5; Proposition 4 gives the control-variate Lyapunov recursion under A6-A7. The final bound controls Ē_t, which is explicitly defined in terms of h and the projected SA step; convergence is not 'predicted' from a fitted value, and no equation in the proof is assumed equal to the quantity being bounded. The S-space aggregation claim is a structural identity: by linearity of the surrogate parameterization, s_{t+1} = Σ_i μ_i E_{π_i}[S(Z,T(s_t))] = E_π[S(Z,T(s_t))], so averaging surrogate parameters is exactly averaging the surrogate functions. This is a derivation from the framework, not a renamed empirical pattern. Self-citations—FedEM (Dieuleveut et al., 2021), the SA survey (Dieuleveut et al., 2023), and the block-quantization example—are used as instances, related work, or illustrative references; none is the load-bearing justification for Theorem 1, which does not invoke those papers in its proof. The genuine concerns are not circularity: A6(ii) can fail for Example 1 when f is quadratic and g=0 because h(s) = -ρ∇f(s) is unbounded, and Appendix E.3 explicitly concedes that the variational/dictionary-learning surrogate has rank-deficient B(s), so A7(ii) fails and the paper states that extending the proof is 'out of the scope of this paper.' Section 6 then runs FedMM with B_t=I, which is outside the theorem's projection assumption. These are correctness and scope limitations, not reductions of the result to its own assumptions.

Assumptions & free parameters 0 free parameters · 10 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; the 'surrogate space S' is a parameterization of known majorizing functions. The main unverified load-bearing assumptions are A6(ii) and A7(ii), which restrict the theory's scope more than the paper's narrative acknowledges. The control variates are standard in the optimization literature.

assumptions (10)
  • domain assumption MM-1: the surrogate functions are linearly parameterized and majorize f, with measurable S̄, ψ, φ satisfying the inequality (3).
    This is the defining structural assumption of the framework, invoked at the start of Section 2. It is not proven and must hold for all examples.
  • domain assumption MM-2: the map T(s) = argmin(g + ψ - ⟨s, φ⟩) is well-defined and measurable.
    Needed for the algorithm to produce iterates; stated in Section 2 as a standing condition.
  • domain assumption A1: for any s, θ↦ψ(θ)-⟨s,φ(θ)⟩ is convex, and ψ, φ are C1 on intΘ.
    Used in Proposition 1 to apply subdifferential calculus; stated in Section 2.
  • domain assumption A2: f is C1 on intΘ.
    Used in Proposition 1 for the stationarity equivalence.
  • domain assumption A3: local oracles are unbiased with uniformly bounded variance.
    Standard SA oracle assumption, stated in Section 5.
  • domain assumption A4(ω): compression is unbiased with E||Quant(s)-s||² ≤ ω||s||².
    Standard unbiased compression model, includes quantization and sparsification.
  • domain assumption A5(p): each worker is selected independently with probability p.
    Models partial participation and is used to fold participation into compression.
  • domain assumption A6: each h_i is globally Lipschitz and sup_S ||h|| < ∞.
    The boundedness part is strong and not verified for the quadratic or dictionary-learning examples; it is used in Proposition 2 and Theorem 1.
  • domain assumption A7: ϕ∘T is smooth in the sense of the estimates with a uniformly positive definite B(s), v_min>0.
    The positive-definiteness fails for the dictionary-learning variational surrogate (Appendix E.3), making Theorem 1 inapplicable to that example.
  • domain assumption A8: S is closed convex and the projection in Algorithm 2 uses B_t = B(Ŝ_t).
    The projection geometry is part of the analysis; experiments violate it by using B_t = I.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Majorize-Minimization: Beyond Parameter Aggregation." pith.science (2026). https://pith.science/paper/SS72YVRI

@misc{pith2026250717534,
  author       = {Pith},
  title        = {Pith review of: Federated Majorize-Minimization: Beyond Parameter Aggregation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SS72YVRI}},
  note         = {Machine review of arXiv:2507.17534}
}
read the original abstract

This paper proposes a unified approach for designing stochastic optimization algorithms that robustly scale to the federated learning setting. Our work studies a class of Majorize-Minimization (MM) problems, which possesses a linearly parameterized family of majorizing surrogate functions. This framework encompasses (proximal) gradient-based algorithms for (regularized) smooth objectives, the Expectation Maximization algorithm, and many problems seen as variational surrogate MM. We show that our framework motivates a unifying algorithm called Stochastic Approximation Stochastic Surrogate MM (\SSMM), which includes previous stochastic MM procedures as special instances. We then extend \SSMM\ to the federated setting, while taking into consideration common bottlenecks such as data heterogeneity, partial participation, and communication constraints; this yields \QSMM. The originality of \QSMM\ is to learn locally and then aggregate information characterizing the \textit{surrogate majorizing function}, contrary to classical algorithms which learn and aggregate the \textit{original parameter}. Finally, to showcase the flexibility of this methodology beyond our theoretical setting, we use it to design an algorithm for computing optimal transport maps in the federated setting.

Figures

Figures reproduced from arXiv: 2507.17534 by the authors.

Figure 1
Figure 1. Impact of Aggregation Space. The columns (left to right) display the objective value, parameter update size (E p t and E p,s t ), and surrogate update size (E s t and E s,p t ) as a function of communication rounds, t ∈ 1, . . . , Tmax. The rows correspond to different data settings (from top to bottom): synthetic homogeneous, synthetic heterogeneous, and MovieLens 1M. Results are averaged over 10 runs, with the sha… view at source ↗
Figure 2
Figure 2. Impact of Control Variates. The columns (left to right) display the objective value, parameter update size (E p,s t ), and surrogate update size (E s t ) as a function of commu￾nication rounds, t ∈ 1, . . . , Tmax. The rows correspond to different data settings (from top to bottom): synthetic homogeneous, synthetic heterogeneous, and MovieLens 1M. Results are averaged over 10 runs, with the shaded regions indicating… view at source ↗
Figure 3
Figure 3. Performance is measured by (L UVP 2 ), where lower is better, versus communication rounds for dimensions d = 16, d = 32, and d = 64. In all settings, our proposed FedMM-OT algorithm converges faster than the FedAdam baseline. 8 Conclusion This paper investigates Stochastic Surrogate MM methods that iteratively construct surrogate functions in a parametric space. This principle allows us to introduce FedMM, a unified… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

93 extracted references · 73 canonical work pages

  1. [1]

    Alistarh, D

    D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic. QSGD : Communication - Efficient SGD via Gradient Quantization and Encoding . Advances in Neural Information Processing Systems, 30: 0 1709--1720, 2017

    2017

  2. [2]

    B. Amos, L. Xu, and J. Kolter. Input Convex Neural Networks . In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70, pages 146--155. PMLR, 06--11 Aug 2017

    2017

  3. [3]

    Andrieu, E

    C. Andrieu, E. Moulines, and P. Priouret. Stability of Stochastic Approximation under Verifiable Conditions . SIAM Journal on Control and Optimization, 44 0 (1): 0 283--312, 2005

    2005

  4. [4]

    Atchad \'e , G

    Y. Atchad \'e , G. Fort, and E. Moulines. On Perturbed Proximal Gradient Algorithms . Journal of Machine Learning Research, 18 0 (10): 0 1--33, 2017

    2017

  5. [5]

    H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer Publishing Company, Incorporated, 2011

    2011

  6. [6]

    Beck and M

    A. Beck and M. Teboulle. A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems . SIAM journal on imaging sciences, 2 0 (1): 0 183--202, 2009 a

    2009

  7. [7]

    Beck and M

    A. Beck and M. Teboulle. Gradient-based algorithms with applications to signal-recovery problems, page 42–88. Cambridge University Press, 2009 b

    2009

  8. [8]

    Benveniste, M

    A. Benveniste, M. M \' e tivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations . Springer Verlag, 1990

    1990

Show all 93 references
  1. [9]

    V. S. Borkar. Stochastic Approximation. A Dynamical Systems Viewpoint . Cambridge University Press, Cambridge; Hindustan Book Agency, New Delhi, 2008

    2008

  2. [10]

    Bradley, K

    P. Bradley, K. Bennett, and A. Demiriz. Constrained K-Means Clustering . Microsoft Research, Redmond, 20 0 (0): 0 0, 2000

    2000

  3. [11]

    L. D. Brown. Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory . Lecture notes-monograph series Fundamentals of statistical exponential families. Institute of Mathematical Statistics, 1986

    1986

  4. [12]

    T. Canh, T. Nguyen, and N. Josh. Personalized Federated Learning with Moreau Envelopes . In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 21394--21405. Curran Associates, Inc., 2020

    2020

  5. [13]

    Capp \'e and \'E

    O. Capp \'e and \'E . Moulines. On-Line Expectation–Maximization Algorithm for Latent Data Models . Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71 0 (3): 0 593--613, 2009

    2009

  6. [14]

    Celeux and J

    G. Celeux and J. Diebolt. A stochastic approximation type EM algorithm for the mixture problem . Stochastics and Stochastic Reports, 41 0 (1-2): 0 119--134, 1992

    1992

  7. [15]

    J. Chen, J. Zhu, Y. Teh, and T. Zhang. Stochastic Expectation Maximization with Variance Reduction . In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 7967--7977, 2018

    2018

  8. [16]

    Collins, R

    M. Collins, R. E. Schapire, and Y. Singer. Logistic Regression, AdaBoost and Bregman Distances . Machine Learning, 48 0 (1): 0 253--285, 2002

    2002

  9. [17]

    P. L. Combettes and J.-C. Pesquet. Proximal Splitting Methods in Signal Processing, pages 185--212. Springer New York, New York, NY, 2011

    2011

  10. [18]

    Condat and P

    L. Condat and P. Richt \'a rik. RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updates . In The Eleventh International Conference on Learning Representations. OpenReview.net, 2023

    2023

  11. [19]

    X. Dai, X. Yan, K. Zhou, H. Yang, K. K. Ng, J. Cheng, and Y. Fan. Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning . Technical report, arXiv:1911.04655, 2019

    1911 arXiv

  12. [20]

    Della Pietra, V

    S. Della Pietra, V. Della Pietra, and J. Lafferty. Duality and Auxiliary Functions for Bregman Distances . Technical report, Carnegie-Mellon University, 2001

    2001

  13. [21]

    Delyon, M

    B. Delyon, M. Lavielle, and E. Moulines. Convergence of a Stochastic Approximation Version of the EM Algorithm . Ann. Statist., 27 0 (1): 0 94--128, 1999

    1999

  14. [22]

    A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum Likelihood from Incomplete Data via the EM Algorithm . Journal of the Royal Statistical Society: Series B, 39: 0 1--38, 1977

    1977

  15. [23]

    Dieuleveut, G

    A. Dieuleveut, G. Fort, E. Moulines, and G. Robin. Federated-EM with heterogeneity mitigation and variance reduction . In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 29553--29...

    2021

  16. [24]

    Dieuleveut, G

    A. Dieuleveut, G. Fort, E. Moulines, and H.-T. Wai. Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning . IEEE Transactions on Signal Processing, 71: 0 3117--3148, 2023

    2023

  17. [25]

    C. Dinh, T. T. Vu, N. H. Tran, M. N. Dao, and H. Zhang. FedU: A Unified Framework for Federated Multi-Task Learning with Laplacian Regularization . Technical report, arXiv:2102.07148, 2021

    2021 arXiv

  18. [26]

    R. M. Dudley. The speed of mean glivenko-cantelli convergence. The Annals of Mathematical Statistics, 40 0 (1): 0 40--50, 1969

    1969

  19. [27]

    Efron, T

    B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least Angle Regression . The Annals of Statistics, 32 0 (2), April 2004

    2004

  20. [28]

    Flanagan, W

    A. Flanagan, W. Oyomno, A. Grigorievskiy, K. E. Tan, S. Khan, and M. Ammad-Ud-Din. Federated Multi-view Matrix Factorization for Personalized Recommendations . In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 324--347. Springer, 2020

    2020

  21. [29]

    Fort and E

    G. Fort and E. Moulines. Convergence of the Monte Carlo Expectation Maximization for Curved Exponential Families . Ann. Statist., 31 0 (4): 0 1220--1259, 2003

    2003

  22. [30]

    Fort and E

    G. Fort and E. Moulines. Stochastic variable metric proximal gradient with variance reduction for non-convex composite optimization. Stat Comput, 33 0 (65), 2023

    2023

  23. [31]

    G. Fort, P. Gach, and E. Moulines. Fast incremental expectation maximization for finite-sum optimization: nonasymptotic convergence . Statistics and Computing, 31, 2021 a

    2021

  24. [32]

    G. Fort, E. Moulines, and H.-T. Wai. Geom-Spider-EM: Faster Variance Reduced Stochastic Expectation Maximization for Nonconvex Finite-Sum Optimization . In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3135--3139, 2021 b

    2021

  25. [33]

    D. Gao, B. Tan, C. Ju, V. W. Zheng, and Q. Yang. Privacy Threats Against Federated Matrix Factorization . Technical report, arXiv:2007.01587, 2020

    2007 arXiv

  26. [34]

    Ghadimi and G

    S. Ghadimi and G. Lan. Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming . SIAM J. Optimiz., 23 0 (4): 0 2341--2368, 2013

    2013

  27. [35]

    Ghosh, J

    A. Ghosh, J. Chung, D. Yin, and K. Ramchandran. An Efficient Framework for Clustered Federated Learning . Advances in Neural Information Processing Systems, 33: 0 19586--19597, 2020

    2020

  28. [36]

    Gorbunov, Z

    E. Gorbunov, Z. Burlachenko, K. P.and Li, and P. Richt \'a rik. MARINA: Faster Non-Convex Distributed Learning with Compression . In International Conference on Machine Learning, pages 3788--3798. PMLR, 2021

    2021

  29. [37]

    Hanzely, S

    F. Hanzely, S. Hanzely, S. Horv \'a th, and P. Richt \'a rik. Lower Bounds and Optimal Algorithms for Personalized Federated Learning . Advances in Neural Information Processing Systems, 33: 0 2304--2315, 2020

    2020

  30. [38]

    Harper and J

    F. Harper and J. A. Konstan. The Movielens Datasets: History and context . ACM Transactions on interactive intelligent systems, 5 0 (4): 0 1--19, 2015

    2015

  31. [39]

    Heged u s, G

    I. Heged u s, G. Danner, and M. Jelasity. Decentralized Recommendation Based on Matrix Factorization: A Comparison of Gossip and Federated Learning . In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 317--332. Springer, 2019

    2019

  32. [40]

    Islamov, X

    R. Islamov, X. Qian, and P. Richt \'a rik. Distributed Second Order Methods with Fast Rates and Compressed Communication . In International Conference on Machine Learning, pages 4617--4628. PMLR, 2021

    2021

  33. [41]

    A. K. Jain, M. N. Murty, and P. J. Flynn. Data clustering: a review . ACM computing surveys (CSUR), 31 0 (3): 0 264--323, 1999

    1999

  34. [42]

    Kairouz, H

    P. Kairouz, H. McMahan, et al. Advances and Open Problems in Federated Learning. New Foundations and Trends, 2021

    2021

  35. [43]

    S. P. Karimireddy, Q. Rebjock, S. Stich, and M. Jaggi. Error Feedback Fixes SignSGD and other Gradient Compression Schemes . In International Conference on Machine Learning , pages 3252--3261. PMLR, May 2019. ISSN: 2640-3498

    2019

  36. [44]

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning . In International Conference on Machine Learning, pages 5132--5143. PMLR, 2020

    2020

  37. [45]

    Khaled and P

    A. Khaled and P. Richtárik. Gradient Descent with Compressed Iterates . Technical report, arXiv:1909.04716, 2020

    1909 arXiv

  38. [46]

    D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization . arXiv:1412.6980 [cs], January 2017. arXiv: 1412.6980

    2017 arXiv

  39. [47]

    Koloskova, S

    A. Koloskova, S. Stich, and M. Jaggi. Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication . In International conference on machine learning, pages 3478--3487. PMLR, 2019

    2019

  40. [48]

    Konečný, H

    J. Konečný, H. B. McMahan, F. X. Yu, P. Richtarik, A. T. Suresh, and D. Bacon. Federated Learning : Strategies for Improving Communication Efficiency . In NIPS Workshop on Private Multi - Party Machine Learning , 2016

    2016

  41. [49]

    Korotin, V

    A. Korotin, V. Egiazarian, A. Asadulaev, A. Safin, and E. Burnaev. Wasserstein-2 Generative Networks . In International Conference on Learning Representations, 2021 a

    2021

  42. [50]

    Korotin, L

    A. Korotin, L. Li, A. Genevay, J. Solomon, A. Filippov, and E. Burnaev. Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark . In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021 b

    2021

  43. [51]

    Kunstner, R

    F. Kunstner, R. Kumar, and M. Schmidt. Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent . In International Conference on Artificial Intelligence and Statistics, pages 3295--3303. PMLR, 2021

    2021

  44. [52]

    K. Lange. Optimization. Springer Texts in Statistics. Springer New York, 2013

    2013

  45. [53]

    Lange, D

    K. Lange, D. Hunter, and I. Yang. Optimization Transfer Using Surrogate Objective Functions . Journal of computational and graphical statistics, 9 0 (1): 0 1--20, 2000

    2000

  46. [54]

    Leconte, A

    L. Leconte, A. Dieuleveut, E. Oyallon, E. Moulines, and G. Pages. DoStoVoQ: Doubly Stochastic Voronoi Vector Quantization SGD for Federated Learning . Technical report, openreview.net, 2021

    2021

  47. [55]

    Lee and H

    D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401 0 (6755): 0 788--791, 1999

    1999

  48. [56]

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang. On the Convergence of FedAvg on Non - IID Data . In International Conference on Learning Representations, 2020

    2020

  49. [57]

    J. Mairal. Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization . In C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013

    2013

  50. [58]

    J. Mairal. Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning . SIAM Journal on Optimization, 25 0 (2): 0 829--855, 2015

    2015

  51. [59]

    Mairal, F

    J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res., 11: 0 19–60, 2010

    2010

  52. [60]

    Makkuva, A

    A. Makkuva, A. Taghvaei, S. Oh, and J. Lee. Optimal transport mapping via input convex neural networks. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 6672-...

    2020

  53. [61]

    Mansour, M

    Y. Mansour, M. Mohri, J. Ro, and A. Suresh. Three Approaches for Personalization with aApplications to Federated Learning . Technical report, arXiv:2002.10619, 2020

    2002 arXiv

  54. [62]

    Marfoq, G

    O. Marfoq, G. Neglia, A. Bellet, L. Kameni, and R. Vidal. Federated Multi-Task Learning under a Mixture of Distributions . Advances in Neural Information Processing Systems, 34, 2021

    2021

  55. [63]

    McLachlan and T

    G. McLachlan and T. Krishnan. The EM Algorithm and Extensions . Wiley series in probability and statistics. Wiley, 2008

    2008

  56. [64]

    McMahan, E

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Aguera y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data . In Artificial intelligence and statistics, pages 1273--1282. PMLR, 2017 a

    2017

  57. [65]

    McMahan, E

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas. Communication- Efficient Learning of Deep Networks from Decentralized Data . In Artificial Intelligence and Statistics , pages 1273--1282. PMLR, April 2017 b . ISSN: 2640-3498

    2017

  58. [66]

    Mishchenko, E

    K. Mishchenko, E. Gorbunov, M. Takáč, and P. Richtárik. Distributed Learning with Compressed Gradient Differences . Technical report, arXiv:1901.09269, 2019

    1901 arXiv

  59. [67]

    Mishchenko, G

    K. Mishchenko, G. Malinovsky, S. Stich, and P. Richtarik. P rox S kip: Yes! L ocal gradient steps provably lead to communication acceleration! F inally! In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, editors, Proceedings of the 39th International C...

    2022

  60. [68]

    J.-J. Moreau. Fonctions convexes duales et points proximaux dans un espace hilbertien. C. R. Acad. Sci., Paris, 255: 0 2897--2899, 1962

    1962

  61. [69]

    R. M. Neal and G. E. Hinton. A View of the EM Algorithm that Justifies Incremental, Sparse, and other Variants . In M. I. Jordan, editor, Learning in Graphical Models, pages 355--368. Springer Netherlands, Dordrecht, 1998

    1998

  62. [70]

    Nguyen, F

    H. Nguyen, F. Forbes, G. Fort, and O. Capp \'e . An Online Minorization-Maximization Algorithm . In P. Brito, J. G. Dias, B. Lausen, A. Montanari, and R. Nugent, editors, Classification and Data Science in the Digital Age, pages 263--271. Springer International Publishing, 2023

    2023

  63. [71]

    Paatero and U

    P. Paatero and U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 5 0 (2): 0 111--126, 1994

    1994

  64. [72]

    Parikh and S

    N. Parikh and S. Boyd. Proximal Algorithms . Found. Trends Optim., 1 0 (3): 0 127–239, 2014

    2014

  65. [73]

    Philippenko and A

    C. Philippenko and A. Dieuleveut. Preserved central model for faster bidirectional compression in distributed settings. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021

    2021

  66. [74]

    R. Polyak. Introduction to Continuous Optimization . Springer Optimization and Its Applications. Springer, April 2021

    2021

  67. [75]

    Reddi, Z

    S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Kone c n \'y , K. S., and H. McMahan. Adaptive Federated Optimization . In International Conference on Learning Representations, 2021

    2021

  68. [76]

    Richt \'a rik, I

    P. Richt \'a rik, I. Sokolov, and I. Fatkhullin. EF 21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback . In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021

    2021

  69. [77]

    Robbins and S

    H. Robbins and S. Monro. A Stochastic Approximation Method . Annals of Mathematical Statistics, 22 0 (3): 0 400--407, 1951

    1951

  70. [78]

    Sato and S

    M.-A. Sato and S. Ishii. On-line EM Algorithm for the Normalized Gaussian Network . Neural Computation, 12 0 (2): 0 407--432, 2000

    2000

  71. [79]

    Sattler, S

    F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek. Robust and Communication - Efficient Federated Learning From Non -i.i.d. Data . IEEE Transactions on Neural Networks and Learning Systems, pages 1--14, 2019

    2019

  72. [80]

    Sattler, K.-R

    F. Sattler, K.-R. M \"u ller, and W. Samek. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization under Privacy Constraints . IEEE Transactions on Neural Networks and Learning Systems, 32 0 (8): 0 3710--3722, 2020

    2020

  73. [81]

    Singhal, H

    K. Singhal, H. Sidahmed, Z. Garrett, S. Wu, J. Rush, and S. Prakash. Federated Reconstruction: Partially Local Federated Learning . In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pa...

    2021

  74. [82]

    S. Stich. Local SGD Converges Fast and Communicates Little . In International Conference on Learning Representations, 2019

    2019

  75. [83]

    Y. Tian, H. Weng, and Y. Feng. Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms . In Forty-first International Conference on Machine Learning, 2024

    2024

  76. [84]

    C. Villani. Topics in Optimal Transportation , volume 58. American Mathematical Soc., 2021

    2021

  77. [85]

    J. Wang, Z. Charles, et al. A Field Guide to Federated Optimization . Technical report, arXiv:2107.06917, 2021

    2021 arXiv

  78. [86]

    Wang and T.-H

    S. Wang and T.-H. Chang. Federated Matrix Factorization: Algorithm Design and Application to Data Clustering . IEEE Transactions on Signal Processing, 70: 0 1625--1640, 2022

    2022

  79. [87]

    Wangni, J

    J. Wangni, J. Wang, J. Liu, and T. Zhang. Gradient Sparsification for Communication-Efficient Distributed Optimization . In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31....

    2018

  80. [88]

    Wei and M

    G. Wei and M. Tanner. A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms . J. Am. Stat. Assoc., 85 0 (411): 0 699--704, 1990

    1990

  81. [89]

    Woodworth, K

    B. Woodworth, K. Patel, S. Stich, Z. Dai, B. Bullins, B. Mcmahan, O. Shamir, and N. Srebro. Is Local SGD Better than Minibatch SGD ? In International Conference on Machine Learning , pages 10334--10343. PMLR, 2020

    2020

  82. [90]

    S. J. Wright, R. Nowak, and M. Figueiredo. Sparse Reconstruction by Separable Approximation . IEEE Transactions on signal processing, 57 0 (7): 0 2479--2493, 2009

    2009

  83. [91]

    H. Yuan, M. Zaheer, and S. Reddi. Federated Composite Optimization . In International Conference on Machine Learning, pages 12253--12266. PMLR, 2021

    2021

  84. [92]

    Zhixu, C

    T. Zhixu, C. Rajita, and K. Sanjeev. On the Convergence of a Federated Expectation-Maximization Algorithm . Technical report, arXiv:2408.05819, 2024

    2024

  85. [93]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.