Pith. sign in

REVIEW 5 major objections 5 minor 37 references

Adaptive collaboration for online personalized distributed learning with heterogeneous clients

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A binary gradient-similarity collaboration rule reaches oracle-optimal sample complexity in personalized distributed learning without knowing the target accuracy.

desk verdict The dynamic-weight schema and cancellation identity are genuinely novel, but the optimality claim currently rests on a pointwise-vs-expectation gap in Lemma 2, plus an unproven stochastic estimator. read the letter →

arxiv 2507.06844 v1 pith:4C2Y56SP submitted 2025-07-09 stat.ML cs.LG

Constantin Philippenko , Batiste Le Bars , Kevin Scaman , Laurent Massoulié This is my paper
classification stat.MLcs.LG
keywords personalizedfederatedlearningdecentralizedoptimizationadaptivecollaborationgradientsimilarityvariancereductionsamplecomplexityheterogeneousclientsPolyak–Łojasiewiczcondition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that in online personalized distributed learning, each client can decide on the fly which other clients are worth collaborating with by comparing stochastic gradients, and that this adaptive choice can match the performance of an oracle that already knows the best collaboration. The paper studies the All-for-one weighted-averaging algorithm with time-varying weights and proves excess-loss bounds for smooth objectives that are strongly convex, satisfy the Polyak–Łojasiewicz condition, or are merely non-convex. The analysis identifies collaboration as a variance-reduction mechanism: the speed-up is controlled by a 'sufficient variance' that shrinks as the set of similar collaborators grows. The headline result is that a binary hard-threshold version of the criterion reaches the same sample complexity as the oracle scheme of Even et al. [8] up to constants, while removing the oracle's dependence on the target accuracy $\epsilon$ and on pre-known heterogeneity bounds. If correct, this means heterogeneous clients can dynamically cluster themselves and still get an optimal statistical efficiency, without knowing the target error in advance.

What carries the argument

The engine is the descent lemma for All-for-one with time-varying weights (Lemma 1), combined with the similarity ratio $r_{ik}^t = (1-\|\nabla R_k(\theta_i^{t-1})-\nabla R_i(\theta_i^{t-1})\|^2/\|\nabla R_i(\theta_i^{t-1})\|^2)_+$ and the weight rule $\alpha_{ik}^t = \phi(r_{ik}^t)(\sigma_{i,\psi}^t/\sigma_k)^2$, with $\sigma_{i,\psi}^t = (\sum_k \sigma_k^{-2}\psi(r_{ik}^t))^{-1/2}$ and $\psi(x)=x\phi(x)$. This particular form makes the bias terms cancel in the descent inequality, so only an effective variance $(\sigma_{i,\psi}^t)^2$ times the squared step size remains. The sufficient cluster $N_i^\star(\epsilon)$ and sufficient variance $\sigma_{i,\mathrm{suf}}^2(\epsilon)$ then convert the per-iteration bound into a target-independent sample-complexity statement by showing that every client in the sufficient cluster stays in the active collaboration set while the excess loss is at least $\epsilon$.

What would settle it

Run the binary collaboration scheme with the stochastic estimators of Eq. (1) on a synthetic two-cluster least-squares problem where the true similarity ratios are computable, and measure the number of iterations needed to reach excess loss $\epsilon$ against $\beta\sigma^2/(2\mu^2\epsilon N_i^\star(\epsilon))$; if the stochastic version fails to beat Local training or its iteration count does not scale with the sufficient-cluster size in the regime $N\gg b_\alpha$, the transfer of optimality to the practical algorithm is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that collaboration weights of the form $\alpha_{ik}^t = \phi(r_{ik}^t)(\sigma_{i,\psi}^t/\sigma_k)^2$ turn the All-for-one update into a variance-reduction method, where $r_{ik}^t$ is the similarity ratio between client $i$'s gradient and client $k$'s gradient and $\phi$ is any nondecreasing criterion function with $\phi(0)=0$ and $\phi(x)\le x$. These weights make the gradient-dissimilarity bias terms in the descent lemma collapse exactly to the local gradient norm, leaving an effective variance $(\sigma_{i,\psi}^t)^2$ in place of the full noise variance. Using an affine heterogeneity assumption $\|\nabla R_i(\theta)-\nabla R_k(\theta)\|^2\le b_{ik}^2+c_{ik}\|\nabla R_i(\theta)\|^2$, the paper defines a sufficient cluster $N_i^\star(\epsilon)$ and a sufficient variance $\sigma_{i,\mathrm{suf}}^2(\epsilon)$ that are fixed in time and control convergence to precision $\epsilon$. With the binary criterion $\phi_{\mathrm{bin}}(x)=\lambda\mathbf{1}_{x\ge\lambda}$, Corollary 1 gives the iteration bound $T_\epsilon^i=\beta\sigma_{i,\mathrm{suf}}^2(\epsilon)C/(2\mu^2\epsilon)$, which under uniform variance and $c_{ik}=0$ matches the lower bound of Even et al. [8] up to constants without oracle knowledge of the heterogeneity or of $\epsilon$.

Load-bearing premise

The sample-optimality theorem assumes that the collaboration weights are computed from exact gradients and exact per-client gradient variances; the algorithm actually run in experiments substitutes finite-batch stochastic estimates, and no theorem shows those estimates preserve the optimal sample complexity.

Editorial extensions

If this is right

  • Under the binary criterion, All-for-one attains the sample-optimal rate of Even et al. [8] up to constants, with no need to know the target accuracy $\epsilon$ or the heterogeneity bounds in advance.
  • The adaptive weights produce dynamic collaboration: as a client's excess loss decreases, clients whose gradients are no longer similar enough are automatically dropped, interpolating between broad collaboration early in training and local fine-tuning late in training.
  • The speed-up from collaboration is governed by the sufficient variance $\sigma_{i,\mathrm{suf}}^2(\epsilon)$, showing that collaboration acts as variance reduction rather than as simple cluster averaging.
  • The convergence guarantees cover strongly convex, PL, and smooth non-convex objectives, recovering classical SGD rates with the collaboration effect embedded in the variance term.
  • The continuous criterion minimizes the per-step descent upper bound and removes the extra threshold hyperparameter, although its worst-case sample-complexity bound is weaker than the binary version's.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the stochastic estimators $\hat{r}_{ik}^t$ from Eq. (1) are shown to concentrate around the true similarity ratios, the optimality guarantee would transfer to the implemented algorithm; proving that transfer is the natural next step the paper leaves open.
  • The binary criterion can be read as an online clustering rule whose clusters shrink monotonically with precision, so the same machinery could serve as an anomaly or malicious-client detector during federated training.
  • The sufficient variance $\sigma_{i,\mathrm{suf}}^2(\epsilon)$ quantifies each client's marginal contribution to another client's variance reduction, which could be used as a principled basis for incentive or reward allocation in collaborative learning systems.
  • A testable extension is to vary the frequency of weight updates (e.g., logarithmic schedule) and check whether the empirical speed-up degrades gracefully; the paper hints at this but does not test it systematically.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies online personalized decentralized learning with N heterogeneous clients, extending the All-for-one weighted averaging scheme of Even et al. (2022). It introduces a gradient-similarity-based collaboration criterion: for each client i, weights α_ik^t are defined from a similarity ratio r_ik^t = (1 − ||∇R_k(θ_i^{t−1})−∇R_i(θ_i^{t−1})||² / ||∇R_i(θ_i^{t−1})||²)_+ through a nondecreasing function φ, with variance normalization. The main theoretical results are a descent lemma (Lemma 1), a variance-reduction bound (Theorem 1) for strongly convex/PL objectives, definitions of a sufficient cluster N_i^*(ε) and sufficient variance σ_{i,suf}²(ε), upper bounds for constant, horizon-dependent and decreasing step sizes (Theorem 2), a non-convex bound (Theorem 3), and a sample-complexity corollary (Corollary 1) that recovers, up to constants and under c_ik=0 and uniform variance, the Even et al. lower bound. Two practical instantiations are proposed—a binary hard-threshold and a continuous soft-threshold criterion—and experiments on synthetic and real datasets compare them with FedAvg, Local, Ditto, Cobo, Wga-bc and APFL.

Significance. If the main theorems were fully rigorous, the paper would make a valuable contribution: the cancellation identity in Definition 1 is elegant, the notion of sufficient cluster gives intuitive, algorithm-dependent speed-up, and the claim that a binary adaptive schema attains the Even et al. lower bound without knowing ε or the cluster structure in advance is significant. The paper is also honest about the gap between the oracle weights used in the theory and the stochastic estimator used in practice, and it provides reproducible experimental code and detailed experimental settings. However, as detailed below, the proof of the central variance-reduction bound mixes pointwise and expectation quantities, so the optimality claim is not yet established even for the oracle version. The contribution is therefore promising but currently incomplete.

major comments (5)
  1. [Section 2, Appendix B.1 (Lemma 1)] The proof of Lemma 1 in Appendix B.1 writes "R_i(θ_t^i) − R_i(θ_{t−1}^i) = ε_t^i − ε_{t−1}^i", but Section 2 defines ε_t^i = E[R_i(θ_t^i) − R_i(θ_*^i)], so this identity is not true pointwise. The conditional inequality (S1)–(S2) therefore needs to be stated either for the random variable r_i^t := R_i(θ_t^i)−R_i(θ_*^i) with ε_t^i = E[r_i^t], or with a separate argument that the expectation can pass through the smoothness expansion. As written, the descent lemma that underpins all subsequent theorems is not justified.
  2. [Appendix B.2, Eq. (S2) and Theorem 1] In Eq. (S2) the right-hand side contains (σ_{i,eff}^t)^2 = (σ_{i,ψ}^t)^2, which depends on the random parameter θ_{t−1}^i through r_ik^t. Taking full expectations gives E[ε_t^i] ≤ (1−η_i μ) E[ε_{t−1}^i] + (η_i^2 β/2) E[(σ_{i,eff}^t)^2], not the display in Theorem 1 where (σ_{i,ψ}^t)^2 appears outside the expectation. Since all later uses of the sufficient variance replace σ_{i,eff}^t by σ_{i,suf}(ε), an expectation-level bound on the effective variance is needed; the current proof does not provide one.
  3. [Lemma 2 (Section 3)] The proof of Lemma 2 claims ∥∇R_i(θ_t^i)∥^2 ≥ 2μ ε_t^i from strong convexity or the PL inequality. Strong convexity/PL only implies the pointwise inequality ∥∇R_i(θ)∥^2 ≥ 2μ(R_i(θ)−R_i(θ_*^i)) for each realization, while ε_t^i is the expectation of R_i(θ_t^i)−R_i(θ_*^i); hence the pointwise gradient norm need not dominate 2μ ε_t^i. Consequently the inclusion N_*^i(ε) ⊂ N_i^t and the bound σ_{i,eff}^t ≤ σ_{i,suf}(ε) are not established for all trajectories. This gap affects Theorem 2, Corollary 1, and the claimed match to the Even et al. lower bound even in the exact-gradient oracle setting.
  4. [Section 4, Eq. (1) and Section 6, Limitations] The implemented algorithm replaces the similarity ratio r_ik^t by the stochastic estimator \hat r_ik^t from Eq. (1), but no convergence result is proved for these estimated weights. The optimality statement in Section 4 and Corollary 1 apply to the weights of Definition 1, which require exact gradients and exact variances; the manuscript acknowledges this in Section 6, but as a result the main practical claim—that the binary schema preserves All-for-one's optimality while removing dependence on ε—has no theoretical support for the algorithm actually evaluated in Section 5.
  5. [Corollary 1] Corollary 1 derives T_i^ε from the decreasing step-size bound in Theorem 2, but that bound is conditional on ε_{T-1}^i ≥ ε. The proof sets T_i^ε so that the upper bound equals ε and does not verify that ε_{T-1}^i ≥ ε holds all the way up to T_i^ε; a valid sample-complexity guarantee needs a high-probability or expectation argument showing the sufficient-cluster condition is in force before the target accuracy is reached. As stated, T_i^ε is not justified as a stopping time.
minor comments (5)
  1. [Lemma 2 (Section 3)] In the proof of Lemma 2, the line applying Assumption 3 writes ∥∇R_i(θ_t^i) − ∇R_k(θ_t^i)∥² ≤ b_{ik}² + c_{ik}∥∇R_i(θ_t^k)∥²; the last gradient should be evaluated at θ_t^i rather than θ_t^k.
  2. [Appendix B.2] In the proof of Theorem 3, the notation σ_{i,bin}(ε) is used in the step-size and in the final bound, whereas the theorem statement and Theorem 2 use σ_{i,suf}(ε); please make the notation consistent.
  3. [Table 1] For the horizon-dependent step size, the bound contains ln(2T μ² ε_0^i / σ_{i,suf}²(ε)), while the optimized step size contains ln(2T μ² ε_0^i / (β σ_{i,suf}²(ε))); please check whether the β in the logarithm has been intentionally absorbed into the constants.
  4. [Section 4, Eq. (1)] The batch size b_α used in Eq. (1) is not defined in the main text; it first appears in Table S1, and the notation m is used both for the number of samples in Eq. (1) and for the momentum in Table S1.
  5. [Section 4] The statement that the binary schema 'preserves the sampling optimality of All-for-one' should be qualified as 'up to constants and for the exact-gradient weights defined in Definition 1', since the comparison with Even et al. is asymptotic and the implemented estimator in Eq. (1) is not covered by the analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the adaptive weights cancel bias by construction, but the claimed optimality is an analytically derived variance bound matched to an external lower bound.

full rationale

The derivation chain is self-contained rather than circular. Lemma 1 is a standard smoothness-based descent inequality. Definition 1 intentionally chooses weights α_t_ik = φ(r_t_ik)(σ_{i,ψ}/σ_k)^2 with ψ = rφ, so that in the proof of Theorem 1 (Appendix B.2) the bias term Σ_k α(‖∇R_k−∇R_i‖² − ‖∇R_i‖²) cancels exactly to −‖∇R_i‖². This is a design choice that makes the analysis tractable, not a prediction equivalent to an input: the resulting effective variance (σ_{i,ψ})² is a function of the current gradient similarities, and Lemma 2 plus Definition 2 bound it by the sufficient variance σ²_{i,suf}(ε) using Assumption 3 and strong convexity/PL. Corollary 1 then converts the upper bound into T_i^ε = βσ²_{i,suf}(ε)C/(2μ²ε). No fitted parameter is renamed as a prediction: b_ik, c_ik, and σ_k are stated assumptions, and Remark 2 explicitly calls the sufficient cluster theoretical. The comparison with Even et al. [8] is an external benchmark, used only after the upper bound is derived. Although two current authors are coauthors of [8], the cited lower bound is a published, parameter-free mathematical statement whose assumptions do not include the present result, so it is independent evidence rather than load-bearing self-citation. The manuscript's own limitation statement (Section 4: estimating α_t_ik from Definition 1 is not possible without knowing true gradients, replaced by stochastic estimates with no convergence guarantee) is an acknowledged practical gap, not a circular reduction. Similarly, the possible expected-vs-pointwise issue in Lemma 2 is a correctness concern about the proof, not an identity between the claimed conclusion and an input assumption.

Assumptions & free parameters 4 free parameters · 8 assumptions · 0 invented entities

The central claim rests on standard smoothness, variance, and heterogeneity assumptions plus exact knowledge of gradients and variances for the theoretical weights. No free parameters are fitted to data in the theory; lambda, C, b_alpha, and update frequency are hand-chosen or scheduling constants. No new physical or model entities are introduced; the similarity ratio and sufficient cluster are analytical constructs defined from existing quantities.

free parameters (4)
  • lambda (binary threshold) = unspecified constant with 0 < lambda <= 1
    Hyperparameter in the binary collaboration function phi_bin(x) = lambda * 1{x >= lambda}; controls how similar gradients must be before collaboration. Chosen by hand; the continuous scheme removes it.
  • C (decreasing step-size constant) = any C > 1, minimized at C = 2
    Appears in Theorem 2 and Corollary 1; the sample complexity bound inherits a factor C^2/(C-1), minimized to 4 at C = 2.
  • b_alpha (stochastic batch size for weight estimation) = 1 (synthetic), 512 (MNIST), 16 (CIFAR-10), 1 (Heart), 8 (Ixi)
    Number of stochastic gradients used to estimate Z_t_ik and Z_t_i in Eq. (1); controls estimator noise but adds computational cost.
  • Inner iteration update frequency = 50 batches (MNIST/CIFAR-10), one local epoch (Heart/Ixi)
    Weights are refreshed every inner iterations rather than each iteration; the theory assumes per-iteration updates.
assumptions (8)
  • domain assumption Assumption 1: each R_i is beta-smooth.
    Used in Lemma 1 descent inequality and throughout Theorems 1 to 3.
  • domain assumption Assumption 2: stochastic gradient variance is uniformly bounded by sigma_i^2.
    Used to bound the variance term in Lemma 1 and to define weights via sigma_k.
  • domain assumption Assumption 3: pairwise gradient discrepancy satisfies norm squared <= b_ik^2 + c_ik * norm local gradient squared.
    Powers the sufficient-cluster results in Definition 2 and Lemma 2; generalizes the uniform bound c_ik = 0 of Even et al.
  • domain assumption Assumption 4 (strong convexity) or Assumption 5 (PL condition) for the linear-rate theorems.
    Needed to convert the gradient norm lower bound into excess loss contraction in Theorem 1, Theorem 2, and Lemma 2.
  • domain assumption Exact access to gradients and variances when computing alpha_t_ik (Definition 1).
    Theorems 1 to 3 and Corollary 1 are proven for these exact weights; the practical stochastic estimator in Eq. (1) is not covered.
  • standard math Stochastic gradients of different clients are independent given theta_i^{t-1}.
    Used in the Lemma 1 proof to decompose the variance of the weighted average of stochastic gradients.
  • standard math Existence of minimizers theta_i^* for each local loss.
    Excess loss epsilon_i^t is defined relative to theta_i^*.
  • standard math Theorem 4.7 of Bottou et al. for decreasing step-size SGD rate.
    Applied in the proof of Theorem 2 after upper-bounding the effective variance by the sufficient variance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive collaboration for online personalized distributed learning with heterogeneous clients." pith.science (2026). https://pith.science/paper/4C2Y56SP

@misc{pith2026250706844,
  author       = {Pith},
  title        = {Pith review of: Adaptive collaboration for online personalized distributed learning with heterogeneous clients},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4C2Y56SP}},
  note         = {Machine review of arXiv:2507.06844}
}
abstract

We study the problem of online personalized decentralized learning with $N$ statistically heterogeneous clients collaborating to accelerate local training. An important challenge in this setting is to select relevant collaborators to reduce gradient variance while mitigating the introduced bias. To tackle this, we introduce a gradient-based collaboration criterion, allowing each client to dynamically select peers with similar gradients during the optimization process. Our criterion is motivated by a refined and more general theoretical analysis of the All-for-one algorithm, proved to be optimal in Even et al. (2022) for an oracle collaboration scheme. We derive excess loss upper-bounds for smooth objective functions, being either strongly convex, non-convex, or satisfying the Polyak-Lojasiewicz condition; our analysis reveals that the algorithm acts as a variance reduction method where the speed-up depends on a sufficient variance. We put forward two collaboration methods instantiating the proposed general schema; and we show that one variant preserves the optimality of All-for-one. We validate our results with experiments on synthetic and real datasets.

Figures

Figures reproduced from arXiv: 2507.06844 by the authors.

Figure 1
Figure 1. Size of the sufficient cluster N ⋆ 1 (ε) us￾ing synthetic data (N = 20) and the binary col￾laboration function ϕbin defined in Section 4. The variance v of models (θ ⋆ i )i∈[N] is defined in Sec￾tion A: lower v means lower heterogeneity. Notice that the function σ 2 i,suf is decreasing and that it admits a limit as ε decreases. In￾deed, limε→0 σ 2 i,suf(ε) = (P k∈[N] σ 2 kψ((1 − cik)+)1bik=0) −1 – it corresponds to … view at source ↗
Figure 2
Figure 2. Test loss for synthetic dataset. We now evaluate the empirical per￾formance of the collaboration scheme from Section 4. Our code is provided on Github. Complete experiments’ settings are detailed in Section A. All experiments are performed without any tuning of the al￾gorithms (e.g. same set of hyperparameters for every algorithms), for three different seeds, without any pretraining. We use five datasets: one synthe… view at source ↗
Figure 3
Figure 3. Test loss for real dataset. X-axis: number of epochs. Y-axis: logarithm of the test loss. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 21 canonical work pages

  1. [1]

    M. G. Arivazhagan, V . Aggarwal, A. K. Singh, and S. Choudhary. Federated learning with personalization layers.arXiv preprint arXiv:1912.00818, 2019

    arXiv 1912

  2. [2]

    WAFFLE: Weighted Averaging for Personalized Federated Learning

    M. Beaussart, F. Grimberg, M.-A. Hartley, and M. Jaggi. Waffle: Weighted averaging for personalized federated learning.arXiv preprint arXiv:2110.06978, 2021

    work page Pith review arXiv 2021

  3. [3]

    Bottou, F

    L. Bottou, F. E. Curtis, and J. Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60(2):223–311, 2018

    2018

  4. [4]

    Capitaine, E

    A. Capitaine, E. Boursier, A. Scheid, E. Moulines, M. Jordan, E.-M. El-Mhamdi, and A. Durmus. Unravelling in collaborative learning.Advances in Neural Information Processing Systems, 37: 97231–97260, 2024

    work page 2024

  5. [5]

    E. M. Chayti, S. P. Karimireddy, S. U. Stich, N. Flammarion, and M. Jaggi. Linear speedup in personalized collaborative learning.arXiv preprint arXiv:2111.05968, 2021

    arXiv 2021

  6. [6]

    Collins, H

    L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai. Exploiting shared representations for personalized federated learning. In M. Meila and T. Zhang, editors,Proceedings of the 38th International Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Research, pages 2089–2099. PMLR, 18–24 Jul 2021. URL https://proceedings. mlr.pre...

    work page 2021

  7. [7]

    Y . Deng, M. M. Kamani, and M. Mahdavi. Adaptive personalized federated learning.arXiv preprint arXiv:2003.13461, 2020. 10

    arXiv 2003

  8. [8]

    M. Even, L. Massoulié, and K. Scaman. On sample optimality in personalized collaborative and federated learning.Advances in Neural Information Processing Systems, 35:212–225, 2022

    work page 2022

Show all 37 references
  1. [9]

    Fallah, A

    A. Fallah, A. Mokhtari, and A. Ozdaglar. Personalized federated learning with theoretical guar- antees: A model-agnostic meta-learning approach.Advances in neural information processing systems, 33:3557–3568, 2020

    2020

  2. [10]

    Ghosh, J

    A. Ghosh, J. Chung, D. Yin, and K. Ramchandran. An efficient framework for clustered federated learning.Advances in neural information processing systems, 33:19586–19597, 2020

    2020

  3. [11]

    Hanzely, S

    F. Hanzely, S. Hanzely, S. Horváth, and P. Richtárik. Lower bounds and optimal algorithms for personalized federated learning.Advances in Neural Information Processing Systems, 33: 2304–2315, 2020

    2020

  4. [12]

    Hashemi, L

    D. Hashemi, L. He, and M. Jaggi. Cobo: Collaborative learning via bilevel optimization.arXiv preprint arXiv:2409.05539, 2024

    2024 arXiv

  5. [13]

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh. Scaffold: Stochastic controlled averaging for federated learning. InInternational conference on machine learning, pages 5132–5143. PMLR, 2020

    2020

  6. [14]

    S. P. Karimireddy, W. Guo, and M. I. Jordan. Mechanisms that incentivize data sharing in federated learning.arXiv preprint arXiv:2207.04557, 2022

    2022 arXiv

  7. [15]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  8. [16]

    Koneˇcný, H

    J. Koneˇcný, H. B. McMahan, F. X. Yu, P. Richtarik, A. T. Suresh, and D. Bacon. Federated Learning: Strategies for Improving Communication Efficiency. InNIPS Workshop on Private Multi-Party Machine Learning, 2016

    2016

  9. [17]

    Krizhevsky, G

    A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  10. [18]

    Kulkarni, M

    V . Kulkarni, M. Kulkarni, and A. Pant. Survey of personalization techniques for federated learning. In2020 fourth world conference on smart trends in systems, security and sustainability (WorldS4), pages 794–797. IEEE, 2020

    2020

  11. [19]

    Lecun, L

    Y . Lecun, L. Bottou, Y . Bengio, and P. Haffner. Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, Nov. 1998. ISSN 1558-2256. doi: 10.1109/5.726791. Conference Name: Proceedings of the IEEE

    1998 doi

  12. [20]

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith. Federated optimization in heterogeneous networks.Proceedings of Machine learning and systems, 2:429–450, 2020

    2020

  13. [21]

    T. Li, S. Hu, A. Beirami, and V . Smith. Ditto: Fair and robust federated learning through personalization. InInternational conference on machine learning, pages 6357–6368. PMLR, 2021

    2021

  14. [22]

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang. On the convergence of fedavg on non-iid data.arXiv preprint arXiv:1907.02189, 2019

    1907 arXiv

  15. [23]

    P. P. Liang, T. Liu, L. Ziyin, N. B. Allen, R. P. Auerbach, D. Brent, R. Salakhutdinov, and L.-P. Morency. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523, 2020

    2001 arXiv

  16. [24]

    J. Liu, J. Wu, J. Chen, M. Hu, Y . Zhou, and D. Wu. Feddwa: Personalized federated learning with dynamic weight adjustment.arXiv preprint arXiv:2305.06124, 2023

    2023 arXiv

  17. [25]

    Z. Liu, Y . Zhang, P. Li, Y . Liu, and D. Yang. Dynamic llm-agent network: An llm-agent collaboration framework with agent team optimization.arXiv preprint arXiv:2310.02170, 2023

    2023 arXiv

  18. [26]

    Mansour, M

    Y . Mansour, M. Mohri, J. Ro, and A. T. Suresh. Three approaches for personalization with applications to federated learning.arXiv preprint arXiv:2002.10619, 2020. 11

    2002 arXiv

  19. [27]

    Marfoq, G

    O. Marfoq, G. Neglia, A. Bellet, L. Kameni, and R. Vidal. Federated multi-task learning under a mixture of distributions.Advances in Neural Information Processing Systems, 34:15434–15447, 2021

    2021

  20. [28]

    McMahan, E

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. InArtificial Intelligence and Statistics, pages 1273–1282. PMLR, Apr. 2017. ISSN: 2640-3498

    2017

  21. [29]

    Muhammad, Q

    K. Muhammad, Q. Wang, D. O’Reilly-Morgan, E. Tragos, B. Smyth, N. Hurley, J. Geraci, and A. Lawlor. Fedfast: Going beyond average for faster training of federated recommender systems. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data m...

    2020

  22. [30]

    Ogier du Terrail, S.-S

    J. Ogier du Terrail, S.-S. Ayed, E. Cyffers, F. Grimberg, C. He, R. Loeb, P. Mangold, T. Marchand, O. Marfoq, E. Mushtaq, B. Muzellec, C. Philippenko, S. Silva, M. Tele ´nczuk, S. Albarqouni, S. Avestimehr, A. Bellet, A. Dieuleveut, M. Jaggi, S. P. Karimireddy, M. Lorenzi, G. ...

    2022

  23. [31]

    Philippenko, K

    C. Philippenko, K. Scaman, and L. Massoulié. In-depth analysis of low-rank matrix factorisation in a federated setting. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 19904–19912, 2025

    2025

  24. [32]

    Robbins and S

    H. Robbins and S. Monro. A stochastic approximation method.The annals of mathematical statistics, pages 400–407, 1951

    1951

  25. [33]

    Sattler, K.-R

    F. Sattler, K.-R. Müller, and W. Samek. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints.IEEE transactions on neural networks and learning systems, 32(8):3710–3722, 2020

    2020

  26. [34]

    T Dinh, N

    C. T Dinh, N. Tran, and J. Nguyen. Personalized federated learning with moreau envelopes. Advances in neural information processing systems, 33:21394–21405, 2020

    2020

  27. [35]

    A. Z. Tan, H. Yu, L. Cui, and Q. Yang. Towards personalized federated learning.IEEE transactions on neural networks and learning systems, 34(12):9587–9603, 2022

    2022

  28. [36]

    Werner, L

    M. Werner, L. He, M. Jordan, M. Jaggi, and S. P. Karimireddy. Provably personalized and robust federated learning.Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URLhttps://openreview.net/forum?id=B0uBSSUy0G

    2023

  29. [37]

    T. Yu, E. Bagdasaryan, and V . Shmatikov. Salvaging federated learning by local adaptation. arXiv preprint arXiv:2002.04758, 2020. 12 Supplementary material In this appendix, we provide additional information to supplement our work. In Section A, we provide complete details on...

    2002 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.