Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity

Mehdi Bennis; Yiwei Li; Zhaoyang Zhang; Zhuojun Tian

REVIEW 4 major objections 4 minor 43 references

Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that heterogeneous subnetworks inside one randomly initialized network can fit each agent's local data, so decentralized agents need exchange only binary masks.

desk verdict The MCE-PL algorithm is a plausible and useful extension of mask-based personalized FL, but the proof of its central DSLTH theorem contains a real inequality error that leaves the heterogeneity claim unproven. read the letter →

arxiv 2504.17520 v1 pith:QFYLEO7E submitted 2025-04-24 cs.LG cs.DCcs.MA

Zhuojun Tian , Zhaoyang Zhang , Yiwei Li , Mehdi Bennis This is my paper · ORCID

classification cs.LGcs.DCcs.MA

keywords distributedstronglotterytickethypothesispersonalizedlearningdataheterogeneitynodebinarymaskpruningcommunicationefficiencydecentralizedstructuredsparsity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a distributed strong lottery ticket hypothesis: in a network of agents with different data distributions and different computational capacities, one randomly initialized over-parameterized neural network contains a separate sparse subnetwork for each agent that fits that agent's local data well. If the hypothesis holds, every agent can keep the same global real-valued weights $w$ fixed and personalize only by learning a binary mask $m_i$, with the local model written as the Hadamard product $v_i = w \odot m_i$. The authors build MCE-PL on this foundation: agents update real-valued mask tensors by backpropagation, apply group-sparsity regularization and a filter pruning rule to obtain structured binary masks, aggregate neighboring masks through an intermediate aggregation tensor, and add a personalized fine-tuning step. On CIFAR-10 with AlexNet, the method matches or beats baseline algorithms (decentralized SGD, FedProx, LotteryFL, Dis-PFL) while transmitting far fewer bits. The theoretical proof of the hypothesis is narrower than the general claim, covering only pairs of nodes and a restricted convolutional architecture under assumptions about the trained models.

What carries the argument

The load-bearing object is the shared random over-parameterized CNN $g$ with fixed weights $w$, together with the mask parameterization $v_i = w \odot m_i$, where $\odot$ is the Hadamard product and $m_i$ is a per-agent binary tensor. Because binary tensors are not directly differentiable, each agent maintains a real-valued mask tensor $z_i$, updates it with a sign-based gradient approximation, and thresholds it layerwise by the agent's retention ratio $r_i$ to produce $m_i$. Structured sparsity comes from a group-Lasso penalty on output filters plus an extra rule that zeros any filter with too few nonzero entries. The aggregation mechanism is an intermediate tensor $y_i = z_i + \mathrm{mean}[\mathrm{abs}(\tilde{z}_i)]\, \mathrm{sign}(\tilde{z}_i) \odot \frac{1}{|\mathcal{N}_i|}\sum_{j\in\mathcal{N}_i} m_j$, which injects neighbor mask information into the local real-valued tensor at an adaptively chosen amplitude; Theorem 1 is obtained by instantiating Lemma 1, the SLTH for CNNs, separately for each node and combining the approximation and triangle-inequality bounds.

What would settle it

Train two AlexNet agents on disjoint CIFAR-10 label subsets, fix a random shared initialization $w$, and run MCE-PL's mask search at retention ratios 0.1, 0.3, and 0.5; also record the trained weights' layerwise $\ell^1$ norms and the sup-norm output distance between $f_1$ and $f_2$ on a shared test set. If, for any agent, no binary mask reaches the fully-trained accuracy within a chosen tolerance, or if $\|v_l\|_1 > 1$ or the output distance violates the assumed $\alpha$-bounds, then the DSLTH as stated is not supported in that setting.

Watch

Extended reading notes

Core claim

The central claim, DSLTH, is stated informally in Section III-A and proved as Theorem 1 for an arbitrary pair of independently trained local models $f_1$ and $f_2$. Theorem 1 shows that with probability at least $(1-\delta)^2$ there exist binary mask tensors $m_1$ and $m_2$ such that the pruned versions of one random $2L$-layer CNN $g$ approximate $f_1$ and $f_2$ with accuracy $\varepsilon_1$ and $\varepsilon_2$, respectively; moreover, the two pruned networks' outputs are correlated (distance at most $\varepsilon_1 + \varepsilon_2 + \alpha_u$) yet still heterogeneous (distance at least $\min\{|\varepsilon_1 + \varepsilon_2 - \alpha_l|, |\varepsilon_1 + \varepsilon_2 - \alpha_u|, |\alpha_l|\}$). The quantities $\alpha_u$ and $\alpha_l$ come from Assumption 3, which bounds the output distance of the two trained models. The authors read these bounds as establishing that personalization can be achieved purely by choosing masks in one shared random network, with inter-agent similarity and local distinctiveness both guaranteed. They also state explicitly that the theorem does not cover system heterogeneity and applies only to a restricted CNN form without pooling, stride, or bias.

Load-bearing premise

The proof's load-bearing premise is that the two fully trained personal models $f_1$ and $f_2$ lie in the restricted class $\mathcal{F}$ with $\|v_l\|_1 \le 1$ and that their output distance is bounded by $\alpha_u$ and $\alpha_l$ (Assumptions 2 and 3); no evidence is given that realistic trained CNNs satisfy these, so if they fail, Theorem 1 — and with it the theoretical foundation for MCE-PL — does not apply.

Editorial extensions

If this is right

Communication per round drops to one bit per retained parameter: agents exchange binary masks only, not 32-bit floats, so total cost is set by the sparsity ratios $r_i$ and the graph degree.
Node heterogeneity is expressed as differing retention ratios $r_i$, so each agent's model size scales with its own capacity instead of forcing all agents to a common architecture.
The aggregation-tensor step avoids the intersection-fusion failure mode in which the shared model collapses toward the most pruned agent, preserving the diversity of retained filters across agents.
Personalized fine-tuning each iteration steers the aggregated mask toward the local data distribution, which the experiments link to faster convergence and higher accuracy than independent mask training.
Because Theorem 1 gives both an upper and a lower bound on the distance between pruned subnetworks, it provides quantitative guarantees that the learned personalized models remain correlated yet distinct.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The pairwise proof structure suggests a natural extension to arbitrary graphs: if each edge is treated as a pair $(i,j)$, the correlation bound (25) is exactly the quantity an averaging-based consensus analysis would need, so DSLTH may be convertible into a convergence guarantee for mask aggregation.
The weakest spot is measurable: for a trained AlexNet on CIFAR-10 one can compute the layerwise $\ell^1$ norms $\|v_l\|_1$ and the output-distance bounds $\alpha_u, \alpha_l$; if these contradict Assumptions 2 and 3, Theorem 1's hypotheses fail in the paper's own experimental setting even if the empirical DSLTH holds.
Mask-based updates use only the sign of $z_i$ for gradient approximation; one could test whether including gradient magnitude or a straight-through estimator with temperature changes mask quality and convergence, which the paper does not explore.
If the hypothesis extends to heterogeneous depths and dynamically estimated retention ratios, the framework could directly address the open problems the paper names: agents with different network depths and unknown capacity constraints.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know about this paper is that the algorithm is worth a look, but the theory does not hold up as written. MCE-PL fixes a shared random network and has each agent learn only a personalized binary mask, communicating masks rather than weights. That is a legitimate extension of FedMask, LotteryFL, and Dis-PFL, and the aggregation tensor plus per-iteration fine-tuning is a sensible response to the node-heterogeneity problem. The experiments cover data and node heterogeneity, compare against sensible baselines, and show real communication savings; the authors also ship code. Credit where due: the empirical story is the strongest part of the paper.

The soft spot is Theorem 1. The heterogeneity lower bound in Eq. (26) does not follow from the stated inequalities. From the reverse triangle inequality you get inf ||gm1 - gm2|| >= max(0, alpha_l - eps1 - eps2), not the min-of-absolute-values expression printed. With alpha_l = 1, eps1 = eps2 = 1, the paper claims a lower bound of 1 while the actual bound is 0. This is not a small typo; it is the only step in the theorem that establishes that the two pruned subnetworks are actually different. Without it, DSLTH reduces to two independent applications of the SLTH plus an upper bound on their distance, and the name 'distributed strong lottery ticket hypothesis' overclaims what is proven. Assumptions 2 and 3 are also asserted for trained CNNs without evidence, so even the per-node approximation part rests on shaky ground.

The experiments are useful but not bulletproof: learning rates are selected using test accuracy, the Fil threshold is unspecified, and results appear to be single runs without variance or multiple seeds. These are fixable issues, but they make the empirical claims less solid than the tables suggest.

My take: the paper deserves a serious referee, but not acceptance in its current form. The algorithm is plausible and the empirical direction is valuable. The authors should either repair or remove the flawed lower bound—if the theory stays, it needs a correct proof, and if it goes, the paper can still stand as an empirical methods contribution. They also need to tighten the experiments. I would send it to review with the expectation of major revision, and I would not cite the DSLTH theorem until it is fixed.

Referee Report

4 major / 4 minor

Summary. The paper proposes MCE-PL, a decentralized learning method in which agents keep a shared randomly initialized weight tensor fixed and communicate only personalized binary pruning masks. A real-valued mask tensor is updated by backpropagation, thresholded to a binary mask, and aggregated through an intermediate aggregation tensor with a personalized fine-tuning step. The authors state a distributed strong lottery ticket hypothesis (DSLTH) and claim a proof under a restricted CNN setting (Theorem 1), based on applying a centralized SLTH lemma to each of two agents and then bounding the correlation and heterogeneity between the resulting pruned subnetworks via Assumptions 2 and 3. Experiments on CIFAR-10 with AlexNet compare accuracy and communication cost under data and node heterogeneity with several baselines.

Significance. The paper addresses a real and timely problem, personalized decentralized learning under both statistical and system heterogeneity, and the idea of transmitting only binary masks is well motivated and clearly described. The authors provide code, compare against multiple baselines, and are transparent about some limitations, including the fact that Theorem 1 does not cover system heterogeneity. If DSLTH could be established, it would be a valuable distributed analogue of SLTH, and the intermediate-aggregation-tensor design is a sensible engineering contribution. However, the theoretical result as stated is not valid: the assumptions are mutually inconsistent for the stated no-bias CNN class, the derivation of Eq. (26) in the proof is incorrect, and the theorem is an existence statement not connected to the actual MCE-PL updates. The empirical study may support a systems-oriented paper, but not the theoretical claims made here.

major comments (4)

[Section IV, Assumptions 2 and 3] Assumptions 2 and 3 are mutually inconsistent for the function class F defined in Section IV. The networks in F are defined in (20) as no-bias convolutional ReLU networks, so for the zero input 0 in the domain [0,1]^{D x D x O0}, every f in F satisfies f(0)=0. Hence for any f1,f2 in F, inf_{x in X} ||f1(x)-f2(x)||_max = 0. Assumption 3 can therefore hold only with alpha_l = 0; with alpha_l = 0 the right-hand side of (26) collapses to 0, so Theorem 1 cannot establish any positive heterogeneity between the pruned subnetworks. The proof's use of a positive alpha_l is not justified and contradicts Assumption 2.
[Appendix A, Eq. (26)] Even if a positive alpha_l were granted, the displayed lower bound in (26) does not follow from the stated inequalities. From sup ||(f1-f2)-(gm1-gm2)|| <= epsilon1+epsilon2 and inf ||f1-f2|| >= alpha_l, the reverse triangle inequality gives pointwise ||gm1-gm2|| >= ||f1-f2|| - ||(f1-f2)-(gm1-gm2)||, and hence inf ||gm1-gm2|| >= max(0, alpha_l - epsilon1 - epsilon2). The expression min{|epsilon1+epsilon2-alpha_l|, |epsilon1+epsilon2-alpha_u|, |alpha_l|} is not implied. For example, with alpha_l=0.1, alpha_u=1, and epsilon1+epsilon2=0.15, the paper's bound is 0.05 while the valid bound is 0. Since the heterogeneity half of DSLTH rests entirely on (26), this gap invalidates the proof as written.
[Sections III-C and IV, Theorem 1 versus MCE-PL] Theorem 1 is an existence result and does not analyze the MCE-PL algorithm. The theorem guarantees, via the external SLTH lemma, that some binary masks approximate f1 and f2; it says nothing about whether the specific gradient-based update (14), the aggregation rules (16)-(19), or the personalized fine-tuning step (17) converge to such masks or preserve their guarantees. The claim that Theorem 1 establishes the theoretical foundation of MCE-PL therefore overstates what is proved. The authors acknowledge that node heterogeneity is not covered by the theorem, but the title and abstract nevertheless present a joint theory for data and node heterogeneity.
[Section IV, Assumption 2] Assumption 2 is not justified for the trained models considered in this work. Requiring f1,f2 in F means that every convolution weight tensor has entries in [-1,1] and ||v_l||_1 <= 1; trained AlexNet models in the experiments are not shown to satisfy this, and scaling weights into F changes the function because the network is nonlinear. The sentence 'Assumption 2 can be satisfied through proper initialization and training process' is asserted without a construction or proof, so Lemma 1 cannot be invoked for the models of interest.

minor comments (4)

[Section II-B, Eq. (4)] The notation nabla f_i / nabla v_i is nonstandard and dimensionally unclear; gradients should be written as gradients with respect to the relevant tensors, and the replacement of nabla m_i / nabla z_i by sign(z_i) mixes tensor calculus with a heuristic approximation that should be explained more carefully.
[Section V-A] The sentence 'the analytical results in IV cannot be practically applied here' is an honest caveat, but it also means the experiments do not test the assumptions of Theorem 1; this should be stated more prominently in the main text, not only in the simulation section.
[Throughout] There are several typographical and reference issues: 'We defer to future works' should be 'We refer to future work', and reference [14] appears to duplicate [13] with a different first author.
[Section V-B, Tables I and II] The numerical comparisons are reported without standard deviations or multiple seeds; given the small differences between some methods, the conclusions would be more robust if the experiments reported variability across runs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DSLTH rests on the external SLTH lemma of da Cunha et al.; assumptions are explicit and self-citations are not load-bearing.

full rationale

The paper's central claim, DSLTH, is not produced by fitting or by a self-citation chain. Theorem 1 in Section IV is an application of Lemma 1, which is taken from the external SLTH result of da Cunha, Natale and Viennot [41]. The local approximation bounds (23) and (24) are exactly Lemma 1's guarantee applied to the two assumed target networks f1 and f2. The correlation bound (25) follows by the triangle inequality, and the heterogeneity bound (26) is intended to follow from Assumption 3's alpha_l via the reverse triangle inequality. No parameter is fitted on the data whose performance is then presented as a prediction, and no result is defined in terms of its own conclusion. Assumptions 2 and 3 are explicit modeling assumptions about the fully trained personalized models; they are inputs, not outputs, of the theorem. The self-citations in the paper ([2], [3], [16]) are background related-work references and are not used to justify the DSLTH proof, the uniqueness of any construction, or the correctness of the MCE-PL updates. The proof does not analyze the convergence of the specific MCE-PL update rules, but that is a scope limitation rather than a circular reduction. A separate mathematical concern exists: Eq. (26) does not appear to follow from the reverse triangle inequality, which would give max(0, alpha_l - epsilon1 - epsilon2) rather than the stated min expression. However, an invalid or questionable inequality is a correctness issue, not a circularity issue, because the conclusion is still being derived from, rather than identified with, the assumptions. No circular step is therefore identified.

Assumptions & free parameters 4 free parameters · 8 assumptions · 2 invented entities

The method rests on five classes of assumptions: the external SLTH lemma, restrictive conditions on the trained models, the straight-through gradient approximation, the heuristic aggregation model, and the fixed-graph synchronous setup. Free parameters are dominated by learning-rate selection against test accuracy and the unspecified Fil threshold, which weaken the empirical claims.

free parameters (4)

Learning rate eta for mask update = 1.0 (MCE-PL, ind-mask), 0.1 (LotteryDSGD), 0.01 (DSGD, FL, DSGDProx)
Chosen as the value in [1.0, 0.1, 0.01, 0.001] with best test accuracy in Section V-B; this selects hyperparameters on test performance and may inflate reported accuracy.
Group-sparsity coefficient lambda = 0.001
Fixed in all experiments without sensitivity analysis; it controls the strength of the regularization term R(z_i) in eq (7) and therefore the structure of the learned masks.
Model retention ratios r_i = Random draw from {0.1, 0.2, 0.3, 0.4} per agent
Per-agent system-capacity inputs; in the node-heterogeneity experiment they are randomly generated, not optimized, and directly set the threshold T_a in eq (3).
Fil filter threshold = Not specified in the paper
The operation Fil[...] in eqs (10), (12), (16b), and (19) zeroes entire filters whose non-zero count is below a threshold, but the threshold value is never defined, leaving mask construction under-specified.

assumptions (8)

domain assumption Lemma 1 (SLTH for CNNs, Theorem 1 of [41]): an overparameterized random CNN contains a pruned subnetwork approximating any target network in F to accuracy epsilon with probability 1-delta.
External theorem used as the starting point of Theorem 1; the paper does not verify its width requirements in the simulations.
ad hoc to paper Assumption 2: the trained local models f1 and f2 both lie in F, meaning every convolution weight tensor satisfies ||v_l||_1 <= 1.
Restricts the theorem to a function class not shown to contain realistically trained CNNs; the paper only asserts proper initialization and training can satisfy it.
ad hoc to paper Assumption 3: there exist constants alpha_u and alpha_l such that sup_x ||f1(x)-f2(x)||_max <= alpha_u and inf_x ||f1(x)-f2(x)||_max >= alpha_l.
Imposes both an upper and positive lower bound on model disagreement; the lower bound is needed for the heterogeneity statement (26) but no data-dependent construction is provided.
domain assumption The two local models are sufficiently and independently trained from the same initialization f0 (Section IV, opening paragraph).
Defines the approximation targets for DSLTH; no training loss, number of steps, or convergence condition is specified.
domain assumption Gradient approximation del(m_i)/del(z_i) approximately equals sign(z_i) in eqs (4) and (14).
Straight-through estimator borrowed from binary neural networks; MCE-PL's mask updates depend on this approximation and no error control is given.
ad hoc to paper The shared random network w simultaneously contains good subnetworks for all agents (the DSLTH premise itself, Section III-A).
The algorithm fixes w and learns only masks; if no single random w contains good masks for every local task, MCE-PL cannot succeed. The theorem only supports this for the restricted class F.
domain assumption Aggregation amplitude is modeled as mean of abs(z_i,l) in eq (11), with neighboring binary masks averaged before injection.
A heuristic because the explicit amplitude could not be accurately computed; the paper provides no convergence or bias analysis for this aggregation rule.
domain assumption The communication graph is connected, static, and updates are synchronous (Section II-A).
Simplifies the algorithm and experiments; excludes time-varying topologies, async delays, and node churn common in IoT.

invented entities (2)

Aggregation tensor y_i
purpose: Intermediate real-valued tensor that mixes the local mask tensor z_i with averaged neighbor masks before thresholding (eqs 11, 16a, 18).
Novel algorithmic state introduced here; its benefits appear only in the paper's own experiments and are not supported by a convergence theorem.
DSLTH as a named distributed hypothesis
purpose: Justifies fixing global weights and updating only binary masks, claiming one random shared network contains good subnetworks for all heterogeneous agents.
Despite the name, Theorem 1 reduces to the centralized SLTH applied node-by-node plus a triangle inequality; no independent falsifiable prediction about distributed systems is made outside the paper's own simulations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity." pith.science (2026). https://pith.science/paper/QFYLEO7E

@misc{pith2026250417520,
  author       = {Pith},
  title        = {Pith review of: Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QFYLEO7E}},
  note         = {Machine review of arXiv:2504.17520}
}

read the original abstract

To jointly tackle the challenges of data and node heterogeneity in decentralized learning, we propose a distributed strong lottery ticket hypothesis (DSLTH), based on which a communication-efficient personalized learning algorithm is developed. In the proposed method, each local model is represented as the Hadamard product of global real-valued parameters and a personalized binary mask for pruning. The local model is learned by updating and fusing the personalized binary masks while the real-valued parameters are fixed among different agents. To further reduce the complexity of hardware implementation, we incorporate a group sparse regularization term in the loss function, enabling the learned local model to achieve structured sparsity. Then, a binary mask aggregation algorithm is designed by introducing an intermediate aggregation tensor and adding a personalized fine-tuning step in each iteration, which constrains model updates towards the local data distribution. The proposed method effectively leverages the relativity among agents while meeting personalized requirements in heterogeneous node conditions. We also provide a theoretical proof for the DSLTH, establishing it as the foundation of the proposed method. Numerical simulations confirm the validity of the DSLTH and demonstrate the effectiveness of the proposed algorithm.

Figures

Figures reproduced from arXiv: 2504.17520 by the authors.

**Figure 2.** Illustration of the lottery ticket hypothesis, stro [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The algorithm flowchart of the proposed MCE-PL in the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: The verification of the DSLTH: comparison of converge [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: The comparison of communication cost. C. Comparison on Convergence and Communication Cost in Node Heterogeneity All of the above baseline methods cannot be applied in node heterogeneity settings. To further show the superiority of the proposed method, we additionally c…

**Figure 6.** Figure 6: The convergence curves of different algorithms. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: The comparison of communication cost. denser model than the intended sparsity. Furthermore, MCEPL achieves higher accuracy at the same communication cost, demonstrating its communication efficiency. D. Comparison of Learned Model Then given the above simulation settin…

**Figure 8.** Figure 8: The learned personalized models in different agents [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

**Figure 9.** Figure 9: The learned personalized models in different agents [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 40 canonical work pages

[1]

Differential ly private federated clustering over non-iid data,

Y . Li, S. Wang, C.-Y . Chi, and T. Q. S. Quek, “Differential ly private federated clustering over non-iid data,” IEEE Internet of Things Journal, vol. 11, no. 4, pp. 6705—6721, 2024

work page 2024
[2]

Distributed Multi-View Sparse V ector Recovery,

Z. Tian, Z. Zhang, and L. Hanzo, “Distributed Multi-View Sparse V ector Recovery,” IEEE Trans. Signal Process. , vol. 71, pp. 1448–1463, 2023

work page 2023
[3]

Distributed Learning Over Networks With Graph-Attention-Based Person al- ization,

Z. Tian, Z. Zhang, Z. Yang, R. Jin and H. Dai, “Distributed Learning Over Networks With Graph-Attention-Based Person al- ization,” IEEE Trans. Signal Process. , vol. 71, pp. 2071–2086, 2023

work page 2023
[4]

Can decentralized algorithms outperform centrali zed algorithms? A case study for decentralized parallel stocha stic gradient descent,

X. Lian, C. Zhang, H. Zhang, C. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centrali zed algorithms? A case study for decentralized parallel stocha stic gradient descent,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 5330–5340, 2017

work page 2017
[5]

Personalize d federated learning with theoretical guarantees: A model-a gnostic meta-learning approach,

A. Fallah, A. Mokhtari, and A. E. Ozdaglar, “Personalize d federated learning with theoretical guarantees: A model-a gnostic meta-learning approach,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS), pp. 3557–3568, 2020

work page 2020
[6]

Cluster-driven graph federated learning ov er multiple domains

D. Caldarola, M. Mancini, F. Galasso, M. Ciccone, E. Rodo la, and B. Caputo, “Cluster-driven graph federated learning ov er multiple domains.” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) W orkshops, pp. 2743–2752, 2021

work page 2021
[7]

Federated optimization in heterogeneous networ ks,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networ ks,” in Proc. Machine learning and systems , vol. 2, pp. 429–450, 2020

work page 2020
[8]

An efﬁcie nt framework for clustered federated learning,

A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efﬁcie nt framework for clustered federated learning,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 19586–19597, 2020

work page 2020

Show all 43 references

[9]

Personalized federat ed learning with moreau envelopes,

C. T. Dinh, N. Tran, and J. Nguyen, “Personalized federat ed learning with moreau envelopes,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 21394–21405, 2020

2020
[10]

Group knowledge transfer: Federated learning of large cnns at the edge,

C. He, M. Annavaram and S. Avestimehr, “Group knowledge transfer: Federated learning of large cnns at the edge,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 14068–14080, 2020

2020
[11]

Distributed learning of deep ne ural network over multiple agents,

O. Gupta and R. Raskar, “Distributed learning of deep ne ural network over multiple agents,” in Jour . Network and Computer Applications, vol. 116, pp. 1–8, 2018

2018
[12]

Heteroﬂ: Computation a nd communication efﬁcient federated learning for heterogene ous clients,

E. Diao, J. Ding, and V . Tarokh, “Heteroﬂ: Computation a nd communication efﬁcient federated learning for heterogene ous clients,” arXiv: 2010.01264 , 2020

2010 arXiv
[13]

Fjord: Fair and accurate federated lear ning under heterogeneous targets with ordered dropout,

S. Horvath, S. Laskaridis, M. Almeida, I. Leontiadis, S . V e- nieris and N. Lane, “Fjord: Fair and accurate federated lear ning under heterogeneous targets with ordered dropout,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 12876–12889, 2021

2021
[14]

Fjord: Fair and accurate federated learning under heterog eneous targets with ordered dropout,

Y . Deng, W. Chen, J. Ren, F. Lyu, Y . Liu, Y . Liu and Y . Zhao, “Fjord: Fair and accurate federated learning under heterog eneous targets with ordered dropout,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 12876–12889, 2021

2021
[15]

Lotteryﬂ: Personalized and communication-efﬁcient fede rated learning with lottery ticket hypothesis on non-iid dataset s,

A. Li, J. Sun, B. Wang, L. Duan, S. Li, Y . Chen, and H. Li, “Lotteryﬂ: Personalized and communication-efﬁcient fede rated learning with lottery ticket hypothesis on non-iid dataset s,” arXiv: 2008.03371, 2020

2008 arXiv
[16]

S. Seo, S. W. Ko, J. Park, S. L. Kim, and M. Ben- nis, ”Communication-efﬁcient and personalized federated lottery ticket learning,” in 2021 IEEE 22nd Int. W orkshop Signal Pro- cess. Advances in Wireless Communications (SPAWC) , pp. 581- 585, Sep. 2021

2021
[17]

Model pruning enables efﬁcient federate d learning on edge devices,

Y . Jiang, S. Wang, V . V alls, B. J. Ko, W. H. Lee, K. K. Leung , and L. Tassiulas, “Model pruning enables efﬁcient federate d learning on edge devices,” in IEEE Trans. Neural Networks and Learning Systems , vol. 34, no. 12, pp. 10374–10386, 2022

2022
[18]

Fedhm: Efﬁcient federated learning for heterogene ous models via low-rank factorization,

D. Yao, W. Pan, M. J. O’Neill, Y . Dai, Y . Wan, H. Jin, and L. Sun, “Fedhm: Efﬁcient federated learning for heterogene ous models via low-rank factorization,” arXiv: 2111.14655 , 2021

2021 arXiv
[19]

Federated learning of large models at the edge via principa l sub-model training,

Y . Niu, S. Prakash, S. Kundu, S. Lee, and S. Avestimehr, “Federated learning of large models at the edge via principa l sub-model training,” arXiv: 2208.13141 , 2022

2022 arXiv
[20]

Resource-adaptive federated learning with all-in-one neural composition,

Y . Mei, P . Guo, M. Zhou and V . Patel, “Resource-adaptive federated learning with all-in-one neural composition,” i n Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 4270–4284, 2022

2022
[21]

Fully decent ralized joint learning of personalized models and collaboration gr aphs,

V . Zantedeschi, A. Bellet and M. Tommasi, “Fully decent ralized joint learning of personalized models and collaboration gr aphs,” in International Conf. Artif. Intell. Statist. , pp. 864–874, 2020

2020
[22]

Enhancing Decentralized and Personalized Federated Learning with To pol- ogy Construction,

S. Chen, Y . Xu, H. Xu, Z. Ma and Z. Wang, “Enhancing Decentralized and Personalized Federated Learning with To pol- ogy Construction,” in IEEE Trans. Mobile Computing , pp. 1–16, 2024

2024
[23]

DePRL: Achiev- ing Linear Convergence Speedup in Personalized Decentrali zed Learning with Shared Representations,

G. Xiong, G. Yan, S. Wang, Z. Ma and J. Li, “DePRL: Achiev- ing Linear Convergence Speedup in Personalized Decentrali zed Learning with Shared Representations,” in Proc. AAAI Conf. Artif. Intell. , vol. 38, pp. 16103–16111, 2024

2024
[24]

Dispﬂ: Towards communication-efﬁcient personalized federated learning via de- centralized sparse training,

R. Dai, L. Shen, F. He, X. Tian and D. Tao, “Dispﬂ: Towards communication-efﬁcient personalized federated learning via de- centralized sparse training,” in Int. Conf. Mach. Learning , pp. 4587–4604, 2022

2022
[25]

Dlion: Decentralized distribu ted deep learning in micro-clouds,

R. Hong and A. Chandra, “Dlion: Decentralized distribu ted deep learning in micro-clouds,” in Proc. 30th Int. Symposium on High- Performance Parallel and Distributed Computing , pp. 227–238, 2021

2021
[26]

Mitigating stragglers i n the decentralized training on heterogeneous clusters,

D. Yang, W. Rang, and D. Cheng, “Mitigating stragglers i n the decentralized training on heterogeneous clusters,” in Proc. 21st Int. Middleware Conf. , pp. 386–399, 2020

2020
[27]

Federated learni ng via over-the-air computation,

K. Yang, T. Jiang, Y . Shi, and Z. Ding, “Federated learni ng via over-the-air computation,” in IEEE Trans. Wireless Commun., vol. 19, no. 3, pp. 2022–2035, 2020

2022
[28]

Adaptive federated learning in resource constrained edge computing systems,

S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He and K. Chan, “Adaptive federated learning in resource constrained edge computing systems,” in IEEE Jour . Selected Areas Commun. , vol. 37, no. 6, pp. 1205–1221, 2019

2019
[29]

Communication-efﬁcie nt fed- erated learning based on compressed sensing,

C. Li, G. Li, and P . K. V arshney, “Communication-efﬁcie nt fed- erated learning based on compressed sensing,” in IEEE Internet of Things Journal , vol. 8, no. 20, pp. 15531-15541, 2021

2021
[30]

Privacy-preserving federated primal-dual learning for n on- convex and non-smooth problems with model sparsiﬁcation,

Y . Li, C.-W. Huang, S. Wang, C.-Y . Chi, and T. Q. Quek, “Privacy-preserving federated primal-dual learning for n on- convex and non-smooth problems with model sparsiﬁcation,” in IEEE Internet of Things Journal , vol. 11, no. 15, pp. 25853–25866, 2024

2024
[31]

Nonlinear perturbation-based non-convex optimiza tion over time-varying networks,

M. Doostmohammadian, Z. R. Gabidullina, and H. R. Ra- biee, “Nonlinear perturbation-based non-convex optimiza tion over time-varying networks,” in IEEE Trans. Network Science and Engineering , 2024

2024
[32]

Log-Scale Quantization in Distributed First-Order Methods: Gradient-based Learnin g from Distributed Data,

M. Doostmohammadian, M. I. Qureshi, M. H. Khalesi, H. R. Rabiee, and U. A. Khan, “Log-Scale Quantization in Distributed First-Order Methods: Gradient-based Learnin g from Distributed Data,” in IEEE Trans. Automation Science and En- gineering, 2025

2025
[33]

Fedmas k: Joint computation and communication-efﬁcient personaliz ed fed- erated learning via heterogeneous masking,

A. Li, J. Sun, X. Zeng, M. Zhang, H. Li and Y . Chen, “Fedmas k: Joint computation and communication-efﬁcient personaliz ed fed- erated learning via heterogeneous masking,” in Proc. 19th ACM Conf. Embedded Networked Sensor Systems , pp. 42–55, 2021

2021
[34]

A lightweight an d secure deep learning model for privacy-preserving federated lear ning in intelligent enterprises,

R. Fotohi, F. S. Aliee and B. Farahani, “A lightweight an d secure deep learning model for privacy-preserving federated lear ning in intelligent enterprises,” IEEE Internet of Things Journal , 2024

2024
[35]

Decentralized a nd robust privacy-preserving model using blockchain-enabled feder ated deep learning in intelligent enterprises,

R. Fotohi, F. S. Aliee and B. Farahani, “Decentralized a nd robust privacy-preserving model using blockchain-enabled feder ated deep learning in intelligent enterprises,” Applied Soft Computing , vol. 161, 2024

2024
[36]

Deconstructing lottery tickets: Zeros, signs, and the supermask,

H. Zhou, J. Lan, R. Liu and J. Y osinski, “Deconstructing lottery tickets: Zeros, signs, and the supermask,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , vol. 32, 2019. 15

2019
[37]

Proving the lottery ticket hypothesis: Pruning is all you n eed,

E. Malach, G. Yehudai, S. Shalev-Schwartz, and O. Shami r, “Proving the lottery ticket hypothesis: Pruning is all you n eed,” in Int. Conf. Mach. Learning , pp. 6682–6691, 2020

2020
[38]

The Lottery Ticket Hypothes is: Finding Sparse, Trainable Neural Networks,

J. Frankle, and M. Carbin, “The Lottery Ticket Hypothes is: Finding Sparse, Trainable Neural Networks,” in Int. Conf. Learn- ing Represent., 2018

2018
[39]

What’s hidden in a randomly weighted neural network?,

V . Ramanujan, M. Wortsman, A. Kembhavi, A. Farhadi, and M. Rastegari, “What’s hidden in a randomly weighted neural network?,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 11893–11902, 2020

2020
[40]

Optimal lottery tickets via subset sum: Logarit hmic over-parameterization is sufﬁcient,

A. Pensia, S. Rajput, A. Nagle, H. Vishwakarma, and D. Pa pail- iopoulos, “Optimal lottery tickets via subset sum: Logarit hmic over-parameterization is sufﬁcient,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , vol. 33, pp. 2599–2610, 2020

2020
[41]

Proving the stro ng lottery ticket hypothesis for convolutional neural networ ks,

A. da Cunha, E. Natale, and L. Viennot, “Proving the stro ng lottery ticket hypothesis for convolutional neural networ ks,” in Int. Conf. Learning Represent. , 2022

2022
[42]

Group sparse regularization for deep neural networks,

S. Scardapane, D. Comminiello, A. Hussain, and A. Uncin i, “Group sparse regularization for deep neural networks,” in Neu- rocomputing, vol. 241, pp. 81–89, 2017

2017
[43]

Hierarchical grou p sparse regularization for deep convolutional neural networks,

K. Mitsuno, J. Miyao, and T. Kurita, “Hierarchical grou p sparse regularization for deep convolutional neural networks,” i n 2020 Int Joint Conf. Neural Networks (IJCNN) , pp. 1–8, 2020

2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Differential ly private federated clustering over non-iid data,

Y . Li, S. Wang, C.-Y . Chi, and T. Q. S. Quek, “Differential ly private federated clustering over non-iid data,” IEEE Internet of Things Journal, vol. 11, no. 4, pp. 6705—6721, 2024

work page 2024

[2] [2]

Distributed Multi-View Sparse V ector Recovery,

Z. Tian, Z. Zhang, and L. Hanzo, “Distributed Multi-View Sparse V ector Recovery,” IEEE Trans. Signal Process. , vol. 71, pp. 1448–1463, 2023

work page 2023

[3] [3]

Distributed Learning Over Networks With Graph-Attention-Based Person al- ization,

Z. Tian, Z. Zhang, Z. Yang, R. Jin and H. Dai, “Distributed Learning Over Networks With Graph-Attention-Based Person al- ization,” IEEE Trans. Signal Process. , vol. 71, pp. 2071–2086, 2023

work page 2023

[4] [4]

Can decentralized algorithms outperform centrali zed algorithms? A case study for decentralized parallel stocha stic gradient descent,

X. Lian, C. Zhang, H. Zhang, C. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centrali zed algorithms? A case study for decentralized parallel stocha stic gradient descent,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 5330–5340, 2017

work page 2017

[5] [5]

Personalize d federated learning with theoretical guarantees: A model-a gnostic meta-learning approach,

A. Fallah, A. Mokhtari, and A. E. Ozdaglar, “Personalize d federated learning with theoretical guarantees: A model-a gnostic meta-learning approach,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS), pp. 3557–3568, 2020

work page 2020

[6] [6]

Cluster-driven graph federated learning ov er multiple domains

D. Caldarola, M. Mancini, F. Galasso, M. Ciccone, E. Rodo la, and B. Caputo, “Cluster-driven graph federated learning ov er multiple domains.” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) W orkshops, pp. 2743–2752, 2021

work page 2021

[7] [7]

Federated optimization in heterogeneous networ ks,

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networ ks,” in Proc. Machine learning and systems , vol. 2, pp. 429–450, 2020

work page 2020

[8] [8]

An efﬁcie nt framework for clustered federated learning,

A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efﬁcie nt framework for clustered federated learning,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS) , pp. 19586–19597, 2020

work page 2020