Pith. sign in

REVIEW 5 major objections 9 minor 40 references

Distilling A Universal Expert from Clustered Federated Learning

T0 review · 5 major / 9 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Distilling a universal expert model from cluster-specific teachers lets a single federated model outperform eleven baselines on SVHN, CIFAR-10, and CIFAR-100 while preserving per-client personalization.

desk verdict The idea is a plausible incremental extension, but Algorithm 1 as written never uses the distilled universal expert for training—a load-bearing bug that makes the reported results inexplicable. read the letter →

arxiv 2506.20285 v1 pith:FBELENUS submitted 2025-06-25 cs.LG

Zeqi Leng , Chunxu Zhang , Guodong Long , Riting Xia , Bo Yang This is my paper
classification cs.LG
keywords federatedlearningclustereddata-freeknowledgedistillationnon-IIDdatamodelpersonalizationtransferaffinitypropagationuniversalexpert
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DisUE, a clustered federated learning framework that treats the cluster-specific expert models produced by client clustering as teachers and distills a single 'universal expert' model from them using data-free knowledge distillation. The universal expert is designed to capture the knowledge shared across all clusters, and it is sent back to every client as the initialization for the next round, so each client keeps its personalization while benefiting from global generalization. The paper reports that DisUE achieves higher test accuracy than eleven federated learning, clustered federated learning, and data-free knowledge distillation baselines on SVHN, CIFAR-10, and CIFAR-100 under both IID and non-IID data splits, with the largest gains in the most skewed settings. It also shows that the inter-group aggregation module can be attached to existing clustered federated learning optimizers and improve their reported accuracy, supporting the claim that cross-cluster knowledge transfer is the missing ingredient in current CFL designs.

What carries the argument

The load-bearing mechanism is the inter-group aggregation (IGA) module, a cluster-level data-free knowledge distillation loop. A server-side generator $G$, conditioned on class labels, synthesizes pseudo-samples $\hat{x}=G(\theta_G; z, y)$ from Gaussian noise $z$, and the cluster specialist models act as teachers for the global model $\omega_G$, which is the student; a minimax objective alternately makes the generator produce hard-classifiable samples and makes the student align its softmax predictions with the teachers' logits. Two adaptive components, the Group Label Sampler (GLS) and Group Weighting Factors (GWF), use the per-cluster category counts $n_k^y$ to bias label sampling and cluster weighting toward under-represented classes, and the diversity regularizer $L_{\mathrm{div}}$ keeps the pseudo-samples varied. This module is what transfers knowledge across clusters and is also the plug-in that the paper shows can be added to existing CFL optimizers such as IFCA, CFL, CFL-GP, and PACFL.

What would settle it

On SVHN with $\mathrm{Dir}(\epsilon=0.01)$, the paper's ablation (Table 3) shows that removing GLS or GWF drops accuracy from 83.04% to 82.23% and 82.14%, respectively, while removing the whole IGA module drops it to 72.76%. A decisive experiment is to run the full method with the per-cluster counts $n_k^y$ replaced by shuffled values: if accuracy stays near 83%, the adaptive category statistics are not the active ingredient; if it falls toward the 82% ablation level, they are.

Watch

Extended reading notes

Core claim

The central claim is that the strict inter-cluster separation in clustered federated learning discards shared statistical structure, and that this structure can be recovered by distilling a universal expert model from all cluster specialists at once. In DisUE, clients train locally on private data, the server clusters them with affinity propagation over encrypted cosine similarities, aggregates each cluster into a specialist model, and then runs a data-free adversarial distillation in which a server-side generator synthesizes pseudo-samples while the global model learns to match the specialists' predictions. Adaptive weights computed from per-cluster label counts, the Group Label Sampler (GLS) and Group Weighting Factors (GWF), steer both the generator and the ensemble so that classes under-represented in some clusters are not drowned out. The authors report that this three-step paradigm preserves fine-grained non-IID characteristics while incorporating shared knowledge, and that the resulting global model outperforms eleven baselines across the three image benchmarks.

Load-bearing premise

The method assumes the server can obtain, for every cluster, the number of training examples of each class ($n_k^y$) that feeds the GLS and GWF equations, without violating the privacy that federated learning promises; the paper does not describe how those counts are collected or protected.

Editorial extensions

If this is right

  • A single global model can serve all clients in a federated system even when local data distributions differ, because cluster-specific knowledge is distilled into it rather than averaged away.
  • The inter-group aggregation module can be layered on top of existing clustered federated learning methods such as IFCA, CFL, CFL-GP, and PACFL, improving their reported accuracy by 1.15 to 3.44 percentage points in the non-IID setting tested.
  • Data-free knowledge distillation across clusters removes the need for a public auxiliary dataset, which is one of the main practical obstacles to knowledge distillation in federated learning.
  • Clustered federated learning with this distillation step converges faster in the early communication rounds than other data-free distillation methods, because clustering suppresses non-IID noise before distillation begins.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The IGA module is model-agnostic in design, so it could likely be grafted onto any future clustering-based federated method besides the four optimizers tested; this generalization is a natural next experiment.
  • If privacy rules forbid sharing ground-truth label counts, the adaptive weights could be re-estimated from the aggregate model's own soft predictions, preserving the mechanism without exposing labels; the paper does not explore this variant.
  • The paper's privacy claim covers similarity values but not label statistics, so a fully private deployment would need an additional protocol to aggregate the $n_k^y$ counts, and the accuracy impact of that protocol is untested.
  • Because the universal expert is trained by ensemble distillation from cluster specialists, it may inherit better prediction calibration or robustness to input distribution shift than standard FedAvg models, but these properties are not measured.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 9 minor

Summary. The paper proposes DisUE, a clustered federated learning framework that (i) clusters clients via Affinity Propagation over cosine similarities, (ii) aggregates client models within clusters, and (iii) distills a 'universal expert' from the cluster models through data-free knowledge distillation, with an adaptive label sampler (GLS) and group weighting factors (GWF) to handle inter-cluster label shift. A Secure Similarity Computing (SSC) protocol is claimed to protect parameter similarity values. Experiments on SVHN, CIFAR-10, and CIFAR-100 under IID and Dirichlet non-IID settings compare DisUE against 11 FL, CFL, and DFKD baselines, and include a compatibility study, an ablation, and a sensitivity analysis.

Significance. If the method is implemented as the text describes, the idea of feeding a distilled universal expert back into clients is a plausible way to transfer shared knowledge across clusters while retaining personalization, and the compatibility results suggest the inter-group aggregation module could be a useful plug-in for existing CFL methods. Strengths include broad empirical coverage, a clean ablation isolating IGA (Table 3), a compatibility study over four CFL optimizers (Table 2), and averaging over five random seeds. However, the value of these contributions depends on resolving the algorithmic and formulation inconsistencies described below; the paper also does not release code, which makes the reported results hard to audit.

major comments (5)
  1. [Algorithm 1 (Section 4.4)] As printed, Algorithm 1 does not implement the paper's core mechanism. Line 5 initializes round-t local training from ω_G^(t−1); line 19 sets ω_G^(t) ← FedAvg(...); line 21 sets ω_G^(t+1) ← IGA(...). Therefore at round t+1 clients train from ω_G^(t), the pre-distillation FedAvg model, and the IGA output ω_G^(t+1) is never used for training. After T rounds the function returns ω_G^(T), which was computed by FedAvg at line 19, not by IGA. Taken literally, the universal expert distillation has no effect on either the returned model or the clients' initializations, yet Table 3 attributes a 10-point accuracy gain to IGA. This is a load-bearing inconsistency: the pseudocode must be corrected so that the distilled model is actually used as the next-round initialization and is the returned model, and the experiments must be re-verified against the corrected algorithm, or the central claim is unsupported.
  2. [Eqs. (5)-(6), Section 4.2] The adaptive GLS and GWF components require the server to know per-cluster label counts n_k^y for every class y and cluster k. The paper never explains how the server obtains these counts under the federated setting: clients upload only encrypted parameters (Algorithm 1, lines 6-8), and the Secure Similarity Computing protocol concerns parameter similarity, not label distributions. Without a concrete mechanism for clients to report n_k^y (e.g., secure aggregation or differentially private histograms), Eqs. (5)-(6) are underspecified, and the privacy claim is overstated because label histograms are arguably more sensitive than parameter similarity values.
  3. [Eq. (13) vs Eq. (10), Section 4.4] The composite objective in Eq. (13), min over ω_G and max over θ_G of L_cd + β_cf L_cf + β_div L_div, is inconsistent with Eq. (10), which minimizes L_cf with respect to θ_G. As written, the generator is incentivized to maximize L_cf, producing label-inconsistent pseudo-samples, contrary to the text's statement that dropping L_cf blurs pseudo-data. The paper should state the alternating updates precisely: for fixed θ_G, ω_G should minimize L_cd; for fixed ω_G, θ_G should maximize L_cd while minimizing L_cf and maximizing L_div (or an equivalent sign convention). This is central to the IGA mechanism.
  4. [Section 4.3 and Section 1 (C3)] The paper claims that the Secure Similarity Computing protocol prevents direct exposure of similarity values, but no cryptographic construction is given beyond a citation to PCSC [Lu et al., 2014]. It is not specified whether Affinity Propagation in Eq. (4) operates on encrypted values, what the security model is, or how the protocol interacts with the plaintext cosine similarity in Eq. (3). The privacy claim should be scaled back to what is actually shown, and the label counts required by Eqs. (5)-(6) must be covered by the privacy analysis.
  5. [Table 1 and Section 5.1] The paper reports 'average over five random seeds' but gives no standard deviations or confidence intervals. Several of the reported margins over the strongest baseline are small (e.g., CIFAR-10 dir(0.1): 76.22 vs 75.39 for SCAFFOLD; CIFAR-100 dir(0.01): 31.87 vs 30.96 for PACFL and 30.92 for FedAvg), so the claim of 'consistently outperforms' is not statistically supported as presented. Please add error bars or per-seed tables and, where margins are small, a significance test.
minor comments (9)
  1. [Eq. (11)] In Eq. (11), the right-hand side sums over k, so L_k^cf is the same for every k and the outer weighted sum in Eq. (10) reduces to a squared weight. Use a separate index for the teacher sum, e.g., L_k^cf = Σ_{j=1}^K α_j^y CE(σ(C(ω_j; x̂)), y), or remove the inner sum.
  2. [Table 3] The text says that removing IGA 'reverts DisUE to FedAvg', but the reported -IGA accuracy (72.76) is lower than the FedAvg baseline in Table 1 (73.83) on the same SVHN dir(0.01) setting; please explain this discrepancy.
  3. [References] References [Lin et al., 2020a] and [Lin et al., 2020b] are the same paper (Ensemble distillation for robust model fusion in federated learning); please merge them and use one citation key.
  4. [Eq. (12)] The diversity loss in Eq. (12) contains '1/Q*Q', which should presumably be '1/Q^2'; please clarify the normalization.
  5. [Section 5.2 and Figure 2] In Figure 2b, the quantity 'act(·)' is not defined in the caption; Section 5.2 refers to an active fraction while Section 5.1 defines 'Act=0.15'. Please unify the notation and define the symbol.
  6. [Section 5.5] Please clarify whether the hyperparameters β_cf and β_div were selected on a validation split of SVHN dir(0.01) and then transferred to all other datasets, or tuned per dataset on the test set; the current text does not state this.
  7. [Section 4.2, paragraph 'Cluster-level DFKD Aggregation Mechanism'] The text describes the aggregated cluster models as 'serving as student networks' and then 'subsequently function as teacher networks'; please clarify that in Eqs. (8)-(9) the teachers are the cluster models ω_Ck and the student is the global model ω_G.
  8. [Algorithm 1] The pseudocode does not specify the internal optimization loops for IGA (e.g., generator update steps and the number of pseudo-samples Q per round); please add these details for reproducibility.
  9. [Throughout] There are several typographical issues, e.g., 'adptaive' (Section 4.2), 'a significantly challenge' (Section 1), 'on-IID' (Section 5.2), 'conducing' (Section 4.4), and the missing space in Table 1's PACFL row ('34.3743.14'); these should be corrected in revision.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: DisUE's empirical claims rest on external benchmarks and its components are not defined in terms of the target result; the Algorithm 1 inconsistency is a correctness flaw, not a circular derivation.

full rationale

The central claim is an empirical performance comparison against 11 external FL, CFL, and DFKD baselines on SVHN, CIFAR-10, and CIFAR-100 (Table 1), with additional compatibility and ablation studies in Tables 2 and 3. These targets are not constructed from DisUE's own outputs: the Dirichlet partitions, baseline implementations, and public datasets are independent of the method's fitted hyperparameters (beta_cf, beta_div, noise dimension), and hyperparameter selection on the evaluation datasets is standard practice rather than circular. The only self-citations (Zhang et al. 2023; Long et al. 2023) appear in the introduction and related work as general FL/CFL context and are not invoked as a load-bearing uniqueness theorem or as justification for the main result. Equations (5)-(12) define GLS, GWF, and the distillation losses from cluster label statistics and model outputs; while the server-side access to per-cluster category counts n_k^y is underspecified from a privacy standpoint, this is an implementation/privacy gap, not a circular derivation. Separately, Algorithm 1's IGA output omega_G^(t+1) is never used by clients at line 5 in the following round, so the described mechanism may not be faithfully implemented; this is a correctness/reproducibility flaw that would affect the paper's experimental claims, but it does not make the derivation equivalent to its own inputs. Overall, the derivation chain is self-contained and externally falsifiable, so no circularity step is present; the minor self-citations are not load-bearing and the score reflects only that contextual self-citation, not any circular reduction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on several unstated assumptions: the server can access per-cluster label counts for GLS/GWF, the AP clustering yields meaningful groups from high-dimensional model updates, and the generator-based distillation transfers knowledge across heterogeneous clusters. Hyperparameters beta_cf, beta_div, and noise dimension are tuned on the evaluation datasets. No new physical entities are introduced.

free parameters (3)
  • beta_cf = 1.0 (default); searched over 0.5, 0.75, 1, 1.25, 1.5
    Weight for cluster fidelity loss in Eq. (13); selected by sensitivity analysis on SVHN, not derived from first principles.
  • beta_div = 1.0 (default); searched over 0.5, 0.75, 1, 1.25, 1.5
    Weight for diversity loss in Eq. (13); tuned on SVHN via sensitivity analysis.
  • noise_dimension = 100 (CIFAR-10, SVHN), 256 (CIFAR-100)
    Dimension of generator input; chosen via sensitivity analysis on SVHN, following settings in Zhang et al. 2022.
assumptions (4)
  • domain assumption Affinity Propagation correctly groups clients into meaningful clusters based on cosine similarity of parameter updates (Eqs. 3-4).
    The method's intra-group aggregation assumes these clusters reflect true label distributions; no analysis of clustering stability under non-IID is provided.
  • ad hoc to paper The server can obtain per-cluster label distribution counts n_k^y used in Eqs. (5) and (6) for GLS and GWF.
    No privacy-preserving mechanism for label counts is described, and this assumption is not stated in the paper.
  • domain assumption A server-side generator trained with L_cf and L_div produces pseudo-samples that transfer knowledge between heterogeneous cluster models (Eqs. 7-12).
    This is the core effectiveness assumption of the DFKD component; it is only empirically tested, not theoretically justified.
  • domain assumption Clients within a cluster have identically distributed data, justifying FedAvg per cluster (Eq. 2).
    Standard CFL assumption inherited from prior work, not verified in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Distilling A Universal Expert from Clustered Federated Learning." pith.science (2026). https://pith.science/paper/FBELENUS

@misc{pith2026250620285,
  author       = {Pith},
  title        = {Pith review of: Distilling A Universal Expert from Clustered Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FBELENUS}},
  note         = {Machine review of arXiv:2506.20285}
}
read the original abstract

Clustered Federated Learning (CFL) addresses the challenges posed by non-IID data by training multiple group- or cluster-specific expert models. However, existing methods often overlook the shared information across clusters, which represents the generalizable knowledge valuable to all participants in the Federated Learning (FL) system. To overcome this limitation, this paper introduces a novel FL framework that distills a universal expert model from the knowledge of multiple clusters. This universal expert captures globally shared information across all clients and is subsequently distributed to each client as the initialization for the next round of model training. The proposed FL framework operates in three iterative steps: (1) local model training at each client, (2) cluster-specific model aggregation, and (3) universal expert distillation. This three-step learning paradigm ensures the preservation of fine-grained non-IID characteristics while effectively incorporating shared knowledge across clusters. Compared to traditional gradient-based aggregation methods, the distillation-based model aggregation introduces greater flexibility in handling model heterogeneity and reduces conflicts among cluster-specific experts. Extensive experimental results demonstrate the superior performance of the proposed method across various scenarios, highlighting its potential to advance the state of CFL by balancing personalized and shared knowledge more effectively.

Figures

Figures reproduced from arXiv: 2506.20285 by the authors.

Figure 1
Figure 1. Overview of DisUE workflow. (1) Local Training: Clients first train local models using private data. (2) Secure Similarity Computing: Clients employ the Secure Similarity Computing protocol to encrypt model parameters. (3) Clustering and Intra-group Ag￾gregation: The server partition clients into clusters. Each cluster performs FedAvg averaging within its group. (4) Inter-group Aggregation: The framework distills a … view at source ↗
Figure 2
Figure 2. (a) Test accuracy versus data heterogeneity measured by [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Performance of DisUE using different hyperparameters on SVHN, dir [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 32 canonical work pages

  1. [1]

    Federated learn- ing based on dynamic regularization.arXiv preprint arXiv:2111.04263,

    [Acaret al., 2021 ] Durmus Alp Emre Acar, Yue Zhao, Ra- mon Matas Navarro, Matthew Mattina, Paul N What- mough, and Venkatesh Saligrama. Federated learn- ing based on dynamic regularization.arXiv preprint arXiv:2111.04263,

    arXiv 2021

  2. [4]

    Density-based spatial cluster- ing of applications with noise

    [Esteret al., 1996 ] Martin Ester, Hans-Peter Kriegel, J ¨org Sander, and Xiaowei Xu. Density-based spatial cluster- ing of applications with noise. InInt. Conf. knowledge discovery and data mining, volume 240,

    work page 1996

  3. [9]

    Data-free ensemble knowledge distillation for privacy-conscious multimedia model com- pression

    [Haoet al., 2021 ] Zhiwei Hao, Yong Luo, Han Hu, Jianping An, and Yonggang Wen. Data-free ensemble knowledge distillation for privacy-conscious multimedia model com- pression. InProceedings of the 29th ACM International Conference on Multimedia, pages 1803–1811,

    work page 2021

  4. [10]

    Deep residual learning for image recog- nition

    [Heet al., 2016 ] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun

    work page 2016

  5. [13]

    Swiftagg: Communication-efficient and dropout-resistant secure aggregation for federated learning with worst-case security guarantees

    [Jahani-Nezhadet al., 2022 ] Tayyebeh Jahani-Nezhad, Mo- hammad Ali Maddah-Ali, Songze Li, and Giuseppe Caire. Swiftagg: Communication-efficient and dropout-resistant secure aggregation for federated learning with worst-case security guarantees. In2022 IEEE International Sympo- sium on Information Theory (ISIT), pages 103–108. IEEE,

    work page 2022

  6. [15]

    Clustered federated learning via gradient- based partitioning

    [Kimet al., 2024 ] Heasung Kim, Hyeji Kim, and Gustavo De Veciana. Clustered federated learning via gradient- based partitioning. InForty-first International Conference on Machine Learning,

    work page 2024

  7. [16]

    Learning multiple layers of features from tiny im- ages

    [Krizhevskyet al., 2009 ] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages

    2009

  8. [18]

    Model-contrastive federated learning

    [Liet al., 2021 ] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10713–10722,

    2021

Show all 40 references
  1. [19]

    Casa: Clustered fed- erated learning with asynchronous clients

    [Liuet al., 2024 ] Boyi Liu, Yiming Ma, Zimu Zhou, Yexuan Shi, Shuyuan Li, and Yongxin Tong. Casa: Clustered fed- erated learning with asynchronous clients. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Dis- covery and Data Mining, pages 1851–1862,

    2024

  2. [20]

    Multi-center federated learning: clients clustering for better personal- ization.World Wide Web, 26(1):481–500,

    [Longet al., 2023 ] Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang. Multi-center federated learning: clients clustering for better personal- ization.World Wide Web, 26(1):481–500,

    2023

  3. [21]

    Toward efficient and privacy- preserving computing in big data era.IEEE Network, 28(4):46–50,

    [Luet al., 2014 ] Rongxing Lu, Hui Zhu, Ximeng Liu, Joseph K Liu, and Jun Shao. Toward efficient and privacy- preserving computing in big data era.IEEE Network, 28(4):46–50,

    2014

  4. [23]

    Structured federated learning through clustered additive modeling.Advances in Neural Information Processing Systems, 36:43097–43107,

    [Maet al., 2023 ] Jie Ma, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. Structured federated learning through clustered additive modeling.Advances in Neural Information Processing Systems, 36:43097–43107,

    2023

  5. [24]

    hdbscan: Hierarchical density based clus- tering.J

    [McInneset al., 2017 ] Leland McInnes, John Healy, Steve Astels, et al. hdbscan: Hierarchical density based clus- tering.J. Open Source Softw., 2(11):205,

    2017

  6. [26]

    Reading digits in natural images with unsupervised feature learning

    [Netzeret al., 2011 ] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. InNIPS workshop on deep learning and unsu- pervised feature learning, volume 2011, page

    2011

  7. [28]

    Fedsoft: Soft clustered federated learning with proximal local updating,

    [Ruan and Joe-Wong, 2022] Yichen Ruan and Carlee Joe- Wong. Fedsoft: Soft clustered federated learning with proximal local updating,

    2022

  8. [29]

    [Sattleret al., 2020 ] Felix Sattler, Klaus-Robert M ¨uller, and Wojciech Samek. Clustered federated learning: Model- agnostic distributed multitask optimization under privacy constraints.IEEE transactions on neural networks and learning systems, 32(8):3710–3722,

    2020

  9. [30]

    Teacher as a lenient expert: Teacher-agnostic data-free knowledge distillation

    [Shin and Choi, 2024] Hyunjune Shin and Dong-Wan Choi. Teacher as a lenient expert: Teacher-agnostic data-free knowledge distillation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14991–14999,

    2024

  10. [31]

    Turbo-aggregate: Breaking the quadratic ag- gregation barrier in secure federated learning.IEEE Jour- nal on Selected Areas in Information Theory, 2(1):479– 489,

    [Soet al., 2021 ] Jinhyun So, Bas ¸ak G ¨uler, and A Salman Avestimehr. Turbo-aggregate: Breaking the quadratic ag- gregation barrier in secure federated learning.IEEE Jour- nal on Selected Areas in Information Theory, 2(1):479– 489,

    2021

  11. [32]

    Entrocfl: Entropy-based clustered federated learning with incentive mechanism.IEEE Internet of Things Journal, 12(1):986–1001,

    [Tuet al., 2025 ] Kaifei Tu, Xuehe Wang, and Xiping Hu. Entrocfl: Entropy-based clustered federated learning with incentive mechanism.IEEE Internet of Things Journal, 12(1):986–1001,

    2025

  12. [34]

    [Wang and Yoon, 2021] Lin Wang and Kuk-Jin Yoon. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks.IEEE transactions on pattern analysis and machine intelligence, 44(6):3048–3068,

    2021

  13. [35]

    Bridging model heterogeneity in federated learning via uncertainty-based asymmetrical reciprocity learning

    [Wanget al., 2024 ] Jiaqi Wang, Chenxu Zhao, Lingjuan Lyu, Quanzeng You, Mengdi Huai, and Fenglong Ma. Bridging model heterogeneity in federated learning via uncertainty-based asymmetrical reciprocity learning. CoRR, abs/2407.03247,

    2024 arXiv

  14. [36]

    Data-free knowledge amalga- mation via group-stack dual-gan

    [Yeet al., 2020 ] Jingwen Ye, Yixin Ji, Xinchao Wang, Xin Gao, and Mingli Song. Data-free knowledge amalga- mation via group-stack dual-gan. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12516–12525,

    2020

  15. [37]

    Knowledge extraction with no observable data.Advances in Neural Information Processing Systems, 32,

    [Yooet al., 2019 ] Jaemin Yoo, Minyong Cho, Taebum Kim, and U Kang. Knowledge extraction with no observable data.Advances in Neural Information Processing Systems, 32,

    2019

  16. [38]

    Fine-tuning global model via data-free knowledge distillation for non-iid fed- erated learning

    [Zhanget al., 2022 ] Lin Zhang, Li Shen, Liang Ding, Dacheng Tao, and Ling-Yu Duan. Fine-tuning global model via data-free knowledge distillation for non-iid fed- erated learning. InProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 10174–10183,

    2022

  17. [39]

    Dual personalization on federated recommen- dation

    [Zhanget al., 2023 ] Chunxu Zhang, Guodong Long, Tianyi Zhou, Peng Yan, Zijian Zhang, Chengqi Zhang, and Bo Yang. Dual personalization on federated recommen- dation. InProceedings of the Thirty-Second Interna- tional Joint Conference on Artificial Intelligence, IJCAI ’23,

    2023

  18. [40]

    Data-free knowledge distillation for heterogeneous federated learning

    [Zhuet al., 2021 ] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. InInternational conference on ma- chine learning, pages 12878–12889. PMLR, 2021

    2021

  19. [1996]

    Taking advantage of the mistakes: Rethinking clustered federated learning for iot anomaly detection.IEEE Transactions on Parallel and Distributed Systems, 35(6):862–876,

    [Fanet al., 2024 ] Jiamin Fan, Kui Wu, Guoming Tang, Yang Zhou, and Shengqiang Huang. Taking advantage of the mistakes: Rethinking clustered federated learning for iot anomaly detection.IEEE Transactions on Parallel and Distributed Systems, 35(6):862–876,

    2024

  20. [2007]

    An efficient framework for clustered federated learning.Advances in Neural In- formation Processing Systems, 33:19586–19597,

    [Ghoshet al., 2020 ] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning.Advances in Neural In- formation Processing Systems, 33:19586–19597,

    2020

  21. [2009]

    Federated optimization in heterogeneous networks.Pro- ceedings of Machine learning and systems, 2:429–450,

    [Liet al., 2020 ] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks.Pro- ceedings of Machine learning and systems, 2:429–450,

    2020

  22. [2011]

    Federated learning for internet of things: A comprehensive survey.IEEE Communications Surveys & Tutorials, 23(3):1622–1658,

    [Nguyenet al., 2021 ] Dinh C Nguyen, Ming Ding, Pub- udu N Pathirana, Aruna Seneviratne, Jun Li, and H Vin- cent Poor. Federated learning for internet of things: A comprehensive survey.IEEE Communications Surveys & Tutorials, 23(3):1622–1658,

    2021

  23. [2014]

    On the conver- gence of clustered federated learning.arXiv preprint arXiv:2202.06187,

    [Maet al., 2022 ] Jie Ma, Guodong Long, Tianyi Zhou, Jing Jiang, and Chengqi Zhang. On the conver- gence of clustered federated learning.arXiv preprint arXiv:2202.06187,

    2022 arXiv

  24. [2016]

    Measuring the effects of non-identical data distribution for federated visual classification.arXiv preprint arXiv:1909.06335,

    [Hsuet al., 2019 ] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification.arXiv preprint arXiv:1909.06335,

    2019 arXiv

  25. [2017]

    Communication-efficient learning of deep networks from decentralized data

    [McMahanet al., 2017 ] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. InArtificial intelligence and statistics, pages 1273–1282. PMLR,

    2017

  26. [2019]

    Active client selection for clustered feder- ated learning.IEEE Transactions on Neural Networks and Learning Systems, 35(11):16424–16438,

    [Huanget al., 2024 ] Honglan Huang, Wei Shi, Yanghe Feng, Chaoyue Niu, Guangquan Cheng, Jincai Huang, and Zhong Liu. Active client selection for clustered feder- ated learning.IEEE Transactions on Neural Networks and Learning Systems, 35(11):16424–16438,

    2024

  27. [2020]

    Fedrc: Tackling diverse distribution shifts challenge in federated learning by robust clustering,

    [Guoet al., 2024 ] Yongxin Guo, Xiaoying Tang, and Tao Lin. Fedrc: Tackling diverse distribution shifts challenge in federated learning by robust clustering,

    2024

  28. [2021]

    Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges.IEEE Communications Surveys & Tutorials, 25(4):2983–3013,

    [Beltr´anet al., 2023 ] Enrique Tom ´as Mart ´ınez Beltr ´an, Mario Quiles P ´erez, Pedro Miguel S ´anchez S ´anchez, Sergio L ´opez Bernal, G ´erˆome Bovet, Manuel Gil P ´erez, Gregorio Mart´ınez P ´erez, and Alberto Huertas Celdr ´an. Decentralized federated learning: Fundam...

    2023

  29. [2022]

    Scaffold: Stochastic controlled averaging for federated learning

    [Karimireddyet al., 2020 ] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. InInterna- tional conference on machine learning, pages 5132–5143. PMLR,

    2020

  30. [2023]

    Flexible clustered federated learning for client- level data distribution shift.IEEE Transactions on Parallel and Distributed Systems, 33(11):2661–2674,

    [Duanet al., 2021 ] Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren. Flexible clustered federated learning for client- level data distribution shift.IEEE Transactions on Parallel and Distributed Systems, 33(11):2661–2674,

    2021

  31. [2024]

    Clustering by passing messages between data points.sci- ence, 315(5814):972–976,

    [Frey and Dueck, 2007] Brendan J Frey and Delbert Dueck. Clustering by passing messages between data points.sci- ence, 315(5814):972–976,

    2007

  32. [2025]

    [Vahidianet al., 2023 ] Saeed Vahidian, Mahdi Morafah, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, and Bill Lin. Efficient distribution sim- ilarity identification in clustered federated learning via principal angles between client data subspaces.Proceed- ings...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.