REVIEW 1 major objections 2 minor 32 references
Decentralized federated clustering matches central-server accuracy within about one percent.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 09:26 UTC pith:WEGV53BF
load-bearing objection DFCA is a plausible, well-tested decentralization of IFCA with a real theory–implementation gap: the convergence proof analyzes full-model gossip, while the algorithm only sends the assigned cluster model. the 1 major comments →
DFCA: Decentralized Federated Clustering Algorithm
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
DFCA shows that the iterative federated clustering mechanism — assign a cluster by taking the argmin over local losses, then train only the assigned model — can be carried over to a fully decentralized setting without a central aggregator. The discovery is that a sequential running average of incoming neighbor models per cluster, combined with these local assignments, is enough to maintain cluster specialization while propagating all k models across the network. On MNIST, EMNIST, CIFAR-10 and FEMNIST with rotated data distributions, the authors report mean test accuracy within roughly 1% of the centralized baseline, and consistently above decentralized averaging baselines with momentum and a
What carries the argument
The central device is the running-average aggregation step, which replaces synchronous batch averaging with an online update θ_i,j ← (r/(r+1))θ_i,j + (1/(r+1))θ_m,j as each neighbor message arrives for cluster j. Worked into the IFCA-style loop of cluster assignment and local gradient steps, this sequential average lets the network behave like k independent runs of decentralized SGD after assignments stabilize. The analysis treats the communication as a row-stochastic gossip matrix W^(j) per cluster and depends on it being doubly-stochastic so that cluster-level network averages are preserved and disagreement contracts.
Load-bearing premise
The convergence proof assumes clients exchange all k cluster models with every neighbor so that the per-cluster gossip matrix is doubly-stochastic and the network-wide average of each cluster model is preserved, but the implemented algorithm sends only the assigned cluster's model, so the actual aggregation matrix is only row-stochastic and the average-preservation premise may fail.
What would settle it
Run DFCA on a graph divided into two disconnected components, each containing clients of a different true cluster, and check whether a client in one component ever acquires the other cluster's model; if it does not, the 'all clients obtain all k models' claim is false. Alternatively, compute the column sums of the implemented W^(j) for a cluster j; if any column sum is not 1, the network average of that cluster is not preserved, violating Lemma 3 and the rates in Theorem 1.
If this is right
- If correct, DFCA removes the single point of failure in clustered federated learning: no central server is needed to aggregate or redistribute models, so training can continue even when a coordinator would be unavailable.
- Every client ends up with all k trained cluster models, not just its own cluster's model, which enables per-client inference to switch clusters when local data changes.
- The sequential running-average scheme is memory-efficient, storing only current estimates per cluster, and naturally supports asynchronous message arrival, suiting intermittently connected devices.
- Sparse connectivity is sufficient: performance stays close to the centralized baseline even with Erdős–Rényi graphs of connectivity probability 0.1–0.2, lowering the topology requirements for decentralized clustered training.
Where Pith is reading between the lines
- If the central claim holds, DFCA's sequential averaging could combine with local differential privacy or compressed communication without architectural changes, since it already processes one model message at a time.
- The same cluster-assignment-plus-running-average recipe might extend to soft clustering (clients with mixed distributions) by maintaining multiple assigned models per client, though the paper does not make this claim.
- A directly testable extension is to measure how per-cluster disagreement Disp_j evolves under the actual one-model exchange rule on graphs where same-cluster subgraphs are weakly connected; the theory predicts contraction, so the rate is observable.
- The analysis assumes a fixed number of clusters k known in advance; an open question is whether DFCA can detect the number of clusters online, since its assignment step requires k as input.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DFCA, a fully decentralized clustered federated learning algorithm. Each client maintains k cluster-specific models, assigns itself to the cluster whose model has the lowest local loss, performs local SGD only on the assigned model, and then aggregates neighbor models via a sequential running average. The authors claim that DFCA matches centralized IFCA accuracy within about 1% across MNIST, EMNIST, CIFAR-10, and FEMNIST, while outperforming decentralized baselines (DFedAvgM, FedSPD). A convergence analysis (Theorem 1 and Appendix B) argues that DFCA behaves like k independent decentralized SGD runs after a finite stabilization phase, with all clients eventually holding all k cluster models.
Significance. If the central claim holds, DFCA would be a valuable practical contribution: it removes the central server from clustered federated learning, preserves cluster specialization in sparse networks, and uses a communication-efficient asynchronous aggregation scheme. The empirical results are promising and suggest that decentralized clustered FL is feasible. However, the paper's theoretical and empirical support is weakened by a major mismatch between the analyzed aggregation and the implemented algorithm, as well as by inconsistencies in reported experimental settings and the absence of publicly available code. A corrected analysis or a revised algorithm that matches the proof would make the contribution solid.
major comments (1)
- [§4, Theorem 1 and Eq. (12)–(13)] The convergence rates in Theorem 1 depend on the consensus factor λ (or λ̃) and on the assumption that after stabilization the system becomes 'k independent copies of decentralized SGD'. That conclusion relies on the per-cluster gossip being over the full graph with a doubly-stochastic mixing matrix. Since the implemented algorithm does not satisfy that condition, the stated rates do not apply to DFCA as specified in Algorithm 1. This is a load-bearing gap: the paper's headline theoretical contribution is a rate guarantee that does not cover the actual method.
minor comments (2)
- [Algorithm 1] The local update loop (lines 15–17) uses the same symbol T for the number of local epochs, which conflicts with the global iteration count T defined on line 7. This should be τ to match the text.
- [References] The reference for MNIST is given as Krizhevsky & Hinton (2009), but the citation is misplaced; MNIST is commonly credited to LeCun et al. (1998). The CIFAR-10 entry also appears to have swapped authors/dataset.
Circularity Check
No significant circularity: the convergence analysis is a conditional composition of external IFCA and decentralized-SGD results; the main proof-algorithm gap is a correctness/rigor issue, not a circular reduction.
full rationale
The paper's derivation chain does not reduce to its inputs. Cluster assignment (Eq. 4) and the loss definitions make Lemma 1's non-increase statement a direct consequence of the argmin definition, but the paper uses it only as a valid descent step, not as a hidden prediction. Convergence (Theorem 1) is explicitly conditional on assumptions (A1)-(A5), including the graph-mixing assumption (A3). The async part of Lemma 3 closely restates the async clause of A3, but A3 is presented as an adopted assumption, not as a result proved from the algorithm; the unverified burden is whether the W_t^(j) induced by Algorithm 1, which sends only the assigned cluster model theta_{i,c(i)}, satisfies A3. That is a missing verification, not an equivalence-by-construction. No fitted parameter is renamed as a prediction: the empirical claims are comparisons against external benchmarks (IFCA, FedSPD, DFedAvgM) on MNIST/EMNIST/CIFAR-10/FEMNIST. All theory citations (Ghosh et al. 2021; Lian et al. 2017; Koloskova et al. 2019a; Boyd et al. 2006; Nedic and Olshevsky 2016) are external, and no load-bearing self-citation or imported uniqueness theorem appears. The LLM note concerns proof drafting and is not a circular step. The substantive gap - Eq. 10 models full-exchange doubly-stochastic gossip while Algorithm 1 exchanges only the assigned cluster model, and Remark (iii) asserts without proof that non-trainers mixing theta_{i,j} preserves averages - is a correctness/rigor problem outside the circularity definition under hard rule 1, so it does not raise the circularity score.
Axiom & Free-Parameter Ledger
free parameters (4)
- number of clusters k =
k = 2 or 4 per dataset
- learning rate γ =
0.1 (MNIST/EMNIST/FEMNIST), 0.25 (CIFAR-10)
- graph connection probability p =
0.15 (EMNIST), 0.1 (MNIST), 0.2 (CIFAR-10/FEMNIST)
- local epochs τ and global iterations T =
τ = 5, T = 150
axioms (7)
- domain assumption A1: Each F_client(·, D_i) is L-smooth.
- domain assumption A2: Unbiased stochastic gradients with bounded variance σ².
- domain assumption A3 (sync): A symmetric, doubly-stochastic W respecting G with spectral gap 1−λ > 0.
- ad hoc to paper A3 (async): Over any B consecutive rounds disagreement contracts by λ̃.
- domain assumption A4: Each cluster objective satisfies PL or convexity.
- domain assumption A5: Separability margin δ for correct cluster assignment (Eq 11).
- ad hoc to paper The analyzed full-model-exchange gossip (Eq 10) represents the implemented single-model exchange (Eq 7–8, Algorithm 1).
read the original abstract
Clustered Federated Learning has emerged as an effective approach for handling heterogeneous data across clients by partitioning them into clusters with similar or identical data distributions. However, most existing methods, including the Iterative Federated Clustering Algorithm (IFCA), rely on a central server to coordinate model updates, which creates a bottleneck and a single point of failure, limiting their applicability in more realistic decentralized learning settings. In this work, we introduce DFCA, a fully decentralized clustered FL algorithm that enables clients to collaboratively train cluster-specific models without central coordination. DFCA uses a sequential running average to aggregate models from neighbors as updates arrive, providing a communication-efficient alternative to batch aggregation while maintaining clustering performance. Our experiments on various datasets demonstrate that DFCA outperforms other decentralized algorithms and performs comparably to centralized IFCA, even under sparse connectivity, highlighting its robustness and practicality for dynamic real-world decentralized networks.
Figures
Reference graph
Works this paper leans on
-
[4]
URL https://arxiv.org/abs/1812.01097. Ken Chang, Niranjan Balachandar, Carson Lam, Darvin Yi, James Brown, Andrew Beers, Bruce Rosen, Daniel Rubin, and Jayashree Kalpathy-Cramer. Distributed deep learning networks among institutions for medical imaging.Journal of the American Medical Informatics Association : JAMIA, 25, 03
-
[5]
Vishnu Pandi Chellapandi, Antesh Upadhyay, Abolfazl Hashemi, and Stanislaw H
doi: 10.1093/jamia/ocy017. Vishnu Pandi Chellapandi, Antesh Upadhyay, Abolfazl Hashemi, and Stanislaw H. ˙Zak. De- centralized federated learning: Model update tracking under imperfect information sharing. In2024 IEEE International Conference on Big Data (BigData), pp. 7697–7706,
-
[10]
Istv´an Heged˝us, G´abor Danner, and M´ark Jelasity
doi: 10.1109/TBDATA.2022.3167994. Istv´an Heged˝us, G´abor Danner, and M´ark Jelasity. Gossip learning as a decentralized alternative to federated learning. In Jos ´e Pereira and Laura Ricci (eds.),Distributed Applications and Interop- erable Systems, pp. 74–90, Cham,
arXiv 2022
-
[11]
URLhttps://arxiv.org/abs/1908.07782. Yixing Huang, Christoph Bert, Stefan Fischer, Manuel Schmidt, Arnd D ¨orfler, Andreas Maier, Rainer Fietkau, and Florian Putz. Continual learning for peer-to-peer federated learning: A study on automated brain metastasis identification,
Pith/arXiv arXiv 1908
-
[13]
Anastasia Koloskova, Sebastian U. Stich, and Martin Jaggi. Decentralized stochastic optimization and gossip algorithms with compressed communication.CoRR, abs/1902.00340, 2019b. URL http://arxiv.org/abs/1902.00340. Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images.Tech- nical Report,
Pith/arXiv arXiv 1902
-
[14]
Peer-to-peer federated learning on graphs.arXiv preprint arXiv:1901.11173,
Anusha Lalitha, Osman Cihan Kilinc, Tara Javidi, and Farinaz Koushanfar. Peer-to-peer federated learning on graphs.arXiv preprint arXiv:1901.11173,
Pith/arXiv arXiv 1901
-
[15]
I-Cheng Lin, Osman Yagan, and Carlee Joe-Wong
URLhttps://arxiv.org/abs/1710.06952. I-Cheng Lin, Osman Yagan, and Carlee Joe-Wong. FedSPD: A soft-clustering approach for per- sonalized decentralized federated learning. InThe 41st Conference on Uncertainty in Artificial Intelligence,
-
[17]
Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh
URLhttps://arxiv.org/abs/2101.01995. Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh. Three approaches for personalization with applications to federated learning,
-
[18]
Othmane Marfoq, Chuan Xu, Giovanni Neglia, and Richard Vidal
URLhttps://arxiv.org/ abs/2002.10619. Othmane Marfoq, Chuan Xu, Giovanni Neglia, and Richard Vidal. Throughput-optimal topol- ogy design for cross-silo federated learning,
Pith/arXiv arXiv 2002
-
[19]
URLhttps://arxiv.org/abs/2010. 12229. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Aarti Singh and Jerry Zhu (eds.),Proceedings of the 20th International Conference on Artifi- cial Intelligence and Statistics, volume 54 ofProceedings of...
2010
-
[21]
Christodoulos Pappas, Dimitris Chatzopoulos, Spyros Lalis, and Manolis Vavalis
doi: 10.1109/INFOCOM.2019.8737602. Christodoulos Pappas, Dimitris Chatzopoulos, Spyros Lalis, and Manolis Vavalis. Ipls : A frame- work for decentralized federated learning,
arXiv 2019
-
[22]
Yichen Ruan and Carlee Joe-Wong
URL https://arxiv.org/abs/1905.06731. Yichen Ruan and Carlee Joe-Wong. Fedsoft: Soft clustered federated learning with proximal local updating,
Pith/arXiv arXiv 1905
-
[23]
Felix Sattler, Klaus-Robert M ¨uller, and Wojciech Samek
URLhttps://arxiv.org/abs/2112.06053. Felix Sattler, Klaus-Robert M ¨uller, and Wojciech Samek. Clustered federated learning: Model- agnostic distributed multi-task optimization under privacy constraints,
-
[24]
URLhttps:// arxiv.org/abs/1910.01991. Micah Sheller, G. Reina, Brandon Edwards, Jason Martin, and Spyridon Bakas.Multi-institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revis...
Pith/arXiv arXiv 1910
-
[25]
doi: 10.1007/978-3-030-11723-8
ISBN 978-3-030-11722-1. doi: 10.1007/978-3-030-11723-8
-
[26]
Yandong Shi, Yong Zhou, and Yuanming Shi
doi: 10.1038/s41598-020-69250-1. Yandong Shi, Yong Zhou, and Yuanming Shi. Over-the-air decentralized federated learning,
-
[27]
Tao Sun, Dongsheng Li, and Bao Wang
URLhttps://arxiv.org/abs/2106.08011. Tao Sun, Dongsheng Li, and Bao Wang. Decentralized federated averaging,
-
[28]
Yan Sun, Li Shen, and Dacheng Tao
URLhttps: //arxiv.org/abs/2104.11375. Yan Sun, Li Shen, and Dacheng Tao. Which mode is better for federated learning? centralized or decentralized,
-
[29]
Shuai Wang, Dan Li, Jinkun Geng, Yue Gu, and Yang Cheng
doi: 10.1109/TSP.2022.3212536. Shuai Wang, Dan Li, Jinkun Geng, Yue Gu, and Yang Cheng. Impact of network topology on the performance of dml: Theoretical analysis and practical factors. InIEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp. 1729–1737,
arXiv 2022
-
[30]
doi: 10.1109/INFOCOM.2019. 8737595. Liangqi Yuan, Yunsheng Ma, Lu Su, and Ziran Wang. Peer-to-peer federated continual learning for naturalistic driving action recognition,
-
[32]
running average
dataset. The training is done onN= 400clients, who each get data from one distinct writer, withk= 4clusters with a graph connection probability of 0.2and all other parameters equal to the MNIST experiments. B CONVERGENCEANALYSIS We provide a proof template that reuses standard ingredients from clustered FL (e.g., Ghosh et al. (2021)) for the assignment an...
2021
-
[2006]
doi: 10.1109/TIT.2006. 874516. Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Koneˇcn´y, H. Brendan McMa- han, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings,
-
[2009]
We use a simple Multilayer Perceptron (MLP) with one hidden layer of size 2048 followed by a relu activation func- tion
is conducted onN= 240clients andk= 4clusters and data distributions (0, 90, 180, 270 degree rotations). We use a simple Multilayer Perceptron (MLP) with one hidden layer of size 2048 followed by a relu activation func- tion. The other training parameters stay consistent with the EMNIST experimental setting, with the exception of reducing the connection pr...
2048
-
[2016]
See also network optimization surveys (2018+)
doi: 10.1109/TAC.2016.2529644. See also network optimization surveys (2018+). 11 Preprint. Under Review. Giovanni Neglia, Gianmarco Calbi, Don Towsley, and Gayane Vardoyan. The role of network topology for distributed machine learning. InIEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp. 2350–2358,
arXiv 2016
-
[2017]
Omar El-Rifai, Michael Ben Ali, Imen Megdiche, Andr ´e Peninou, and Olivier Teste
URLhttps://arxiv.org/abs/1702.05373. Omar El-Rifai, Michael Ben Ali, Imen Megdiche, Andr ´e Peninou, and Olivier Teste. A survey on cluster-based federated learning,
-
[2018]
Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah
URLhttps://arxiv.org/abs/1705.08435. Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. Randomized gossip algorithms. IEEE Transactions on Information Theory, 52(6):2508–2530,
-
[2019]
Aur´elien Bellet, Rachid Guerraoui, Mahsa Taziki, and Marc Tommasi
URLhttps://arxiv.org/abs/1811.10792. Aur´elien Bellet, Rachid Guerraoui, Mahsa Taziki, and Marc Tommasi. Personalized and private peer-to-peer machine learning,
-
[2020]
URLhttps://arxiv. org/abs/2012.03788. Anastasia Koloskova, Sebastian U. Stich, and Martin Jaggi. Decentralized stochastic optimization and gossip algorithms with compressed communication. InProceedings of the 36th International Conference on Machine Learning (ICML), 2019a. Extended version in JMLR,
Pith/arXiv arXiv 2012
-
[2021]
Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, and Xiaojiang Chen
URLhttps://arxiv.org/abs/2006.04088. Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, and Xiaojiang Chen. Adaptive client clustering for efficient federated learning over non-iid and imbalanced data.IEEE Transactions on Big Data, 10(6):1051–1065,
Pith/arXiv arXiv 2006
-
[2022]
doi: 10.1007/s11280-022-01046-x
ISSN 1573-1413. doi: 10.1007/s11280-022-01046-x. URLhttp://dx.doi. org/10.1007/s11280-022-01046-x. Francesco Malandrino and Carla Fabiana Chiasserini. Federated learning at the network edge: When not all nodes are created equal,
-
[2024]
Shuzhen Chen, Dongxiao Yu, Yifei Zou, Jiguo Yu, and Xiuzhen Cheng
doi: 10.1109/BigData62323.2024.10825274. Shuzhen Chen, Dongxiao Yu, Yifei Zou, Jiguo Yu, and Xiuzhen Cheng. Decentralized wireless federated learning with differential privacy,
arXiv 2024
-
[2025]
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran
URLhttps://arxiv.org/abs/2501.17512. Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.