REVIEW 5 major objections 5 minor 21 references
The paper claims that clustered federated learning can be made fully automatic by triggering one-shot client clustering the first time a temperature-like scalar starts rising, and that this beats prior methods.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
OCFL automatically picks the clustering round by detecting a rise in the p-norm of the pairwise cosine-distance matrix of client gradients, and with density-based clustering it recovers client cohorts earlier and more accurately than prior methods.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection A practical, hyperparameter-light CFL method with honest limitations, but the evidence would be a lot stronger with repeated runs. the 5 major comments →
One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
OCFL is a one-shot, hyperparameter-free clustering algorithm for federated learning. It runs ordinary FedOpt training, computes the divergence matrix Γ whose entries are cosine distances between client gradient deltas, and defines a temperature T(Γ) as the scaled p-norm of Γ. The algorithm clusters the client population exactly once, at the earliest round where the temperature increases relative to the previous round—either after an initial fall or from the very beginning. The paper's central empirical claim is that this trigger fires early and reliably: across five datasets, four data-split types, and 15 or 30 clients, OCFL combined with HDBSCAN or Mean Shift achieves near-perfect cluster r
What carries the argument
The Divergence Matrix Γ, whose (i,j) entry is the cosine distance between the flattened gradient updates of clients i and j, and the Clustering Temperature Function T(Γ) = ||Γ||_p / λ, where λ normalises by the maximal divergence constant. The temperature condenses the full pairwise client-disagreement structure into one time series; the algorithm fires when T rises above its previous value. The clustering itself is delegated to a plug-in clustering algorithm, with density-based methods HDBSCAN and Mean Shift carrying the main empirical results because they infer the number of clusters automatically.
Load-bearing premise
The whole method rests on the heuristic that the first round in which the temperature scalar rises is the right moment to cluster; the paper acknowledges that individual runs can show different temperature patterns and that a moving-average detector would be more stable.
What would settle it
Run OCFL with HDBSCAN on a synthetic federated task where three client subgroups are statistically identical for the first several rounds and only diverge later because of a delayed feature shift. If the temperature profile stays flat or noisy early, the first-rise trigger will fire before the subgroups are distinguishable, and the recovered Rand Index will remain near chance while a later fixed-round clustering recovers the true partition.
If this is right
- A federated learning practitioner no longer needs to pre-specify a clustering round, the number of clusters, or a distance threshold; OCFL supplies the trigger and density-based clustering infers the partition.
- Because clustering fires in the first 2–9 rounds, most of the training budget is spent on personalised in-cluster models, which is where the measured F1 gains come from.
- Density-based clustering on full-gradient cosine distances is empirically strong, challenging the narrative that high-dimensional gradient spaces require dimensionality reduction before clustering.
- Personalised cluster models improve local test performance without sacrificing much global generalisation, as measured by the learning gap between local and orchestrator test sets.
- Local explainability improves with correct personalisation: insertion and deletion scores of saliency maps are better for OCFL-personalised models than for the baseline or the compared CFL methods.
Where Pith is reading between the lines
- The single-round temperature-rise trigger is the least protected part of the method; the paper itself notes that individual runs can show different temperature patterns, so a moving-average version would likely fire more robustly at the cost of a slightly later clustering round.
- The same temperature signal could serve as a general diagnostic for non-IID severity in federated learning, predicting when federated averaging will struggle, even in settings where clustering is not the goal.
- In a dynamic client population, OCFL naturally becomes few-shot clustering: after the one-shot split, new clients must be assigned to existing clusters by comparing their gradients with the learned cluster structure, but the paper does not evaluate this scenario.
- The cosine-distance temperature is tested only on full neural-network parameter updates; an obvious extension is to test whether the same trigger works on lower-dimensional embeddings or on non-vision modalities, where the geometry of gradient updates may differ.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes One-Shot Clustered Federated Learning (OCFL), a method that performs a single clustering step early in federated training without requiring the number of clusters a priori. It computes the pairwise cosine distance between client gradient updates, forms a Divergence Matrix, and defines a Temperature Function T(Γ) as the p-norm of that matrix. The algorithm triggers clustering at the first round in which temperature increases (Algorithm 1, line 20), under the empirical claim that this point marks the earliest suitable moment to cluster. OCFL is combined with HDBSCAN, Mean Shift, and Affinity Propagation, and compared against SCL and BCL baselines plus an idealized K-means variant, across five datasets, four split types, and 15/30 clients (280 runs total). The paper reports high RAND/AMI/COM scores for OCFL-HDB and OCFL-MS, improved personalized F1, and a novel evaluation of cluster-level explainability using insertion/deletion metrics.
Significance. If the central claim holds, OCFL would be a useful practical contribution: a simple, clustering-agnostic rule for deciding when to cluster in FL, requiring no predefined cluster count and working with standard density-based clustering. The paper also provides a substantial empirical study spanning 40 tasks, releases code and numerical results, and extends evaluation to explainability, which is a relatively unexplored direction. The reported gains are large (RAND/AMI often above 0.9) and the algorithmic idea is simple enough to be adopted by practitioners. However, the current evidence does not yet establish the load-bearing claims. The evaluation is based on single runs with no variance measures, the 'hyperparameter-free' assertion is undermined by unspecified/ hand-set parameters (norm order p, HDBSCAN min_cluster_size), and the temperature trigger is explicitly conceded in §6.3 to be fragile in individual runs. These issues are fixable with additional experiments, but they are central to the paper's main claims.
major comments (5)
- [§4.1, Tables 1–2; Figure 2] The paper reports 280 runs, which is one run per scenario-algorithm cell. Tables 1 and 2 show single point estimates with no standard deviations, confidence intervals, or significance tests. Since the abstract claims OCFL 'achieves better results than State-of-the-Art CFL algorithms', the evidence must be shown to be stable across random seeds. The currently reported values could be seed-specific, especially for clustering round selection where the temperature trigger is the only deciding factor. I request multi-seed repetitions (at least 5–10 seeds) and, where appropriate, paired significance tests (e.g., Wilcoxon).
- [Definition 7; Algorithm 1 line 17; §4.2] The paper repeatedly describes OCFL as 'free of any hyperparameters' (Abstract, §1, §4.6), but the Temperature Function depends on an unspecified norm order p in Definition 7, and no sensitivity analysis or default value is provided. Moreover, HDBSCAN's min_cluster_size is hand-set to 20% of the sample size (§4.2), and the algorithm requires a clustering algorithm A as input. The claim that no hyperparameters need adjustment is therefore not supported. At minimum, the authors should state the chosen p, justify it, and show robustness to p and to min_cluster_size.
- [Algorithm 1 line 20; §6.3] The temperature trigger is a single-round comparison: clustering is performed at the first round where T_t ≥ T_{t−1}. Since clustering occurs exactly once and there is no correction mechanism, a false early trigger freezes a possibly incorrect partition. The authors themselves note in §6.3 that individual runs exhibit 'slightly different patterns' and that a moving-average detector 'would be more robust than the static switch condition'. The paper does not quantify how often the single-round trigger selects a suboptimal round, nor does it report the accuracy of the trigger across runs. This is the load-bearing component of the 'automatically detect the earliest suitable moment' claim and requires direct evaluation, e.g., distribution of clustering rounds and comparison against an oracle/ moving-average trigger.
- [§4.3, §4.4] The evaluation protocol explicitly rewards early clustering: §4.3 states 'Computing the score each round rewards clustering algorithms that can detect the correct structure at an early stage', and §4.4 averages F1 scores over all training rounds. This is a circular bias in favor of OCFL's objective (early one-shot clustering) and against post-processing baselines such as SCL, which intentionally cluster later. As a result, the aggregated RAND/AMI and PF1 comparisons conflate clustering-quality with clustering-time, making it difficult to attribute the reported gains to better final partitions rather than to earlier clustering. The authors should also report scores at a fixed evaluation round (e.g., the final round) or after each algorithm's chosen clustering round.
- [§4.2, Table 1] The baselines SCL and BCL are only partially compared: SCL fails to cluster on CIFAR10, PathMNIST, and BloodMNIST, and BCL returns empty entries for several settings. While the authors describe hyperparameter tuning efforts, the comparison is uneven. The strong claims 'better results than State-of-the-Art CFL algorithms' are based on scenarios where the baselines effectively did not run. The paper should either include more competitive baseline implementations that work on the tested datasets or limit the comparative claim to the settings where the baselines were functional.
minor comments (5)
- [Definition 6] Definition 6 states 'Divergence Matrix Θ ∈ R^{n×n}' but the intended symbol is Γ; the matrix is written as Γ in Definition 7 and elsewhere. Please fix the inconsistent notation.
- [Algorithm 1 line 17] The line 'λ ← p√(n(n−1)2p)' is malformed. In the text, λ is referred to as the normalizing constant 1/Γmax, but the pseudocode uses a different expression. Clarify the exact definition and make it consistent with Definition 7.
- [Figures 3 and 4 and Appendix A.3] The captions in Appendix A.3 state 'Iterations are placed on the y-axis, while the value of the monitored function is placed on the x-axis', which appears reversed relative to the plotted figures. Please check and correct the axis descriptions.
- [§4.2 and Table 3] The description of the local optimizer for PathMNIST and BloodMNIST mentions Adam with very small learning rates, and Table 3 reports these. However, the text says 'To rule out the possibility that excessive hyperparameter fine-tuning influences the clustering results, SGD was employed'—this is inconsistent with the use of Adam for two datasets. A short explanation of why Adam was needed there would help.
- [§6.3] The moving-average extension is described as 'a special case' when window=1, but a moving average is a more general smoother, not a strict extension of the pointwise condition. Consider clarifying the relationship.
Circularity Check
No significant circularity: OCFL's temperature trigger is a parameter-free heuristic evaluated against ground-truth partitions, not a fitted input renamed as a prediction.
full rationale
The paper's central mechanism is an empirically tested heuristic: cluster once when the p-norm of the pairwise cosine-distance matrix of client gradients rises (Algorithm 1, line 20). No target cluster labels are used to set this trigger; the condition is a relative comparison (T_t >= T_{t-1}) with no fitted threshold, and the clustering uses the same gradient matrix available at that round. The phrase "earliest suitable moment" is an operational description of the trigger rather than a prediction of an externally defined optimal round, so the claim does not reduce to its inputs by construction. The authors' own Section 6.3 concedes that a moving-average detector might be more robust than the static single-round switch, but this is a fragility/robustness limitation, not a circularity. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation; baselines SCL and BCL are external algorithms compared using their authors' recommendations. The evaluation protocol's per-round averaging rewards early clustering, but that is an evaluation choice and does not make the derivation circular. Consequently, the central empirical claim remains self-contained and independently falsifiable against the ground-truth partitions used in the experiments.
Axiom & Free-Parameter Ledger
free parameters (2)
- Norm order p in Temperature Function
- HDBSCAN min_cluster_size =
20% of sample size
axioms (4)
- domain assumption Gradients (or model deltas) of clients trained on different data-generating distributions are separable by cosine distance.
- domain assumption A rise in the Temperature Function marks the earliest suitable clustering moment.
- domain assumption Density-based clustering can discover the true cohort structure in the high-dimensional gradient space.
- domain assumption The overlapping/non-overlapping balanced/imbalanced splits simulate real-world incongruent client populations.
invented entities (1)
-
Temperature Function T(Γ)
no independent evidence
Cite this review
Pith. "Pith review of One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption." pith.science (2026). https://pith.science/paper/77DWMDYP
@misc{pith2026250901587,
author = {Pith},
title = {Pith review of: One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption},
year = {2026},
howpublished = {\url{https://pith.science/paper/77DWMDYP}},
note = {Machine review of arXiv:2509.01587}
}
read the original abstract
Federated Learning (FL) is a widespread and well-adopted paradigm of decentralised learning that allows training one model from multiple sources without the need to transfer data between participating clients directly. Since its inception in 2015, it has been divided into numerous subfields that deal with application-specific issues, such as data heterogeneity or resource allocation. One such sub-field, Clustered Federated Learning (CFL), deals with the problem of clustering the population of clients into separate cohorts to deliver personalised models. Although a few remarkable works have been published in this domain, the problem remains largely unexplored, as its basic assumptions and settings differ slightly from those of standard FL. In this work, we present One-Shot Clustered Federated Learning (OCFL), a clustering-agnostic algorithm that can automatically detect the earliest suitable moment for clustering. Our algorithm is based on computing the cosine distance between the gradients of the clients and a temperature measure that detects when the federated model starts to converge. We empirically evaluate our methodology by testing various one-shot clustering algorithms for over forty different tasks on five benchmark datasets. Our experiments showcase the good performance of our approach when used to perform CFL in an automated manner without the need to adjust hyperparameters. We also revisit the practical feasibility of CFL algorithms based on the gradients of the clients, providing firm evidence of the high efficiency of density-based clustering methods when used to differentiate between the loss surfaces of neural networks trained on different distributions. Moreover, by inspecting the feasibility of local explanations generated with the help of GradCAM, we can provide more insights into the relationship between personalisation and the explainability of local predictions.
Figures
Reference graph
Works this paper leans on
-
[8]
doi: 10.1109/JIOT.2021.3113927. URL https://doi. org/10.1109/JIOT.2021.3113927. T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith. Federated opti- mization in heterogeneous networks. In MLSys. mlsys.org,
-
[12]
V. Petsiuk, A. Das, and K. Saenko. RISE: randomized input sampling for explanation of black-box models. In British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018 , page
work page 2018
-
[13]
URL http://bmvc2018.org/ contents/papers/1064.pdf. S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Koneˇ cn´ y, S. Kumar, and H. B. McMahan. Adaptive federated optimization. CoRR, abs/2003.00295,
Pith/arXiv arXiv 2003
-
[14]
org/10.1007/978-3-030-28954-6_14
URL https://doi. org/10.1007/978-3-030-28954-6_14 . D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Y. Bengio and Y. LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings ,
-
[15]
Y. Ruan and C. Joe-Wong. Fedsoft: Soft clustered federated learning with proximal local up- dating. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty- Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Vir- tual...
work page 2022
-
[17]
URL https://arxiv.org/abs/ 2106.12773. F. Sattler, K. M¨ uller, and W. Samek. Clustered federated learning: Model-agnostic dis- tributed multitask optimization under privacy constraints. IEEE Trans. Neural Networks Learn. Syst., 32(8):3710–3722,
work page internal anchor Pith review Pith/arXiv arXiv
-
[18]
doi: 10.18653/v1/2021.privatenlp-1.1
As- sociation for Computational Linguistics. doi: 10.18653/v1/2021.privatenlp-1.1. URL https://aclanthology.org/2021.privatenlp-1.1. P. Tian, W. Liao, W. Yu, and E. Blasch. WSCC: A weight-similarity-based client clustering approach for non-iid federated learning. IEEE Internet Things J. , 9(20):20243–20256,
-
[19]
J. Wang, Z. Charles, Z. Xu, G. Joshi, H. B. McMahan, B. A. y Arcas, M. Al-Shedivat, G. An- drew, S. Avestimehr, K. Daly, D. Data, S. N. Diggavi, H. Eichner, A. Gadhikar, Z. Gar- rett, A. M. Girgis, F. Hanzely, A. Hard, C. He, S. Horv´ ath, Z. Huo, A. Ingerman, M. Jaggi, T. Javidi, P. Kairouz, S. Kale, S. P. Karimireddy, J. Koneˇ cn´ y, S. Koyejo, T. Li, L...
-
[20]
Z. Wang, Y. Song, and E. Zuazua. Approximate and weighted data reconstruction attack in federated learning. CoRR, abs/2308.06822,
work page internal anchor Pith review Pith/arXiv arXiv
-
[21]
H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR, abs/1708.07747,
-
[22]
G. Zhu, D. Li, H. Gu, Y. Han, Y. Yao, L. Fan, and Q. Yang. Evaluating membership inference attacks and defenses in federated learning. CoRR, abs/2402.06289,
work page internal anchor Pith review Pith/arXiv arXiv
-
[2009]
URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf . Y. LeCun and C. Cortes. MNIST handwritten digit database. CoRR,
work page 2009
-
[2015]
URL http://arxiv.org/abs/1412.6980. A. Krizhevsky. Learning multiple layers of features from tiny images. CoRR,
-
[2017]
doi: 10.1109/ICCV.2017.371. URL https://ieeexplore. ieee.org/document/8237633. ISSN: 2380-7504. M. Fontana, F. Naretto, and A. Monreale. A new approach for cross-silo federated learning and its privacy risks. In PST, pages 1–10. IEEE,
-
[2018]
46 One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption J. Ma, G. Long, T. Zhou, J. Jiang, and C. Zhang. On the convergence of clustered federated learning. CoRR, abs/2202.06187,
work page internal anchor Pith review Pith/arXiv arXiv
-
[2019]
doi: 10.1007/978-3-030-28954-6 \
-
[2020]
Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh. Three approaches for personalization with applications to federated learning. CoRR, abs/2002.10619,
Pith/arXiv arXiv 2002
-
[2021]
R. C. Fong and A. Vedaldi. Interpretable Explanations of Black Boxes by Meaningful Perturbation. In 2017 IEEE International Conference on Computer Vision (ICCV) , pages 3449–3457, Oct
work page 2017
-
[2022]
B. Gong, T. Xing, Z. Liu, J. Wang, and X. Liu. Adaptive clustered federated learning for heterogeneous data in edge computing. Mob. Networks Appl. , 27(4):1520–1530, 2022a. B. Gong, T. Xing, Z. Liu, W. Xi, and X. Chen. Adaptive client clustering for efficient federated learning over non-iid and imbalanced data. IEEE Transactions on Big Data , pages 1–1, 2...
-
[2023]
doi: 10.1109/TNNLS.2022. 3160699. URL https://doi.org/10.1109/TNNLS.2022.3160699. O. D. Thakkar, S. Ramaswamy, R. Mathews, and F. Beaufays. Understanding unin- tended memorization in language models under federated learning. In O. Feyisetan, S. Ghanavati, S. Malmasi, and P. Thaine, editors, Proceedings of the Third Work- shop on Privacy in Natural Languag...
arXiv 2022
-
[2024]
Y. Long, V. Bindschaedler, L. Wang, D. Bu, X. Wang, H. Tang, C. A. Gunter, and K. Chen. Understanding membership inferences on well-generalized learning models. CoRR, abs/1802.04889,
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.