REVIEW 4 major objections 5 minor 1 cited by
Recommender Engine Driven Client Selection in Federated Brain Tumor Segmentation
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A recommender engine based on non-negative matrix factorization selects federated-learning clients by past performance and expertise; a new harmonic-mean aggregation (HSimAgg) blends their model updates, and the method reports…
desk verdict A plausible idea undone by missing baselines and a questionable aggregation formula. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The two load-bearing pieces are the NNMF-based recommender and the HSimAgg aggregator. The recommender decomposes a normalized client-by-metric matrix (4 metrics, 33 clients here) into two low-rank latent factors and uses the first latent component to score collaborators; the schedule alternates between the lowest and highest scorers across rounds, with a random-selection fallback for the cold start. HSimAgg computes, for each selected client, the inverse distance of its parameter vector to the unweighted average model (Eqs. 2–3), normalizes these similarity weights, blends them with sample-size weights (Eq. 4), and finally combines the updates through the harmonic-mean operation in Eq. 6, which is meant to reduce the pull of outlier models.
What would settle it
Run HSimAgg on a trained U-Net whose parameter tensors contain an exact zero (for example after pruning or ReLU dead neurons) and check whether the master model in Eq. 6 becomes NaN; if it does, the aggregation rule is not computable as stated without extra safeguards.
Extended reading notes
Core claim
The paper's central claim is that a recommender engine built on non-negative matrix factorization can rank federated collaborators from a compact matrix of performance and participation metrics, and that alternating between the lowest-ranked and highest-ranked fifth of collaborators across rounds produces faster, more stable convergence than batch-wise random selection. Aggregating the selected models with HSimAgg, which weights each collaborator by its harmonic similarity to the unweighted average model and by its local sample size, then suppresses divergent parameter updates. The reported external-validation Dice scores—0.7298 for enhancing tumor, 0.7424 for tumor core, and 0.8218 for whole tumor—are presented as evidence that task-aligned collaborator selection improves federated brain tumor segmentation.
Load-bearing premise
The load-bearing premise is that the aggregation in Eq. 6, $p_m = \sum_c w_c p_c \,/\, \sum_c (w_c/p_c)$, is defined for real neural-network parameters; if any parameter is zero or negative, the division is undefined or the harmonic mean becomes erratic.
Editorial extensions
If this is right
- Federated segmentation can operate with only 20% of sites contributing in each round while still achieving competitive Dice scores.
- New or seldom-seen clients can join through a random-selection fallback, addressing the cold-start problem without requiring prior performance history.
- Because HSimAgg weights down parameter vectors that lie far from the average, training becomes less sensitive to outlier or corrupted client updates.
- The NNMF-based selection scheme generalizes to other federated tasks wherever per-client validation metrics such as Dice and loss are logged.
Reading between the lines
- The paper does not include a head-to-head comparison against random selection or standard FedAvg on the same data split, so the claimed improvement in precision is supported by convergence curves and final scores rather than an ablation; a controlled comparison would separate the contribution of selection from that of aggregation.
- Equation 6 divides element-wise by model parameters, and if any parameter is zero the reciprocal is undefined; negative parameters would also make a harmonic mean behave unexpectedly, so a reliable deployment would need clipping or an absolute-value-based similarity.
- The odd-round policy of selecting the bottom 20% could slow convergence in highly heterogeneous federations, since consistently poor clients are forced in every other round; annealing the exploration rate over time would be a natural extension.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a client-selection protocol for federated learning (FL) that uses non-negative matrix factorization (NNMF) on historical client metrics to rank collaborators, alternating between top-20% and bottom-20% selection in successive rounds, together with a new aggregation rule called HSimAgg (harmonic similarity weighted aggregation). The method is evaluated on the FeTS 2022 brain tumor segmentation data (1,251 training mpMRI scans, 219 external validation scans) across 33 simulated collaborators. The reported external validation Dice scores are 0.7298 for enhancing tumor, 0.7424 for tumor core, and 0.8218 for whole tumor after 20 communication rounds. The central claim, stated in the abstract and Section 1, is that the proposed approach significantly improves the precision and efficiency of federated learning for this task.
Significance. If the comparative claims were substantiated, the paper would offer a useful contribution to client selection in federated medical FL, a topic of active interest. The paper has some strengths: it uses a public benchmark (FeTS/BraTS), reports wall-clock time, memory, and energy consumption, provides pseudocode for the aggregation algorithm, and evaluates on an external validation cohort. However, the significance is not yet established. The experimental section reports only the proposed configuration; there is no comparison to FedAvg, random client selection, the authors' prior SimAgg method [8], or any other selection policy. In addition, the aggregation equation labeled as a harmonic mean is not a harmonic mean and involves element-wise division by model parameters, which are not guaranteed to be nonzero or positive. Because both the selection and aggregation components are load-bearing for the claimed improvement, the current evidence is insufficient to support the abstract's 'significantly improves' statement.
major comments (4)
- [§3, Tables 2 and 3] The central comparative claim is unsupported because no baseline or ablation is reported. Table 2 and Table 3 contain only the HSimAgg column, and Figure 2 shows training curves for one configuration. There is no comparison to random 20% client selection, to plain FedAvg, to the authors' earlier SimAgg algorithm [8], or to a variant without NNMF selection. Since client selection and data partitioning are stochastic, a single run without error bars or statistical testing cannot support the word 'significantly' used in the abstract and Section 1. The authors should report results for at least FedAvg with random selection, random selection with HSimAgg, and the previous SimAgg selection policy, with multiple seeds and variance estimates.
- [§2.2, Eq. (6)] Equation (6) does not define the harmonic mean and is not a well-defined operation on neural-network parameter vectors. A weighted harmonic mean of values p_c with weights w_c is (Σ w_c) / (Σ w_c / p_c), while Eq. (6) computes (1 / Σ (w_c / p_c)) · Σ (w_c · p_c), which is the ratio of a reciprocal sum and a weighted sum. More seriously, the expression divides element-wise by model parameters p_c, which can be zero or negative in a trained neural network, making the aggregated model undefined or numerically unstable. Since HSimAgg is a named contribution and its correctness depends on this equation, the authors must either correct the formula to an actual harmonic-style aggregation over parameter vectors with a well-defined positive transformation, or remove the harmonic-mean claim and describe a mathematically consistent aggregation.
- [§2.2, Recommender Engine Collaborator Selection] The selection algorithm is underspecified at load-bearing points. The text states that NNMF with two components is applied to a 33×4 matrix, that the first latent variable is used, and that odd rounds select the bottom 20% while even rounds select the top 20%, but it never defines how the NNMF scores are converted into a ranking, how the 'exploration and exploitation' weighting is computed, or how the fallback random selection is triggered beyond 'does not yield a result.' The alternating top/bottom schedule is also not justified; selecting the bottom 20% in half of all rounds appears to deliberately choose poor performers and may harm convergence. A precise algorithmic specification of the selection rule and an ablation showing its effect on the final Dice scores are needed.
- [§3.1, Table 1] Several free hyperparameters of the proposed method are not reported or analyzed. Table 1 lists learning rate, epochs per round, and communication rounds, but not the NNMF latent dimension (set to 2 in the text), the selection fraction (set to 20%), the epsilon in Eq. (2), or the optimizer and loss function. Since the paper claims that NNMF-based selection improves results, a sensitivity analysis over the latent dimension and selection fraction is necessary to show that the reported performance is not an artifact of a single favorable setting.
minor comments (5)
- [§2.2, Eqs. (2)–(5)] The mathematical notation in the similarity-weight equations is inconsistent: Eq. (2) introduces a quantity written as 's im' with subscripts, while Eq. (3) and Algorithm 1 use different symbols for the same quantity. Please unify the notation and ensure every symbol is defined before use.
- [§3.2] The phrase 'collaborator selection via reinforcement approaches' is inaccurate, because no reinforcement learning is used in the paper. This should be reworded to describe the NNMF-based recommender selection.
- [§2.1] The preprocessing description lists both 'brain extraction' and 'skull stripping,' which appear to be redundant; please clarify whether these are distinct steps or the same step described twice.
- [§2.2, HSimAgg] The text calls HSimAgg 'an advancement of the SimAgg algorithm' but does not state what HSimAgg changes relative to SimAgg beyond the selection mechanism. A brief comparison to the SimAgg formulation in [8] would help the reader assess the novelty.
- [General] The paper would benefit from a reproducibility statement indicating whether the code, model weights, and experimental configuration are publicly available.
Circularity Check
No significant circularity: NNMF client selection uses historical per-collaborator metrics as inputs, HSimAgg is defined in-paper, and final evaluation is on an external validation set.
full rationale
The paper's derivation chain is a proposed method, not a mathematical reduction. The recommender selection step takes a 33×4 matrix of historical metrics (Dice, loss, participation frequency, total contribution time) from previous FL rounds and factorizes it with NNMF (k=2) to rank collaborators (Section 2). The aggregation step defines HSimAgg explicitly in Algorithm 1 and Equations 1–6, starting from the selected collaborators' parameters and sample sizes. The final model is evaluated on 219 external validation scans (Section 3.2), which are not the same records used to form the historical selection matrix; the selection therefore does not fit the reported outcome. The self-citation of the authors' prior work [8] for the earlier SimAgg and batch-wise selection is present but not load-bearing: HSimAgg's aggregation rule is stated and testable on its own, and no uniqueness or correctness theorem is imported from [8] to forbid alternatives. The absence of baselines or ablations, and the element-wise division by model parameters in Eq. 6, are evaluation and numerical-robustness concerns rather than circularity. No step in the paper reduces, by construction or by fitted input, its claimed prediction to its own inputs.
Assumptions & free parameters
free parameters (4)
- NNMF latent dimension =
2
- Selection fraction =
20% (top/bottom)
- epsilon in Eq. 2 =
1e-5
- Learning rate =
5e-5
assumptions (3)
- domain assumption Historical validation metrics (Dice, loss, frequency, time) are predictive of future collaborator utility
- ad hoc to paper Harmonic mean of model parameter vectors is well-defined
- domain assumption NNMF with rank 2 captures meaningful latent patterns in collaborator behavior
Cite this review
Pith. "Pith review of Recommender Engine Driven Client Selection in Federated Brain Tumor Segmentation." pith.science (2026). https://pith.science/paper/QWCRTFUN
@misc{pith2026241220250,
author = {Pith},
title = {Pith review of: Recommender Engine Driven Client Selection in Federated Brain Tumor Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/QWCRTFUN}},
note = {Machine review of arXiv:2412.20250}
}
read the original abstract
This study presents a robust and efficient client selection protocol designed to optimize the Federated Learning (FL) process for the Federated Tumor Segmentation Challenge (FeTS 2024). In the evolving landscape of FL, the judicious selection of collaborators emerges as a critical determinant for the success and efficiency of collective learning endeavors, particularly in domains requiring high precision. This work introduces a recommender engine framework based on non-negative matrix factorization (NNMF) and a hybrid aggregation approach that blends content-based and collaborative filtering. This method intelligently analyzes historical performance, expertise, and other relevant metrics to identify the most suitable collaborators. This approach not only addresses the cold start problem where new or inactive collaborators pose selection challenges due to limited data but also significantly improves the precision and efficiency of the FL process. Additionally, we propose harmonic similarity weight aggregation (HSimAgg) for adaptive aggregation of model parameters. We utilized a dataset comprising 1,251 multi-parametric magnetic resonance imaging (mpMRI) scans from individuals diagnosed with glioblastoma (GBM) for training purposes and an additional 219 mpMRI scans for external evaluations. Our federated tumor segmentation approach achieved dice scores of 0.7298, 0.7424, and 0.8218 for enhancing tumor (ET), tumor core (TC), and whole tumor (WT) segmentation tasks respectively on the external validation set. In conclusion, this research demonstrates that selecting collaborators with expertise aligned to specific tasks, like brain tumor segmentation, improves the effectiveness of FL networks.
Figures
Forward citations
Cited by 1 Pith paper
-
Similarity Weighted Aggregation with Global Differential Privacy for Federated Brain Lesion Segmentation
DP-SimAgg claims per-round (epsilon, delta)-DP for federated brain tumor segmentation by adding Gaussian noise after similarity-weighted aggregation, but the noise scale relies on an empirically estimated sensitivity ...
Reference graph
Works this paper leans on
-
[8]
Khan, M.I., Jafaritadi, M., Alhoniemi, E., Kontio, E., Khan, S.A.: Adaptive weight aggregation in federated learning for brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. pp. 455–469. Springer International Publishing, Cham (2022)
work page 2022
-
[1]
arXiv preprint arXiv:2107.02314 (2021)
Baid, U., Ghodasara, S., Mohan, S., Bilello, M., Calabrese, E., Colak, E., Farahani, K., Kalpathy-Cramer, J., Kitamura, F.C., Pati, S., et al.: The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classifica- tion. arXiv preprint arXiv:2107.02314 (2021)
arXiv 2021
-
[2]
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Farahani, K., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the tcga-gbm collection. the cancer imaging archive. Nat Sci Data 4, 170117 (2017)
work page 2017
-
[3]
The cancer imaging archive286 (2017)
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Farahani, K., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the tcga-lgg collection. The cancer imaging archive286 (2017)
work page 2017
-
[4]
Scientific data 4(1), 1–13 (2017)
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data 4(1), 1–13 (2017)
2017
-
[5]
Bakas, S., Pati, S., Sheller, M., Karargyris, A., Mattson, P., Edwards, B., Baid, U., Chen, Y., Shinohara, R.T., Martin, J., Menze, B., Zenk, M., Maier-Hein, K., Floca, R., Reinke, A., Maier-Hein, L., Isensee, F., Zimmerer, D., Chen, Y.: The Federated Tumor Segmentation (FeTS) Challenge 2022 (Mar 2022).https://doi. org/10.5281/zenodo.6362409
-
[6]
International Journal of Machine Learning and Cybernetics pp
Ji, S., Tan, Y., Saravirta, T., Yang, Z., Liu, Y., Vasankari, L., Pan, S., Long, G., Walid, A.: Emerging trends in federated learning: From model fusion to federated x learning. International Journal of Machine Learning and Cybernetics pp. 1–22 (2024)
work page 2024
-
[7]
Foundations and Trends® in Machine Learning 14(1–2), 1–210 (2021)
Kairouz, P., McMahan, H.B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A.N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al.: Advances and open problems in federated learning. Foundations and Trends® in Machine Learning 14(1–2), 1–210 (2021)
work page 2021
Show all 20 references
-
[9]
IEEE signal processing magazine37(3), 50–60 (2020)
Li, T., Sahu, A.K., Talwalkar, A., Smith, V.: Federated learning: Challenges, meth- ods, and future directions. IEEE signal processing magazine37(3), 50–60 (2020)
2020
-
[10]
In: Artificial intelligence and statistics
McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics. pp. 1273–1282. PMLR (2017)
2017
-
[11]
IEEE transactions on medical imaging 34(10), 1993–2024 (2014) 12 Khan et al., 2024
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34(10), 1993–2024 (2014) 12 Khan et al., 2024
2014
-
[12]
In: ICC 2019-2019 IEEE international conference on communications (ICC)
Nishio, T., Yonetani, R.: Client selection for federated learning with heterogeneous resources in mobile edge. In: ICC 2019-2019 IEEE international conference on communications (ICC). pp. 1–7. IEEE (2019)
2019
-
[13]
arXiv preprint arXiv:2105.05874 (2021)
Pati, S., Baid, U., Zenk, M., Edwards, B., Sheller, M., Reina, G.A., Foley, P., Gruzdev, A., Martin, J., Albarqouni, S., et al.: The federated tumor segmentation (fets) challenge. arXiv preprint arXiv:2105.05874 (2021)
2021 arXiv
-
[14]
arXiv preprint arXiv:2105.06413 (2021)
Reina, G.A., Gruzdev, A., Foley, P., Perepelkina, O., Sharma, M., Davidyuk, I., Trushkin, I., Radionov, M., Mokrov, A., Agapov, D., et al.: Openfl: An open-source framework for federated learning. arXiv preprint arXiv:2105.06413 (2021)
2021 arXiv
-
[15]
IEEE transactions on neural networks and learning systems32(8), 3710–3722 (2020)
Sattler, F., Müller, K.R., Samek, W.: Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE transactions on neural networks and learning systems32(8), 3710–3722 (2020)
2020
-
[16]
IEEE transactions on neural net- works and learning systems31(9), 3400–3413 (2019)
Sattler, F., Wiedemann, S., Müller, K.R., Samek, W.: Robust and communication- efficient federated learning from non-iid data. IEEE transactions on neural net- works and learning systems31(9), 3400–3413 (2019)
2019
-
[17]
In: Proceedings of the 12th ACM workshop on artificial intelligence and security
Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Zhang, R., Zhou, Y.: A hybrid approach to privacy-preserving federated learning. In: Proceedings of the 12th ACM workshop on artificial intelligence and security. pp. 1–11 (2019)
2019
-
[18]
IEEE Internet of Things Journal7(7), 6360–6368 (2020)
Zhan, Y., Li, P., Qu, Z., Zeng, D., Guo, S.: A learning-based incentive mechanism for federated learning. IEEE Internet of Things Journal7(7), 6360–6368 (2020)
2020
-
[19]
arXiv preprint arXiv:1806.00582 (2018)
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018)
2018 arXiv
-
[20]
Computer Networks p
Zhu, K., Zhang, F., Jiao, L., Xue, B., Zhang, L.: Client selection for federated learning using combinatorial multi-arm bandit under long-term energy constraint. Computer Networks p. 110512 (2024)
2024
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.