REVIEW 3 major objections 6 minor 1 cited by
ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read ClusMFL keeps multimodal federated learning accurate when hospitals or patients lack MRI or PET.
desk verdict A sensible prototype-clustering extension for missing-modality federated learning, but the evaluation's fixed test mix undercuts the 'varying incompleteness' claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the global pool of cluster centers per modality-label pair. Each client runs a parameter-free first-neighbor clustering algorithm (FINCH) on its local feature embeddings for each modality and label, producing multiple cluster centers and sizes, which are uploaded, concatenated server-side, and redistributed as global pools. These centers have two jobs in local training: they act as positive anchors in a supervised contrastive loss that aligns local embeddings with global structure, and as weighted proxies for the absent modality so that a PET-only instance can still be classified through the MRI encoder and vice versa. The final piece is modality-aware aggregation, which assigns each encoder an aggregation weight proportional to the number of instances of that modality across clients, rather than a single uniform weight for the whole model.
What would settle it
Run ClusMFL on a naturally incomplete multimodal clinical cohort where the reasons for missing scans are recorded, or simulate missingness whose probability depends on the diagnosis and disease severity. If the accuracy edge over prototype-based and generative baselines disappears or reverses under such label-correlated missingness, the paper's central claim about realistic settings fails.
Extended reading notes
Core claim
The central claim is that a cluster-enhanced multimodal federated learning framework can maintain classification accuracy under realistic modality incompleteness that operates at two levels: whole clients missing one modality and individual instances missing one modality. ClusMFL builds a global pool of cluster centers from local feature embeddings for each modality-label pair using a parameter-free clustering algorithm, uses those centers in a supervised contrastive loss for feature alignment, and treats weighted cluster centers as proxy embeddings for the missing modality in a completion loss. A modality-aware aggregation step weights each encoder by the number of modality-specific instances contributed across clients. The authors argue this finer-grained representation beats class-level prototypes and generative reconstruction, with the largest margin in severe incompleteness settings.
Load-bearing premise
The evaluation creates missingness by deleting MRI or PET from originally complete scans, and the reported gains depend on that simulated incompleteness matching real clinical absence; if missingness in practice tracks disease stage, scanner type, or institutional practice, the advantage may shrink.
Editorial extensions
If this is right
- Institutions missing an entire imaging modality can still contribute to training both modality encoders, because cluster centers from other sites proxy the absent modality during local updates.
- Model performance stays comparatively stable as the fraction of single-modality instances rises, whereas prototype-based and generative baselines decline.
- The framework converges in fewer communication rounds than the baselines, reducing the coordination burden of federated training.
- Because no generative model is used to synthesize missing scans, training avoids adversarial instability and the comparatively high time cost of GAN-based completion.
Reading between the lines
- Editorial inference: Because only cluster centers, sizes, and model parameters are shared, the same cluster-proxy recipe should transfer to other modality pairs, such as CT with pathology images, without changing the privacy posture.
- Editorial inference: Using several cluster centers rather than one prototype could be exploited at inference time to output a distribution over candidate predictions, and the spread of that distribution might flag low-confidence cases for human review.
- Editorial inference: The modality-aware aggregation weights could be reused as a lightweight indicator of a client's modality mix at test time, enabling models to be selected or calibrated by expected missingness without retraining.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ClusMFL, a cluster-enhanced multimodal federated learning framework for brain imaging under modality incompleteness. The method uses FINCH clustering to build a pool of modalitity-label cluster centers, aligns local embeddings with these centers via supervised contrastive learning, approximates missing modalities with cluster centers in a completion loss, and aggregates encoder modules with modality-aware weights. Experiments on ADNI MRI/PET data with simulated client-level and instance-level missingness compare ClusMFL against FedAvg, FedProx, FedMed-GAN, FedMI, MFCPL, and PmcmFL, reporting consistent improvements across several alpha/beta settings, an ablation study, and convergence curves.
Significance. If the reported results hold, ClusMFL would be a useful practical contribution to healthcare federated learning: it addresses a realistic form of missingness that most prior work oversimplifies, avoids generative models that are unstable under scarce complete data, and leverages a parameter-free clustering algorithm (FINCH) plus a modality-aware aggregation rule. The paper studies a public dataset, reports five metrics, includes an ablation, and compares against several relevant baselines. However, the evaluation protocol has a load-bearing mismatch between training and test incompleteness, the empirical claims lack significance testing, and key implementation details are unreported, so the current evidence does not firmly establish the headline claim.
major comments (3)
- [Section IV.A and Table I] The evaluation does not measure performance at the claimed varying levels of test-time modality incompleteness. Section IV.A fixes the test set so that multimodal, PET-only, and MRI-only instances each constitute 1/3 regardless of alpha and beta, while the training distribution follows Eq. (1). For example, with alpha=0.4 and beta=0.4 the training mix is roughly 48% PET-only, 48% MRI-only, and only 4% multimodal, but the test set remains 33% multimodal. Thus Table I varies only the training-side incompleteness, and the conclusion that ClusMFL outperforms baselines 'across varying levels of modality incompleteness' conflates training-time and test-time incompleteness. Please re-evaluate with test sets whose incompleteness distribution matches each alpha/beta setting, or explicitly reframe the claims as being about robustness to training-side incompleteness.
- [Section III.E, Section IV.A, and Table I] The central empirical claim is not reproducible because several load-bearing experimental details are missing. The encoder architectures f_P and f_M are never specified, the input representation is only partially described (90 AAL-90 ROI features per modality, without dimensions or preprocessing of those features), and the values of the regularization coefficients lambda1 and lambda2 in Eq. (14) and the temperature tau in the contrastive loss are not reported. Since these coefficients directly control the balance between the three loss terms, their omission makes it impossible to assess sensitivity or to reproduce the reported numbers. Please report all hyperparameters, architectures, and preferably release code.
- [Table I and Section IV.B] The claim that ClusMFL 'consistently outperforms' all baselines is not supported by statistical evidence. Many of the reported mean differences are smaller than the standard deviations; for instance, under alpha=0.4, beta=0.2, ClusMFL achieves accuracy 56.37 +/- 4.10 versus FedMI's 54.95 +/- 2.69, and several other comparisons show overlapping intervals. No significance tests or effect sizes are reported for the five-fold results. Please add paired tests across folds (or equivalent) and discuss which differences are statistically meaningful, or soften the claim to the observed mean improvements.
minor comments (6)
- [Section IV.A] The sentence 'we first split the dataset into a training set and a test set with a ratio of 1:4' is ambiguous: it is unclear whether this means 1 part training to 4 parts test or the reverse, and the subsequent 5-fold cross-validation is not reconciled with this split. Please clarify the data partitioning.
- [Section III.B] There is a typo in the text after Eq. (4): 'cancatenated' should be 'concatenated'.
- [Section III.E] In the sentence defining the prediction vector, 'the m instances' should be 'the n_i instances' to match the notation introduced earlier.
- [Section III.C and Eq. (6)] The symbol 'direct sum' is used for concatenation of feature embeddings with multiple cluster centers, but the notation is not defined, and the extension of the label vector in Eq. (7) is described verbally rather than with clear set notation. Please define these operations explicitly.
- [Table II] The ablation table uses checkmarks to indicate which components are enabled, but the rows are hard to read and the baseline (no checkmarks) row is missing. Consider reformatting with explicit 'yes/no' entries and include the no-component baseline.
- [Section IV.D and Fig. 3] The text describes convergence in terms of 'test loss', but the figure panel labels are not described; please state what is plotted in each panel and on which axis.
Circularity Check
No circularity: the central SOTA claim is an external test-set comparison; no prediction is constructed from its own definition.
full rationale
ClusMFL's central claim is an empirical accuracy/F1/AUC comparison against FedAvg, FedProx, FedMed-GAN, FedMI, MFCPL, and PmcmFL on the ADNI test split (Table I). The model's components—FINCH cluster centers (Eq. 2), supervised contrastive loss (Eq. 8), modality-completion loss (Eqs. 10-12), and modality-aware aggregation (Eqs. 16-18)—are training objectives, not fitted parameters renamed as predictions. The reported metrics are measured on held-out test instances, and the ADNI data and baseline methods are external to the framework. The only related concern, that the test set is fixed at 1/3 multimodal, 1/3 PET-only, and 1/3 MRI-only while training incompleteness varies with alpha and beta (Section IV.A), is an evaluation-protocol mismatch about external validity, not a circularity: varying training availability is not the same as varying test-time incompleteness, but no tested number is forced by construction. No load-bearing self-citation appears; references [18] and [19] are external algorithm and objective citations. Hence the derivation chain is self-contained against external benchmarks.
Assumptions & free parameters
free parameters (4)
- lambda1 (contrastive regularization coefficient)
- lambda2 (modality completion coefficient)
- tau (contrastive temperature)
- encoder architecture hyperparameters
assumptions (4)
- domain assumption FINCH clustering produces stable, informative cluster centers from local feature embeddings under non-IID federated data.
- domain assumption Shared cluster centers do not leak private patient information and are safe to exchange.
- domain assumption Simulated modality dropping from complete ADNI instances is a faithful proxy for real clinical modality incompleteness.
- domain assumption Zero-padding the missing modality embedding before classification is a valid inference strategy.
Cite this review
Pith. "Pith review of ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis." pith.science (2026). https://pith.science/paper/HWHRADHM
@misc{pith2026250212180,
author = {Pith},
title = {Pith review of: ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/HWHRADHM}},
note = {Machine review of arXiv:2502.12180}
}
read the original abstract
Multimodal Federated Learning (MFL) has emerged as a promising approach for collaboratively training multimodal models across distributed clients, particularly in healthcare domains. In the context of brain imaging analysis, modality incompleteness presents a significant challenge, where some institutions may lack specific imaging modalities (e.g., PET, MRI, or CT) due to privacy concerns, device limitations, or data availability issues. While existing work typically assumes modality completeness or oversimplifies missing-modality scenarios, we simulate a more realistic setting by considering both client-level and instance-level modality incompleteness in this study. Building on this realistic simulation, we propose ClusMFL, a novel MFL framework that leverages feature clustering for cross-institutional brain imaging analysis under modality incompleteness. Specifically, ClusMFL utilizes the FINCH algorithm to construct a pool of cluster centers for the feature embeddings of each modality-label pair, effectively capturing fine-grained data distributions. These cluster centers are then used for feature alignment within each modality through supervised contrastive learning, while also acting as proxies for missing modalities, allowing cross-modal knowledge transfer. Furthermore, ClusMFL employs a modality-aware aggregation strategy, further enhancing the model's performance in scenarios with severe modality incompleteness. We evaluate the proposed framework on the ADNI dataset, utilizing structural MRI and PET scans. Extensive experimental results demonstrate that ClusMFL achieves state-of-the-art performance compared to various baseline methods across varying levels of modality incompleteness, providing a scalable solution for cross-institutional brain imaging analysis.
Figures
Forward citations
Cited by 1 Pith paper
-
Multimodal Online Federated Learning with Modality Missing in Internet of Things
Introduces MMO-FL, an online federated multimodal learning framework with a prototype-based algorithm, PMM, for compensating missing sensor modalities.
Reference graph
Works this paper leans on
-
[21]
Stage-wise training: An improved feature learning strategy for deep models,
E. Barshan and P. Fieguth, “Stage-wise training: An improved feature learning strategy for deep models,” in Feature extraction: modern questions and challenges . PMLR, 2015, pp. 49–59
work page 2015
-
[22]
Graphene and graphene oxide: synthesis, properties, and applications,
Y . Zhu, S. Murali, W. Cai, X. Li, J. W. Suk, J. R. Potts, and R. S. Ruoff, “Graphene and graphene oxide: synthesis, properties, and applications,” Advanced materials , vol. 22, no. 35, pp. 3906–3924, 2010
work page 2010
-
[1]
Multimodal federated learning in healthcare: a review,
J. Thrasher, A. Devkota, P. Siwakotai, R. Chivukula, P. Poudel, C. Hu, B. Bhattarai, and P. Gyawali, “Multimodal federated learning in healthcare: a review,” arXiv preprint arXiv:2310.09650 , 2023
arXiv 2023
-
[2]
Multimodal federated learning: A survey,
L. Che, J. Wang, Y . Zhou, and F. Ma, “Multimodal federated learning: A survey,” Sensors, vol. 23, no. 15, p. 6986, 2023
2023
-
[3]
A survey of multimodal federated learning: background, applications, and perspectives,
H. Pan, X. Zhao, L. He, Y . Shi, and X. Lin, “A survey of multimodal federated learning: background, applications, and perspectives,” Mul- timedia Systems, vol. 30, no. 4, p. 222, 2024
work page 2024
-
[4]
A multimodal federated learning framework for modality incomplete scenarios in healthcare,
Y . An, Y . Bai, Y . Liu, L. Guo, and X. Chen, “A multimodal federated learning framework for modality incomplete scenarios in healthcare,” in International Symposium on Bioinformatics Research and Applica- tions. Springer, 2024, pp. 245–256
work page 2024
-
[5]
Cross-modal prototype based multimodal federated learning under severely missing modality,
H. Q. Le, C. M. Thwal, Y . Qiao, Y . L. Tun, M. N. Nguyen, and C. S. Hong, “Cross-modal prototype based multimodal federated learning under severely missing modality,” arXiv preprint arXiv:2401.13898 , 2024
arXiv 2024
-
[6]
Multimodal federated learning with missing modality via prototype mask and contrast,
G. Bao, Q. Zhang, D. Miao, Z. Gong, L. Hu, K. Liu, Y . Liu, and C. Shi, “Multimodal federated learning with missing modality via prototype mask and contrast,” arXiv preprint arXiv:2312.13508 , 2023
arXiv 2023
Show all 28 references
-
[7]
Meta-baseline: Exploring simple meta-learning for few-shot learning,
Y . Chen, Z. Liu, H. Xu, T. Darrell, and X. Wang, “Meta-baseline: Exploring simple meta-learning for few-shot learning,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 9062–9071
2021
-
[8]
Free lunch for few-shot learning: Distribution calibration,
S. Yang, L. Liu, and M. Xu, “Free lunch for few-shot learning: Distribution calibration,” in International Conference on Learning Representations, 2021
2021
-
[9]
Epl: Empirical prototype learning for deep face recognition,
W. Fan, J. Wen, X. Jia, L. Shen, J. Zhou, and Q. Li, “Epl: Empirical prototype learning for deep face recognition,” 2024. [Online]. Available: https://arxiv.org/abs/2405.12447
2024 arXiv
-
[10]
Fedmed-gan: Federated domain translation on unsupervised cross- modality brain image synthesis,
J. Wang, G. Xie, Y . Huang, J. Lyu, F. Zheng, Y . Zheng, and Y . Jin, “Fedmed-gan: Federated domain translation on unsupervised cross- modality brain image synthesis,” Neurocomputing, vol. 546, p. 126282, 2023
2023
-
[11]
Client-adaptive cross-model reconstruction network for modality-incomplete multi- modal federated learning,
B. Xiong, X. Yang, Y . Song, Y . Wang, and C. Xu, “Client-adaptive cross-model reconstruction network for modality-incomplete multi- modal federated learning,” in Proceedings of the 31st ACM Interna- tional Conference on Multimedia , 2023, pp. 1241–1249
2023
-
[12]
Deep multi- modal learning with missing modality: A survey,
R. Wu, H. Wang, H.-T. Chen, and G. Carneiro, “Deep multi- modal learning with missing modality: A survey,” arXiv preprint arXiv:2409.07825, 2024
2024 arXiv
-
[13]
Generative adversarial net- works,
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial net- works,” Communications of the ACM , vol. 63, no. 11, pp. 139–144, 2020
2020
-
[14]
Ht-fed-gan: Federated generative model for decentralized tabular data synthesis,
S. Duan, C. Liu, P. Han, X. Jin, X. Zhang, T. He, H. Pan, and X. Xiang, “Ht-fed-gan: Federated generative model for decentralized tabular data synthesis,” Entropy, vol. 25, no. 1, p. 88, 2022
2022
-
[15]
Fligan: Enhancing federated learning with incomplete data using gan,
P. J. Maliakel, S. Ilager, and I. Brandic, “Fligan: Enhancing federated learning with incomplete data using gan,” 2024. [Online]. Available: https://arxiv.org/abs/2403.16930
2024 arXiv
-
[16]
Fedmac: Tackling partial-modality missing in federated learning with cross-modal aggregation and contrastive regularization,
M. D. Nguyen, T. T. Nguyen, H. H. Pham, T. N. Hoang, P. L. Nguyen, and T. T. Huynh, “Fedmac: Tackling partial-modality missing in federated learning with cross-modal aggregation and contrastive regularization,” 2024. [Online]. Available: https://arxiv.org/abs/2410.03070
2024
-
[17]
Adaptive hyper-graph aggregation for modality- agnostic federated learning,
Q. Fan and L. Shuai, “Adaptive hyper-graph aggregation for modality- agnostic federated learning,” in 2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , 2024, pp. 12 312– 12 321
2024
-
[18]
Efficient parameter- free clustering using first neighbor relations,
S. Sarfraz, V . Sharma, and R. Stiefelhagen, “Efficient parameter- free clustering using first neighbor relations,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 8934–8943
2019
-
[19]
Supervised contrastive learning,
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y . Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” 2021. [Online]. Available: https://arxiv.org/abs/2004.11362
2021 arXiv
-
[20]
The alzheimer’s disease neuroimaging initiative,
S. G. Mueller, M. W. Weiner, L. J. Thal, R. C. Petersen, C. Jack, W. Jagust, J. Q. Trojanowski, A. W. Toga, and L. Beckett, “The alzheimer’s disease neuroimaging initiative,” Neuroimaging Clinics , vol. 15, no. 4, pp. 869–877, 2005
2005
-
[23]
V oxel-based morphometry—the methods,
J. Ashburner and K. J. Friston, “V oxel-based morphometry—the methods,” Neuroimage, vol. 11, no. 6, pp. 805–821, 2000
2000
-
[24]
Automated anatom- ical labeling of activations in spm using a macroscopic anatomical parcellation of the mni mri single-subject brain,
N. Tzourio-Mazoyer, B. Landeau, D. Papathanassiou, F. Crivello, O. Etard, N. Delcroix, B. Mazoyer, and M. Joliot, “Automated anatom- ical labeling of activations in spm using a macroscopic anatomical parcellation of the mni mri single-subject brain,” Neuroimage, vol. 15, no. 1...
2002
-
[25]
Communication-efficient learning of deep networks from decentral- ized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentral- ized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273–1282
2017
-
[26]
Federated optimization in heterogeneous networks,
T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine learning and systems , vol. 2, pp. 429–450, 2020
2020
-
[27]
Unpaired image- to-image translation using cycle-consistent adversarial networks,
J.-Y . Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image- to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2223–2232
2017
-
[28]
Adam: A method for stochastic optimization,
D. P. Kingma, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.