Pith. sign in

REVIEW 4 major objections 4 minor 62 references

MMiC: Mitigating Modality Incompleteness in Clustered Federated Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MMiC claims a three-part fix for missing modalities in clustered federated learning and reports top global and personalized results on three multimodal benchmarks.

desk verdict Plausible modular framework for missing modalities in clustered FL, but the reported gains are uninterpretable until the authors disclose whether the per-round returns that drive BPI and MPO were computed on the test set. read the letter →

arxiv 2505.06911 v3 pith:4TZLC5MY submitted 2025-05-11 cs.LG cs.AI

Lishan Yang , Wei Emma Zhang , Quan Z. Sheng , Lina Yao , Weitong Chen , Ali Shakeri This is my paper · ORCID
classification cs.LGcs.AI
keywords federatedlearningmultimodalmissingmodalitiesclusteredclientselectionBanzhafPowerIndexMarkowitzportfoliooptimizationpersonalized
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning over multimodal data usually assumes every client has every modality; in practice some clients lack text, images, or sensor channels, and existing fixes either reconstruct the missing modality or require task-specific architectures. MMiC tries to make the training process itself robust instead: within each cluster it detects the single network layer most disturbed by missing-modality batches, replaces that poverty layer with a weighted average of parameters from complete-modality clients in the same cluster, then uses the Banzhaf Power Index to bias client selection toward contributors, and a portfolio-optimization rule to down-weight risky clusters during global aggregation. The paper claims that this three-part recipe outperforms standard federated, clustered-federated, and missing-modality baselines in both global and personalized evaluation across classification and retrieval datasets. A sympathetic reading is that the authors are trying to show that missing modalities can be handled by editing and selection inside an existing clustered federated loop, without rebuilding the base model.

What carries the argument

Three coupled mechanisms carry the argument. The poverty layer is identified by $\rho_{k,t}^{e,b}(p)$, the mean relative parameter change of layer $p$ between a batch with complete modalities and the following batch with a missing modality, and is edited via $\boldsymbol{\theta}_{k,t}(P_{k,t}) = \sum_j w_j \boldsymbol{\theta}_{j,t}(P_{k,t})$ using parameters of complete-modality clients in the cluster. The Banzhaf Power Index appears through a per-round return $a_{i,t}$ and a core-member test: client $i$ is core if excluding it pushes the cluster's return below its threshold, and selection probabilities are $\mathrm{prob}(i) = e^{\tau \phi(i)/T(i)} / \sum_{i \in \mathcal{M}_m} e^{\tau \phi(i)/T(i)}$. The portfolio module computes intra-cluster covariance $\bar{\sigma}_{m,t}$, a risk-adjusted return coefficient $\gamma_{m,t} = \lambda \bar{\sigma}_{m,t} - (1-\lambda) A_{m,t}$, and a dynamic $\beta^*_{m,t}$ that retains more historical updates when a cluster is risky. The poverty substitution is the model-agnostic repair, BPI biases which clients participate, and MPO gates how much each cluster's update counts in global aggregation.

What would settle it

Inspect the released code for the data source behind $a_{i,t}$ in Eq. (7). If $a_{i,t}$ is the accuracy, F1, or RSum computed on the cluster test set visible to all clients or on the global test set, rerun the comparison with $a_{i,t}$ computed only from local training batches or a held-out per-client validation split; the central claim collapses if MMiC's edge over FedAvg, FedOpt, and PACFL shrinks or vanishes under that correction.

Watch

Extended reading notes

Core claim

The central claim is that modality incompleteness can be mitigated at the level of the federated optimization loop rather than at the data or architecture level. For each client, MMiC identifies the layer whose parameters change most sharply when a missing-modality batch follows a complete-modality batch, calls it the poverty layer, and substitutes a weighted average of corresponding parameters from cluster peers that trained on complete data. Client selection then replaces uniform randomness with a probability based on the Banzhaf Power Index: a client is a core member when removing it from the selected set drops the cluster's return below a threshold, and historical core counts are balanced against selection counts to set future probabilities. Global aggregation extends adaptive federated optimization with a risk-return score per cluster, adjusting how much of the current cluster update is retained. The paper reports that MMiC and its two variants, one with modality reconstruction and one with geometric multimodal contrastive fusion, achieve the top global and personalized results across CrisisMMD, UPMC-Food101, and Flickr30k under three missing-modality settings.

Load-bearing premise

The load-bearing premise is that the per-client performance returns $a_{i,t}$ used by both the Banzhaf selection and the portfolio aggregation are honest signals, but the paper does not say whether they are computed on the cluster or global test sets described in Section 5.1.3; if they are, the method is being selected and weighted with the evaluation answer key in hand.

Editorial extensions

If this is right

  • If the poverty-layer signal is reliable, missing modalities can be repaired by editing one parameter layer per client, so a cluster model can keep training without reconstructing the absent data.
  • Because BPI changes only which clients join, the framework can cut communication rounds: on Flickr30k, random selection needed 12, 24, or 16 extra rounds to match BPI selection across the three missing-modality settings.
  • MPO-based dynamic $\beta^*$ slows performance decline after convergence, so the same mechanism that hedges missing modalities also mitigates client overfitting.
  • The recipe transfers across modalities: on accelerometer-gyroscope activity data (UCI-HAR and KU-HAR), MMiC beats FedOpt by about 6 and 2 percent respectively, supporting the modality-agnostic claim.
  • Because MMiC is model-independent, the three modules can be layered onto different multimodal base models, including a lightweight concatenation model, CLIP, SCAN, and GMC fusion, without changing the core training objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Replacing only the single most-affected layer is a communication-saving choice; the paper's own conclusion flags substituting more poverty parameters, so a direct extension is to make the number and identity of replaced layers adaptive.
  • Since the poverty-layer signal is defined by a batch-to-batch parameter change, the same detector could be used to schedule when to activate reconstruction modules or when to pause a client's contribution, not just where to edit.
  • The Banzhaf and portfolio modules both depend on a scalar per-client return; one could test cheaper proxies such as gradient norm or loss change, which would make the mechanism applicable when per-client evaluation labels are unavailable.
  • The paper fixes missing-modality rates per client; a stronger stress test would vary the missing rates round by round, exercising the selection and aggregation modules under the non-stationary conditions they are designed for.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes MMiC, a clustered federated learning framework for multimodal data with missing modalities. MMiC combines three modules: poverty parameter substitution (PPS), which replaces the most affected layer of a client model with a weighted combination of complete-modality clients' parameters; a Banzhaf Power Index (BPI) based client selection mechanism; and a Markowitz Portfolio Optimization (MPO) based dynamic global aggregation scheme. The authors evaluate MMiC and two variants on CrisisMMD, UPMC-Food101, Flickr30k, and two HAR datasets, reporting consistent improvements over FedAvg, FedOpt, FedSoft, PACFL, CreamFL, and PmcmFL in both global and personalized performance.

Significance. If the experimental evaluation is free of data leakage, the paper addresses a relevant and under-explored problem: missing modalities in clustered multimodal federated learning. The proposed modules are model-independent and modality-agnostic in principle, the code is released, and the experimental coverage spans classification, retrieval, and sensor-based activity recognition. However, the central outperformance claim depends on an unresolved question about what data the per-round performance signal a_{i,t} is computed on, and there are additional reproducibility gaps in the method description. The contribution is therefore potentially useful but currently not fully assessed.

major comments (4)
  1. [Section 5.1.3 and Eqs. (7)-(15)] The paper never states what data the per-client per-round performance a_{i,t} in Eq. (7) is computed on. Section 5.1.3 says 'Each cluster had a test set visible to all clients within the cluster. Additionally, the global server provided accessibility to the test set for all clients.' If a_{i,t} is evaluated on this visible test set, then BPI core-member identification (Eqs. 9-11) and MPO aggregation weights (Eqs. 12-15) are functions of the exact labels and metric used for the final evaluation in Table 1. Baselines do not use such per-round outcomes in their selection or aggregation procedures, so the reported comparison would be circular and the headline outperformance claim would be invalid. Please state the exact evaluation split used to compute a_{i,t}, confirm that it is disjoint from the final test set, and explain how all baselines are given the same information.
  2. [Section 4.2, Eq. (9)] The core-member identification condition in Eq. (9) uses a threshold \alpha_m^t that is not defined anywhere in the paper. The authors define A_{S_m,t} and A_{S'_m,t}, but the threshold \alpha_m^t is introduced without explanation, making the BPI selection step not reproducible as written.
  3. [Section 4.1, Eq. (6)] Equation (6) is presented as an Adam-based calculation of parameter variation, but it does not follow from the Adam update rule. For an Adam step, \theta_b = \theta_{b-1} - \eta \hat{m}_b / (\sqrt{\hat{\nu}_b} + \epsilon), so the relative parameter change involves \theta_{b-1} in the denominator, not \theta_b as written in Eq. (6). Since the poverty-layer identification in Eqs. (2)-(4) depends on this quantity, the equation should be corrected or explicitly justified as a rough approximation, and the text should state whether saved parameters or the approximation is used in the experiments.
  4. [Tables 1 and 2] The main results and the ablation results are inconsistent. For Flickr30k under (mm@0.2, mc@0.5), Table 1 reports MMiC global RSum 8.322 and personalized RSum 75.814, whereas Table 2 reports an MMiC RSum of 11.02 for the same setting. In addition, CrisisMMD is evaluated with F1 in Table 1 but with accuracy in Table 2. These discrepancies need to be reconciled, because the ablation conclusions about the contribution of each module are not fully supported by the reported numbers.
minor comments (4)
  1. [Section 5.4] The claim that MMiC is modality-agnostic is supported only by comparing MMiC with FedOpt on two sensor datasets; a comparison with the full baseline set would make the generalization claim stronger.
  2. [Eq. (12)] The covariance formula in Eq. (12) reuses t both as the current global round and as the summation index, and \bar{\alpha}_{i,t-1} is described as the historical average return for round t-1 without a clear definition of the averaging window. Please rewrite with distinct indices and define all quantities precisely.
  3. [Section 5.1.4] The missing-modality simulation is described ambiguously: clients are assigned a fixed number of missing samples by the mm rate, but the mc rate is applied per round to selected clients. The precise relationship between the global mc/mm parameters and the actual per-client missing status in each round should be stated.
  4. [Section 5.8] The time-complexity derivation would benefit from defining M and K explicitly at the start and clarifying whether prefix sums are maintained per cluster or globally; as written, the index conventions are easy to misread.

Circularity Check

1 steps flagged · score 6.0 of 10

BPI/MPO consume the evaluation metric itself: a_{i,t} is accuracy/F1/RSum and the only described evaluation set is visible to clients and server, so reported gains are not independent.

  1. fitted input called prediction [Section 4.2 (Eqs. 7–11) and Section 4.3 (Eqs. 12–15); evaluation setup in Section 5.1.3 and Table 1.]
    "We can obtain the i-th client’s local model performance a_{i,t} (i.e., accuracy, F1 score or RSum) ... α_{i,t}=a_{i,t}−a_{i,t−1} ... Each cluster had a test set visible to all clients within the cluster. Additionally, the global server provided accessibility to the test set for all clients."

    Eq. (7) defines every BPI input (Eqs. 9–11) and every MPO/RARC input (Eqs. 12–15) as α_{i,t}=a_{i,t}−a_{i,t−1}, with a_{i,t} explicitly 'accuracy, F1 score or RSum'. Table 1 reports exactly these three metrics as the headline global and personalized results. Section 5.1.3 makes the test set visible to all clients and the server and defines no validation split for a_{i,t}. Hence, as written, MMiC's selection and aggregation weights are optimized on the same test outcomes used to score it, so the reported superiority is an artifact of test-set access rather than an independent prediction. Table 2 shows removing BPI/MPO sharply degrades performance, making this signal load-bearing.

full rationale

The parameter-substitution module (Eqs. 2–5) is self-contained and does not use test labels. The circularity, if present, is confined to BPI and MPO, which both consume per-round client performance returns. The paper never states the data split for a_{i,t}; the only evaluation set described in Section 5.1.3 is a test set visible to all clients and the server, and the metrics defining a_{i,t} are identical to the metrics reported in Table 1 (F1, accuracy, RSum). Under the protocol as written, the control loop is self-referential: the same test-set outcomes choose which clients and clusters influence the model and then score the resulting model. This makes the headline comparison unfair relative to baselines that do not use such signals. A single additional sentence specifying a held-out validation set disjoint from the final test set would resolve the issue; absent that sentence, the comparison is not independently assessable. No other circular step is evident: the citation of the authors' prior LSH work [52] is used only as a clustering tool, and neither BPI nor MPO is justified by a self-citation chain.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework rests on four classes of unproven premises: (1) that per-round performance returns are valid contribution signals, and that they are not computed on the evaluation test set; (2) that clusters are coherent enough to make one-layer substitution beneficial; (3) that the randomly toggled missingness simulation matches real missing-modality settings; (4) that several tuned hyperparameters (lambda, tau, beta, cluster counts, learning rates) are reasonable. No convergence theory is provided for the BPI/MPO control loop.

free parameters (5)
  • lambda (risk tolerance) = 0.5
    In Eq 13, gamma = lambda*sigma - (1-lambda)*A; set by default to 0.5 on all datasets, and directly changes the MPO down-weighting of clusters.
  • tau (temperature) = 10-20
    In Eq 11 and Eq 15, tau controls selection probability disparity and beta smoothing; the authors suggest [10,20] but do not report the exact value per experiment.
  • beta (FedAdagrad momentum) = not reported
    Base momentum in Eq 14/15 balances historical and current updates; the experiments do not state its value, yet it is central to FedAdagrad.
  • cluster hyperparameters (LSH k, SVD threshold) = LSH 3 or 7 clusters; SVD thresholds 10 or 14
    Chosen per dataset by silhouette coefficient (Sec 5.7), and directly affects which clients share parameters in PPS and how BPI/MPO operate.
  • client and global learning rates = global 0.1/0.01/0.05; client Adam 0.005
    Set by 'simple tuning in early stage' (Sec 5.1.1); these are standard but still fitted to each dataset.
assumptions (5)
  • domain assumption Clients and server know which clients and samples have missing modalities and can distinguish complete-modality clients in the same cluster.
    The PPS step (Eq 5) substitutes parameters from clients that trained with the complete modality round, and the authors state 'We are aware of the missing modality status of the data' (footnote in Sec 4.1). In real FL, this information may be available locally but not reliably at cluster level.
  • domain assumption Same-cluster clients have sufficiently similar model parameters that replacing one layer of a poverty client with an average of other clients' layers improves performance.
    Eq 5 and Sec 5.7 state PPS 'heavily relies on the quality of client clustering'; no theoretical condition is given for when substitution helps.
  • domain assumption Per-round local model performance (accuracy/F1/RSum) is an informative and unbiased signal for client contribution.
    Eqs 7 to 9 and 12 to 13 use these performance returns to define core members and risk; if the performance metric is noisy or computed on the test set, the selection and weighting are biased.
  • ad hoc to paper Random missingness activated per round (mc and mm rates) adequately represents real modality incompleteness.
    Sec 5.1.4 randomly toggles missing data at the round level for selected clients; real missingness is usually static per client or per dataset, and this dynamic setup may make the problem easier.
  • ad hoc to paper The layer with the highest parameter change rate during missing-modality batches is the layer whose substitution best restores performance.
    Eq 4 defines P_{k,t} by argmax change rate; no evidence is provided that this layer is the best or sufficient one to replace, and Sec 6 says the authors plan to substitute 'additional poverty parameters' in future work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MMiC: Mitigating Modality Incompleteness in Clustered Federated Learning." pith.science (2026). https://pith.science/paper/4TZLC5MY

@misc{pith2026250506911,
  author       = {Pith},
  title        = {Pith review of: MMiC: Mitigating Modality Incompleteness in Clustered Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4TZLC5MY}},
  note         = {Machine review of arXiv:2505.06911}
}
read the original abstract

In the era of big data, data mining has become indispensable for uncovering hidden patterns and insights from vast and complex datasets. The integration of multimodal data sources further enhances its potential. Multimodal Federated Learning (MFL) is a distributed approach that enhances the efficiency and quality of multimodal learning, ensuring collaborative work and privacy protection. However, missing modalities pose a significant challenge in MFL, often due to data quality issues or privacy policies across the clients. In this work, we present MMiC, a framework for Mitigating Modality incompleteness in MFL within the Clusters. MMiC replaces partial parameters within client models inside clusters to mitigate the impact of missing modalities. Furthermore, it leverages the Banzhaf Power Index to optimize client selection under these conditions. Finally, MMiC employs an innovative approach to dynamically control global aggregation by utilizing Markovitz Portfolio Optimization. Extensive experiments demonstrate that MMiC consistently outperforms existing federated learning architectures in both global and personalized performance on multimodal datasets with missing modalities, confirming the effectiveness of our proposed solution. Our code is available at https://github.com/gotobcn8/MMiC.

Figures

Figures reproduced from arXiv: 2505.06911 by the authors.

Figure 1
Figure 1. The MMiC architecture is divided into three lay [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The number of additional training rounds needed [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. The two plots on the left depict the SCV of LSH clustering under two different hyperparameter (number of clusters), [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: We compared two clustering algorithms, LSH and [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 53 canonical work pages

  1. [1]

    Guangyin Bao, Qi Zhang, Duoqian Miao, Zixuan Gong, Liang Hu, Ke Liu, Yang Liu, and Chongyang Shi. 2024. Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast. arXiv:2312.13508 [cs.LG] https: //arxiv.org/abs/2312.13508

    arXiv 2024

  2. [2]

    Guy E Blelloch. 1990. Prefix sums and their applications. (1990)

    work page 1990

  3. [3]

    Liwei Che, Jiaqi Wang, Yao Zhou, and Fenglong Ma. 2023. Multimodal Federated Learning: A Survey. Sensors 23, 15 (2023). doi:10.3390/s23156986

    work page doi:10.3390/s23156986 2023

  4. [4]

    Jiayi Chen and Aidong Zhang. 2022. FedMSplit: Correlation-Adaptive Federated Multi-Task Learning across Multimodal Split Networks. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’22)

    work page 2022

  5. [5]

    Yae Jee Cho, Jianyu Wang, and Gauri Joshi. 2020. Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies.CoRR abs/2010.01243 (2020)

    arXiv 2020

  6. [6]

    Don Kurian Dennis, Tian Li, and Virginia Smith. 2021. Heterogeneity for the Win: One-Shot Federated Clustering. InProceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 2611–2620

    work page 2021

  7. [7]

    Pradeep Dubey and Lloyd S. Shapley. 1979. Mathematical Properties of the Banzhaf Power Index. Math. Oper. Res. 4, 2 (1979), 99–131

    work page 1979

  8. [8]

    Fleet, Jamie Ryan Kiros, and Sanja Fidler

    Fartash Faghri, David J. Fleet, Jamie Ryan Kiros, and Sanja Fidler. 2018. VSE++: Improving Visual-Semantic Embeddings with Hard Negatives. In Proceedings of the British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018. BMVA Press, 12

    work page 2018

Show all 62 references
  1. [9]

    Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. 2020. Personalized Fed- erated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach. In Proceedings of the Advances in Neural Information Processing Sys- tems, NeurIPS 2020, H. Larochelle, M. Ranzato...

    2020

  2. [10]

    Tiantian Feng, Digbalay Bose, Tuo Zhang, Rajat Hebbar, Anil Ramakrishna, Rahul Gupta, Mi Zhang, Salman Avestimehr, and Shrikanth Narayanan. 2023. FedMultimodal: A Benchmark for Multimodal Federated Learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discove...

    2023

  3. [11]

    Ignazio Gallo, Gianmarco Ria, Nicola Landro, and Riccardo La Grassa. 2020. Image and Text fusion for UPMC Food-101 using BERT and CNNs. In Proceedings of the 35th International Conference on Image and Vision Computing New Zealand, IVCNZ 2020, Wellington, New Zealand, November ...

    2020

  4. [12]

    Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. 2020. An Efficient Framework for Clustered Federated Learning. In Proceedings of Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020...

    2020

  5. [13]

    Avishek Ghosh, Justin Hong, Dong Yin, and Kannan Ramchandran. 2019. Robust Federated Learning in a Heterogeneous Environment. CoRR abs/1906.06629 (2019). http://arxiv.org/abs/1906.06629

    2019 arXiv

  6. [14]

    Cyril Goutte and Éric Gaussier. 2005. A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation. In Proceedings of the 27th European Conference on IR Research, ECIR 2005, Santiago de Compostela, Spain, March 21-23, 2005, Proceedings (Lect...

    2005

  7. [15]

    Shubham Gupta, Nandini Saini, Suman Kundu, and Debasis Das. 2024. CrisisKAN: Knowledge-Infused and Explainable Multimodal Attention Network for Crisis Event Classification. InProceedings of the 46th European Conference on Information Retrieval, ECIR 2024, Glasgow, UK, March 24...

    2024

  8. [16]

    László A Jeni, Jeffrey F Cohn, and Fernando De La Torre. 2013. Facing imbalanced data–recommendations for the use of performance metrics. In Proceedings of the 2013 Humaine association conference on affective computing and intelligent interaction. IEEE, 245–251

    2013

  9. [17]

    Stephen C Johnson. 1967. Hierarchical clustering schemes. Psychometrika 32, 3 (1967), 241–254

    1967

  10. [18]

    Makowski, Daniel Rueckert, and Rickmer Braren

    Georgios Kaissis, Marcus R. Makowski, Daniel Rueckert, and Rickmer Braren

  11. [19]

    Tapas Kanungo, David M Mount, Nathan S Netanyahu, Christine D Piatko, Ruth Silverman, and Angela Y Wu. 2002. An efficient k-means clustering algorithm: Analysis and implementation. IEEE transactions on pattern analysis and machine intelligence 24, 7 (2002), 881–892

    2002

  12. [20]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimiza- tion. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

    2015

  13. [21]

    Annick Laruelle and Federico Valenciano. 2001. Shapley-Shubik and Banzhaf Indices Revisited. Math. Oper. Res. 26, 1 (2001), 89–104

    2001

  14. [22]

    Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. 2018. Stacked Cross Attention for Image-Text Matching. In Proceedings of the 15th European Conference on Computer Vision, ECCV 2018 (Lecture Notes in Computer Science, Vol. 11208). Springer, 212–228

    2018

  15. [23]

    Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang

  16. [24]

    Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. Pmet: Precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18564–18572

    2024

  17. [25]

    In Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

    On the Convergence of FedAvg on Non-IID Data. In Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id= HJxNAnVtDS

    2020

  18. [26]

    Yunlong Lu, Xiaohong Huang, Yueyue Dai, Sabita Maharjan, and Yan Zhang

  19. [27]

    Zichang Liu, Zhaozhuo Xu, Benjamin Coleman, and Anshumali Shrivastava. 2023. One-Pass Distribution Sketch for Measuring Data Heterogeneity in Federated Learning. In Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information...

    2023

  20. [28]

    Harry M Markowitz. 1991. Foundations of portfolio theory. The journal of finance 46, 2 (1991), 469–477

    1991

  21. [29]

    IEEE Trans

    Blockchain and Federated Learning for Privacy-Preserved Data Sharing in Industrial IoT. IEEE Trans. Ind. Informatics 16, 6 (2020), 4177–4186

    2020

  22. [30]

    Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh. 2020. Three Approaches for Personalization with Applications to Federated Learning. CoRR abs/2002.10619 (2020)

    2020 arXiv

  23. [31]

    Takayuki Nishio and Ryo Yonetani. 2019. Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge. In Proceedings of 2019 IEEE Inter- national Conference on Communications, ICC 2019, Shanghai, China, May 20-24,

    2019

  24. [32]

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-Efficient Learning of Deep Net- works from Decentralized Data. InProceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, ...

    2017

  25. [33]

    Merton H. Miller. 1960. Portfolio selection: efficient diversification of investments. The Journal of Business 33, 4 (1960), 391–393. http://www.jstor.org/stable/2350928

    1960

  26. [34]

    Melo, Ana Paiva, and Danica Kragic

    Petra Poklukar, Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva, and Danica Kragic. 2022. Geometric Multimodal Contrastive Representation Learning. In Proceedings of the International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA (Pr...

    2022

  27. [35]

    Ferda Ofli, Firoj Alam, and Muhammad Imran. 2020. Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response. In Proceedings of the 17th International Conference on Information Systems for Crisis Response and Management, ISCRAM 2020, May 2020 . ISCRAM D...

    2020

  28. [36]

    Shiva Raj Pokhrel and Jinho Choi. 2020. Federated Learning With Blockchain for Autonomous Vehicles: Analysis and Design Challenges. IEEE Trans. Commun. 68, 8 (2020), 4734–4746

    2020

  29. [37]

    Amin Rasekh, Chien-An Chen, and Yan Lu. 2014. Human activity recognition using smartphone. arXiv preprint arXiv:1401.8212 (2014)

    2014 arXiv

  30. [38]

    Fan Qi and Shuai Li. 2024. Adaptive hyper-graph aggregation for modality- agnostic federated learning. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition

    2024

  31. [39]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings...

    2021

  32. [40]

    Yichen Ruan and Carlee Joe-Wong. 2022. FedSoft: Soft Clustered Federated Learn- ing with Proximal Local Updating. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applica- tions of Artificial Intelligence,...

    2022

  33. [41]

    Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan

    Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan. 2021. Adaptive Federated Optimization. In Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event...

    2021

  34. [42]

    Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan

    Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan. 2021. Adaptive Fed- erated Optimization. In Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Eve...

    2021

  35. [43]

    Niloy Sikder and Abdullah-Al Nahid. 2021. KU-HAR: An open dataset for het- erogeneous human activity recognition. Pattern Recognition Letters 146 (2021), 46–54

    2021

  36. [44]

    Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. 2017. Federated Multi-Task Learning. In Proceedings of Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, C...

    2017

  37. [45]

    Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Moham- mad Norouzi. 2022. Photorealistic Text-to-Image Diffusion Models...

    2022

  38. [46]

    Ketan Rajshekhar Shahapure and Charles Nicholas. 2020. Cluster Quality Analysis Using Silhouette Score. In Proceedings of the 7th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2020, Sydney, Australia, October 6-9,

    2020

  39. [47]

    Saeed Vahidian, Mahdi Morafah, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, and Bill Lin. 2023. Efficient Distribution Similarity Identifi- cation in Clustered Federated Learning via Principal Angles between Client Data Subspaces. In Proceedings of the 37th AAA...

    2023

  40. [48]

    Shiqiang Wang and Mingyue Ji. 2024. A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging. In Proceedings of The 12th International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net

    2024

  41. [49]

    Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. 2023. Towards Personal- ized Federated Learning. IEEE Trans. Neural Networks Learn. Syst. 34, 12 (2023), 9587–9603. https://doi.org/10.1109/TNNLS.2022.3160699

    2023

  42. [50]

    Yee Whye Teh. 2010. Dirichlet Process. In Encyclopedia of Machine Learning , Claude Sammut and Geoffrey I. Webb (Eds.). Springer, 280–287

    2010

  43. [51]

    Baochen Xiong, Xiaoshan Yang, Yaguang Song, Yaowei Wang, and Changsheng Xu. 2023. Client-Adaptive Cross-Model Reconstruction Network for Modality- Incomplete Multimodal Federated Learning. In Proceedings of the 31st ACM In- ternational Conference on Multimedia (MM), MM 2023, O...

    2023

  44. [52]

    Lishan Yang, Alireza Seyed Shakeri, Liangxi Pu, Weitong Chen, and Yanjun Shu

  45. [53]

    Qiong Wu, Kaiwen He, and Xu Chen. 2020. Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework. IEEE Open J. Comput. Soc. 1 (2020), 35–44

    2020

  46. [54]

    Renjie Wu, Hu Wang, and Hsiang-Ting Chen. 2024. A Comprehensive Survey on Deep Multimodal Learning with Missing Modality. CoRR abs/2409.07825 (2024). arXiv:2409.07825 doi:10.48550/ARXIV.2409.07825

    2024 doi arXiv

  47. [55]

    Songcan Yu, Junbo Wang, Walid Hussein, and Patrick C. K. Hung. 2024. Robust multimodal federated learning for incomplete modalities. Comput. Commun. 214 (2024), 234–243. https://doi.org/10.1016/j.comcom.2023.12.003

    2024 doi

  48. [56]

    Linlin Zong, Qiujie Xie, Jiahui Zhou, Peiran Wu, Xianchao Zhang, and Bo Xu

  49. [58]

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Trans. Assoc. Comput. Linguistics 2 (2014), 67–78

    2014

  50. [59]

    Qiying Yu, Yang Liu, Yimu Wang, Ke Xu, and Jingjing Liu. 2023. Multimodal Federated Learning via Contrastive Representation Ensemble. In Proceedings of the 11th International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    2023

  51. [2020]

    Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2, 6 (2020), 305–311

    2020

  52. [2021]

    In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021

    FedCMR: Federated Cross-Modal Retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021 . ACM, 1672–1676

    2021

  53. [2022]

    AAAI Press, 8124–8131

  54. [2024]

    In International Conference on Advanced Data Mining and Applications

    Efficient Clustered Federated Learning by Locality Sensitive Hashing. In International Conference on Advanced Data Mining and Applications . Springer, 177–191

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.