Pith. sign in

REVIEW 3 major objections 6 minor 69 references

FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FedMHO claims that heterogeneous one-shot federated learning can beat data-free baselines by averaging strong classifiers and training a global model on synthetic samples from lightweight client generators, with distillation to prevent…

desk verdict A solid empirical one-shot FL paper with a sensible role-split by client capacity; the main gap is that it never measures label correctness of the synthetic samples that are supposed to carry the weak clients' knowledge. read the letter →

arxiv 2502.08518 v1 pith:RC6QVCD2 submitted 2025-02-12 cs.LG cs.AIcs.DC

Dezhong Yao , Yuexin Shi , Tongtong Liu , Zhiqiang Xu This is my paper · ORCID
classification cs.LGcs.AIcs.DC
keywords one-shotfederatedlearningmodelheterogeneitydata-freeknowledgedistillationconditionalvariationalautoencoderresource-constrainededgedevicesforgettingsyntheticdataoptimizationnon-IID
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedMHO is a one-shot federated learning method for clients with unequal computing power: some run deep classifiers, some run lightweight generative models. The paper claims that a single communication round can produce a global model that beats state-of-the-art data-free one-shot baselines, by averaging the classifiers to initialize the server model and then training that model on synthetic images produced by small conditional variational autoencoders. Two variants, FedMHO-MD and FedMHO-SD, add a distillation term that keeps the global model from forgetting classifier knowledge while it adapts to synthetic data. Across MNIST, Fashion-MNIST, SVHN, and EMNIST under several non-IID partitions, the reported average gains over the best baseline are 5.17%, 8.35%, and 8.25% for the three variants. The practical point is that weak clients can contribute through cheap generators rather than through unreliable classifiers or extra communication.

What carries the argument

The load-bearing machinery is a two-stage server-side pipeline. Stage one generates synthetic samples: each uploaded CVAE decoder draws latent vectors from a standard Gaussian and produces images conditioned on that client's local label distribution, and an unsupervised K-means filter keeps only samples closest to each class's cluster center. Stage two fuses knowledge: the global model starts from $w_0 = \frac{1}{|\{K_C\}|}\sum_{k\in\{K_C\}} w_k$, the mean of the uploaded classifiers, and is optimized with $L_g = \lambda L_{CE} + (1-\lambda)L_{KL}$, where cross-entropy fits synthetic data and either multi-teacher distillation (average logits of all classifiers) or self-distillation (logits of the initialized model) resists forgetting. The CVAE is defined by encoder $q_\phi(z|x,c)$ and decoder $p_\theta(x|z,c)$ with reconstruction plus KL loss; the small decoder is cheap enough for wristband-class devices.

What would settle it

Oracle-label the synthetic samples: for a resource-constrained client, generate samples per class, apply the K-means filter, and have a human or a trusted strong classifier label them. If the filtered set contains a substantial fraction of samples whose true class differs from the conditioning label, then FedMHO's data-generation path is injecting label noise, and removing generative clients should leave accuracy unchanged or higher.

Watch

Extended reading notes

Core claim

The paper's central claim is that model-heterogeneous one-shot federated learning can be solved without auxiliary public data by splitting roles according to client capability: resource-sufficient clients train and upload deep classifiers, while resource-constrained clients train and upload only lightweight CVAE decoders plus local label distributions. On the server, the global model is initialized as the average of the uploaded classifiers, then trained on synthetic samples generated by the decoders and filtered by an unsupervised K-means procedure. Because the weak clients' classifiers would give misleading logits, they are never used as teachers; their knowledge enters only through generated data. The multi-teacher and self-distillation variants prevent the global model from overwriting classifier knowledge while learning from synthetic data. The paper reports that these methods consistently rank first or second across twelve dataset/partition settings and improve average Top-1 accuracy over the optimal baseline by 5.17% (FedMHO), 8.35% (FedMHO-MD), and 8.25% (FedMHO-SD).

Load-bearing premise

The method stands on assuming that a lightweight CVAE trained on highly non-IID local data produces synthetic samples whose class labels remain trustworthy after K-means filtering, even when the same client's classifier would give misleading logits; if that fails, the weak clients' contribution dissolves.

Editorial extensions

If this is right

  • A single communication round suffices to build a competitive global model even when half the clients can only train lightweight generators, so iterative FL rounds and public auxiliary datasets are unnecessary.
  • Weak clients contribute through cheap decoders and local label distributions, avoiding the misleading logits their classifiers would produce and adding negligible communication cost.
  • Averaging strong clients' classifiers before training on synthetic data gives the global model a strong initialization, so it reaches target accuracy in fewer server-side epochs than methods that train generators from scratch.
  • The distillation regularizer (multi-teacher or self-distillation) keeps the global model from forgetting classifier knowledge, and the two variants perform comparably, so either can be chosen based on deployment preference.
  • The reported FID values (about 100 to 150) support the paper's claim that synthetic samples preserve useful class information while remaining distinct enough from real data to mitigate privacy leakage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the method's success rests on generated samples carrying correct class information; if a client's local data is so skewed that its CVAE learns a class-conditional style instead of the class, K-means filtering cannot recover the labels, so performance would degrade toward using only strong clients.
  • Editorial inference: the reported FID range could be treated as a tunable privacy-utility knob, letting a deployment choose how close synthetic samples may come to real data before risking leakage.
  • Editorial inference: the same two-stage recipe might transfer to other modalities (text, tabular, audio) wherever a cheap conditional generator exists, by replacing the pixel-space K-means filter with a modality-appropriate outlier filter.
  • Editorial inference: a natural stress test is to vary the ratio of strong to weak clients beyond the tested 50/50 split; the paper does not report whether accuracy degrades gracefully when the generator population dominates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FedMHO, a one-shot federated learning framework for heterogeneous clients. Clients with sufficient resources train deep classifiers (VGG-9) and send their full models to the server, while resource-constrained clients train lightweight conditional variational autoencoders and send only the decoders together with their local label distributions. On the server, the global model is initialized by averaging the classifiers' parameters (Eq. 4), synthetic samples are generated per class from the decoders, filtered by an unsupervised K-means-based step (Algorithm 2, retaining the 80% of samples closest to each class centroid), and the global model is fine-tuned on the filtered synthetic samples. Two variants, FedMHO-MD (multi-teacher distillation from the local classifiers) and FedMHO-SD (self-distillation from the initialized global model), add a KL regularization term with lambda equal to 0.5 to counter knowledge forgetting during global training. Experiments on MNIST, Fashion-MNIST, SVHN, and EMNIST under Dir(0.5/0.3/0.1) partitions compare FedMHO and its variants against FedAvg, FedDF, DENSE, Co-Boosting, and FEDCV AE. The paper claims average accuracy improvements of 5.17%, 8.35%, and 8.25% over the best baseline, and presents ablations of the distillation loss, the data-optimization step, and sensitivity studies over local epochs, synthetic-sample counts, client counts, the retention ratio Rth, and the loss weights.

Significance. If the central premise holds, this is a practical contribution: a single-round, data-free protocol that lets weak clients participate through lightweight generators while strong clients contribute classifiers, and the empirical evaluation is unusually complete, with twelve dataset/partition settings, ablations for every component (Tables 4-8), a homogeneous-large-model comparison (Table 9), a diverse-architecture extension (Table 10), an FID-based analysis (Table 11), and convergence curves (Figures 5-8). The reported standard deviations are small, and I verified that the headline improvement figures (5.17%, 8.35%, 8.25%) are reproducible from Table 4 as the differences between each variant's average accuracy and the average accuracy of FEDCV AE, which is a point in the paper's favor. The conceptual novelty is moderate, since the method combines parameter averaging, CVAE-based generation, and existing distillation ideas, but the combination is nontrivial and the gains over strong baselines are consistent and often large (5-17 points in several settings).

major comments (3)
  1. [4.4 / Algorithm 2 (with 5.7, 5.8, Table 4)] The load-bearing premise that resource-constrained clients contribute through synthetic samples with correct class labels is never directly verified. Algorithm 2 clusters each class's synthetic samples into one cluster and keeps the 80% closest to the centroid by flattened-pixel Euclidean distance; with a single cluster this is a distance-to-class-mean filter, and it cannot correct systematic label confusion, because class-swapped or class-confused samples need not be Euclidean outliers. Under the Dir(0.1) partition, the per-class synthetic pool can be generated by only one or two weak clients that actually hold that class, so the class centroid itself may be an unreliable reference. The paper's evidence for label accuracy is visual inspection (Section 5.7) and FID (Section 5.8), neither of which measures per-sample agreement between the conditioning label and the content of the generated image. Table 4 is consistent with the risk: FedMHO without the distillation loss falls below FEDCV AE on MNIST (88.48 vs 89.73) and Fashion-MNIST (62.14 vs 64.01) at Dir(0.1), suggesting the weak-client stream can be net negative at high heterogeneity. I ask the authors to add (i) a quantitative label-agreement analysis of the generated samples (e.g., agreement between the conditioning labels and the predictions of a reference classifier, before and after filtering, per dataset and per partition), and (ii) an ablation that removes the weak-client synthetic stream (e.g., training the initialized global model without the weak-client decoders' samples) to establish that the advertised gains actually come from the weak clients' generated data rather than solely from the strong-classifier initialization combined with distillation regularization.
  2. [5.8 (Table 11)] The privacy claim is overclaimed. The text states that FID values between 100 and 150 indicate that 'synthetic data diverges sufficiently from real data, ensuring privacy protection without compromising global model performance.' FID is a distribution-level similarity measure and does not bound membership-inference, model-inversion, or reconstruction risk; moreover, in FedMHO the server receives decoders, per-client label distributions, and full classifiers, all of which are potential attack surfaces that Section 5.8 does not consider. I recommend rewording the claim to state only what is measured (synthetic and real samples are far apart in FID) and to defer privacy guarantees explicitly to future work, as the Conclusion already does.
  3. [5.6 (Table 10)] The extension experiment with diverse large models (EfficientNet-V2, ResNet-50, MobileNet-V2, VGG-9) is not reproducible from the method description. The text says that FedMHO variants 'employ ensemble distillation for global model initialization' in this setting, but Section 4 defines initialization only as parameter averaging in Eq. (4), which requires homogeneous architectures. The teachers, the data used for the distillation, the loss, and the number of steps are not specified, so the Table 10 results cannot be evaluated or reproduced. Please either specify this procedure in Section 4 or clearly mark Table 10 as a separate protocol with full details.
minor comments (6)
  1. [5.3.2] The text says the default setting yields '96,000' retained samples for EMNIST; since EMNIST uses 12,000 synthetic samples with an 80% retention ratio, the correct number is 9,600.
  2. [Abstract and Section 1] The phrase 'compared to the optimal baseline' is ambiguous; the claimed improvements of 5.17%, 8.35%, and 8.25% match the differences between each FedMHO variant's average accuracy and FEDCV AE's average accuracy over the 12 settings of Table 4, and the text should say so explicitly.
  3. [5.2] The per-dataset Dir(0.5) improvements (3.06%, 5.44%, 7.23%, 6.12%) are not all consistent with Table 4 when the best baseline is taken per dataset (for example, Fashion Dir(0.5) gives 4.75 and SVHN Dir(0.5) gives 7.32); please reconcile the sentence with the table or state which baseline is used.
  4. [4.3, Eq. (7)] The notation hk(w0, bs) uses the per-client index k for the initialized global model's logits, which conflicts with Eq. (6) where hk denotes the k-th local classifier; use h(w0, bs) instead.
  5. [5.1.1, 5.1.2] The auxiliary public dataset used by FedDF is not specified, and the number of random seeds underlying the reported mean plus/minus standard deviation values is not stated.
  6. [4.4] Since Algorithm 2 runs K-means with a single cluster per class, it is equivalent to filtering by distance to the class mean; describing it that way would make the underlying modeling assumption explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FedMHO's central claims are empirical, benchmarked on held-out test sets, and no load-bearing step reduces to its own inputs or to a self-citation.

full rationale

The paper's central claim is an empirical performance comparison: FedMHO, FedMHO-MD, and FedMHO-SD are evaluated on MNIST, Fashion-MNIST, SVHN, and EMNIST test sets against external baselines (DENSE, Co-Boosting, FEDCV AE, FedAvg, FedDF). The reported accuracies are measured on data not used in training or hyperparameter selection, so the claim is externally grounded rather than derived from the method's own definitions. The initialization in Eq. (4) is an average of classification models, and the global model is then trained on synthetic samples using Eq. (5) and Eq. (8); this is a constructive algorithm, not a tautology. The hyperparameters Rth = 0.8 and lambda = 0.5 are selected via ablations on the same benchmark family, but they are tuning choices for an empirical system, not fitted quantities renamed as predictions. The paper cites Co-Boosting (reference [26]), one of whose co-authors overlaps with the current paper, but this citation is used only as a baseline for comparison and is not load-bearing for any derivation or justification of the proposed method. The K-means filtering in Algorithm 2 assumes per-class Euclidean proximity as a proxy for label correctness; this is a substantive empirical assumption whose failure would weaken the method, but it is not a circular reduction because success is judged by held-out accuracy, independently of the filter's own output. Overall, no step in the paper equates a prediction with an input by construction, no fitted parameter is renamed as a prediction, and no load-bearing uniqueness or ansatz is imported from the authors' prior work.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on three practical assumptions: private local training, the validity of parameter averaging for initialization, and the reliability of K-means-filtered synthetic samples for knowledge transfer. No new physical entities are introduced. The only hand-chosen hyperparameters that affect the results are the retention ratio Rth and the distillation weight lambda.

free parameters (3)
  • Rth (retention ratio for K-means filtering) = 0.8
    Set by hand and tuned via ablation on EMNIST (Section 5.4.4); the central claim depends on filtering improving sample quality, though the value itself is not a fitted constant in a derivation.
  • lambda (trade-off between CE and KL losses) = 0.5
    Set by hand after observing similar loss magnitudes (Section 4.3); the method's performance is relatively insensitive according to Table 5.
  • Number of synthetic samples per dataset = 6,000 (12,000 for EMNIST)
    Chosen to match FEDCVAE baseline (Section 5.1.4); the claim that synthetic data drives gains depends on this quantity, but it is a benchmark configuration rather than a fitted constant.
assumptions (4)
  • domain assumption Local models are trained privately and no raw data leaves clients.
    The entire FL framing assumes this; invoked implicitly in Section 4.2.1 and Figure 1.
  • domain assumption Averaging the parameters of same-architecture local classifiers yields a useful global initialization (FedAvg-style).
    Equation (4) directly uses this; it is a standard FL assumption with known limitations under non-IID data.
  • ad hoc to paper K-means clustering on flattened pixels with a single cluster and Euclidean distance can separate mislabeled synthetic samples from correctly labeled ones.
    This is the paper's own heuristic, Algorithm 2; the paper provides no independent evidence beyond the ablation in Section 5.3.2 and 5.4.4.
  • domain assumption The global model's performance on synthetic samples transfers to real test data.
    Implicit in training on synthetic data (Eq. 5); standard in data-free KD literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices." pith.science (2026). https://pith.science/paper/RC6QVCD2

@misc{pith2026250208518,
  author       = {Pith},
  title        = {Pith review of: FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RC6QVCD2}},
  note         = {Machine review of arXiv:2502.08518}
}
read the original abstract

Federated Learning (FL) is increasingly adopted in edge computing scenarios, where a large number of heterogeneous clients operate under constrained or sufficient resources. The iterative training process in conventional FL introduces significant computation and communication overhead, which is unfriendly for resource-constrained edge devices. One-shot FL has emerged as a promising approach to mitigate communication overhead, and model-heterogeneous FL solves the problem of diverse computing resources across clients. However, existing methods face challenges in effectively managing model-heterogeneous one-shot FL, often leading to unsatisfactory global model performance or reliance on auxiliary datasets. To address these challenges, we propose a novel FL framework named FedMHO, which leverages deep classification models on resource-sufficient clients and lightweight generative models on resource-constrained devices. On the server side, FedMHO involves a two-stage process that includes data generation and knowledge fusion. Furthermore, we introduce FedMHO-MD and FedMHO-SD to mitigate the knowledge-forgetting problem during the knowledge fusion stage, and an unsupervised data optimization solution to improve the quality of synthetic samples. Comprehensive experiments demonstrate the effectiveness of our methods, as they outperform state-of-the-art baselines in various experimental setups.

Figures

Figures reproduced from arXiv: 2502.08518 by the authors.

Figure 1
Figure 1. The heterogeneous one-shot Federated Learning (FL) frame [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Top-1 test accuracy (%) on the EMNIST dataset. ‘small’ or [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of FedMHO. Resource-sufficient clients train deep classification models, and resource-constrained clients train lightweight [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: A toy example of the knowledge-forgetting problem. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Global training epochs to reach target Top-1 test accuracy on the EMNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Top-1 test accuracy (%) curve of FedMHOs on the EMNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Top-1 test accuracy (%) of FedMHOs on the EMNIST dataset under various numbers of local epoch of classification models. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Top-1 test accuracy (%) of FedMHO-MD and FedMHO-SD under various number of synthetic samples. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Visualization of synthetic samples after unsupervised data optimization. 5 synthetic samples are present for each category. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 47 canonical work pages

  1. [1]

    Topology- aware federated learning in edge computing: A comprehensive survey,

    J. Wu, F. Dong, H. Leung, Z. Zhu, J. Zhou, and S. Drew, “Topology- aware federated learning in edge computing: A comprehensive survey,” ACM Computing Surveys, 2023

    work page 2023

  2. [2]

    Decentralized federated learning with intermediate results in mobile edge computing,

    S. Chen, Y . Xu, H. Xu, Z. Jiang, and C. Qiao, “Decentralized federated learning with intermediate results in mobile edge computing,” IEEE Transactions on Mobile Computing, vol. 23, no. 1, pp. 341–358, 2022

    work page 2022

  3. [3]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. of the 20th International Conference on Artificial Intelli- gence and Statistics, 2017, pp. 1273–1282

    work page 2017

  4. [4]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al. , “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021

    work page 2021

  5. [5]

    A field guide to federated optimization,

    J. Wang, Z. Charles, Z. Xu, G. Joshi, H. B. McMahan, M. Al-Shedivat, G. Andrew, S. Avestimehr, K. Daly, D. Data et al. , “A field guide to federated optimization,” arXiv preprint arXiv:2107.06917, 2021

    arXiv 2021

  6. [6]

    Federated learning for healthcare: Systematic review and architecture proposal,

    R. S. Antunes, C. Andr ´e da Costa, A. K¨uderle, I. A. Yari, and B. Eskofier, “Federated learning for healthcare: Systematic review and architecture proposal,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 4, pp. 1–23, 2022

    work page 2022

  7. [7]

    Fine-grained preference-aware personalized federated poi recommendation with data sparsity,

    X. Zhang, Z. Ye, J. Lu, F. Zhuang, Y . Zheng, and D. Yu, “Fine-grained preference-aware personalized federated poi recommendation with data sparsity,” in Proc. of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 413–422

    work page 2023

  8. [8]

    Federated meta-learning for fraudulent credit card detection,

    W. Zheng, L. Yan, C. Gou, and F.-Y . Wang, “Federated meta-learning for fraudulent credit card detection,” in Proc. of the 29th International Conference on International Joint Conferences on Artificial Intelligence, 2021, pp. 4654–4660

    work page 2021

Show all 69 references
  1. [9]

    Communication-efficient federated learning via knowledge distillation,

    C. Wu, F. Wu, L. Lyu, Y . Huang, and X. Xie, “Communication-efficient federated learning via knowledge distillation,” Nature communications, vol. 13, no. 1, p. 2032, 2022

    2022

  2. [10]

    Feddm: Iterative distribution matching for communication-efficient federated learning,

    Y . Xiong, R. Wang, M. Cheng, F. Yu, and C.-J. Hsieh, “Feddm: Iterative distribution matching for communication-efficient federated learning,” in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 16 323–16 332

    2023

  3. [11]

    Privacy and robustness in federated learning: Attacks and defenses,

    L. Lyu, H. Yu, X. Ma, C. Chen, L. Sun, J. Zhao, Q. Yang, and S. Y . Philip, “Privacy and robustness in federated learning: Attacks and defenses,” IEEE transactions on neural networks and learning systems , 2022

    2022

  4. [12]

    Efficient privacy-preserving federated learning under dishonest-majority setting,

    Y . Miao, D. Kuang, Lihuiwang, T. Leng, X. Liu, and J. F. Ma, “Efficient privacy-preserving federated learning under dishonest-majority setting,” Science China Information Sciences, vol. 67, no. 5, 2024

    2024

  5. [13]

    Man-in-the-middle attacks against machine learning classifiers via malicious generative models,

    D. Wang, C. Li, S. Wen, S. Nepal, and Y . Xiang, “Man-in-the-middle attacks against machine learning classifiers via malicious generative models,” IEEE Transactions on Dependable and Secure Computing , vol. 18, no. 5, pp. 2074–2087, 2020

    2020

  6. [14]

    See through gradients: Image batch recovery via gradinversion,

    H. Yin, A. Mallya, A. Vahdat, J. M. Alvarez, J. Kautz, and P. Molchanov, “See through gradients: Image batch recovery via gradinversion,” in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 16 337–16 346

    2021

  7. [15]

    One-shot federated learning,

    N. Guha, A. Talwalkar, and V . Smith, “One-shot federated learning,” arXiv preprint arXiv:1902.11175, 2019

    1902 arXiv

  8. [16]

    Model decomposition and reassembly for purified knowledge transfer in personalized federated learning,

    J. Zhang, S. Guo, X. Ma, W. Xu, Q. Zhou, J. Guo, Z. Hong, and J. Shan, “Model decomposition and reassembly for purified knowledge transfer in personalized federated learning,” IEEE Transactions on Mobile Com- puting, 2024

    2024

  9. [17]

    Achieving linear speedup in asynchronous federated learning with heterogeneous clients,

    X. Wang, Z. Li, S. Jin, and J. Zhang, “Achieving linear speedup in asynchronous federated learning with heterogeneous clients,” IEEE Transactions on Mobile Computing, 2024

    2024

  10. [18]

    Flrce: Resource-efficient federated learning with early-stopping strategy,

    Z. Niu, H. Dong, A. K. Qin, and T. Gu, “Flrce: Resource-efficient federated learning with early-stopping strategy,” IEEE Transactions on Mobile Computing, 2024

    2024

  11. [19]

    Dense: Data-free one-shot federated learning,

    J. Zhang, C. Chen, B. Li, L. Lyu, S. Wu, S. Ding, C. Shen, and C. Wu, “Dense: Data-free one-shot federated learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 21 414–21 428, 2022

    2022

  12. [20]

    Data-free one-shot feder- ated learning under very high statistical heterogeneity,

    C. E. Heinbaugh, E. Luz-Ricca, and H. Shao, “Data-free one-shot feder- ated learning under very high statistical heterogeneity,” in The Eleventh International Conference on Learning Representations, 2023

    2023

  13. [21]

    Distilled one-shot federated learning,

    Y . Zhou, G. Pu, X. Ma, X. Li, and D. Wu, “Distilled one-shot federated learning,” arXiv preprint arXiv:2009.07999, 2020

    2009 arXiv

  14. [22]

    Dataset distillation,

    T. Wang, J.-Y . Zhu, A. Torralba, and A. A. Efros, “Dataset distillation,” arXiv preprint arXiv:1811.10959, 2018

    2018 arXiv

  15. [23]

    Practical one-shot federated learning for cross-silo setting,

    Q. Li, B. He, and D. X. Song, “Practical one-shot federated learning for cross-silo setting,” in International Joint Conference on Artificial Intelligence, 2020

    2020

  16. [24]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015. UNDER REVIEW 13

    2015 arXiv

  17. [25]

    Does knowledge distillation really work?

    S. Stanton, P. Izmailov, P. Kirichenko, A. A. Alemi, and A. G. Wilson, “Does knowledge distillation really work?” Advances in Neural Informa- tion Processing Systems, vol. 34, pp. 6906–6919, 2021

    2021

  18. [26]

    Enhancing one-shot federated learning through data and ensemble co-boosting,

    R. Dai, Y . Zhang, A. Li, T. Liu, X. Yang, and B. Han, “Enhancing one-shot federated learning through data and ensemble co-boosting,” in Proceedings of the 12th International Conference on Learning Represen- tations, 2024

    2024

  19. [27]

    Learning structured output representation using deep conditional generative models,

    K. Sohn, H. Lee, and X. Yan, “Learning structured output representation using deep conditional generative models,” Advances in neural informa- tion processing systems, vol. 28, 2015

    2015

  20. [28]

    Open set learning with counterfactual images,

    L. Neal, M. Olson, X. Fern, W.-K. Wong, and F. Li, “Open set learning with counterfactual images,” in Proc. of the European Conference on Computer Vision (ECCV), 2018, pp. 613–628

    2018

  21. [29]

    Learning placeholders for open- set recognition,

    D.-W. Zhou, H.-J. Ye, and D.-C. Zhan, “Learning placeholders for open- set recognition,” inProc. of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4401–4410

    2021

  22. [30]

    One-shot federated learning via synthetic distiller-distillate communication,

    J. Zhang, S. Liu, and X. Wang, “One-shot federated learning via synthetic distiller-distillate communication,” arXiv preprint arXiv:2412.05186 , 2024

    2024 arXiv

  23. [31]

    Expanding the reach of federated learning by reducing client resource requirements,

    S. Caldas, J. Kone ˇcny, H. B. McMahan, and A. Talwalkar, “Expanding the reach of federated learning by reducing client resource requirements,” arXiv preprint arXiv:1812.07210, 2018

    2018 arXiv

  24. [32]

    Dropout: a simple way to prevent neural networks from overfitting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdi- nov, “Dropout: a simple way to prevent neural networks from overfitting,” The journal of machine learning research, vol. 15, no. 1, pp. 1929–1958, 2014

    1929

  25. [33]

    Heterofl: Computation and com- munication efficient federated learning for heterogeneous clients,

    E. Diao, J. Ding, and V . Tarokh, “Heterofl: Computation and com- munication efficient federated learning for heterogeneous clients,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . New York: OpenReview.net, 2021

    2021

  26. [34]

    Fjord: Fair and accurate federated learning under heteroge- neous targets with ordered dropout,

    S. Horvath, S. Laskaridis, M. Almeida, I. Leontiadis, S. Venieris, and N. Lane, “Fjord: Fair and accurate federated learning under heteroge- neous targets with ordered dropout,” Advances in Neural Information Processing Systems, vol. 34, pp. 12 876–12 889, 2021

    2021

  27. [35]

    Feddse: Distribution-aware sub-model extraction for federated learning over resource-constrained devices,

    H. Wang, Y . Jia, M. Zhang, Q. Hu, H. Ren, P. Sun, Y . Wen, and T. Zhang, “Feddse: Distribution-aware sub-model extraction for federated learning over resource-constrained devices,” in Proceedings of the ACM on Web Conference 2024, 2024, pp. 2902–2913

    2024

  28. [36]

    Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction,

    S. Alam, L. Liu, M. Yan, and M. Zhang, “Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction,” in Advances in Neural Information Processing Systems. New York: MIT Press, 2022

    2022

  29. [37]

    Efficient split-mix federated learning for on-demand and in-situ customization,

    J. Hong, H. Wang, Z. Wang, and J. Zhou, “Efficient split-mix federated learning for on-demand and in-situ customization,” in International Conference on Learning Representations, 2022

    2022

  30. [38]

    Fedmd: Heterogenous federated learning via model distillation,

    D. Li and J. Wang, “Fedmd: Heterogenous federated learning via model distillation,” arXiv preprint arXiv:1910.03581, 2019

    1910 arXiv

  31. [39]

    Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,

    J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in International conference on machine learning . PMLR, 2020, pp. 6028–6039

    2020

  32. [40]

    Group knowledge transfer: Federated learning of large cnns at the edge,

    C. He, M. Annavaram, and S. Avestimehr, “Group knowledge transfer: Federated learning of large cnns at the edge,” Advances in Neural Information Processing Systems, vol. 33, pp. 14 068–14 080, 2020

    2020

  33. [41]

    Exploring the distributed knowledge congruence in proxy-data-free federated distillation,

    Z. Wu, S. Sun, Y . Wang, M. Liu, Q. Pan, J. Zhang, Z. Li, and Q. Liu, “Exploring the distributed knowledge congruence in proxy-data-free federated distillation,” ACM Transactions on Intelligent Systems and Technology, vol. 15, no. 2, pp. 1–34, 2024

    2024

  34. [42]

    Distributed learning of deep neural network over multiple agents,

    O. Gupta and R. Raskar, “Distributed learning of deep neural network over multiple agents,” Journal of Network and Computer Applications , vol. 116, pp. 1–8, 2018

    2018

  35. [43]

    Federated mutual learning: a collaborative machine learning method for heterogeneous data, models, and objectives,

    T. Shen, J. Zhang, X. Jia, F. Zhang, Z. Lv, K. Kuang, C. Wu, and F. Wu, “Federated mutual learning: a collaborative machine learning method for heterogeneous data, models, and objectives,” Frontiers of Information Technology & Electronic Engineering , vol. 24, no. 10, pp. 1390...

    2023

  36. [44]

    Pervasivefl: Pervasive federated learning for heterogeneous iot systems,

    J. Xia, T. Liu, Z. Ling, T. Wang, X. Fu, and M. Chen, “Pervasivefl: Pervasive federated learning for heterogeneous iot systems,”IEEE Trans- actions on Computer-Aided Design of Integrated Circuits and Systems , vol. 41, no. 11, pp. 4100–4111, 2022

    2022

  37. [45]

    Deep mutual learning,

    Y . Zhang, T. Xiang, T. M. Hospedales, and H. Lu, “Deep mutual learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4320–4328

    2018

  38. [46]

    Ensemble distillation for robust model fusion in federated learning,

    T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 2351–2363, 2020

    2020

  39. [47]

    Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data,

    S. Itahara, T. Nishio, Y . Koda, M. Morikura, and K. Ya- mamoto, “Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data,” IEEE Transactions on Mobile Computing , vol. 22, no. 1, pp. 191–205, 2023

    2023

  40. [48]

    Heteroge- neous ensemble knowledge transfer for training large models in federated learning,

    Y . J. Cho, A. Manoel, G. Joshi, R. Sim, and D. Dimitriadis, “Heteroge- neous ensemble knowledge transfer for training large models in federated learning,” in International Joint Conferences on Artificial Intelligence Organization, 7 2022, pp. 2881–2887

    2022

  41. [49]

    Data-free learning of student networks,

    H. Chen, Y . Wang, C. Xu, Z. Yang, C. Liu, B. Shi, C. Xu, C. Xu, and Q. Tian, “Data-free learning of student networks,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 3514– 3522

    2019

  42. [50]

    Momentum adversarial distillation: Handling large distribution shifts in data-free knowledge distillation,

    K. Do, T. H. Le, D. Nguyen, D. Nguyen, H. Harikumar, T. Tran, S. Rana, and S. Venkatesh, “Momentum adversarial distillation: Handling large distribution shifts in data-free knowledge distillation,” Advances in Neural Information Processing Systems , vol. 35, pp. 10 055–10 067, 2022

    2022

  43. [51]

    Fine-tuning global model via data-free knowledge distillation for non-iid federated learning,

    L. Zhang, L. Shen, L. Ding, D. Tao, and L.-Y . Duan, “Fine-tuning global model via data-free knowledge distillation for non-iid federated learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 174–10 183

    2022

  44. [52]

    Dfrd: Data-free robustness distillation for heterogeneous federated learning,

    S. Wang, Y . Fu, X. Li, Y . Lan, M. Gaoet al., “Dfrd: Data-free robustness distillation for heterogeneous federated learning,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  45. [53]

    One-shot fed- erated learning: theoretical limits and algorithms to achieve them,

    S. Salehkaleybar, A. Sharifnassab, and S. J. Golestani, “One-shot fed- erated learning: theoretical limits and algorithms to achieve them,” The Journal of Machine Learning Research , vol. 22, no. 1, pp. 8485–8531, 2021

    2021

  46. [54]

    Distilling knowledge via knowledge review,

    P. Chen, S. Liu, H. Zhao, and J. Jia, “Distilling knowledge via knowledge review,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 5008–5017

    2021

  47. [55]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  48. [56]

    On information and sufficiency,

    S. Kullback and R. A. Leibler, “On information and sufficiency,” The annals of mathematical statistics, vol. 22, no. 1, pp. 79–86, 1951

    1951

  49. [57]

    Least squares quantization in pcm,

    S. Lloyd, “Least squares quantization in pcm,” IEEE transactions on information theory, vol. 28, no. 2, pp. 129–137, 1982

    1982

  50. [58]

    Gradient-based learning applied to document recognition,

    Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998

    1998

  51. [59]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,

    H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017

    2017 arXiv

  52. [60]

    Reading digits in natural images with unsupervised feature learning,

    N. Yuval, “Reading digits in natural images with unsupervised feature learning,” in Proc. of the NIPS Workshop on Deep Learning and Unsu- pervised Feature Learning, 2011

    2011

  53. [61]

    Emnist: Extending mnist to handwritten letters,

    G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik, “Emnist: Extending mnist to handwritten letters,” in 2017 international joint conference on neural networks (IJCNN). IEEE, 2017, pp. 2921–2926

    2017

  54. [62]

    Measuring the effects of non- identical data distribution for federated visual classification,

    T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the effects of non- identical data distribution for federated visual classification,” arXiv preprint arXiv:1909.06335, 2019

    1909 arXiv

  55. [63]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” Proc. of Machine learning and systems, vol. 2, pp. 429–450, 2020

    2020

  56. [64]

    Scaffold: Stochastic controlled averaging for federated learn- ing,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learn- ing,” in International conference on machine learning . PMLR, 2020, pp. 5132–5143

    2020

  57. [65]

    Data-free knowledge distillation for het- erogeneous federated learning,

    Z. Zhu, J. Hong, and J. Zhou, “Data-free knowledge distillation for het- erogeneous federated learning,” in International conference on machine learning. PMLR, 2023, pp. 12 878–12 889

    2023

  58. [66]

    Efficientnetv2: Smaller models and faster training,

    M. Tan and Q. Le, “Efficientnetv2: Smaller models and faster training,” in International conference on machine learning . PMLR, 2021, pp. 10 096–10 106

    2021

  59. [67]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  60. [68]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proc. of the IEEE conference on computer vision and pattern recognition , 2018, pp. 4510–4520

    2018

  61. [69]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.