Pith. sign in

REVIEW 5 major objections 4 minor 75 references

Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FedDHAD dynamically weights client models by non-IID degree and applies neuron-adaptive dropout, achieving up to 6.7% higher accuracy, 2.02x faster training, and 15% lower computation cost than state-of-the-art federated learning methods.

desk verdict A plausible heuristic with a solid experimental sweep, but the convergence theory does not cover the actual algorithm and the reported gains are not rigorously supported. read the letter →

arxiv 2507.10430 v2 pith:VJFI3JKF submitted 2025-07-14 cs.DC cs.AIcs.LG

Ji Liu , Beichen Ma , Qiaolin Yu , Ruoming Jin , Jingbo Zhou , Yang Zhou , Huaiyu Dai , Haixun Wang
show 2 more authors
Dejing Dou Patrick Valduriez
This is my paper
classification cs.DCcs.AIcs.LG
keywords federatedlearningnon-IIDdatadynamicaggregationadaptivedropoutdeviceheterogeneitymodelclientweighting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning loses accuracy when devices hold very different data, and it slows down when some devices are much weaker than others. This paper tries to show that both problems can be fixed at once. Its aggregation step, FedDH, weights each uploaded model inversely to the device's estimated non-IID degree, where that degree is a learned linear transformation of the Jensen-Shannon divergence between the device's label distribution and the global label distribution. Its dropout step, FedAD, prunes each device's sub-model neuron-by-neuron based on measured neuron importance and device speed. If the framework is right, it converges to a better final model in less wall-clock time and with lower compute cost than a broad set of baselines, and the advantage grows as data skew, bandwidth limits, and device count increase.

What carries the argument

The load-bearing object is the dynamic non-IID degree $D^{\text{non-IID}}_k(P_k,\upsilon_k,b_k)=\upsilon_k D_{JS}(P_k)+b_k$: a Jensen-Shannon divergence between the device's label distribution and the global label distribution, rescaled and shifted by two learnable parameters updated by gradient descent on the server. This feeds Formula 11's ratio weighting, which inverts the degree so that more strongly skewed devices count less during aggregation. The supporting machinery is FedAD: rank- and weight-based neuron importance scores, a Hessian-eigenvalue gap rule to set a device-level dropout rate, and per-neuron masks that shrink the sub-model for slow devices while preserving the most influential filters and neurons.

What would settle it

Run FedDHAD with the server given oracle non-IID degrees $\Gamma_k = \mathbb{E}[F_k(w^*)] - \mathbb{E}[F_k(w_k^*)]$ instead of the learned proxy, and compare accuracy and convergence time against FedDHAD with the proxy; if the proxy version does not match the oracle version, the learned estimate is not delivering the theoretical benefit claimed by Theorem 5.7.

Watch

Extended reading notes

Core claim

The central claim is that aggregation weights should follow the ratio $q_k = \frac{n_k / D^{\text{non-IID}}_k}{\sum_{k'\in S_t} n_{k'}/D^{\text{non-IID}}_{k'}}$, where $D^{\text{non-IID}}_k = \upsilon_k D_{JS}(P_k) + b_k$ is the JS divergence between the device's label distribution and the global distribution, rescaled and shifted by learnable control parameters. Theorem 5.7 shows that when weights take the ideal form $p_{k,t}=n_k/\Gamma_{k,t}$ with $\Gamma_{k,t} = \mathbb{E}[F_k(w^*)] - \mathbb{E}[F_k(w_k^*)]$, the convergence upper bound is no larger than with uniform sample-count weights, because the weighted heterogeneity term shrinks; the learned proxy is meant to realize that ideal. FedAD's contribution is a per-neuron dropout rule driven by rank- or weight-based importance scores plus a Hessian-eigenvalue gap rule for choosing each device's dropout rate, so slow devices train smaller sub-models without discarding important neurons. The claimed net result is higher accuracy reached faster and at lower FLOP count across CIFAR-10, CIFAR-100, SVHN, and TinyImageNet.

Load-bearing premise

The theoretical speedup rests on the learned linear transformation of JS divergence faithfully representing the true non-IID degree; the paper's own measured correlation of $R=0.552$ means most of the true degree's variation is not captured by the proxy.

Editorial extensions

If this is right

  • Devices whose label distribution deviates most from the global distribution will contribute less to the global model, reducing the drag that non-IID data exerts on convergence.
  • Adaptive dropout reduces communication and computation roughly in proportion to each device's capacity, so the slowest devices become a smaller bottleneck in each round.
  • The framework's advantages become larger under poor bandwidth, high non-IID degree, and larger device counts, as shown by the paper's environment experiments.
  • The server-side cost of the dynamic weighting remains small, because the control parameters need only label-distribution metadata or a gradient-based estimate when privacy rules out transferring metadata.
  • FedDHAD can be combined with asynchronous or decentralized aggregation schemes, an extension the paper names as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own correlation analysis ($R=0.552$ between JS divergence and true non-IID degree) means the learned linear proxy explains only about 30% of the variance in $\Gamma$; the authors do not dwell on that gap, but it implies the convergence theorem justifies the weighting only to the extent the proxy tracks $\Gamma$, so part of the empirical gain may come from the control parameters learnin
  • A direct test of the weighting hypothesis is to give the server oracle values of $\Gamma$ instead of the learned proxy; if the oracle version does not beat the proxy version, then the theoretical motivation is not what is driving the measured gains.
  • The neuron-adaptive dropout recipe could be transferred to transformers by treating attention heads as the units being pruned, with per-head importance scores replacing per-filter rank scores; the Hessian-gap rule would then need to be re-derived for attention layers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. This paper proposes FedDHAD, a federated learning framework that combines dynamic aggregation weights based on an estimated non-IID degree (FedDH) with device-adaptive neuron dropout (FedAD). The authors claim significant improvements in accuracy, training time, and computation cost, and they provide convergence theorems (Theorems 5.6 and 5.7) plus experiments on four datasets (CIFAR-10/100, SVHN, TinyImageNet) and four model architectures (LeNet, CNN, VGG, ResNet).

Significance. The problem addressed is relevant, and the experimental study is broad, covering thirteen baselines and several model/dataset combinations. The combination of heterogeneity-aware weighting with structured dropout is a reasonable design idea. However, the theoretical analysis fails to connect to the implemented algorithm, the assumptions do not cover the experimental settings, and the evidence for the non-IID degree proxy is weak (R^2 ≈ 0.30). As a result, the claimed advantages are not convincingly attributed to the proposed mechanisms.

major comments (5)
  1. [Section 5.1.2, Eq. (11); Section 5.1.3, Theorem 5.7] Theorem 5.7 proves a smaller bound only for weights p_{k,t}=n_k/Γ_{k,t} with Γ_{k,t} defined in Eq. (12), but the aggregation in Algorithm 1 and Eq. (11) uses q_k ∝ n_k/(υ_k D_JS(P_k)+b_k). No step in the proof, the appendix, or the experimental section shows that the learned (υ_k,b_k) from Eq. (10) produce weights equal to or close to p_{k,t}; therefore the theorem does not apply to FedDH as implemented.
  2. [Appendix, Convergence Proof; Theorem 5.6] Theorem 5.6 contains undefined symbols m, β, G^2_b, σ^2_b, and Q; the proof introduces Q without definition, and the theorem's statement E||F(w^{T,H})−F(w*)|| ≤ (L/2)·υ/(TH+γ) does not match the derived bound E[F(w^{t,h})−F(w*)] ≤ (L/2)·υ/((t−1)H+h+γ). These inconsistencies make the proof unverifiable.
  3. [Section 5.1.3, Assumptions 5.1–5.5; Section 6.1] The convergence analysis assumes each F_k is L-smooth and μ-strongly convex with bounded gradients, but the experiments use LeNet, CNN, VGG, and ResNet on image data, none of which satisfy these conditions. The paper does not provide any argument that the theory extends to non-convex deep networks, so the theorems do not support the empirical accuracy claims.
  4. [Appendix, Correlation Analysis] The reported Pearson correlation of 55.2% (R^2 ≈ 0.30) between JS divergence and the non-IID degree Γ is weak evidence for the linear relationship assumed in Eq. (4). Since most of the variance is unexplained and no validation is given for the learned transformation against Γ, the claim that FedDH weights by a meaningful non-IID degree is not supported.
  5. [Section 5.1.2, Eq. (10)] Updating the control parameters υ_k and b_k by gradient descent on the global loss F(w(Q)) optimizes the aggregation weights for the same objective used to train the model. The paper offers no convergence analysis or baseline comparison against directly learned weights, so the reported gains could stem from this model-fitting feedback rather than from the proposed non-IID-degree mechanism.
minor comments (4)
  1. [Section 3, Related Work] There is a typo: 'servre' should be 'server' in the description of Federated Dropout.
  2. [Table 2 and Section 6.4.3] Table 2 reports server computation overhead as values around 0.1 (seconds, presumably), while Section 6.4.3 states overheads of 'up to 10.4%' and 'up to 15.6%' without explaining the conversion; the relationship between the table entries and the percentages is unclear.
  3. [Appendix, Comparative Experiments] Tables 6 and 7 appear only in the appendix and are not referenced in the main text; the main text refers to Table 2 for server overhead, making the table numbering inconsistent.
  4. [Section 6.2.2] The text states that FedAD's training time is 'up to 27.6%' longer than FjORD, but Table 3 shows FedAD (1394 s) shorter than FjORD (1663 s) for LeNet on CIFAR-10; this numerical inconsistency should be corrected.

Circularity Check

2 steps flagged · score 6.0 of 10

The FedDH convergence claim is not a derivation: the 'non-IID degree' is fitted to the global loss while the theorem applies to a different Γ-based weighting; the claimed advantage reduces to the fit.

  1. self definitional [Section 5.1.1-5.1.2, Formulas 4, 10, 11]
    "We exploit a linear transformation of JS divergence in Formula 4 while the parameters 𝑣𝑘 and 𝑏𝑘 are dynamically updated within FedDH based on Formula 10 to minimize the global loss function. ... The control parameters are updated on the server using Formula 10, where ... ∇Υ𝑡𝐹(𝑤𝑡(𝑄𝑡)) and ∇𝑏𝑡𝐹(𝑤𝑡(𝑄𝑡)) refer to the partial derivative of the global model with respect to control parameters Υ𝑡 and 𝑏𝑡, respectively, which can be calculated based on the sub network of each device."

    The quantity called the non-IID degree, D_nonIID = υ_k D_JS(P_k)+b_k, is not measured or independently validated; its parameters are optimized to minimize F(w(Q)), the same global loss whose reduction is presented as the method's benefit. Formula 3 even defines Q = argmin_Q F(w,Q). So 'weighting by the learned degree gives smaller loss' is true by construction of the fitting procedure, and cannot serve as evidence that the JS-based heterogeneity proxy is the cause. The theorem cited is about different weights p=n/Γ, so it does not supply the missing independent content.

  2. fitted input called prediction [Section 5.1.3, Theorem 5.7 and Formula 11; Appendix Correlation Analysis]
    "Theorem 5.7. ... when the weights 𝑝𝑘,𝑡 = 𝑛𝑘/Γ𝑘,𝑡 / sum, the upper bound of 𝐸∥ 𝐹(𝑤𝑇,𝐻)− 𝐹(𝑤∗)∥ is equal or smaller than that when 𝑝𝑘,𝑡 = 𝑛𝑘 / sum ... . ... Γ𝑘,𝑡 = 𝐸[𝐹𝑘(𝑤∗)]− 𝐸[𝐹𝑘(𝑤𝑘∗)] (12). ... The analysis yields a Pearson correlation coefficient of 𝑅∗∗ = 55.2% ... which implies strong linear relationship between JS divergence and the non-IID degree."

    The theorem's optimal aggregation weights use Γ from Formula 12, but the algorithm's Formula 11 uses q_k = n_k/(υ_k D_JS(P_k)+b_k) with (υ,b) gradient-fitted to F. No step in the paper shows that the fitted D_nonIID approaches Γ; the only empirical link (JS vs Γ) has R=0.552 (R^2≈0.30), and the linear transform's parameters are fit to the loss, not to Γ. Thus when the paper says 'Formula 11 can incur higher accuracy with smaller loss compared with traditional aggregation methods (see theoretical analysis in Section 5.1.3)', it is renaming a fitted parameter as the theorem's input: the claimed prediction is not derived from the theorem.

full rationale

FedDHAD's experimental comparison is self-contained against external baselines; the dropout component is not circular, and no load-bearing self-citation chain was found. However, the central theoretical claim for FedDH is partially circular. Formula 10 fits υ_k,b_k by gradient descent on the global loss, and Formula 11 then defines aggregation weights through this same fitted quantity, so 'the FedDH weighting reduces the loss' is partly true by construction. The convergence theorem that is invoked (Theorem 5.7) requires weights p=n/Γ with the true non-IID degree Γ, not the fitted JS linear transform; the paper never proves q approximates p, and its own correlation evidence (R=55.2%) accounts for only about 30% of the variance. The claim that the dynamic non-IID weighting, rather than the server-side fit, drives the gains is therefore not established by the derivation. This warrants a partial circularity score of 6, not a higher one, because the empirical benchmarks are external and the dropout contribution is independent.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central mechanism relies on 2N learnable control parameters (υ,b) per device, an ad hoc eigenvalue threshold, and strong-convexity assumptions that are not satisfied by the neural networks tested. The only 'entity' introduced is the linearized non-IID degree formula, which is a fitted model rather than an independent law.

free parameters (5)
  • Per-device control parameters υ_k, b_k = Initialized to 1 and 0, then updated by Formula 10 via gradient descent
    These 2N parameters are fit to the global loss and directly determine the aggregation weights in Formula 11.
  • Learning rates λ_Υ, λ_B and decays for control parameters = Table 5 lists different values per model/dataset
    Tuned per experiment; they control the update of the bias parameters.
  • Threshold 4L_k in Hessian eigenvalue gap = 4
    Ad hoc constant used to select the dropout rate d_k=m_k/h_k in Section 5.2.1; no theoretical justification.
  • Dropout update interval C and variation condition = Unspecified
    The paper states dropout rates are updated 'when certain conditions are met' and 'after C rounds', but C and the variation threshold are never defined.
  • Beta in gradient-based distribution estimation (FedDHE) = Set as hyperparameter, not specified in Table 5
    Used in Formula 8 to convert gradient norms to class proportions; value is not given.
assumptions (6)
  • standard math Assumptions 5.1-5.5: L-smoothness, μ-strong convexity, unbiased stochastic gradients, bounded variance, bounded gradient for each local loss F_k
    Invoked in Theorem 5.6 and its proof; strong convexity is violated by deep neural networks used in the experiments.
  • domain assumption Label distribution metadata P_k can be transmitted to the server without privacy violation
    Assumed in Section 5.1.1 with citations [52-54]; the paper does not quantify the privacy leakage.
  • ad hoc to paper JS divergence is linearly related to the true non-IID degree Γ_k
    Formula 4 defines this linear model; the Appendix correlation analysis shows only R=0.552, so a large fraction of variance is unexplained.
  • domain assumption The bi-level optimization between local models and control parameters converges
    The paper cites [60,61] instead of proving convergence for the specific alternating scheme used in Algorithm 1.
  • ad hoc to paper Hessian eigenvalue gap criterion d_k=m_k/h_k preserves accuracy
    Section 5.2.1 justifies it by citation [64]; no derivation or ablation isolating this choice is given.
  • domain assumption Randomly selected devices are unbiased samples of the device population
    Used in the proof of Theorem 5.6 to eliminate the A3 cross-term; the paper does not discuss how client selection bias would affect the bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout." pith.science (2026). https://pith.science/paper/VJFI3JKF

@misc{pith2026250710430,
  author       = {Pith},
  title        = {Pith review of: Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VJFI3JKF}},
  note         = {Machine review of arXiv:2507.10430}
}
read the original abstract

Federated Learning (FL) is a promising distributed machine learning approach that enables collaborative training of a global model using multiple edge devices. The data distributed among the edge devices is highly heterogeneous. Thus, FL faces the challenge of data distribution and heterogeneity, where non-Independent and Identically Distributed (non-IID) data across edge devices may yield in significant accuracy drop. Furthermore, the limited computation and communication capabilities of edge devices increase the likelihood of stragglers, thus leading to slow model convergence. In this paper, we propose the FedDHAD FL framework, which comes with two novel methods: Dynamic Heterogeneous model aggregation (FedDH) and Adaptive Dropout (FedAD). FedDH dynamically adjusts the weights of each local model within the model aggregation process based on the non-IID degree of heterogeneous data to deal with the statistical data heterogeneity. FedAD performs neuron-adaptive operations in response to heterogeneous devices to improve accuracy while achieving superb efficiency. The combination of these two methods makes FedDHAD significantly outperform state-of-the-art solutions in terms of accuracy (up to 6.7% higher), efficiency (up to 2.02 times faster), and computation cost (up to 15.0% smaller).

Figures

Figures reproduced from arXiv: 2507.10430 by the authors.

Figure 1
Figure 1. The system model of FedDHAD. Problem formulation. The problem to address in this paper is how to efficiently train a global model of high performance with a FL setting. We break down the problem into two aspects, i.e., model performance and training efficiency. Thus, the first problem is how to achieve high model performance (defined in Formula 1) while considering the data heterogeneity. As data heterogeneity is cr… view at source ↗
Figure 2
Figure 2. The accuracy and training time with FedDHAD and various baseline methods for LeNet and [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. The accuracy and training time with FedDHAD and various baseline methods for LeNet, CNN, [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The accuracy and training rounds with FedDHAD and various baseline methods for LeNet [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Performance of FedDHAD and various baseline methods with LeNet on CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: The accuracy and training time with FedDH, FedDHE, and FedJS for LeNet and CNN on [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Significant positive correlations have been found between JS divergence and non-IID degree [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 70 canonical work pages

  1. [1]

    Brendan McMahan, Brendan Avent, Aurélien Bellet, and Mehdi Bennis et al

    Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, and Mehdi Bennis et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1-2):1–210, 2021

    work page 2021

  2. [2]

    Trustworthy federated learning: Privacy, security, and beyond

    Chunlu Chen, Ji Liu, Haowen Tan, Xingjian Li, Kevin I-Kai Wang, Peng Li, Kouichi Sakurai, and Dejing Dou. Trustworthy federated learning: Privacy, security, and beyond. pages 1–32, 2024

    work page 2024

  3. [3]

    Enhancing trust and privacy in distributed networks: a comprehensive survey on blockchain-based federated learning

    Ji Liu, Chunlu Chen, Yu Li, Lin Sun, Yulun Song, Jingbo Zhou, Bo Jing, and Dejing Dou. Enhancing trust and privacy in distributed networks: a comprehensive survey on blockchain-based federated learning. Knowledge and Information Systems, pages 1–27, 2024

    work page 2024

  4. [4]

    From distributed machine learning to federated learning: a survey

    Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong, and Dejing Dou. From distributed machine learning to federated learning: a survey. Knowledge and Information Systems, 64(4):885–917, 2022

    work page 2022

  5. [5]

    General data protection regulation

    Official Journal of the European Union. General data protection regulation. https://eur-lex.europa.eu/legal-content/EN/ TXT/PDF/?uri=CELEX:32016R0679, 2016. Online; accessed 09/05/2022

    work page 2016

  6. [6]

    California consumer privacy act home page

    Californians for Consumer Privacy. California consumer privacy act home page. https://www.caprivacy.org/, 2020. Online; accessed 09/05/2022

    work page 2020

  7. [7]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics (AISTATS), pages 1273–1282, Fort Lauderdale, FL, USA, 2017. PMLR

    work page 2017

  8. [8]

    Heterps: Distributed deep learning with reinforcement learning based scheduling in heterogeneous environments

    Ji Liu, Zhihua Wu, Danlei Feng, Minxu Zhang, Xinxuan Wu, Xuefeng Yao, Dianhai Yu, Yanjun Ma, Feng Zhao, and Dejing Dou. Heterps: Distributed deep learning with reinforcement learning based scheduling in heterogeneous environments. Future Generation Computer Systems, 148:106–117, 2023

    work page 2023

Show all 75 references
  1. [9]

    Scaling distributed machine learning with the parameter server

    Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. Scaling distributed machine learning with the parameter server. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), pag...

    2014

  2. [10]

    Vincent Poor

    Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 7611–7623, Virtual Event, 2020. Curran ...

    2020

  3. [11]

    Multi-job intelligent scheduling with cross-device federated learning

    Ji Liu, Juncheng Jia, Beichen Ma, Chendi Zhou, Jingbo Zhou, Yang Zhou, Huaiyu Dai, and Dejing Dou. Multi-job intelligent scheduling with cross-device federated learning. IEEE Transactions on Parallel and Distributed Systems, 34(2):535–551, 2022

    2022

  4. [12]

    Efficient device scheduling with multi-job federated learning

    Chendi Zhou, Ji Liu, Juncheng Jia, Jingbo Zhou, Yang Zhou, Huaiyu Dai, and Dejing Dou. Efficient device scheduling with multi-job federated learning. AAAI Conf. on Artificial Intelligence, 36(9):9971–9979, 2022

    2022

  5. [13]

    Federated learning on non-iid data silos: An experimental study

    Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In IEEE Int. Conf. on Data Engineering (ICDE), pages 965–978, Kuala Lumpur, Malaysia, 2022. IEEE

    2022

  6. [14]

    On the convergence of fedavg on non-iid data

    Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. In Int. Conf. on Learning Representations (ICLR), pages 1–26, Addis Ababa, Ethiopia, 16 Sep 2024. OpenReview.net

    2024

  7. [15]

    Federated optimization in heterogeneous networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Machine Learning and Systems (MLSys), volume 2, pages 429–450, Austin, TX, USA,

  8. [16]

    Jensen-shannon divergence and Hilbert space embedding

    Bent Fuglede and Flemming Topsoe. Jensen-shannon divergence and Hilbert space embedding. In Int. Symposium on Information Theory (ISIT), page 31, Chicago, IL, United States, 2004. IEEE

    2004

  9. [17]

    Information theory and statistics

    Solomon Kullback. Information theory and statistics. Courier Corporation, United States, 1997

    1997

  10. [18]

    Multi-center federated learning

    Ming Xie, Guodong Long, Tao Shen, Tianyi Zhou, Xianzhi Wang, Jing Jiang, and Chengqi Zhang. Multi-center federated learning. arXiv preprint arXiv:2108.08647, 26(1):481–500, 2021

    2021 arXiv

  11. [19]

    Towards federated learning at scale: System design

    Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Koneˇcný, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. Towards federated learning at scale: Syst...

  12. [20]

    Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov

    Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014

    1929

  13. [21]

    Fjord: Fair and accurate federated learning under heterogeneous targets with ordered dropout

    Samuel Horvath, Stefanos Laskaridis, Mario Almeida, Ilias Leontiadis, Stylianos Venieris, and Nicholas Lane. Fjord: Fair and accurate federated learning under heterogeneous targets with ordered dropout. In Advances in Neural Information Processing Systems (NeurIPS), volume 34,...

    2021

  14. [22]

    Adaptive federated dropout: Improving communication efficiency and generalization for federated learning

    Nader Bouacida, Jiahui Hou, Hui Zang, and Xin Liu. Adaptive federated dropout: Improving communication efficiency and generalization for federated learning. In IEEE Conf. on Computer Communications Workshops (INFOCOM WKSHPS), pages 1–6, Vancouver, BC, Canada, 2021. IEEE. Effic...

    2021

  15. [23]

    Federated dropout–a simple approach for enabling federated learning on resource constrained devices

    Dingzhu Wen, Ki-Jun Jeon, and Kaibin Huang. Federated dropout–a simple approach for enabling federated learning on resource constrained devices. IEEE Wireless Communications Letters, 11(5):923–927, 2022

    2022

  16. [24]

    Federated learning based on dynamic regularization

    Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. In Int. Conf. on Learning Representations (ICLR), pages 1–36, Vienna, Austria, 2021. OpenReview.net

    2021

  17. [25]

    Model-contrastive federated learning

    Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 10713–10722, Virtual Event, 2021. IEEE

    2021

  18. [26]

    SCAFFOLD: Stochastic controlled averaging for federated learning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Int. Conf. on Machine Learning (ICML), volume 119, pages 5132–5143, Vienna, Austria, 2020. PMLR

    2020

  19. [27]

    Partialfed: Cross-domain personalized federated learning via partial initialization

    Benyuan Sun, Hongxing Huo, YI YANG, and Bo Bai. Partialfed: Cross-domain personalized federated learning via partial initialization. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 23309–23320, Virtual Event, 2021. Curran Associates

    2021

  20. [28]

    Sageflow: Robust federated learning against both stragglers and adversaries

    Jungwuk Park, Dong-Jun Han, Minseok Choi, and Jaekyun Moon. Sageflow: Robust federated learning against both stragglers and adversaries. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 840–851, Virtual Event, 2021. Curran Associates

    2021

  21. [29]

    Efficient asynchronous federated learning with sparsification and quantization

    Juncheng Jia, Ji Liu, Chendi Zhou, Hao Tian, Mianxiong Dong, and Dejing Dou. Efficient asynchronous federated learning with sparsification and quantization. Concurrency and Computation: Practice and Experience, 36(9):e8002, 2024

    2024

  22. [30]

    Aedfl: efficient asyn- chronous decentralized federated learning with heterogeneous devices

    Ji Liu, Tianshi Che, Yang Zhou, Ruoming Jin, Huaiyu Dai, Dejing Dou, and Patrick Valduriez. Aedfl: efficient asyn- chronous decentralized federated learning with heterogeneous devices. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pages 833–841...

    2024

  23. [31]

    Efficient federated learning with timely update dissemination

    Juncheng Jia, Ji Liu, Chao Huo, Yihui Shen, Yang Zhou, Huaiyu Dai, and Dejing Dou. Efficient federated learning with timely update dissemination. Knowledge and Information Systems, pages 1–38, 2025

    2025

  24. [32]

    Fedasmu: Efficient asynchronous federated learning with dynamic staleness-aware model update

    Ji Liu, Juncheng Jia, Tianshi Che, Chao Huo, Jiaxiang Ren, Yang Zhou, Huaiyu Dai, and Dejing Dou. Fedasmu: Efficient asynchronous federated learning with dynamic staleness-aware model update. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13...

    2024

  25. [33]

    Federated machine learning: Concept and applications.ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019

    Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications.ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019

    2019

  26. [34]

    Exploring one-shot semi-supervised federated learning with pre-trained diffusion models

    Mingzhao Yang, Shangchao Su, Bin Li, and Xiangyang Xue. Exploring one-shot semi-supervised federated learning with pre-trained diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16325–16333, 2024

    2024

  27. [35]

    Jamaloddin Golestani

    Saber Salehkaleybar, Arsalan Sharifnassab, and S. Jamaloddin Golestani. One-shot federated learning: Theoretical limits and algorithms to achieve them. Journal of Machine Learning Research, 22(189):1–47, 2021

    2021

  28. [36]

    Semi-cyclic stochastic gradient descent

    Hubert Eichner, Tomer Koren, Brendan McMahan, Nathan Srebro, and Kunal Talwar. Semi-cyclic stochastic gradient descent. In Int. Conf. on Machine Learning (ICML), pages 1764–1773, Long Beach, California, United States, 2019. PMLR

    2019

  29. [37]

    Benchmarking fedavg and fedcurv for image classification tasks

    Bruno Casella, Roberto Esposito, Carlo Cavazzoni, and Marco Aldinucci. Benchmarking fedavg and fedcurv for image classification tasks. arXiv preprint arXiv:2303.17942, 2023

    2023 arXiv

  30. [38]

    Fair federated learning under domain skew with local consistency and domain diversity

    Yuhang Chen, Wenke Huang, and Mang Ye. Fair federated learning under domain skew with local consistency and domain diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12077–12086, 2024

    2024

  31. [39]

    Accelerated federated learning with decoupled adaptive optimization

    Jiayin Jin, Jiaxiang Ren, Yang Zhou, Lingjuan Lv, Ji Liu, and Dejing Dou. Accelerated federated learning with decoupled adaptive optimization. In Int. Conf. on Machine Learning (ICML) , volume 162, pages 10298–10322, Baltimore, Maryland, United States, 2022. PMLR

    2022

  32. [40]

    Adaptive gradient-based meta-learning methods

    Mikhail Khodak, Maria-Florina F Balcan, and Ameet S Talwalkar. Adaptive gradient-based meta-learning methods. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pages 1–12, Vancouver, BC, Canada, 2019. Curran Associates

    2019

  33. [41]

    Federated multi-task learning

    Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pages 1–11, Long Beach, CA, United States, 2017. Curran Associates

    2017

  34. [42]

    Adversarial collaborative learning on non-iid features

    Qinbin Li, Bingsheng He, and Dawn Song. Adversarial collaborative learning on non-iid features. In International Conference on Machine Learning, pages 19504–19526. PMLR, 2023

    2023

  35. [43]

    Generalizable heterogeneous federated cross-correlation and instance similarity learning

    Wenke Huang, Mang Ye, Zekun Shi, and Bo Du. Generalizable heterogeneous federated cross-correlation and instance similarity learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2):712–728, 2023

    2023

  36. [44]

    Bayesian nonparametric federated learning of neural networks

    Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. In Int. Conf. on Machine Learning (ICML), volume 97, 28 Ji Liu, Beichen Ma, Qiaolin Yu, Ruoming Jin, Jingbo Zh...

    2019

  37. [45]

    Ensemble distillation for robust model fusion in federated learning

    Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Information Processing Systems (NeurIPS) , volume 33, pages 2351–2363, Virtual Event, 2020. Curran Associates

    2020

  38. [46]

    An upload-efficient scheme for transferring knowledge from a server-side pre-trained generator to clients in heterogeneous federated learning

    Jianqing Zhang, Yang Liu, Yang Hua, and Jian Cao. An upload-efficient scheme for transferring knowledge from a server-side pre-trained generator to clients in heterogeneous federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  39. [47]

    Augfl: Augmenting federated learning with pretrained models

    Sheng Yue, Zerui Qin, Yongheng Deng, Ju Ren, Yaoxue Zhang, and Junshan Zhang. Augfl: Augmenting federated learning with pretrained models. arXiv preprint arXiv:2503.02154, 2025

    2025 arXiv

  40. [48]

    Grounding foundation models through federated transfer learning: A general framework

    Yan Kang, Tao Fan, Hanlin Gu, Xiaojin Zhang, Lixin Fan, and Qiang Yang. Grounding foundation models through federated transfer learning: A general framework. arXiv preprint arXiv:2311.17431, 2023

    2023 arXiv

  41. [49]

    Big-fed: Bilevel optimization enhanced graph-aided federated learning

    Pengwei Xing, Songtao Lu, Lingfei Wu, and Han Yu. Big-fed: Bilevel optimization enhanced graph-aided federated learning. In Int. Workshop on Federated Learning for User Privacy and Data Confidentiality, pages 1–8, Virtual Event,

  42. [50]

    Model pruning enables efficient federated learning on edge devices

    Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K Leung, and Leandros Tassiulas. Model pruning enables efficient federated learning on edge devices. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 1(1):1–22, 2022

    2022

  43. [51]

    Federated dynamic sparse training: Computing less, communicating less, yet learning better

    Sameer Bibikar, Haris Vikalo, Zhangyang Wang, and Xiaohan Chen. Federated dynamic sparse training: Computing less, communicating less, yet learning better. In AAAI Conf. on Artificial Intelligence, pages 6080–6088, Virtual Event,

  44. [52]

    Federated learning from pre-trained models: A contrastive learning approach

    Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre-trained models: A contrastive learning approach. In Advances in Neural Information Processing Systems (NeurIPS), pages 1–20, New Orleans, Louisiana, United States, 2022. Curran Associates

    2022

  45. [53]

    Oort: Efficient federated learning via guided participant selection

    Fan Lai, Xiangfeng Zhu, Harsha V Madhyastha, and Mosharaf Chowdhury. Oort: Efficient federated learning via guided participant selection. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 19–35, 2021

    2021

  46. [54]

    Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications

    Moming Duan, Duo Liu, Xianzhang Chen, Yujuan Tan, Jinting Ren, Lei Qiao, and Liang Liang. Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications. In IEEE Int. Conf. on Computer Design (ICCD), pages 246–254, 2019

    2019

  47. [55]

    Client selection for federated learning with label noise

    Miao Yang, Hua Qian, Ximin Wang, Yong Zhou, and Hongbin Zhu. Client selection for federated learning with label noise. IEEE Transactions on Vehicular Technology, 71(2):2193–2197, 2021

    2021

  48. [56]

    FedDUAP: Federated learning with dynamic update and adaptive pruning using shared data on the server

    Hong Zhang, Ji Liu, Juncheng Jia, Yang Zhou, and Huaiyu Dai. FedDUAP: Federated learning with dynamic update and adaptive pruning using shared data on the server. In Int. Joint Conf. on Artificial Intelligence (IJCAI), pages 1–7, 2022. To appear

    2022

  49. [57]

    Efficient federated learning using dynamic update and adaptive pruning with momentum on shared server data

    Ji Liu, Juncheng Jia, Hong Zhang, Yuhui Yun, Leye Wang, Yang Zhou, Huaiyu Dai, and Dejing Dou. Efficient federated learning using dynamic update and adaptive pruning with momentum on shared server data. ACM Trans. on Intelligent Systems and Technology, 2024

    2024

  50. [58]

    An improved federated learning algorithm for privacy-preserving in cybertwin-driven 6G system

    Miao Yang, Ximin Wang, Hua Qian, Yongxin Zhu, Hongbin Zhu, Mohsen Guizani, and Victor Chang. An improved federated learning algorithm for privacy-preserving in cybertwin-driven 6G system. IEEE Transactions on Industrial Informatics, 18(10):6733–6742, 2022

    2022

  51. [59]

    Parallelized stochastic gradient descent

    Martin Zinkevich, Markus Weimer, Alexander J Smola, and Lihong Li. Parallelized stochastic gradient descent. In Advances in Neural Information Processing Systems (NeurIPS), volume 23, pages 1–37, Vancouver, British Columbia, Canada, 2010. Curran Associates

    2010

  52. [60]

    Approximation methods for bilevel programming

    Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246, 2018

    2018 arXiv

  53. [61]

    A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic

    Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic. arXiv preprint arXiv:2007.05170, 2020

    2007 arXiv

  54. [62]

    Hrank: Filter pruning using high-rank feature map

    Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao. Hrank: Filter pruning using high-rank feature map. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1529–1538, Seattle, W A, United States, 2020. IEEE

    2020

  55. [63]

    Seizing critical learning periods in federated learning

    Gang Yan, Hao Wang, and Jian Li. Seizing critical learning periods in federated learning. In AAAI Conf. on Artificial Intelligence, pages 1–8, Virtual Event, 2022. AAAI Press. To appear

    2022

  56. [64]

    Validating the lottery ticket hypothesis with inertial manifold theory

    Zeru Zhang, Jiayin Jin, Zijie Zhang, Yang Zhou, Xin Zhao, Jiaxiang Ren, Ji Liu, Lingfei Wu, Ruoming Jin, and Dejing Dou. Validating the lottery ticket hypothesis with inertial manifold theory. Advances in Neural Information Processing Systems (NeurIPS), 34:30196–30210, 2021

    2021

  57. [65]

    Fedas: Bridging inconsistency in personalized federated learning

    Xiyuan Yang, Wenke Huang, and Mang Ye. Fedas: Bridging inconsistency in personalized federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11986–11995, 2024. Efficient Federated Learning with Heterogeneous Data and Adap...

    2024

  58. [66]

    Learning multiple layers of features from tiny images, 2009

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009

    2009

  59. [67]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, pages 1–9, Granada, Spain, 2011. Curran Associates

    2011

  60. [68]

    Tiny imagenet visual recognition challenge

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015

    2015

  61. [69]

    Handwritten digit recognition with a back-propagation network

    Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence Jackel. Handwritten digit recognition with a back-propagation network. In Advances in Neural Information Processing Systems (NeurIPS), volume 2, pages 1–9, Denver, CO, United...

    1989

  62. [70]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Int. Conf. on Learning Representations (ICLR), pages 1–14, San Diego, CA, United State, 2015. OpenReview.net

    2015

  63. [71]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 770–778, Paradise, Nevada, United States, 2016. IEEE

    2016

  64. [72]

    Fisher information- based efficient curriculum federated learning with large language models

    Ji Liu, Jiaxiang Ren, Ruoming Jin, Zijie Zhang, Yang Zhou, Patrick Valduriez, and Dejing Dou. Fisher information- based efficient curriculum federated learning with large language models. In Empirical Methods in Natural Language Processing (EMNLP), pages 1–27, 2024

    2024

  65. [73]

    Federated learning of large language models with parameter-efficient prompt tuning and adaptive optimization

    Tianshi Che, Ji Liu, Yang Zhou, Jiaxiang Ren, Jiwen Zhou, Victor S Sheng, Huaiyu Dai, and Dejing Dou. Federated learning of large language models with parameter-efficient prompt tuning and adaptive optimization. In Empirical Methods in Natural Language Processing (EMNLP), page...

    2023

  66. [74]

    Optimal distributed online prediction using mini-batches

    Ofer Dekel, Ran Gilad-Bachrach, Ohad Shamir, and Lin Xiao. Optimal distributed online prediction using mini-batches. Journal of Machine Learning Research, 13(1):165–202, 2012

    2012

  67. [75]

    On the importance of the pearson correlation coefficient in noise reduction

    Jacob Benesty, Jingdong Chen, and Yiteng Huang. On the importance of the pearson correlation coefficient in noise reduction. IEEE Transactions on Audio, Speech, and Language Processing, 16(4):757–765, 2008. Received 21 June 2024; revised 21 June 2024; accepted 21 June 2024

    2008

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.