Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Federated Dropout: Convergence Analysis and Resource Allocation

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that in Federated Dropout, sub-model gradients are unbiased variance-bounded estimates of the full gradient, so larger dropout rates slow convergence, and derives a joint dropout-rate and bandwidth-allocation rule.

desk verdict The headline convergence analysis is for the dense gradient, not the sparse gradient FedDrop actually computes — a fixable but load-bearing flaw. read the letter →

arxiv 2501.00379 v1 pith:FL7P6TR5 submitted 2024-12-31 cs.LG cs.ITmath.IT

Sijing Xie , Dingzhu Wen , Xiaonan Liu , Changsheng You , Tharmalingam Ratnarajah , Kaibin Huang This is my paper · ORCID
classification cs.LGcs.ITmath.IT
keywords federatedlearningdropoutconvergenceanalysisgradientvariancebandwidthallocationresourcewirelessedgenon-convexoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper gives Federated Dropout a quantitative convergence theory. The authors show that, for small dropout rates, the gradient of a randomly masked sub-model is an unbiased and variance-bounded estimate of the full model's gradient, with variance proportional to $\gamma/(1-\gamma)$ for dropout rate $\gamma$, and they prove that higher dropout rates slow convergence while a maximum rate $\theta$ guarantees per-round loss decrease. They then convert the per-round loss bound into a convex optimization problem that chooses each device's dropout rate and bandwidth allocation to minimize convergence latency under latency, energy, and bandwidth constraints, and solve it in closed form. The practical payoff is a principled trade-off: raising dropout cuts per-round communication and computation costs, but the theory quantifies how many extra rounds that costs.

What carries the argument

The object doing the work is the Bernoulli dropout mask $m_k$ with $m_{k,j}=1/(1-\gamma_k)$ with probability $1-\gamma_k$ and zero otherwise, so $\mathbb{E}[m_k]=1$ and the sub-model is $\hat w_k=w\circ m_k$. The engine is the first-order Taylor expansion of the subnet gradient around the full-network weights, which, once the higher-order term is dropped under Assumption 2, turns dropout randomness into the additive term $H(w)(\hat w_k-w)$; Assumptions 1 and 3 bound the Hessian and weight norms. Lemma 1 uses the mask's mean and variance to bound the gradient variance by $(AG)^2\gamma_k/(1-\gamma_k)$, and Lemma 2 inserts that bound into an $L$-smoothness descent inequality to yield the per-round loss reduction that the optimization problem P1 minimizes.

What would settle it

Train a small nonlinear network, freeze it at a checkpoint, draw many dropout masks at rates $\gamma\in\{0.05,0.1,0.2,0.4\}$, and measure the empirical variance of the subnet gradients around the full-batch gradient; if the variance does not scale approximately as $\gamma/(1-\gamma)$ before the loss stops decreasing, the linearized variance bound is not the operative mechanism.

Watch

Extended reading notes

Core claim

The central claim is Lemma 1: under bounded Hessian, bounded weights, and a small dropout rate, the subnet gradient $\hat g_k(\hat w_k)$ satisfies $\mathbb{E}[\hat g_k]=\tilde g_k(w)$ and $\mathbb{E}[\|\hat g_k-\tilde g_k\|^2]\le (AG)^2\,\gamma_k/(1-\gamma_k)$, making it a variance-bounded estimator of the full network's stochastic gradient. Feeding this into the standard $L$-smooth descent inequality with learning rate $\eta=1/(3\sqrt{T}L)$ gives the per-round loss-reduction bound (27), whose dropout term is the weighted sum of $\gamma_{k,t}/(1-\gamma_{k,t})$, and Theorem 1 turns that into a convergence guarantee for the average squared gradient norm. The paper then minimizes the bound by jointly selecting dropout rates and bandwidth allocations, obtaining closed-form Karush-Kuhn-Tucker solutions in which devices with better channels get lower dropout rates and smaller bandwidth shares.

Load-bearing premise

Everything rests on assuming the dropout rate is small enough that the sub-model is nearly identical to the full model, so the gradient difference can be treated as linear; if that assumption fails, the variance bound is not guaranteed.

Editorial extensions

If this is right

  • Larger dropout rates make convergence slower, with the noise penalty growing like $\gamma/(1-\gamma)$; raising $\gamma$ from 0.1 to 0.2 increases the bound's dropout term by a factor of 2.25.
  • There is a maximum dropout rate $\theta$: above it, the loss-reduction bound no longer guarantees per-round descent, consistent with the small-dropout-rate assumption.
  • Devices with better uplink and downlink channels should receive lower dropout rates and smaller bandwidth shares, while bandwidth allocation moves opposite to the dropout rate.
  • Looser per-round latency or larger system bandwidth pushes all dropout rates down, shrinking the gap to no-dropout federated learning in both underfitting and overfitting settings.
  • In the overfitting setting, a moderate dropout rate can beat no-dropout training on test accuracy even though it increases the number of rounds, so the optimal operating point balances convergence noise against regularization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the $\gamma/(1-\gamma)$ variance bound depends on the mask only through its first two moments; any unbiased mask with the same retention probability should inherit the same noise scaling while the linearized surrogate holds.
  • A direct test of the theory would measure the empirical subnet-gradient variance of a small network at increasing dropout rates; if the $\gamma/(1-\gamma)$ law breaks before the guaranteed-descent threshold, the practical operating range is narrower than the theorem states.
  • Replacing Bernoulli masks with mean- and variance-matched deterministic masks could preserve the formal noise bound at higher sparsity, which would shift the optimal dropout-rate and bandwidth schedule.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies Federated Dropout (FedDrop), in which each device trains and uploads a randomly masked sub-model rather than the full model. The authors claim to prove, by a Taylor expansion, that the sub-model gradient is a variance-bounded estimate of the full gradient with variance scaled by γ/(1−γ), and they use this to state a convergence bound showing that larger dropout rates slow convergence. They then formulate a per-round optimization problem that jointly selects dropout rates and bandwidth allocations, derive closed-form Karush-Kuhn-Tucker solutions, and report experiments on CIFAR-100 with LeNet and AlexNet supporting the qualitative trend that smaller dropout rates converge faster.

Significance. If the convergence analysis were valid for the FedDrop algorithm as described, the paper would fill a real gap in the federated-learning literature and would give a principled basis for trading per-round computation/communication savings against the number of rounds to convergence. The paper has positive features: the convergence argument is explicit and checkable, the resource-allocation problem is practically motivated, and the experiments cover both underfitting and overfitting regimes. However, the main theoretical result is derived for an update rule that differs from the algorithm described in Section II, and the optimization convexity claim has a gap. These issues are load-bearing for the paper's central claims and need substantive correction.

major comments (4)
  1. [Section II-D and Appendix A, Eq. (46)] The convergence analysis does not match the FedDrop aggregation procedure. In the algorithm (Section II-D, step 5), the server constructs the next global model as a weighted average of the zero-padded subnets, i.e., w(t+1) = Σ_k (|D_k|/|D|) (w(t) ◦ m_k(t) − η ĝ_k(ŵ_k(t))). In contrast, Eq. (46) in Appendix A writes w(t+1) = w(t) − η Σ_k (|D_k|/|D|) ĝ_k(ŵ_k(t)). The difference is the mask-dependent term Σ_k (|D_k|/|D|) w(t) ◦ m_k(t) − w(t), which is generally nonzero and has variance that depends on the dropout rate. Lemma 2 and Theorem 1 are derived from (46), so they establish convergence for a different algorithm, not for the FedDrop procedure defined in Section II. This is a load-bearing gap: the paper's main claim concerns the actual FedDrop algorithm.
  2. [Section III-A, Lemma 1, Eqs. (19)–(21)] The proof of unbiasedness (20) requires a precise specification of what ĝ_k(ŵ_k) denotes. If ĝ_k is the gradient of the local loss with respect to the subnet parameter vector ŵ_k evaluated at ŵ_k = w ◦ m_k, then Eq. (22) follows from the Taylor expansion, but this object is not the quantity that determines the global update after zero-padded aggregation (see the previous comment). If, instead, ĝ_k is intended to be the masked gradient of the original network, then E[ĝ_k] = g̃_k(w) does not hold in general: for a quadratic loss f(w) = 0.5(xᵀw − y)² and mask (4), one obtains E[ĝ_k] = g̃_k(w) + x xᵀ w γ/(1−γ), not g̃_k(w), even though the Taylor expansion is exact. The paper should disambiguate the definition of ĝ_k and align it with the local update in Section II-D.
  3. [Section IV, Lemma 3 and constraint C2] The convexity proof of P2 does not cover the energy constraint as written. C2 in (36) contains P^{com,ul}_{k,t}, which by (11) depends on ρ_{k,t} through the exponential term 2^{r_{ul}/(ρ B)}. The Hessian calculation in Lemma 3 treats only the term M_k/ρ_k = x_k² Mori/ρ_k and ignores the ρ-dependence of P^{com,ul}. Consequently, the claimed convexity of C2 is unproven, and the KKT-based closed forms (43)–(44) and Algorithm 1 are not justified for the stated constraints. The authors should either prove convexity of the full C2 or revise the energy model to make the constraint tractable.
  4. [Appendix B, Eqs. (54)–(55) and Theorem 1, Eq. (29)] The displayed coefficients in (55) appear inconsistent with the derivation. After multiplying (53) by 9√T L, the stochastic-variance term should have factor 2K/(√T) σ²/|D| and the dropout term should have factor A²G²/√T times the sum, not 2K√T σ²/|D| and A²G²√T times the sum as printed. As printed, the right side of (55) diverges as T→∞, which contradicts the finite limit claimed in (29). The authors should correct the factors and re-derive the limit in Theorem 1.
minor comments (4)
  1. [Section III-A, Assumption 2] The condition in (17) is asymptotic and unquantified; the proof of Lemma 1 drops the higher-order term in (15) without an explicit error bound, so the admissible range of dropout rates for which (21) holds is not specified. The footnote acknowledges that experiments show convergence for relatively larger dropout rates, but the theorem would be stronger if the approximation error were bounded in terms of γ.
  2. [Section II-C] The mask scaling in (4) is the standard inverted dropout and E[ŵ] = w is correct. The paper should state explicitly whether masks are generated independently across devices and across rounds, since the convergence analysis assumes an expectation over masks while the algorithm description does not specify the correlation structure.
  3. [Table II] The FedDST row is missing entries at sparsity 0. Please indicate whether FedDST is not defined for that setting or provide the corresponding accuracy values so that the comparison is complete.
  4. [Figures 3–6] The caption text in the manuscript appears garbled due to encoding issues; the captions should be regenerated so that the figures are intelligible to readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Lemma 1 derives the variance bound from the dropout mask definition and stated assumptions; no prediction is fitted or reduced to its inputs.

full rationale

The derivation chain is self-contained. Lemma 1 (Eqs. 19-23) computes E[w_hat - w] = 0 and E||w_hat - w||^2 = ||w||^2 gamma/(1-gamma) directly from the Bernoulli mask definition in Eq. (4), so the unbiasedness (20) and variance bound (21) follow by construction from the mask statistics, not from the target conclusion. Lemma 2 and Theorem 1 propagate this bound through standard L-smoothness and variance assumptions; the optimization objective (31) minimizes the derived upper bound rather than using the convergence-rate conclusion as an input. The only self-citations, chiefly [34] for the FedDrop framework and [33] for pruning background, are contextual and not load-bearing: no uniqueness theorem or unverified ansatz is imported from the authors' prior work to force the result. The skeptical objection that the effective local gradient in the implemented algorithm is the masked sparse gradient, so E[g_hat] = g_tilde may fail, is a correctness or modeling-fidelity concern about Lemma 1's applicability to the described algorithm; it is not an instance of the paper's conclusion being equivalent to its inputs by construction, and therefore does not constitute circularity under the specified criteria.

Assumptions & free parameters 1 free parameters · 7 assumptions · 0 invented entities

The central claim rests on six standard assumptions plus a first-order Taylor approximation. No parameters are fitted to data; theta is introduced but never quantified. No new physical entities are posited.

free parameters (1)
  • theta (maximum dropout rate)
    Introduced in the abstract and in constraint C4 as the largest dropout rate preserving per-round loss reduction, but never derived, bounded, or estimated from the model or data.
assumptions (7)
  • domain assumption Assumption 1: bounded Hessian, ||H(w)||_F^2 <= A^2 for all w.
    Used in Lemma 1 to convert the variance expression into the A^2 factor in the bound (21).
  • domain assumption Assumption 2: small dropout rate such that ||w_hat_k - w||^2 -> 0, allowing the higher-order Taylor term to be dropped.
    This is the load-bearing approximation: Eq. (19) replaces the true dropout gradient by first-order Taylor terms; footnote 1 admits experiments work for larger rates than the assumption covers.
  • domain assumption Assumption 3: bounded weight, E[||w||^2] <= G^2.
    Provides the G^2 factor in the variance bound.
  • domain assumption Assumption 4: loss function lower bounded by F*.
    Needed for the telescoping argument in Appendix B.
  • domain assumption Assumption 5: L-smoothness of F.
    Yields the descent lemma (47) used in Lemma 2.
  • domain assumption Assumption 6: stochastic gradients are unbiased with variance bounded by sigma^2/|D_k|.
    Adds the SGD noise term 2K sigma^2/(9TL|D|) to the per-round bound.
  • ad hoc to paper First-order Taylor approximation of the dropout gradient, Eq. (19).
    The expectation and variance identities in Lemma 1 are computed for this surrogate; for nonlinear networks the exact dropout gradient expectation is not equal to g_tilde(w).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Dropout: Convergence Analysis and Resource Allocation." pith.science (2026). https://pith.science/paper/FL7P6TR5

@misc{pith2026250100379,
  author       = {Pith},
  title        = {Pith review of: Federated Dropout: Convergence Analysis and Resource Allocation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FL7P6TR5}},
  note         = {Machine review of arXiv:2501.00379}
}
abstract

Federated Dropout is an efficient technique to overcome both communication and computation bottlenecks for deploying federated learning at the network edge. In each training round, an edge device only needs to update and transmit a sub-model, which is generated by the typical method of dropout in deep learning, and thus effectively reduces the per-round latency. \textcolor{blue}{However, the theoretical convergence analysis for Federated Dropout is still lacking in the literature, particularly regarding the quantitative influence of dropout rate on convergence}. To address this issue, by using the Taylor expansion method, we mathematically show that the gradient variance increases with a scaling factor of $\gamma/(1-\gamma)$, with $\gamma \in [0, \theta)$ denoting the dropout rate and $\theta$ being the maximum dropout rate ensuring the loss function reduction. Based on the above approximation, we provide the convergence analysis for Federated Dropout. Specifically, it is shown that a larger dropout rate of each device leads to a slower convergence rate. This provides a theoretical foundation for reducing the convergence latency by making a tradeoff between the per-round latency and the overall rounds till convergence. Moreover, a low-complexity algorithm is proposed to jointly optimize the dropout rate and the bandwidth allocation for minimizing the loss function in all rounds under a given per-round latency and limited network resources. Finally, numerical results are provided to verify the effectiveness of the proposed algorithm.

Figures

Figures reproduced from arXiv: 2501.00379 by the authors.

Figure 1
Figure 1. The operations of FL with FedDrop in a wireless system. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The Dirichlet distribution of data on all the clients. 2) Effects of Dropout Rate on Performance: • Underfitting Scenario: The effects of different dropout rates on testing accuracy value in underfitting scenarios are shown in the first four lines. It is observed that the testing accuracy decreases with an increasing dropout rate under both data distributions. This is because a simple model is trained with sufficien… view at source ↗
Figure 3
Figure 3. Effects of per-round latency on convergence round in underfitting and overfitting scenarios, respectively. 100 200 300 400 500 System Bandwidth (MHz) 0 20 40 60 80 Convergence Round 78 85 52 67 78 52 63 68 52 59 64 52 56 62 52 Proposed Scheme Bandwidth-aware Scheme Scheme Without Dropout (a) Convergence round v.s. system bandwidth in underfitting sce￾narios 600 650 700 750 800 System Bandwidth (MHz) 0 20 40 60 80 10… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Effects of system bandwidth on convergence round in underfitting and overfitting scenarios, respectively. scheme. The performance gain remains increasing when the latency continuously increases. This is because, with a looser constraint, a smaller dropout rate can meet…
Figure 5
Figure 5. Figure 5: Effects of per-round latency on testing accuracy in underfitting and overfitting scenarios, respectively. 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 Per-round Latency (s) 18 19 20 21 22 23 Testing Accuracy (%) Proposed IID Proposed non-IID (a) Testing accuracy v.s. per-round …
Figure 6
Figure 6. Figure 6: Effects of dynamic wireless environments in underfitting and overfitting scenarios, respectively. overall power constraints, we obtain the optimal dropout rate and bandwidth resource allocation in closed forms. Extensive simulation results show that the proposed scheme…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 43 canonical work pages

  1. [40]

    Adaptive network pruning for wireless federated learning,

    S. Liu, G. Yu, R. Yin, and J. Yuan, “Adaptive network pruning for wireless federated learning,” IEEE Wireless Commun. Lett. , vol. 10, no. 7, pp. 1572–1576, Jul. 2021

    work page 2021

  2. [1]

    Pushing AI to wireless network edge: An overview on integrated sensing, communication, and computation towards 6G,

    G. Zhu, Z. Lyu, X. Jiao, P. Liu, M. Chen, J. Xu, S. Cui, and P. Zhang, “Pushing AI to wireless network edge: An overview on integrated sensing, communication, and computation towards 6G,” Sci. China Inf. Sci., vol. 66, no. 3, p. 130301, 2023

    work page 2023

  3. [2]

    Deep learning inverse analysis of higher order modes in monocone tem cell,

    D. Li, Y . Gu, H. Ma, Y . Li, L. Zhang, R. Li, R. Hao, and E.-P. Li, “Deep learning inverse analysis of higher order modes in monocone tem cell,” IEEE Trans. Microwave Theory Tech. , vol. 70, no. 12, pp. 5332–5339, Dec. 2022

    work page 2022

  4. [3]

    Task- oriented sensing, computation, and communication integration for multi- device edge AI,

    D. Wen, P. Liu, G. Zhu, Y . Shi, J. Xu, Y . C. Eldar, and S. Cui, “Task- oriented sensing, computation, and communication integration for multi- device edge AI,” IEEE Trans. Wireless Commun. , vol. 23, no. 3, pp. 2486–2502, Mar. 2024

    2024

  5. [4]

    Towards effective and interpretable semantic communications,

    Y . Wu, Y . Shi, S. Ma, C. Jiang, W. Zhang, and K. B. Letaief, “Towards effective and interpretable semantic communications,” IEEE Network , pp. 1–1, 2024. 13

    work page 2024

  6. [5]

    Framework and overall objectives of the future development of imt for 2030 and beyond,

    ITU Radiocommunication Sector, “Framework and overall objectives of the future development of imt for 2030 and beyond,” International Telecommunication Union, Recommendation ITU-R M.2160-0, 2023. [Online]. Available: https://www.itu.int/rec/R-REC-M. 2160-0-202311-I/en

    work page 2023

  7. [6]

    Task- oriented communications for 6G: Vision, principles, and technologies,

    Y . Shi, Y . Zhou, D. Wen, Y . Wu, C. Jiang, and K. B. Letaief, “Task- oriented communications for 6G: Vision, principles, and technologies,” IEEE Wireless Commun., vol. 30, no. 3, pp. 78–85, 2023

    work page 2023

  8. [7]

    Adaptive retransmission design for wireless federated edge learning,

    X. Xu, S. Liu, and G. Yu, “Adaptive retransmission design for wireless federated edge learning,” ZTE Communications, vol. 21, no. 1, pp. 3–14, Mar. 2023

    work page 2023

Show all 50 references
  1. [8]

    Delayed random partial gradient averaging for federated learning,

    X. Hu, “Delayed random partial gradient averaging for federated learning,” in IEEE Workshop Signal Process. Adv. Wireless Commun. (SPAWC), 2023, pp. 376–380

    2023

  2. [9]

    Fedcomp: A federated learning compression framework for resource-constrained edge computing devices,

    D. Wu, W. Yang, H. Jin, X. Zou, W. Xia, and B. Fang, “Fedcomp: A federated learning compression framework for resource-constrained edge computing devices,” IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., vol. 43, no. 1, pp. 230–243, Jan. 2024

    2024

  3. [10]

    Adaptive control of local updating and model compression for efficient federated learning,

    Y . Xu, Y . Liao, H. Xu, Z. Ma, L. Wang, and J. Liu, “Adaptive control of local updating and model compression for efficient federated learning,” IEEE Trans. Mob. Comput. , vol. 22, no. 10, pp. 5675–5689, Oct. 2023

    2023

  4. [11]

    Joint user selection and resource allocation for fast federated edge learning,

    Z. Jiang, Y . He, and G. Yu, “Joint user selection and resource allocation for fast federated edge learning,” ZTE Communications, vol. 18, no. 2, pp. 20–30, Jun. 2020

    2020

  5. [12]

    Joint client scheduling and resource allocation under channel uncertainty in federated learning,

    M. M. Wadu, S. Samarakoon, and M. Bennis, “Joint client scheduling and resource allocation under channel uncertainty in federated learning,” IEEE Trans. Commun. , vol. 69, no. 9, pp. 5962–5974, Sep. 2021

    2021

  6. [13]

    Decentralized edge intelligence: A dynamic resource allocation framework for hierarchical federated learning,

    W. Y . B. Lim, J. S. Ng, Z. Xiong, J. Jin, Y . Zhang, D. Niyato, C. Leung, and C. Miao, “Decentralized edge intelligence: A dynamic resource allocation framework for hierarchical federated learning,” IEEE Trans. Parallel Distrib. Syst., vol. 33, no. 3, pp. 536–550, Mar. 2021

    2021

  7. [14]

    Scheduling for cellular federated edge learning with importance and channel aware- ness,

    J. Ren, Y . He, D. Wen, G. Yu, K. Huang, and D. Guo, “Scheduling for cellular federated edge learning with importance and channel aware- ness,” IEEE Trans. Wireless Commun. , vol. 19, no. 11, pp. 7690–7703, Nov. 2020

    2020

  8. [15]

    Scheduling policies for federated learning in wireless networks,

    H. H. Yang, Z. Liu, T. Q. Quek, and H. V . Poor, “Scheduling policies for federated learning in wireless networks,” IEEE Trans. Commun., vol. 68, no. 1, pp. 317–333, Jan. 2020

    2020

  9. [16]

    Optimized power control design for over-the-air federated edge learning,

    X. Cao, G. Zhu, J. Xu, Z. Wang, and S. Cui, “Optimized power control design for over-the-air federated edge learning,” IEEE J. Sel. Areas Commun., vol. 40, no. 1, pp. 342–358, Jan. 2022

    2022

  10. [17]

    Federated learning in massive MIMO 6G networks: convergence analysis and communication-efficient design,

    Y . Mu, N. Garg, and T. Ratnarajah, “Federated learning in massive MIMO 6G networks: convergence analysis and communication-efficient design,” IEEE Trans. Network Sci. Eng. , vol. 9, no. 6, pp. 4220–4234, Nov. 2022

    2022

  11. [18]

    Federated learning with over-the-air ag- gregation over time-varying channels,

    B. Tegin and T. M. Duman, “Federated learning with over-the-air ag- gregation over time-varying channels,” IEEE Trans. Wireless Commun., vol. 22, no. 8, pp. 5671–5684, Aug. 2023

    2023

  12. [19]

    Federated distillation in mas- sive MIMO networks: Dynamic training, convergence analysis, and communication channel-aware learning,

    Y . Mu, N. Garg, and T. Ratnarajah, “Federated distillation in mas- sive MIMO networks: Dynamic training, convergence analysis, and communication channel-aware learning,” IEEE Trans. Cognit. Commun. Networking, vol. 10, no. 4, pp. 1535–1550, 2024

    2024

  13. [20]

    Communication-efficient federated distillation with active data sampling,

    L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Communication-efficient federated distillation with active data sampling,” in IEEE Int. Conf. Commun. (ICC), 2022, pp. 201–206

    2022

  14. [21]

    Federated learning with energy harvesting devices,

    L. Zeng, D. Wen, G. Zhu, C. You, Q. Chen, and Y . Shi, “Federated learning with energy harvesting devices,” IEEE Trans. Green Commun. Networking, vol. 8, no. 1, pp. 190–204, Mar. 2024

    2024

  15. [22]

    Energy-efficient resource management for federated edge learning with CPU-GPU heterogeneous computing,

    Q. Zeng, Y . Du, K. Huang, and K. K. Leung, “Energy-efficient resource management for federated edge learning with CPU-GPU heterogeneous computing,” IEEE Trans. Wireless Commun., vol. 20, no. 12, pp. 7947– 7962, Dec. 2021

    2021

  16. [23]

    Communi- cation and energy efficient decentralized learning over D2D networks,

    S. Liu, G. Yu, D. Wen, X. Chen, M. Bennis, and H. Chen, “Communi- cation and energy efficient decentralized learning over D2D networks,” IEEE Trans. Wireless Commun. , vol. 22, no. 12, pp. 9549–9563, Dec. 2023

    2023

  17. [24]

    De- centralized over-the-air federated learning by second-order optimization method,

    P. Yang, Y . Jiang, D. Wen, T. Wang, C. N. Jones, and Y . Shi, “De- centralized over-the-air federated learning by second-order optimization method,” IEEE Trans. Wireless Commun., vol. 23, no. 6, pp. 5632–5647, Jun. 2024

    2024

  18. [25]

    Enhanced hybrid hierarchical federated edge learning over heterogeneous networks,

    Q. Chen, Z. You, D. Wen, and Z. Zhang, “Enhanced hybrid hierarchical federated edge learning over heterogeneous networks,” IEEE Trans. Veh. Technol., vol. 72, no. 11, pp. 14 601–14 614, Nov. 2023

    2023

  19. [26]

    Optimal user-edge assignment in hierarchical federated learning based on statistical properties and network topology constraints,

    N. Mhaisen, A. A. Abdellatif, A. Mohamed, A. Erbad, and M. Guizani, “Optimal user-edge assignment in hierarchical federated learning based on statistical properties and network topology constraints,” IEEE Trans. Network Sci. Eng. , vol. 9, no. 1, pp. 55–66, Feb. 2022

    2022

  20. [27]

    Perfedmask: Personalized federated learning with optimized masking vectors,

    M. Setayesh, X. Li, and V . W. Wong, “Perfedmask: Personalized federated learning with optimized masking vectors,” in Int. Conf. Learn. Represent. (ICLR), 2023

    2023

  21. [28]

    Com- putation and communication efficient federated learning with adaptive model pruning,

    Z. Jiang, Y . Xu, H. Xu, Z. Wang, J. Liu, Q. Chen, and C. Qiao, “Com- putation and communication efficient federated learning with adaptive model pruning,” IEEE Trans. Mob. Comput. , vol. 23, no. 3, pp. 2003– 2021, Mar. 2024

    2003

  22. [29]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. Int. Conf. Artif. Intell. Statist.(PMLR) , 2017, pp. 1273– 1282

    2017

  23. [30]

    Pruning filters for efficient convnets,

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” in Int. Conf. Learn. Represent. (ICLR) , 2017

    2017

  24. [31]

    Model pruning enables efficient federated learning on edge devices,

    Y . Jiang, S. Wang, V . Valls, B. J. Ko, W.-H. Lee, K. K. Leung, and L. Tassiulas, “Model pruning enables efficient federated learning on edge devices,” IEEE Trans. Neural Networks Learn. Syst., vol. 34, no. 12, pp. 10 374–10 386, Dec. 2023

    2023

  25. [32]

    Adaptive federated prun- ing in hierarchical wireless networks,

    X. Liu, S. Wang, Y . Deng, and A. Nallanathan, “Adaptive federated prun- ing in hierarchical wireless networks,” IEEE Trans. Wireless Commun., vol. 23, no. 6, pp. 5985–5999, Jun. 2024

    2024

  26. [33]

    Adaptive model pruning and personalization for federated learning over wireless networks,

    X. Liu, T. Ratnarajah, M. Sellathurai, and Y . C. Eldar, “Adaptive model pruning and personalization for federated learning over wireless networks,” IEEE Trans. Signal Process. , pp. 1–17, 2024

    2024

  27. [34]

    Federated dropout—A simple ap- proach for enabling federated learning on resource constrained devices,

    D. Wen, K.-J. Jeon, and K. Huang, “Federated dropout—A simple ap- proach for enabling federated learning on resource constrained devices,” IEEE Wireless Commun. Lett. , vol. 11, no. 5, pp. 923–927, May 2022

    2022

  28. [35]

    Dropout: a simple way to prevent neural networks from overfit- ting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut- dinov, “Dropout: a simple way to prevent neural networks from overfit- ting,” J. Mach. Learn. Res. , vol. 15, no. 1, pp. 1929–1958, 2014

    1929

  29. [36]

    Splitfed: When federated learning meets split learning,

    C. Thapa, P. C. M. Arachchige, S. Camtepe, and L. Sun, “Splitfed: When federated learning meets split learning,” in Proc. AAAI Conf. Artif. Intell. (AAAI), vol. 36, no. 8, 2022, pp. 8485–8493

    2022

  30. [37]

    Fjord: Fair and accurate federated learning under hetero- geneous targets with ordered dropout,

    S. Horvath, S. Laskaridis, M. Almeida, I. Leontiadis, S. Venieris, and N. Lane, “Fjord: Fair and accurate federated learning under hetero- geneous targets with ordered dropout,” Adv. neural inf. proces. syst. , vol. 34, pp. 12 876–12 889, 2021

    2021

  31. [38]

    Fedrolex: Model- heterogeneous federated learning with rolling sub-model extraction,

    S. Alam, L. Liu, M. Yan, and M. Zhang, “Fedrolex: Model- heterogeneous federated learning with rolling sub-model extraction,” Adv. neural inf. proces. syst. , vol. 35, pp. 29 677–29 690, 2022

    2022

  32. [39]

    Feddd: Toward communication-efficient federated learning with differential pa- rameter dropout,

    Z. Feng, X. Chen, Q. Wu, W. Wu, X. Zhang, and Q. Huang, “Feddd: Toward communication-efficient federated learning with differential pa- rameter dropout,” IEEE Trans. Mob. Comput., vol. 23, no. 5, pp. 5366– 5384, May 2024

    2024

  33. [41]

    Gradient masked averaging for federated learning,

    I. Tenison, S. A. Sreeramadas, V . Mugunthan, E. Oyallon, I. Rish, and E. Belilovsky, “Gradient masked averaging for federated learning,” Transact. mach. learn. res

  34. [42]

    Joint parameter-and-bandwidth allocation for improving the efficiency of partitioned edge learning,

    D. Wen, M. Bennis, and K. Huang, “Joint parameter-and-bandwidth allocation for improving the efficiency of partitioned edge learning,” IEEE Trans. Wireless Commun. , vol. 19, no. 12, pp. 8272–8286, 2020

    2020

  35. [43]

    Understanding dropout,

    P. Baldi and P. J. Sadowski, “Understanding dropout,” Advances in neural information processing systems , vol. 26, 2013

    2013

  36. [44]

    Energy-efficient resource allocation for mobile-edge computation offloading,

    C. You, K. Huang, H. Chae, and B.-H. Kim, “Energy-efficient resource allocation for mobile-edge computation offloading,” IEEE Trans. Wire- less Commun., vol. 16, no. 3, pp. 1397–1411, Mar. 2017

    2017

  37. [45]

    Federated dynamic sparse training: Computing less, communicating less, yet learning better,

    S. Bibikar, H. Vikalo, Z. Wang, and X. Chen, “Federated dynamic sparse training: Computing less, communicating less, yet learning better,” in Proc. AAAI Conf. Artif. Intell. (AAAI) , vol. 36, no. 6, 2022, pp. 6080– 6088

    2022

  38. [46]

    Dropout training as adaptive regularization,

    S. Wager, S. Wang, and P. S. Liang, “Dropout training as adaptive regularization,” Adv. neural inf. proces. syst. , vol. 26, 2013

    2013

  39. [47]

    Compressed-vfl: Communication-efficient learning with vertically partitioned data,

    T. J. Castiglia, A. Das, S. Wang, and S. Patterson, “Compressed-vfl: Communication-efficient learning with vertically partitioned data,” in Int. Conf. Mach. Learn. (ICML) , 2022, pp. 2738–2766

    2022

  40. [48]

    Stochastic first-and zeroth-order methods for nonconvex stochastic programming,

    S. Ghadimi and G. Lan, “Stochastic first-and zeroth-order methods for nonconvex stochastic programming,” SIAM J. Optim., vol. 23, no. 4, pp. 2341–2368, 2013

    2013

  41. [49]

    On the convergence of fedavg on non-iid data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” arXiv preprint arXiv:1907.02189 , 2019

    1907 arXiv

  42. [50]

    2: Faster non-convex optimization than SGD,

    Z. N. Allen-Zhu, “2: Faster non-convex optimization than SGD,” arXiv preprint arXiv:1708.08694, 2017

    2017 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.