Pith. sign in

REVIEW 4 major objections 5 minor 62 references

Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a new algorithm, FedQVR, removes inter-device variance from the convergence bound of federated learning even when uplink updates are quantized and local update counts vary across devices, achieving O(1/R + σ²/√R)…

desk verdict The convergence theorem that carries the paper has a missing term in the potential-function argument and the quantization parameter formula in Eq. (17) doesn't have consistent dimensions; the algorithm itself is sensible and the experiments are strong enough that a serious referee should look at it, but the main bound is currently unproven. read the letter →

arxiv 2501.11267 v1 pith:JAGBUHT3 submitted 2025-01-20 cs.DC cs.LG

Shuai Wang , Yanqing Xu , Chaoqun You , Mingjie Shao , Tony Q.S. Quek This is my paper · ORCID
classification cs.DCcs.LG
keywords federatedlearningquantizedcommunicationvariancereductiondeviceheterogeneitywirelessedgenetworksconvergenceanalysisresourceallocationcontrolvariates
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedQVR is a federated-learning algorithm for wireless edge networks that couples control-variate variance reduction with stochastic quantization of uplink updates and time-varying numbers of local SGD steps per device. The paper's central claim is that this combination removes the inter-device variance term from the convergence bound: under smooth nonconvex losses, unbiased bounded-variance stochastic gradients, and an unbiased contracting quantizer, the average squared gradient norm decays as $O(1/R + \sigma^2/\sqrt{R})$, and no term in the bound depends on how heterogeneous the data or local update counts are. If the theorem is right, a system using as few as 2-bit quantized uploads and devices doing different amounts of local work reaches $\epsilon$-accuracy in $O(1/\epsilon)$ communication rounds, rather than the $O(1/\epsilon^2)$ of FedAvg and SCAFFOLD. A companion scheme, FedQVR-E, allocates bandwidth and quantization bits per round to keep delay-constrained devices from dropping out. Experiments on CIFAR-10 and MNIST show FedQVR reaching the same test accuracies as unquantized baselines with roughly one-tenth of the upload cost.

What carries the argument

The load-bearing mechanism is the pair of control variates $c_i$ (per device) and $c$ (server) together with the server-side anchor $\theta_0^r = \theta^r - c^r/\gamma$. Each participating device runs a convex combination of a control-variate-corrected SGD step and a pull back toward $\theta_0^r$, which prevents local models from drifting when the control variates are stale, and updates $c_i$ with a decaying stepsize $a/(\eta \tilde{E}_i^r)$ so that the variate accumulates an exponentially weighted average of all historical local stochastic gradients. The quantized quantity is the local update $\Delta_i^{r+1}=Q(\theta_i^{r+1}-\theta_0^r, B_i^r)$, and the same quantized update is used to refresh both control variates, preserving the identity $c=\sum_i p_i c_i$ without extra communication. This construction is what lets the proof build a potential function whose decrease is controlled in every round by $\|\nabla f(\theta^r)\|^2$ plus variance terms that depend only on $\sigma^2$ and quantization error, not on heterogeneity.

What would settle it

Take a concrete vector such as $z=(1,-1)$ with $B=2$ and compute the true ratio $\mathbb{E}\|Q(z,B)-z\|^2/\|z\|^2$; if that ratio is not a dimensionless number in $[0,1]$, or if it disagrees with Eq. (17), then the coefficient $\omega_i^r$ is not the quantity Theorem 1 assumes and the parameter region $a<1/\omega_i^r$ is unjustified.

Watch

Extended reading notes

Core claim

The discovery the authors are trying to establish is Theorem 1: under Assumptions 1–3 (L-smooth lower-bounded losses; unbiased stochastic gradients with variance bounded by $\sigma^2$; and an unbiased quantizer whose error variance is at most $\omega_i^r \mathbb{E}\|\theta_i^{r+1}-\theta_0^r\|^2$ with $0\le \omega_i^r\le 1$), with $\eta$ and $\gamma$ satisfying (22)–(23), FedQVR obeys the bound in (24). The right-hand side of that bound contains no measure of data heterogeneity or of the variation in local update counts, which the authors interpret as intrinsic resilience to both forms of device heterogeneity (Remark 1). Corollary 1 specializes the bound to $O(1/R + \sigma^2/\sqrt{R})$ when the mini-batch size is $\sqrt{R}$, and Corollary 2 gives $O(1/\epsilon)$ communication complexity to reach $\epsilon$-accuracy. The same convergence guarantee is what makes the quantized, partially participating algorithm communication-efficient despite coarse uplink transmission. A companion allocation scheme, FedQVR-E, is proposed for non-ideal wireless channels, where per-round bandwidth and quantization bits are chosen to maximize the minimum bit count under delay constraints.

Load-bearing premise

Everything in the proof rests on the assumption that quantization noise is no larger than a fixed fraction (somewhere between 0 and 1) of the size of the local update being quantized, and the paper's own formula for that fraction does not have the right units to be that fraction.

Editorial extensions

If this is right

  • Under the theorem, FedQVR reaches $\epsilon$-accuracy in $O(1/\epsilon)$ communication rounds and $O(m(d(B+1)+\mu)/\epsilon)$ uploaded bits, matching the best known nonconvex FL complexity while paying only a small per-round bit cost.
  • The absence of heterogeneity terms in (24) means the same round count suffices for strongly non-i.i.d. data and for devices with very different local epoch counts; no extra penalty appears.
  • Quantization bits can be kept small: the bound degrades only through $\omega_i^r$, and experiments with $B_i^r=2$ show the algorithm still converges faster than unquantized baselines.
  • Under delay constraints, FedQVR-E's convex relaxation of bandwidth and quantization-bit allocation avoids transmission failures and preserves convergence where fixed even bandwidth would not.
  • With full gradients ($\sigma=0$), the variance terms vanish and the rate becomes pure $O(1/R)$, meaning the algorithm is exact in the deterministic heterogeneous case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the heterogeneity-free bound is correct, the practical trade-off is between bits per round and rounds, not between heterogeneity and speed: operators can lower quantization bits aggressively and compensate with more rounds, without needing to sample more devices.
  • The theorem's validity hinges on Assumption 3's coefficient; because Eq. (17) appears dimensionally inconsistent, the actual admissible range of $a$ may differ from $a<1/\omega_i^r$, and recomputing that coefficient is a direct test of the proof.
  • The same anchor-and-control-variate construction could be transplanted to decentralized or multi-server FL, where the central $c$ is replaced by a consensus estimate, potentially preserving the no-drift bound in topologies without a single aggregator.
  • The per-round separability of the FedQVR-E allocation problem suggests an online or predictive variant that tracks channel fading, converting the convex solve into a light per-round update.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FedQVR, a federated learning algorithm that combines control-variate-based inter-device variance reduction with stochastic uplink quantization and time-varying heterogeneous local updates. The main theoretical claim is Theorem 1: under smoothness and bounded stochastic gradient variance, and with a quantization variance coefficient ω_i^r satisfying Assumption 3, FedQVR converges at rate O(1/R + σ²/√R) and achieves O(1/ε) communication complexity. A second algorithm, FedQVR-E, adds joint bandwidth and quantization-bit allocation under per-round delay constraints in an FDMA system. The authors support the claims with a supplementary proof of Theorem 1 and with experiments on MNIST and CIFAR-10 comparing against FedAvg, SCAFFOLD, FedDyn, FEDADAM, FedPAQ, FedCOMGATE, and FedCAMS, including a communication-cost comparison table.

Significance. If the central convergence and communication-complexity claims were established, the paper would be significant: it would demonstrate that quantized uplink transmission can be combined with variance reduction without doubling per-round communication, and that the convergence bound can be made independent of explicit device-heterogeneity terms. The paper also provides a structured supplementary proof and a broad experimental comparison with communication-cost accounting, which are useful strengths. However, the proof of Theorem 1 contains a gap in the potential-function argument, and the stated quantization coefficient in Eq. (17) is dimensionally inconsistent; these issues are load-bearing because the convergence conditions and the heterogeneity-robustness interpretation rely on them. The FedQVR-E algorithm also has an aggregation inconsistency after device removal. The paper is not acceptable in its current form, but the claims are of the type that could be repaired with a corrected proof and consistency fixes.

major comments (4)
  1. The proof of Theorem 1 defines the potential P^r = E[f(θ^r)] + C_0^r Σ_i p_i Ξ_i^r with a round-dependent coefficient C_0^r, but the bound displayed as Eq. (57) does not actually bound P^{r+1} - P^r. The quantities being combined in Eqs. (51)-(54) give E[f(θ^{r+1}) - f(θ^r)] + C_0^r Σ_i p_i (Ξ_i^{r+1} - Ξ_i^r), whereas P^{r+1} - P^r contains the additional term (C_0^{r+1} - C_0^r) Σ_i p_i Ξ_i^{r+1}. The proof never shows that this extra term is non-positive or that it is absorbed into the RHS_r terms. Since C_0^r depends on r through \bar{ω}^r in Eqs. (62)-(65), and Theorem 1 explicitly allows \bar{ω}^r to vary across rounds, the telescoping sum Σ_{r=0}^{R-1} (P^r - P^{r+1}) = P^0 - P^R that leads to Eqs. (70)-(72) is not justified. This is a direct gap in the proof of the central bound (24).
  2. The explicit formula for ω_i^r in Eq. (17) is dimensionally inconsistent. The numerator Σ_{j=1}^d (max_k{[z_i^r]_k} - min_k{[z_i^r]_k}) has the units of z (one power of the parameter scale), while the denominator 4(2^{B_i^r}-1)‖z_i^r‖² has units of z², so the claimed ω_i^r has units of 1/z. But Assumption 3 in Eq. (16) requires ω_i^r to be dimensionless because it multiplies E‖θ_i^{r+1}-θ_0^r‖² on both sides. The standard bound for this stochastic quantizer is instead of the form (Σ_j (range_j)²)/( (2^B-1)² ‖z‖² ) or d‖z‖²/(2^B-1)². As written, Eq. (17) cannot be used to determine the admissible interval 0 < a < min{1/ω_i^r, 1}, nor the round-dependent constants \bar{ω}^r appearing in the theorem conditions (22)-(23) and in Remark 2. This directly affects the validity of Assumption 3 and the parameter region in which Theorem 1 is claimed.
  3. The FedQVR-E aggregation is inconsistent after the device-removal step. Line 5 removes from A_r all devices whose allocated quantization bits fall below B, but line 12 still updates θ^{r+1} = θ_0^r + (N/m) Σ_{i∈A_r} p_i Δ_i^{r+1} with the original m, even though |A_r| is now smaller. The update is then not a normalized aggregation over the actually transmitted updates, and the unbiasedness property used in the proof of Lemma 7 no longer holds. In addition, line 11 writes c^{r+1} = c^r - (1/N) Σ_{i∈A_r} a p_i/(η \tilde{E}_i^r) Δ_i^{r+1}, which differs from Eq. (11) by the extra factor 1/N and is not consistent with the identity c^r = Σ_i p_i c_i^r used throughout the analysis. No convergence analysis is provided for the device-removal procedure, so the claim that FedQVR-E 'enhances the convergence of FedQVR' is not supported by the theory.
  4. The parameter conditions in Theorem 1 do not match those used in the proof. The first bound on η in Theorem 1, Eq. (22), is 1/(2γ\bar{E}^r√{N(1+\bar{ω}^r)}), while the corresponding condition in Lemma 8, Eq. (66), is 1/(2γ\bar{E}^r√{N p(1+\bar{ω}^r)}) with p = max_i p_i. Unless p = 1, the theorem's stated condition is weaker than the proof requires, so the proof does not establish Theorem 1 for the stated parameter region. The notation for \bar{E}^r also differs: Theorem 1 defines it as a per-round maximum over devices, while the proof of Lemma 8 needs a uniform bound over rounds. These are not merely cosmetic issues because they determine the admissible stepsize region on which Corollaries 1 and 2 rely.
minor comments (5)
  1. Theorem 1 states that it holds under Assumptions 1 and 2, but the proof and the condition 0 < a < min{1/\bar{ω}^r, 1} also rely on Assumption 3 (the quantization variance bound), which should be listed explicitly in the theorem statement.
  2. The last two sums in Eq. (24) run over i = 0 instead of i = 1, and the definition of P^0 contains garbled notation ('C^0_0 N pi') and uses x_0 where θ_0 is used elsewhere; these need correction.
  3. The lemma and corollary numbering is inconsistent: the main text refers to Lemma 1 and Lemma 2, then to 'Lemma 3' in the discussion of Eq. (31); the supplementary proof of Lemma 1 is labeled Lemma 3, and Corollary 2 in the main text is proved as Corollary 3 in Section X. The algorithm references in Section IV-A also point to 'Algorithm 2' when Algorithm 1 is meant.
  4. The experimental figures report single curves without error bars, confidence intervals, or number of seeds. Given the stochastic quantization, random device sampling, and the text's claims about 'smoother convergence' and stability, the reported differences should be supported by multiple independent runs.
  5. In the quantization rule, the probabilities are written with absolute values but the boundaries c_k are defined on the interval [z, z]; the notation should clarify how negative entries are handled, since the formula currently mixes sign([z]_i) with sub-intervals that may contain both signs.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: Theorem 1's convergence bound is derived from the stated assumptions and lemmas, and the cited quantization bound is external, not the conclusion being proved.

full rationale

I walked the claimed derivation chain. The central result, Theorem 1's inequality (24), is obtained by combining Lemma 4 (smoothness descent), Lemma 5 (local drift bound), Lemma 6 (control-variate error contraction), and Lemma 7 (global update variance), each proved from Assumptions 1-3. The potential P^r = E[f(theta^r)] + C_0^r sum p_i Xi_i^r is a constructed Lyapunov function; the parameter conditions (22)-(23) in Lemma 8 are chosen to make the coefficients D_0^r and D_1^r nonnegative, not fitted to reproduce the stated rate. Corollary 1 follows by substituting S = sqrt(R) into (24), and Corollary 2 follows by choosing eta = sqrt(K1/K2) and S = O(K1/epsilon) so that the three terms in (81) are each O(epsilon). None of these steps assumes the bound being proved. The quantization variance assumption (Assumption 3) is attributed to [18], [33], [52]; although [18] has author overlap, it is a standard scalar-quantization variance bound that is externally checkable and does not contain the FedQVR convergence conclusion, so it is not load-bearing self-citation. The paper also cites [39] when discussing SCAFFOLD limitations, but that point is motivational and not needed for Theorem 1. I therefore find no step where a prediction is, by construction, equal to its input, no fitted parameter renamed as a result, and no uniqueness claim imported from the authors' prior work. Two correctness risks are outside the circularity scope and should be evaluated separately: the proof of Theorem 1 treats C_0^r as if it were constant when telescoping P^r, leaving the extra (C_0^{r+1} - C_0^r) sum p_i Xi_i^{r+1} term uncontrolled, and Eq. (17)'s explicit formula for omega_i^r is dimensionally inconsistent. These concern proof validity, not circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard smoothness and stochastic gradient assumptions, plus an assumption about the quantizer's unbiasedness and contraction. The quantizer assumption is load-bearing and its explicit coefficient formula is inconsistent. No parameters were fitted to data; hyperparameters are chosen by hand but are not used to force the convergence result.

assumptions (4)
  • domain assumption Each local cost function is lower bounded and L-smooth (Assumption 1).
    Used in Lemma 4 and throughout the proof to bound the descent and gradient differences; standard in FL analysis.
  • domain assumption Stochastic gradients are unbiased and have bounded variance σ² (Assumption 2).
    Used to bound the stochastic noise in the local updates and in the control variate estimation.
  • domain assumption The quantization operator is unbiased and contractive with coefficient ω_i^r (Assumption 3).
    Underpins the effect of quantization in every lemma; the explicit formula for ω_i^r in Eq. (17) is suspect and may not hold as stated.
  • domain assumption The wireless uplink rate follows the Shannon capacity model for FDMA systems (Eq. (26)).
    Used in FedQVR-E to formulate the bandwidth and quantization bits allocation problem (30).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks." pith.science (2026). https://pith.science/paper/JAGBUHT3

@misc{pith2026250111267,
  author       = {Pith},
  title        = {Pith review of: Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JAGBUHT3}},
  note         = {Machine review of arXiv:2501.11267}
}
read the original abstract

Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks, but its practical deployment is hindered by the high communication overhead caused by frequent and costly server-device synchronization. Notably, most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance resulting from the prevalent issue of device heterogeneity. This variance severely decelerates algorithm convergence, increasing communication overhead and making it more challenging to achieve a well-performed model. In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices. Comprehensive theoretical analysis justifies that FedQVR is inherently resilient to device heterogeneity and has a comparable convergence rate even with a small number of quantization bits, yielding significant communication savings. Besides, considering non-ideal wireless channels, we propose FedQVR-E which enhances the convergence of FedQVR by performing joint allocation of bandwidth and quantization bits across devices under constrained transmission delays. Extensive experimental results are also presented to demonstrate the superior performance of the proposed algorithms over their counterparts in terms of both communication efficiency and application performance.

Figures

Figures reproduced from arXiv: 2501.11267 by the authors.

Figure 1
Figure 1. Convergence performance of the proposed FedQVR algorithm with various choices of [PITH_FULL_IMAGE:figures/full_fig_p025_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison between the proposed FedQVR algorithm and baseline algorithms without quantization [PITH_FULL_IMAGE:figures/full_fig_p026_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison between the proposed FedQVR algorithm and baseline algorithms with quantization [PITH_FULL_IMAGE:figures/full_fig_p027_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance comparison between the proposed FedQVR algorithm and baseline algorithms with different [PITH_FULL_IMAGE:figures/full_fig_p029_4.png]
Figure 5
Figure 5. Figure 5: Convergence performance of the proposed FedQVR-E algorithm and some baseline algorithms under different settings. [PITH_FULL_IMAGE:figures/full_fig_p030_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 56 canonical work pages

  1. [39]

    Beyond ADMM: A unified client-variance-reduced adaptive federated learning framework,

    S. Wang, Y . Xu, Z. Wang, T.-H. Chang, T. Q. S. Quek, and D. Sun, “Beyond ADMM: A unified client-variance-reduced adaptive federated learning framework,” in Proc. AAAI, Washington, DC, USA, Feb. 7-14, 2023, pp. 10 175–10 183

    work page 2023

  2. [1]

    Federated learning over wireless networks: Convergence analysis and resource allocation,

    W. Shi, S. Zhou, and Z. Niu, “Federated learning over wireless networks: Convergence analysis and resource allocation,” in Proc. IEEE INFOCOM , Paris, France, Apr. 29-May 2, 2029, pp. 1–10. 33

    work page

  3. [2]

    Federated learning with Non-IID data in wireless networks,

    Z. Zhao, C. Feng, W. Hong, J. Jiang, C. Jia, T. Q. S. Quek, and M. Peng, “Federated learning with Non-IID data in wireless networks,” IEEE Trans. Wireless Commun. , vol. 21, no. 3, pp. 1927–1942, Mar. 2022

    work page 1927

  4. [3]

    Age-based scheduling for mobile edge computing: A deep reinforcement learning approach,

    X. He, C. You, and T. Q. S. Quek, “Age-based scheduling for mobile edge computing: A deep reinforcement learning approach,” IEEE Transactions on Mobile Computing , vol. 23, no. 10, pp. 9881–9897, 2024

    work page 2024

  5. [4]

    Federated learning for Internet of things: Recent advances, taxonomy, and open challenges,

    L. U. Khan, W. Saad, Z. Han, E. Hossain, and C. S. Hong, “Federated learning for Internet of things: Recent advances, taxonomy, and open challenges,” IEEE Commun. Surv. Tutorials , vol. 23, no. 3, pp. 1759–1799, Jun. 2021

    work page 2021

  6. [5]

    Federated learning for Internet of things: A comprehensive survey,

    D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, and H. Vincent Poor, “Federated learning for Internet of things: A comprehensive survey,” IEEE Commun. Surv. Tutorials , vol. 23, no. 3, pp. 1622–1658, Apr. 2021

    work page 2021

  7. [6]

    Toward fast personalized semi-supervised federated learning in edge networks: Algorithm design and theoretical guarantee,

    S. Wang, Y . Xu, Y . Yuan, and T. Q. S. Quek, “Toward fast personalized semi-supervised federated learning in edge networks: Algorithm design and theoretical guarantee,” IEEE Transactions on Wireless Communications, vol. 23, no. 2, pp. 1170–1183, 2024

    work page 2024

  8. [7]

    Demystifying model averaging for communication-efficient federated matrix factorization,

    S. Wang and T.-H. Chang, “Demystifying model averaging for communication-efficient federated matrix factorization,” in Proc. IEEE ICASSP , Toronto, Ontario, Canada, June 6-11, 2021, pp. 1–5

    work page 2021

Show all 62 references
  1. [8]

    Federated optimization: Distributed machine learning for on-device intelligence,

    J. K ˇonecn´y, H. B. McMahan, D. Ramage, and P. Richtarik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527 , 2016

    2016 arXiv

  2. [9]

    Federated optimization: Distributed optimization beyond the datacenter,

    J. K ˇonecn´y, H. B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacenter,” in NeuIPS Optimization for Machine Learning Workshop , Montreal, Quebec, Canada, Dec. 7-12, 2015, pp. 1–5

    2015

  3. [10]

    Federated learning in mobile edge networks: A comprehensive survey,

    W. Y . B. Lim, N. C. Luong, D. T. Hoang, Y . Jiao, Y .-C. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” arXiv preprint arXiv:1909.11875 , 2019

    1909 arXiv

  4. [11]

    Mobility-aware cluster federated learning in hierarchical wireless networks,

    C. Feng, H. H. Yang, D. Hu, Z. Zhao, T. Q. S. Quek, and G. Min, “Mobility-aware cluster federated learning in hierarchical wireless networks,” IEEE Transactions on Wireless Communications , vol. 21, no. 10, pp. 8441–8458, 2022

    2022

  5. [12]

    Scheduling policies for federated learning in wireless networks,

    H. H. Yang, Z. Liu, T. Q. S. Quek, and H. V . Poor, “Scheduling policies for federated learning in wireless networks,” IEEE Trans. Commun., vol. 68, no. 1, pp. 317–333, Jan. 2020

    2020

  6. [13]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Sanjabi, M. Zaheer, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” in Proc. MLSys, Austin, TX, USA, Mar. 2-4, 2020, pp. 1–12

    2020

  7. [14]

    Why batch normalization damage federated learning on non-iid data?

    Y . Wang, Q. Shi, and T.-H. Chang, “Why batch normalization damage federated learning on non-iid data?”IEEE Transactions on Neural Networks and Learning Systems , pp. 1–15, 2023

    2023

  8. [15]

    Fedfisher: Leveraging fisher information for one-shot federated learning,

    D. Jhunjhunwala, S. Wang, and G. Joshi, “Fedfisher: Leveraging fisher information for one-shot federated learning,” in Proc. AISTATS, Valencia, Spain, May 2-4 2021, pp. 1–10

    2021

  9. [16]

    Scaffold: Stochastic controlled averaging for federated learning,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learning,” in Proc. ICML, Jul. 13-18 2020, pp. 5132–5143

    2020

  10. [17]

    Tackling the objective inconsistency problem in heterogeneous federated optimization,

    J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V . Poor, “Tackling the objective inconsistency problem in heterogeneous federated optimization,” arXiv preprint arXiv:2007.07481 , 2020

    2007 arXiv

  11. [18]

    Quantized federated learning under transmission delay and outage constraints,

    Y . Wang, Y . Xu, Q. Shi, and T.-H. Chang, “Quantized federated learning under transmission delay and outage constraints,” IEEE J. Sel. Areas Commun. , vol. 40, no. 1, pp. 323–341, Jan. 2022

    2022

  12. [19]

    FedPAQ: A communication-efficient federated learning method with periodic averaging and quantization,

    A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, and R. Pedarsani, “FedPAQ: A communication-efficient federated learning method with periodic averaging and quantization,” in Proc. AISTATS, Virtual Conference, Aug. 26-28 2020, pp. 2021–2031

    2020

  13. [20]

    Federated learning with compression: Unified analysis and sharp guarantees,

    F. Haddadpour, M. M. Kamani, A. Mokhtari, and M. Mahdavi, “Federated learning with compression: Unified analysis and sharp guarantees,” in Proc. AISTATS, Virtual Conference, Apr. 13-15 2021, pp. 1–13. 34

    2021

  14. [21]

    Design and analysis of uplink and downlink communications for federated learning,

    S. Zheng, C. Shen, and X. Chen, “Design and analysis of uplink and downlink communications for federated learning,” IEEE Journal on Selected Areas in Communications , vol. 39, no. 7, pp. 2150 – 2167, Dec. 2021

    2021

  15. [22]

    Communication-efficient federated learning for heterogeneous edge devices based on adaptive gradient quantization,

    H. Liu, F. He, and G. Cao, “Communication-efficient federated learning for heterogeneous edge devices based on adaptive gradient quantization,” in Proc. IEEE INFOCOM , New York, USA, May 17-20 2023, pp. 1–10

    2023

  16. [23]

    Dadaquant: Doubly-adaptive quantization for communication-efficient federated learning,

    R. Honig, Y . Zhao, and R. Mullins, “Dadaquant: Doubly-adaptive quantization for communication-efficient federated learning,” in Proc. ICML, Baltimore, Maryland, USA, Jul. 17-23 2022, pp. 8852–8866

    2022

  17. [24]

    Fedvqcs: Federated learning via vector quantized compressed sensing,

    Y . Oh, Y .-S. Jeon, M. Chen, and W. Saad, “Fedvqcs: Federated learning via vector quantized compressed sensing,” IEEE Transactions on Wireless Communications , vol. 1, no. 1, pp. 1–16, Jun. 2023

    2023

  18. [25]

    Communication-efficient federated learning via quantized compressed sensing,

    Y . Oh, N. Lee, Y .-S. Jeon, M. Chen, and H. V . Poor, “Communication-efficient federated learning via quantized compressed sensing,” IEEE Transactions on Wireless Communications , vol. 22, no. 2, pp. 1087–1100, Feb. 2023

    2023

  19. [26]

    Device scheduling with fast convergence for wireless federated learning,

    W. Shi, S. Zhou, and Z. Niu, “Device scheduling with fast convergence for wireless federated learning,” in Proc. ICC, Dublin, Ireland, Jun. 7-11, 2020, pp. 1–6

    2020

  20. [27]

    Client selection and bandwidth allocation in wireless federated learning networks: A long-term perspective,

    J. Xu and H. .Wang, “Client selection and bandwidth allocation in wireless federated learning networks: A long-term perspective,” IEEE Transactions on Wireless Communications , vol. 20, no. 2, pp. 1188–1200, Oct. 2021

    2021

  21. [28]

    A joint learning and communications framework for federated learning over wireless networks,

    M. Chen, Z. Yang, W. Saad, C. Yin, H. V . Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” IEEE Trans. Wireless Commun. , vol. 20, no. 1, pp. 269–283, Jan. 2021

    2021

  22. [29]

    Federated learning in unreliable and resource-constrained cellular wireless networks,

    M. Salehi and E. Hossain, “Federated learning in unreliable and resource-constrained cellular wireless networks,” IEEE Transactions on Communications , vol. 69, no. 8, pp. 5136–5151, Aug. 2021

    2021

  23. [30]

    Scheduling policies for federated learning in wireless networks,

    H. H. Yang, Z. L. anbd T. Q. S. Quek, and H. V . Poor, “Scheduling policies for federated learning in wireless networks,” IEEE Transactions on Communications , vol. 68, no. 1, pp. 317–333, Jan. 2020

    2020

  24. [31]

    Cost-effective federated learning in mobile edge networks,

    B. Luo, X. Li, S. Wang, J. Huang, and L. Tassiulas, “Cost-effective federated learning in mobile edge networks,” IEEE Journal on Selected Areas in Communications , vol. 99, no. 1, pp. 1–16, Oct. 2021

    2021

  25. [32]

    Automated federated learning in mobile-edge networks—fast adaptation and convergence,

    C. You, K. Guo, G. Feng, P. Yang, and T. Q. S. Quek, “Automated federated learning in mobile-edge networks—fast adaptation and convergence,” IEEE Internet of Things Journal , vol. 10, no. 1, pp. 13 571–13 586, Mar. 2023

    2023

  26. [33]

    Wireless quantized federated learning: A joint computation and communication design,

    P. S. Bouzinis, P. D. Diamantoulakis, and G. K. Karagiannidis, “Wireless quantized federated learning: A joint computation and communication design,” IEEE Transactions on Communications , vol. 71, no. 5, pp. 2756 – 2770, May 2023

    2023

  27. [34]

    Green, quantized federated learning over wireless networks: An energy-efficient design,

    M. Kim, W. Saad, M. Mozaffari, and M. Debbah, “Green, quantized federated learning over wireless networks: An energy-efficient design,” IEEE Transactions on Wireless Communications , vol. 1, no. 1, pp. 1–15, Jun. 2023

    2023

  28. [35]

    On the convergence of FedAvg on non-iid data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of FedAvg on non-iid data,” in Proc. ICLR, Addis Ababa, ETHIOPIA, Apr. 26 - May 1, 2020, pp. 1–11

    2020

  29. [36]

    Variance reduced local sgd with lower communication complexity,

    X. Liang, S. Shen, J. Liu, Z. Pan, E. Chen, and Y . Cheng, “Variance reduced local sgd with lower communication complexity,” arXiv preprint arXiv:1912.12844 , 2020

    1912 arXiv

  30. [37]

    Federated learning based on dynamic regularization,

    D. A. E. Acar, Y . Zhao, R. Matas, M. Mattina, P. Whatmough, and V . Saligrama, “Federated learning based on dynamic regularization,” in Proc. ICLR, May 3-7 2021, pp. 1–6

    2021

  31. [38]

    Mime: Mimicking centralized stochastic algorithms in federated learning,

    S. P. Karimireddy, M. Jaggi, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh, “Mime: Mimicking centralized stochastic algorithms in federated learning,” arXiv preprint arXiv:2008.03606 , 2021

    2008 arXiv

  32. [40]

    Asynchronous federated optimization,

    C. Xie, S. Koyejo, and I. Gupta, “Asynchronous federated optimization,” 2020. [Online]. Available: https: //arxiv.org/abs/1903.03934 35

    2020 arXiv

  33. [41]

    Communication-efficient adaptive federated learning,

    Y . Wang, L. Jin, and J. Chen, “Communication-efficient adaptive federated learning,” in Proc. ICML, Baltimore, Maryland, USA, Jul. 17-23 2022, pp. 1–11

    2022

  34. [42]

    Fast federated learning by balancing communication trade-offs,

    M. . K. Nori, S. Yun, and I.-M. Kim, “Fast federated learning by balancing communication trade-offs,” IEEE Transactions on Communications, vol. 69, no. 8, pp. 5168 – 5182, Aug. 2021

    2021

  35. [43]

    Adaptive control of local updating and model compression for efficient federated learning,

    Y . Xu, Y . Liao, H. Xu, Z. Ma, L. Wang, and J. Liu, “Adaptive control of local updating and model compression for efficient federated learning,” IEEE Transactions on Mobile Computing , vol. 22, no. 10, pp. 5675 – 5689, Oct. 2023

    2023

  36. [44]

    Broadband analog aggregation for low-latency federated edge learning,

    G. Zhu, Y . Wang, and K. Huang, “Broadband analog aggregation for low-latency federated edge learning,” IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 491–506, Jan. 2020

    2020

  37. [45]

    Adaptive federated optimization,

    S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Kone ˇcn´y, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” in Proc. ICLR, Virtual Conference, May 3-7 2021, pp. 1–12

    2021

  38. [46]

    Fedpd: A federated learning framework with adaptivity to non-iid data,

    X. Zhang, M. Hong, S. Dhople, W. Yin, and Y . Liu, “Fedpd: A federated learning framework with adaptivity to non-iid data,” IEEE Transactions on Signal Processing , vol. 1, no. 1, pp. 1–15, 2021

    2021

  39. [47]

    FedADMM: A robust federated deep learning framework with adaptivity to system heterogeneity,

    Y . Gong, Y . Li, and N. M. Freris, “FedADMM: A robust federated deep learning framework with adaptivity to system heterogeneity,” in Proc. ICDE, May 5-12 2022, pp. 2575–2587

    2022

  40. [48]

    Parallel restarted sgd with faster convergence and less communication: Demystifying why model averaging works,

    H. Yu, S. Yang, and S. Zhu, “Parallel restarted sgd with faster convergence and less communication: Demystifying why model averaging works,” in Proc. AAAI 2019 , Honolulu, Hawaii, USA, Jan. 27-Feb. 1 2019, pp. 5693–5700

    2019

  41. [49]

    Local SGD converges fast and communicates little,

    S. U. Stich, “Local SGD converges fast and communicates little,” in Proc. ICLR, New Orleans, LA, USA, May 6 - May 9 2019, pp. 1–5

    2019

  42. [50]

    Communication-efficient learning of deep networks from decentralized data,

    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Areas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. ICML 2017 , Sydney, Australia, Aug. 6-11 2017, pp. 1–10

    2017

  43. [51]

    Federated matrix factorization: Algorithm design and application to data clustering,

    S. Wang and T.-H. Chang, “Federated matrix factorization: Algorithm design and application to data clustering,” IEEE Trans. Signal Process., vol. 70, pp. 1625–1640, Feb. 2022

    2022

  44. [52]

    Federated learning with quantized global model updates,

    M. M. Amiri, D. Gunduz, S. R. Kulkarni, and H. V . Poor, “Federated learning with quantized global model updates,” arXiv preprint arXiv:2006.10672, 2020

    2006 arXiv

  45. [53]

    Federated block coordinate descent scheme for learning global and personalized models,

    R. Wu, A. Scaglione, H.-T. Wai, N. Karakoc, K. Hreinsson, and W.-K. Ma, “Federated block coordinate descent scheme for learning global and personalized models,” in Proc. AAAI, 2021, pp. 10 355–10 362

    2021

  46. [54]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Master’s thesis, Department of Computer Science, University of Toronto , 2009

    2009

  47. [55]

    LeCun, C

    Y . LeCun, C. Cortes, and C. Burges. The mnist database. [Online]. Available: http://yann.lecun.com/exdb/mnist/

  48. [56]

    Nestt: A nonconvex primal-dual splitting method for distributed and stochastic optimization,

    D. Hajinezhad, M. Hong, T. Zhao, and Z. Wang, “Nestt: A nonconvex primal-dual splitting method for distributed and stochastic optimization,” in Proc. NeuIPS, Barcelona, SPAIN, Dec. 5-10 2016, pp. 3215–3223. 36 Supplementary Materials VIII. P ROOF OF LEMMA 1 Lemma 3 For any rou...

    2016

  49. [57]

    − η(gi(θr,Er i −1 i ) − cr i ) 1 + γη . (36) By repeating the above procedure, it holds that θr+1 i − θr 0 = − η 1 + γη Er i −1X t=0 1 1 + γη Er i −1−t(gi(θr,t i ) − cr i ) (37) = − η Er i −1X t=0 1 1 + γη Er i −t(gi(θr,t i ) − cr i ) (38) = − η eEr i Er i −1X t=0 br,t i ∥br i...

  50. [58]

    (53) Next, we scale (47) in Lemma 5 with Cr 2 > 0, taking average of it with respect to all clients, and add it to (53), which yields E[f (θr+1) − f (θr)] + Cr 0 NX i=1 piΞr+1 i − Cr 0 NX i=1 piΞr i ≤ − NX i=1 Dr 0piΞr i − γ − 8Cr 2 2γ2 E[∥∇f (θr)∥2] − γ − 2Cr 1 2γ2 E[∥Br∥2] −...

  51. [59]

    (57) It is natural to consider P r as the potential function because the inequality (57) just delineates its progress made in each round. By letting Dr 0 = Dr 1 = 0, we have ma(1 − a¯ωr)Cr 0 2N − ma(1 − a¯ωr) − 1 2γ − 2N p(1 + ¯ωr)(η ¯Er)2Cr 1 m − 4Cr 2 γ2 − 4(η ¯Er)2Cr 2 = 0,...

  52. [60]

    (72) 42 Then, we need to analyze the terms Cr 0 , Cr 1 , Cr 2 in the RHS of (72)

    (70) Summing (72) up from r = 0 to R − 1, and then dividing it by R yields 1 R R−1X r=0 E[∥∇f (θr)∥2] ≤ 4γ(P 0 − P R) R + 8γma2σ2 SR R−1X r=0 NX i=1 pi(1 + ωr i )∥br i ∥2 2Cr 0 (2N − ma(1 − aωr i ))∥br i ∥2 1 + 4γη 2σ2N mSR R−1X r=0 NX i=1 (1 + ωr i )p2 i ∥br i ∥2 2Cr 1 + 4γη ...

  53. [61]

    (85) Using γη ¯Er ≤ 1 2 √ N p(1+¯ωr), we get zr 1 ≤(γ2 + 4L2) m + 2N p(1 + ¯ωr)(γη ¯Er)2L γ − 2mL2 ≤(γ2 + 4L2) m + L 2γ − 2mL2 ≤ 17γ2 16 + 9L2 4 m ≤ 17m 16 (γ2 + 4L2)

    Since 0 < γη¯Er ≤ 1 2, we have zr 1 >(γ2 + 4L2)m − 4mL2(1 − (γη ¯Er)2) > γ2m. (85) Using γη ¯Er ≤ 1 2 √ N p(1+¯ωr), we get zr 1 ≤(γ2 + 4L2) m + 2N p(1 + ¯ωr)(γη ¯Er)2L γ − 2mL2 ≤(γ2 + 4L2) m + L 2γ − 2mL2 ≤ 17γ2 16 + 9L2 4 m ≤ 17m 16 (γ2 + 4L2). (86) 5Only the uplink communica...

  54. [62]

    In particular, we have γ − 16Cr 2 = γ (γ2 + 4L2)zr 2 (γ2 + 4L2 + 8L2(1 − γη ¯Qr))zr 2 − 8L2m2a2(1 − a¯ωr)(a¯ωr + 3)zr 1 ≥ γ (γ2 + 4L2)zr 2 (γ2 + 8L2)zr 2 − 8L2m2a2(1 − a¯ωr)(a¯ωr + 3)zr 1 (90) ≥ γ (γ2 + 4L2)zr 2 (γ2 + 8L2)zr 2 − 17 2 (γ2 + 4L2)L2m3a2(1 − a¯ωr)(a¯ωr + 3) (91) ≥...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.