Pith. sign in

REVIEW 4 major objections 4 minor 56 references

FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FedWSQ claims that combining weight standardization with a fixed non-uniform quantizer tuned to the standard normal distribution lets federated learning cut uplink communication to 1–2.3 bits per parameter while beating full-precision…

desk verdict Solid engineering, but the quantizer theory has an off-by-one and the Gaussian prior is unverified. read the letter →

arxiv 2506.23516 v3 pith:UNJHLAEY submitted 2025-06-30 cs.LG cs.AIcs.CV

Seung-Wook Kim , Seongyeol Kim , Jiah Kim , Seowon Ji , Se-Ho Lee This is my paper
classification cs.LGcs.AIcs.CV
keywords federatedlearningcommunication-efficientweightstandardizationnon-uniformquantizationdistribution-awareclientdriftdataheterogeneitymixed-precisionallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a federated learning scheme, FedWSQ, built from two pieces: weight standardization, which filters biased directions out of local gradient updates, and a fixed non-uniform quantizer, DANUQ, whose levels are chosen to minimize expected error under a standard normal distribution. The claim is that together these let clients send model updates at 1, 2, or an average of 2.3 bits per parameter without losing accuracy, and in highly non-i.i.d. settings the compressed version can outperform full-precision state-of-the-art methods. The headline number is a 5%-plus accuracy improvement on Tiny-ImageNet at an average of 2.3 bits per parameter compared with leading federated learning methods. If this holds, uplink bandwidth in cross-device federated training can be cut to a few percent of its usual cost while training becomes more stable under skewed data.

What carries the argument

The load-bearing mechanism is the pair (WS, DANUQ). WS reformulates weight standardization as two successive projections: the gradient with respect to the pre-standardized parameter is multiplied by $(I-P_1)(I-P_{\tilde{w}})$, so components along the current weight direction and along the all-ones direction are removed; this is the gradient-filtering step that mitigates client drift. DANUQ then assumes the normalized update $\Delta w/s_{g,l}$ follows $\mathcal{N}(0,1)$, minimizes expected squared quantization error under that density, and uses fixed levels obtained numerically (1-bit: $[-0.798, 0.798]$; 2-bit: $[-1.224, 0, 0.765, 1.724]$; 4-bit values up to $\pm 2.649$). The global scaling vector $s_g$ is updated by an exponential moving average of client layer standard deviations, so no extra per-round quantizer parameters need to be transmitted.

What would settle it

On a real run of FedWSQ over CIFAR-100 at $\alpha=0.1$, collect every layer's normalized LMPUs and run a Kolmogorov–Smirnov test against the standard normal; the supplementary shows only histograms, not a statistical test, so this is the missing evidence. If most layers reject the fit at $p<0.05$, the fixed 1-bit and 2-bit tables are not error-minimal, and per-layer optimized levels should improve accuracy. A concrete spot-check is whether the 1-bit boundary at 0 coincides with the empirical median of each layer's normalized updates.

Watch

Extended reading notes

Core claim

FedWSQ establishes that local model parameter updates, once divided by a per-layer global scaling vector, can be treated as samples from a standard normal distribution $\mathcal{N}(0,1)$, and that fixed quantization levels optimized against that density are enough to make 1-bit and 2-bit communication competitive with, or superior to, 32-bit federated aggregation. Weight standardization contributes by projecting each local gradient onto the subspace orthogonal to the current weight vector and the all-ones vector, which the paper shows removes components that amplify client drift. The quantization levels are obtained numerically by minimizing expected squared error under the standard normal density, yielding tables such as $\pm 0.798$ for 1 bit. The experimental claim is that at 1 bit FedWSQ surpasses full-precision FedRCL and FedACG on CIFAR-100 and Tiny-ImageNet under $\alpha=0.1$, and that low-bit quantization acts as a regularizer under heterogeneity.

Load-bearing premise

The whole quantization scheme rests on the assumption that, after dividing by the global scale, every layer's parameter updates have the same bell-curve shape across clients and rounds; if the real updates are skewed or heavy-tailed, the fixed quantization levels stop being optimal and the reported ultra-low-bit gains could shrink or vanish.

Editorial extensions

If this is right

  • If the central claim is correct, cross-device federated learning can send uplink updates at 1–2 bits per parameter, with mixed allocations averaging 2.33 bits, on image benchmarks.
  • At 1 bit, FedWSQ is reported to outperform full-precision FedRCL and FedACG on CIFAR-100 (62.05 vs. 58.26/58.14 at $\alpha=0.1$) and on Tiny-ImageNet (45.11 vs. 37.86/39.75).
  • The combination is reported to flatten the loss landscape, with a Hessian top eigenvalue of 135.8 versus 671.2 for FedACG, indicating better generalization.
  • Low-bit quantization can improve accuracy rather than merely trade it away: on CIFAR-10 at $\alpha=0.1$, 4-bit FedWSQ reaches 86.84 versus 85.94 for 32-bit FedWS.
  • WS transfers across backbones including ShuffleNet, VGGNet-9, SqueezeNet, and MobileViT, suggesting the benefit is architectural rather than specific to ResNet-18.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the standard-normal prior holds broadly, the published quantization tables could be reused across tasks and models without per-run calibration; a natural test is per-layer normality checks on real LMPUs.
  • The exponential-moving-average global scaling couples quantizer fidelity to client participation patterns, so if participating clients drift toward a subpopulation with different layer standard deviations, normalized updates may stop being standard normal; error feedback or per-layer clipping could hedge this.
  • The paper compresses only the uplink; an implicit extension is whether the same fixed levels also work for downlink compression or for gradient compression in decentralized training.
  • Because 1-bit versions beat 32-bit versions under heterogeneity, part of the gain may come from an implicit regularizing effect of coarse quantization itself, not from optimality of the normal-prior levels; an ablation with uniformly spaced levels at the same bit width would isolate the two effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes FedWSQ, a federated learning method that combines weight standardization (WS) with a fixed non-uniform quantizer (DANUQ). WS is applied to local model weights during local training, and DANUQ compresses local model parameter updates using a global scaling vector and fixed quantization levels derived from a standard-normal prior. The method is evaluated on CIFAR-10, CIFAR-100, and Tiny-ImageNet with 100 clients, 5% participation, and several Dirichlet heterogeneity levels. The central empirical claim is that FedWSQ at 1–4 bits, and at an average of 2.3 bits under mixed-precision allocation, matches or exceeds full-precision state-of-the-art FL methods, with gains of more than 5% on Tiny-ImageNet under high heterogeneity.

Significance. If the empirical results hold, the paper would demonstrate that a fixed, distribution-prior-based quantizer combined with WS can deliver large communication savings without accuracy loss in heterogeneous FL, and even provides a regularization effect at ultra-low bit widths. The main strengths are the clean ablation in Table 2 that isolates the contributions of WS and DANUQ, the breadth of comparisons across datasets, heterogeneity levels, bit-widths, and backbone architectures, and the detailed supplementary derivation of the quantization objective. However, the central quantizer design rests on a normality assumption that is not validated at the per-client or per-layer level, and the quantization-level derivation as written contains an internal inconsistency in the number and symmetry of levels. These issues are load-bearing for the paper's central claim.

major comments (4)
  1. [Section 3.2, Eq. (8)-(9) and the listed quantization levels] The construction is internally inconsistent. With R defined as 2^{B-1}-1 and q0=0 fixed, the scheme in Eq. (8) yields either 2R+1 = 2^B-1 levels (if negative levels are mirrored) or R+1 = 2^{B-1} levels (if Q is the full set). For B=2 this gives 3 or 2 levels, and for B=4 it gives 15 or 8 levels, yet the paper reports 4 and 16 levels, and the reported 2-bit and 4-bit sets are not symmetric about zero. Therefore the listed QLs cannot be the minimizers of Eq. (9) under the stated parameterization. Please restate the optimization variables (R, the symmetry convention, and the role of q0) and provide the brute-force search range and grid so that the QLs are reproducible.
  2. [Section 3.2 and Algorithm 1: standard-normal assumption vs. global scaling] The premise that normalized LMPUs follow a standard normal distribution is not supported by the algorithm. Each client divides its LMPUs by the global scale s_g,l, which is an EMA over clients (Eq. (7)), but the raw LMPUs of client i have layerwise standard deviation s_i,l. The normalized variable therefore has variance (s_i,l/s_g,l)^2, not 1, unless all clients have identical layerwise update variances. Under the paper's extreme heterogeneity setting (alpha=0.1, 100 clients), this is unlikely. The supplementary Figure A shows only an aggregate histogram over clients and layers, which cannot rule out substantial per-client deviations. If the variance deviates from 1, the fixed QLs become MSE-suboptimal. Please provide per-client and per-layer distribution diagnostics, or modify the normalization so that the quantizer input actually has unit variance.
  3. [Section 4.1, Table 5, and Supplementary Section C: model selection and statistical significance] The experimental evaluation is weakened by selection on test accuracy and the absence of error bars. The learning-rate decay for baselines is chosen from {0.995, 0.998, 1} to attain the best accuracy, and the WS hyperparameter rho is selected from the test-set results in Table 5. Moreover, the tables appear to report single runs with no multiple seeds or standard deviations. Since the headline claims are empirical (for example, over 5% improvement on Tiny-ImageNet), please report means and standard deviations over at least three seeds and use a validation split for hyperparameter selection.
  4. [Algorithm 1 and Figure 1: dequantization operation is undefined] The server receives both quantized LMPUs and local scaling vectors s_i, and Algorithm 1 says the server obtains dequantized parameters using (DeltaW_i, s_i), but no dequantization formula is given. If dequantization multiplies by s_i,l, then the effective quantization intervals are determined by s_g,l while the reconstruction levels are determined by s_i,l, so the quantizer is no longer the one optimized in Section 3.2. If dequantization multiplies by s_g,l, the transmitted local scales are not used in reconstruction and the procedure should be stated as such. Please specify the exact dequantization and explain how it remains consistent with the fixed QLs.
minor comments (4)
  1. [Section 4.4 heading and Supplementary Section B] There are typos: 'Comparision' should be 'Comparison', and in the supplementary 'qunatization' appears instead of 'quantization'.
  2. [Table 3] The 1-bit row groups 'NF/FP' as a single row. Please clarify whether NF and FP produce identical 1-bit results or whether one of the methods is omitted.
  3. [Section 4.4, 'Impact on different backbone architectures'] The sentence 'In addition, We evaluate MobileViT...' contains a capitalization error; also, the variance across backbone architectures is reported without error bars, so the stability of the conclusions is unclear.
  4. [Supplementary Section D] The convergence plots include alpha=0.05, which is not defined or discussed in the main text; please either define it or remove it for consistency.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the accuracy and communication-efficiency claims are validated on external benchmarks, while the quantization levels are derived from an explicit Gaussian MSE objective rather than from the target accuracies.

full rationale

The derivation chain is self-contained with respect to circularity. DANUQ's quantization levels are chosen by minimizing the expected squared error under an explicitly stated standard-normal prior in Eqs. (8)-(10), not by fitting target accuracies; the reported gains are then measured on external benchmarks (CIFAR-10/100, Tiny-ImageNet) against published baselines, so the empirical claim is not forced by the construction. WS is taken from Qiao et al. with the algebraic reformulation in Eqs. (5)-(6), and its FL benefit is argued mechanistically and demonstrated experimentally, not derived solely from the definition. No load-bearing self-citation or imported uniqueness theorem appears; the cited [7,45,54] support only the empirical normality premise. Two internal issues are correctness risks rather than circularities: the published 2-bit and 4-bit QL sets do not match the level count R=2^{B-1}-1 implied by Eq. (8)-(9), and the global-EMA scaling in Eq. (7) makes per-client normalized updates have variance (s_i/s_g)^2 rather than 1, which is not verified per client or layer. Also, rho and the learning-rate decay are selected on the evaluation benchmarks (Table 5 and Section C), a model-selection fairness concern that does not make the central claim equivalent to its inputs by construction. Since no prediction reduces to a fitted parameter or to a self-citation chain by the paper's own equations, the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central method rests on the Gaussian prior for local updates and on an interpretive claim about WS projections; the only fitted scalars are hyperparameters chosen by accuracy on benchmarks.

free parameters (4)
  • WS normalization scale rho = 1e-3
    Selected via ablation on CIFAR-10 (Table 5) to maximize test accuracy; the paper states insensitivity across 1e-4 to 1e-1.
  • global scale momentum beta = 0.1
    Set manually for updating the global scaling vector in Eq. (7); no sensitivity study is reported.
  • learning rate decay factor for FedWSQ = 0.995
    Applied per communication round for the proposed method; baselines are tuned over {0.995, 0.998, 1} for best accuracy.
  • QL brute-force search range = restricted range based on empirical observations
    Section 3.2 states the search space is restricted to a reasonable range based on empirical observations; this influences the reported QLs.
assumptions (3)
  • domain assumption Normalized LMPUs follow a standard normal distribution
    Invoked in Section 3.2 to define quantization levels; the paper cites prior work for parameter normality but provides no formal test for LMPUs in FL.
  • domain assumption WS gradient projection removes client drift
    Section 3.1 interprets the two projections as filtering biased components; this is an intuitive argument, not a proven convergence statement.
  • domain assumption Standard deviation is a robust scale for quantizing LMPUs
    Section 3.2 chooses std over absmax to avoid outlier sensitivity; no outlier analysis is supplied.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization." pith.science (2026). https://pith.science/paper/UNJHLAEY

@misc{pith2026250623516,
  author       = {Pith},
  title        = {Pith review of: FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UNJHLAEY}},
  note         = {Machine review of arXiv:2506.23516}
}
read the original abstract

Federated learning (FL) often suffers from performance degradation due to key challenges such as data heterogeneity and communication constraints. To address these limitations, we present a novel FL framework called FedWSQ, which integrates weight standardization (WS) and the proposed distribution-aware non-uniform quantization (DANUQ). WS enhances FL performance by filtering out biased components in local updates during training, thereby improving the robustness of the model against data heterogeneity and unstable client participation. In addition, DANUQ minimizes quantization errors by leveraging the statistical properties of local model updates. As a result, FedWSQ significantly reduces communication overhead while maintaining superior model accuracy. Extensive experiments on FL benchmark datasets demonstrate that FedWSQ consistently outperforms existing FL methods across various challenging FL settings, including extreme data heterogeneity and ultra-low-bit communication scenarios.

Figures

Figures reproduced from arXiv: 2506.23516 by the authors.

Figure 1
Figure 1. The overall framework of FedWSQ, which integrates WS and DANUQ for efficient FL. Clients transmit LMPUs quantized by [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An example of gradient filtering process of WS. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Loss landscape of FL methods trained on CIFAR-100, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 42 canonical work pages

  1. [1]

    Fed- erated learning based on dynamic regularization

    Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Fed- erated learning based on dynamic regularization. In ICLR,

    work page

  2. [2]

    QReg: On Regularization Effects of Quantization

    MohammadHossein AskariHemmat, Reyhane Askari Hem- mat, Alex Hoffman, Ivan Lazarevich, Ehsan Saboori, Olivier Mastropietro, Sudhakar Sah, Yvon Savaria, and Jean-Pierre David. QReg: On regularization effects of quantization. arXiv preprint arXiv:2206.12372, 2022. 7

    work page Pith review arXiv 2022

  3. [3]

    QGen: On the ability to generalize in quantization aware training

    MohammadHossein AskariHemmat, Ahmadreza Jeddi, Rey- hane Askari Hemmat, Ivan Lazarevich, Alexander Hoffman, Sudhakar Sah, Ehsan Saboori, Yvon Savaria, and Jean-Pierre David. QGen: On the ability to generalize in quantization aware training. arXiv preprint arXiv:2404.11769, 2024. 7

    arXiv 2024

  4. [4]

    Qsparse-Local-SGD: Distributed SGD with quantiza- tion, sparsification, and local computations.IEEE Journal on Selected Areas in Information Theory , 1(1):217–226, 2020

    Debraj Basu, Deepesh Data, Can Karakus, and Suhas N Dig- gavi. Qsparse-Local-SGD: Distributed SGD with quantiza- tion, sparsification, and local computations.IEEE Journal on Selected Areas in Information Theory , 1(1):217–226, 2020. 1

    work page 2020

  5. [5]

    Dynamic aggregation for heterogeneous quantization in federated learning

    Shengbo Chen, Cong Shen, Lanxue Zhang, and Yuanmin Tang. Dynamic aggregation for heterogeneous quantization in federated learning. IEEE Transactions on Wireless Com- munications, 20(10):6804–6819, 2021. 2, 6

    work page 2021

  6. [6]

    Pact: Parameterized clipping activation for quantized neural networks

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. Pact: Parameterized clipping activation for quantized neural networks. In ICLR, 2018. 2

    work page 2018

  7. [7]

    Dettmers, M

    T. Dettmers, M. Lewis, S. Shleifer, and L. Zettlemoyer. 8-bit optimizers via block-wise quantization. In ICLR, 2022. 4, 8

    work page 2022

  8. [8]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. In NeurIPS, 2023. 2

    work page 2023

Show all 56 references
  1. [9]

    Esser, Jeffrey L

    Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha. Learned Step Size Quantization. In ICLR, 2020. 2

    2020

  2. [10]

    Gray and David L

    Robert M. Gray and David L. Neuhoff. Quantization. IEEE transactions on information theory, 44(6):2325–2383, 1998. 2

    1998

  3. [11]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR,

  4. [12]

    The non-IID data quagmire of decentralized ma- chine learning

    Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons. The non-IID data quagmire of decentralized ma- chine learning. In ICML, 2020. 6, 3

    2020

  5. [13]

    Mea- suring the effects of non-identical data distribution for feder- ated visual classification

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Mea- suring the effects of non-identical data distribution for feder- ated visual classification. arXiv preprint arXiv:1909.06335,

    1909 arXiv

  6. [14]

    SqueezeNet: AlexNet-level accuracy with 50x fewer pa- rameters and <0.5 MB model size

    Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. SqueezeNet: AlexNet-level accuracy with 50x fewer pa- rameters and <0.5 MB model size. arXiv preprint arXiv:1602.07360, 2016. 8

    2016 arXiv

  7. [15]

    Batch normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In ICML, 2015. 6

    2015

  8. [16]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In CVPR, pages 2704–2713, 2018. 2, 4

    2018

  9. [17]

    Learning to quantize deep networks by opti- mizing quantization intervals with task loss

    Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Youngjun Kwak, Jae-Joon Han, Sung Ju Hwang, and Changkyu Choi. Learning to quantize deep networks by opti- mizing quantization intervals with task loss. In CVPR, pages 4350–4359, 2019. 2

    2019

  10. [18]

    SCAFFOLD: Stochastic controlled averaging for on- device federated learning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for on- device federated learning. In ICML, 2020. 1

    2020

  11. [19]

    First analysis of local GD on heterogeneous data

    Ahmed Khaled, Konstantin Mishchenko, and Peter Richt´arik. First analysis of local GD on heterogeneous data. arXiv preprint arXiv:1909.04715, 2019. 1

    1909 arXiv

  12. [20]

    Communication-efficient federated learning with accel- erated client gradient

    Geeho Kim, Jinkyu Kim, and Bohyung Han. Communication-efficient federated learning with accel- erated client gradient. In CVPR, 2024. 6, 3

    2024

  13. [21]

    Multi-level branched regularization for federated learning

    Jinkyu Kim, Geeho Kim, and Bohyung Han. Multi-level branched regularization for federated learning. In ICML,

  14. [22]

    Brendan McMahan, Felix X

    Jakub Kone ˇcn´y, H. Brendan McMahan, Felix X. Yu, Ananda Theertha Suresh, Dave Bacon, and Peter Richt ´arik. Federated learning: Strategies for improving communication efficiency. In arXiv preprint, 2017. 2

    2017

  15. [23]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Sci- ence, University of Toronto, 2009. 6

    2009

  16. [24]

    Exploring the prac- ticality of federated learning: A survey towards the com- munication perspective

    Khiem Le, Nhan Luong-Ha, Manh Nguyen-Duc, Danh Le- Phuoc, Cuong Do, and Kok-Seng Wong. Exploring the prac- ticality of federated learning: A survey towards the com- munication perspective. arXiv preprint arXiv:2405.20431 ,

    arXiv

  17. [25]

    Tiny ImageNet visual recognition challenge

    Ya Le and Xuan Yang. Tiny ImageNet visual recognition challenge. CS 231N, 7(7):3, 2015. 6

    2015

  18. [26]

    Preservation of the global knowledge by not- true distillation in federated learning

    Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, and Se-Young Yun. Preservation of the global knowledge by not- true distillation in federated learning. In NeurIPS, 2022. 6, 3

    2022

  19. [27]

    Federated optimiza- tion in heterogeneous networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimiza- tion in heterogeneous networks. In MLSys, 2020. 2, 6, 3

    2020

  20. [28]

    On the convergence of FedAvg on non-IID data

    Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of FedAvg on non-IID data. In ICLR, 2020. 1

    2020

  21. [29]

    Fixed point quantization of deep convolutional networks

    Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy. Fixed point quantization of deep convolutional networks. In 9 International conference on machine learning, pages 2849–

  22. [30]

    CMFL: Mitigating communication overhead for federated learning

    Wang Luping, Wang Wei, and Li Bo. CMFL: Mitigating communication overhead for federated learning. In 2019 IEEE 39th international conference on distributed comput- ing systems (ICDCS), pages 954–964. IEEE, 2019. 1

    2019

  23. [31]

    Communication- efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In AISTATS, 2017. 1, 2, 6

    2017

  24. [32]

    Mobilevit: light- weight, general-purpose, and mobile-friendly vision trans- former

    Sachin Mehta and Mohammad Rastegari. Mobilevit: light- weight, general-purpose, and mobile-friendly vision trans- former. arXiv preprint arXiv:2110.02178, 2021. 8

    2021 arXiv

  25. [33]

    A white paper on neural network quantization

    Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yely- sei Bondarenko, Mart van Baalen, and Tijmen Blankevoort. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295, 2021. 2, 4

    2021 arXiv

  26. [34]

    PyTorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An imperative style, high-performance deep learning library. In NeurIPS, 2019. 3

    2019

  27. [35]

    Micro-batch training with batch-channel nor- malization and weight standardization

    Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, and Alan Yuille. Micro-batch training with batch-channel nor- malization and weight standardization. arXiv preprint arXiv:1903.10520, 2019. 1, 3, 6

    1903 arXiv

  28. [36]

    Adaptive federated optimization

    Sashank J Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone ˇcn`y, Sanjiv Kumar, and Hugh Brendan McMahan. Adaptive federated optimization. In ICLR, 2021. 2, 6, 3

    2021

  29. [37]

    FedPAQ: A communication-efficient federated learning method with pe- riodic averaging and quantization

    Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Has- sani, Ali Jadbabaie, and Ramtin Pedarsani. FedPAQ: A communication-efficient federated learning method with pe- riodic averaging and quantization. In AISTATS, 2020. 2, 6, 8

    2020

  30. [38]

    Relaxed contrastive learning for federated learning

    Seonguk Seo, Jinkyu Kim, Geeho Kim, and Bohyung Han. Relaxed contrastive learning for federated learning. In CVPR, 2024. 6, 3

    2024

  31. [39]

    Towards understanding and mitigating dimen- sional collapse in heterogeneous federated learning

    Yujun Shi, Jian Liang, Wenqing Zhang, Vincent YF Tan, and Song Bai. Towards understanding and mitigating dimen- sional collapse in heterogeneous federated learning. InICLR,

  32. [40]

    Very deep convo- lutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. InICLR,

  33. [41]

    Local SGD converges fast and communi- cates little

    Sebastian U Stich. Local SGD converges fast and communi- cates little. In ICLR, 2019. 1

    2019

  34. [42]

    The error- feedback framework: Better rates for SGD with delayed gradients and compressed communication

    Sebastian U Stich and Sai Praneeth Karimireddy. The error- feedback framework: Better rates for SGD with delayed gradients and compressed communication. arXiv preprint arXiv:1909.05350, 2019. 1

    1909 arXiv

  35. [43]

    Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape

    Yan Sun, Li Shen, Shixiang Chen, Liang Ding, and Dacheng Tao. Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape. In International Conference on Machine Learning, pages 32991–33013. PMLR, 2023. 2, 6

    2023

  36. [44]

    Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms

    Jianyu Wang and Gauri Joshi. Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms. Journal of Machine Learning Research, 22(213): 1–50, 2021. 1

    2021

  37. [45]

    Opti- mizing large language model training using fp4 quantization

    Ruizhe Wang, Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, and Peng Cheng. Opti- mizing large language model training using fp4 quantization. In arXiv preprint arXiv:2501.17116, 2025. 4, 8

    2025 arXiv

  38. [46]

    A survey on federated learning: chal- lenges and applications

    Jie Wen, Zhixia Zhang, Yang Lan, Zhihua Cui, Jianghui Cai, and Wensheng Zhang. A survey on federated learning: chal- lenges and applications. International Journal of Machine Learning and Cybernetics, 14(2):513–535, 2023. 1

    2023

  39. [47]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. In ECCV,

  40. [48]

    FedCM: Federated learning with client-level momen- tum

    Jing Xu, Sen Wang, Liwei Wang, and Andrew Chi-Chih Yao. FedCM: Federated learning with client-level momen- tum. arXiv preprint arXiv:2106.10874, 2021. 6, 3

    2021 arXiv

  41. [49]

    Achieving linear speedup with partial worker participation in non-IID feder- ated learning

    Haibo Yang, Minghong Fang, and Jia Liu. Achieving linear speedup with partial worker participation in non-IID feder- ated learning. In ICLR, 2021. 1

    2021

  42. [50]

    Parallel restarted SGD with faster convergence and less communication: De- mystifying why model averaging works for deep learning

    Hao Yu, Sen Yang, and Shenghuo Zhu. Parallel restarted SGD with faster convergence and less communication: De- mystifying why model averaging works for deep learning. In AAAI, 2019. 1

    2019

  43. [51]

    LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

    Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks. In ECCV, 2018. 2

    2018

  44. [52]

    Federated learning with la- bel distribution skew via logits calibration

    Jie Zhang, Zhiqi Li, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Chao Wu. Federated learning with la- bel distribution skew via logits calibration. In ICML, 2022. 6, 3

    2022

  45. [53]

    ShuffleNet: An extremely efficient convolutional neural net- work for mobile devices

    Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. ShuffleNet: An extremely efficient convolutional neural net- work for mobile devices. In CVPR, 2018. 8

    2018

  46. [54]

    Distribution-aware adaptive multi-bit quantization

    Sijie Zhao, Tao Yue, and Xuemei Hu. Distribution-aware adaptive multi-bit quantization. In CVPR, 2021. 4

    2021

  47. [55]

    Chenzhuo Zhu, Song Han, Huizi Mao, and William J. Dally. Trained ternary quantization. In ICLR, 2017. 2

    2017

  48. [56]

    FedWon: Triumphing multi-domain federated learning without normalization

    Weiming Zhuang and Lingjuan Lyu. FedWon: Triumphing multi-domain federated learning without normalization. In ICLR, 2023. 4, 6 10 FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization –Supplementary Document– Paper ID ...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.