Pith. sign in

REVIEW 3 major objections 4 minor 64 references

A federated learning protocol that trains one bit of each quantized weight per round matches uncompressed FedAvg accuracy, using only 1 bit per parameter for uplink and 3–4 bits per parameter for downlink.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 18:05 UTC pith:CUU6OOFL

load-bearing objection FedBiF is a real compression mechanism with solid empirical support—matching FedAvg at 1 bpp uplink on five benchmarks—but the paper's theory and gradient definition lag the deployed algorithm. the 3 major comments →

arxiv 2509.10161 v1 pith:CUU6OOFL submitted 2025-09-12 cs.LG cs.DC

FedBiF: Communication-Efficient Federated Learning via Bits Freezing

Shiwei Li , Qunwei Li , Haozhao Wang , Ruixuan Li , Jianbin Lin , Wenliang Zhong This is my paper
classification cs.LG cs.DC
keywords federated learningcommunication efficiencyquantizationbits freezingvirtual bitsstraight-through estimatormodel sparsitybit-per-parameter
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Federated Bit Freezing (FedBiF), a communication-efficient federated learning method that learns quantized model parameters directly during local training instead of quantizing updates only after training. In each round the server sends an m-bit quantized model; each client trains exactly one binary bit per parameter via a floating-point 'virtual bit' and freezes the remaining bits. Clients upload only the changed bit, so uplink communication is 1 bit-per-parameter and downlink is m bits. On five widely used image datasets under both IID and Non-IID data distributions, FedBiF achieves test accuracy comparable to FedAvg at 1 bpp uplink and 3–4 bpp downlink, while existing post-training quantization baselines lose several points. The method also naturally produces sparse models, which can be exploited for further compression.

Core claim

FedBiF claims that the communication bottleneck in federated learning can be reduced to one bit per parameter per round without sacrificing model accuracy by decoupling parameter precision from communication cost. Instead of quantizing gradients or updates after local training, the server quantizes global parameters into m-bit integers; each client decomposes each integer into binary bits and trains a single 'activated' bit, represented by a sign function of a floating-point virtual bit, while frozen bits are precomputed and reused. Because the uploaded update is just the binary value of the activated bit, uplink is 1 bpp, and the paper reports accuracy within about 0.2–0.4 points of FedAvg

What carries the argument

The central mechanism is bit freezing: each quantized integer parameter is decomposed as a weighted sum of m binary bits, and one bit per parameter is activated each round while the rest are frozen. The activated bit is learned through a floating-point virtual bit v_i, with the binary value given by a step function h(v_i); forward weights are reconstructed cheaply as the precomputed frozen-bit contribution plus the scaled activated bit. Gradients reach the virtual bit through the Straight-Through Estimator, which approximates the derivative of the step function as 1 (dropping the α·2^i scaling), letting gradient descent move the virtual bit across zero so the binary bit flips at the right ti

Load-bearing premise

The load-bearing premise is that the Straight-Through Estimator—treating the non-differentiable step function as identity and setting ∂θ̂/∂v_i = 1, dropping the α·2^i scaling—gives a gradient direction informative enough to flip or retain bits correctly in deep networks with batch normalization and ReLU; if this proxy gradient is poor, the bit-by-bit learning mechanism and the empirical match to FedAvg would not hold.

What would settle it

Run the released code on CIFAR-10 with a standard deep convolutional network under a Non-IID-2 partition at 1 bpp uplink and 4 bpp downlink, and compare the final test accuracy to a full-precision FedAvg run with identical local-update steps; if FedBiF falls more than about one percentage point short of FedAvg, the paper's headline claim of comparable accuracy at 1 bpp is not robust. A more direct check is to replace the STE gradient in Eq. (7) with the true scaled gradient α·2^i and observe whether bit-flip decisions change materially.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Communication per round drops to 1 bpp uplink and m bpp downlink while keeping accuracy close to uncompressed FedAvg, directly addressing the dominant bottleneck in bandwidth-limited federated learning.
  • FedBiF converges at roughly the same speed as FedAvg: the reported convergence curves under equal communication cost show faster progress than quantization-based baselines like SignSGD, FedPAQ, FedDQ, DAdaQuant, and LFL.
  • The learned models are naturally sparse—for example, 47–62% zero-valued parameters on FMNIST and SVHN with a 2-bit representation—offering an automatic pruning effect without manual intervention.
  • Activating one bit per round is the best choice: activating more bits per round reduces accuracy in Non-IID settings, likely because local models overfit on heterogeneous data, and it also increases communication and compute.
  • The method remains robust to the number of clients and model architecture: with 200 clients and with a DenseNet on CIFAR-10, FedBiF still matches FedAvg and outperforms LFL at equal communication cost.
  • The choice of which bit to activate is not sensitive—cyclic and random activation yield comparable accuracy, but always training a fixed bit performs much worse, confirming that all bits must eventually be updated.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The reported per-direction costs imply a total bidirectional cost of m+1 bpp (e.g., 5 bpp for m=4) versus 64 bpp for full-precision FedAvg, a compression factor above 12×; the paper does not state this aggregate figure explicitly.
  • Editorial inference: The convergence analysis assumes all m bits are activated during training, while the empirical protocol activates only one bit per round; showing that the one-bit-activation scheme obeys a similar bound would place the practical algorithm on the same theoretical footing as the theorem.
  • Editorial inference: Because the Straight-Through Estimator drops the α·2^i scaling in Eq. (6), the method implicitly treats every bit as equally easy to flip; a testable extension is to scale the virtual-bit gradient by α·2^i and measure whether convergence or final accuracy changes, which would reveal whether the proxy gradient is the right one.
  • Editorial inference: The natural sparsity of FedBiF models suggests a direct extension: combine bit freezing with sparse masks or top-k bit selection to cut even the downlink cost below m bpp, since a large fraction of parameters can be transmitted as zeros.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FedBiF, a federated learning framework in which model parameters are quantized to m-bit integer representations and each client updates only a single bit per parameter per communication round, while the remaining bits are frozen. The selected bit is represented by a floating-point 'virtual bit' during local training; after local steps, clients upload only the binary values of the activated bits. The server reconstructs the local models from the uploaded activated bit and the precomputed frozen-bit sum, then aggregates them. The paper reports experiments on FMNIST, SVHN, CIFAR-10, CIFAR-100, and TinyImageNet under IID and two non-IID settings, claiming accuracy comparable to FedAvg at 1 bpp uplink and 3-4 bpp downlink, along with natural sparsity of the learned models. Theoretical convergence bounds are provided for an all-bits-activated idealized variant.

Significance. The empirical claim is potentially significant: if a one-bit-per-parameter uplink update can match FedAvg performance while retaining multi-bit precision in the parameter representation, it would be a useful contribution to communication-efficient federated learning. The paper includes a broad set of experiments with error bars, ablations over bit width and bit-selection strategy, and open-source code, which strengthens the empirical contribution. However, the novelty is partially undercut by the authors' own prior FedBAT work, cited in Section II-B, and the theoretical section analyzes a different algorithm from the one deployed. The central empirical result appears plausible and reproducible on the presented benchmarks, but the current manuscript overstates both the exact compression ratio and the degree of formal support for the deployed algorithm.

major comments (3)
  1. [IV-A, Eq. (7)] Equation (7) sets ∂θ̂/∂v_i = 1, but Eq. (6) defines θ̂ = α(Σ 2^i h(v_i) − 2^{m−1}). Applying STE to the step function h would give ∂θ̂/∂v_i = α·2^i, not 1. The sign of the gradient is unchanged, so the descent direction is the same, but the magnitude is wrong by a position-dependent factor. Since a bit flips only when its virtual bit crosses zero, this scaling affects which bits flip and when. The paper should either use the correct α·2^i scaling, or explicitly define a normalized STE and justify it, and state which variant the released code implements. As written, Eq. (7) does not describe a loss-aligned update for the actual quantization defined in Eq. (6).
  2. [IV-C, Theorems 1 and 2] Both theorems explicitly assume 'bits freezing with all m bits activated', while the FedBiF algorithm described in Section IV-B activates only one bit per parameter per round. The error-floor bounds in Eqs. (16)–(17) depend exponentially on m and on the simultaneous update of all m virtual bits; they do not apply to the cyclic one-bit-per-round protocol. The paper should either extend the analysis to the one-bit activation schedule or clearly label the theorems as analyzing an idealized all-bit variant and state that the empirical claims do not rely on these bounds. As written, the convergence section gives a misleading impression of formal support for the actual method.
  3. [Abstract, Table II, Table V] The abstract and introduction claim FedBiF achieves FedAvg-level accuracy with '1 bpp uplink and 3 bpp downlink'. In Table II, this downlink bit rate holds only for FMNIST and SVHN; CIFAR-10, CIFAR-100, and TinyImageNet use 4 bpp downlink. Table V shows that the 3-bpp configuration (FedBiF-1/3) on CIFAR-10 reaches 75.0% versus 78.1% for FedAvg, a 3.1-point drop. Thus the 3-bpp claim is not supported on the larger benchmarks. The reporting should either be restricted to the settings actually used or the accuracy at 3 bpp should be reported for all datasets.
minor comments (4)
  1. [I and II-B] The phrase 'a pioneering direction for CEFL' in Section I is contradicted by Section II-B, which cites the authors' own FedBAT [35] as 'a similar mechanism that directly learns the binary model updates during local training'. The novelty claim should be softened to an incremental contribution over FedBAT.
  2. [V-E] The claim that 'models trained by baseline methods exhibit zero sparsity' should be qualified. Sparsity is reported for the quantized representation used by FedBiF; baselines trained in FP32 may have parameters close to zero but not exactly zero. The comparison would be clearer if sparsity were measured on a common thresholded basis.
  3. [Eq. (12)] The initialization in Eq. (12) should be written more carefully: v_i = (2b_i − 1)|v_i| maps b_i=0 to a negative value and b_i=1 to a positive value, which preserves the sign of the step function. It would help to state explicitly that the absolute value is taken of the Kaiming-initialized value.
  4. [IV-C, Eq. (16)] The bound in Eq. (16) includes factors P and D that are not fully motivated in the text; the proof is deferred to an appendix not included in the manuscript. Please ensure the appendix is provided and that the assumptions on the relation between v and w are stated precisely.

Circularity Check

0 steps flagged

No circular derivation: FedBiF's accuracy claim is measured against external baselines; self-citation to FedBAT is acknowledged and not load-bearing.

full rationale

The paper's central claim — FedBiF reaches FedAvg-comparable accuracy at 1 bpp uplink and 3–4 bpp downlink — is an experimental result compared against FedAvg and external baselines on five datasets, not an output of a fitted constant or a self-citation. The bit decomposition Eq. (4), the reconstruction Eq. (9), and the aggregation Eq. (13) are definitions/architectural identities, not predictions derived from themselves. The straight-through estimator in Eq. (7) is an explicit approximation (∂θ̂/∂v_i = 1), not a hidden input; its mismatch with the α·2^i scaling in Eq. (6) is a reproducibility/correctness concern, not a circular reduction. The convergence theorems (Eqs. 16–17) explicitly assume 'all m bits activated' while the protocol activates one bit per round, so the proof does not cover the executed algorithm; this is a validity gap rather than circularity. The appendix referenced for the proofs is not present in the supplied text, which is an omitted-support issue, not a circular one. The only self-referential element is the citation of the authors' own FedBAT [35] in Sec. II.B, which the paper admits is a similar mechanism; that citation is not used to justify the empirical accuracy claim, and it actually weakens the 'pioneering direction' claim. Hence no equation or fitted parameter reduces to the target result. Score 2 reflects the minor, non-load-bearing self-citation rather than any circular derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 2 invented entities

The central empirical claim depends on tuned hyperparameters (bit width, learning rate, local epochs) and on the STE approximation that lets bits be trained as continuous surrogates. The convergence theory is not directly applicable to the actual algorithm. No new physical entities are postulated; the 'virtual bit' is an algorithmic construct.

free parameters (4)
  • Quantization bit width m (downlink bpp) = 3 for FMNIST/SVHN, 4 for CIFAR-10/CIFAR-100/TinyImageNet
    Tuned among {2,3,4} per Section V.A; affects communication cost and final accuracy. The abstract's 3 bpp claim does not match the table for three datasets.
  • Learning rate = 0.01 for all datasets
    Tuned among {1.0, 0.1, 0.01, 0.001} in Section V.A; chosen to maximize accuracy, and the central training curves depend on it.
  • Number of local epochs = 3 (6 in the 200-client setting)
    Set in Section V.A; increased to 6 in the 200-client robustness test to keep local update steps constant. Determines how much each client moves per round.
  • Bit selection strategy = cyclic activation of bits (FedBiF-1/4)
    A design choice rather than a fitted number, but it determines convergence. Ablations show random activation (FedBiF-R1/4) is slightly worse, and fixed-bit variants degrade sharply.
axioms (3)
  • ad hoc to paper Straight-through estimator sets d theta_hat / d v_i = 1, treating the quantizer as identity in backpropagation.
    Section IV-A, Eq. (7). This is the gradient rule that makes virtual bits trainable; the paper does not justify dropping the alpha * 2^i scaling from Eq. (6).
  • standard math The convergence analysis assumes bounded gradients, L-smoothness, bounded diameter of virtual bits, and bounded model norm (Assumptions 1-4).
    Section IV-C. These are standard optimization assumptions, but the theorems apply to all-bits-activated continuous training, not the step-function one-bit-per-round algorithm.
  • domain assumption Uniform symmetric quantization with alpha = ||theta||_infinity / 2^{m-1} and clamp range [-2^{m-1}, 2^{m-1}-1].
    Section IV-B, Eqs. (10)-(11). This is a standard quantization scheme and orthogonal to the bit-freezing mechanism.
invented entities (2)
  • virtual bit v_i no independent evidence
    purpose: A floating-point surrogate for each binary bit, optimized by gradient descent; the binary bit is its sign via a step function.
    Introduced in Section IV-A. It has no external falsifiable handle; its validity is only shown by the method's internal accuracy, making it an algorithmic device rather than an independently supported entity.
  • precomputed frozen-bits sum s_t no independent evidence
    purpose: Stores the contribution of all non-activated bits, Eq. (8), to make one-bit-per-parameter training memory- and compute-efficient.
    An implementation device to reduce overhead, not a scientific entity; its value is internal to the method and has no external testable consequences.

pith-pipeline@v1.3.0-alltime-deepseek · 18634 in / 14439 out tokens · 132378 ms · 2026-08-04T18:05:19.723341+00:00 · methodology

0 comments
read the original abstract

Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative model training without sharing local data. Despite its advantages, FL suffers from substantial communication overhead, which can affect training efficiency. Recent efforts have mitigated this issue by quantizing model updates to reduce communication costs. However, most existing methods apply quantization only after local training, introducing quantization errors into the trained parameters and potentially degrading model accuracy. In this paper, we propose Federated Bit Freezing (FedBiF), a novel FL framework that directly learns quantized model parameters during local training. In each communication round, the server first quantizes the model parameters and transmits them to the clients. FedBiF then allows each client to update only a single bit of the multi-bit parameter representation, freezing the remaining bits. This bit-by-bit update strategy reduces each parameter update to one bit while maintaining high precision in parameter representation. Extensive experiments are conducted on five widely used datasets under both IID and Non-IID settings. The results demonstrate that FedBiF not only achieves superior communication compression but also promotes sparsity in the resulting models. Notably, FedBiF attains accuracy comparable to FedAvg, even when using only 1 bit-per-parameter (bpp) for uplink and 3 bpp for downlink communication. The code is available at https://github.com/Leopold1423/fedbif-tpds25.

Figures

Figures reproduced from arXiv: 2509.10161 by Haozhao Wang, Jianbin Lin, Qunwei Li, Ruixuan Li, Shiwei Li, Wenliang Zhong.

Figure 1
Figure 1. Figure 1: (a) The framework of Federated Bit Freezing (FedBiF). [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A comparison of Post-Training Quantization (PTQ), Quantization [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Convergence curves on CIFAR-10, CIFAR-100, and TinyImageNet under the same communication costs. The x-axis denotes the sum of the bits-per [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

64 extracted references · 7 linked inside Pith

  1. [1]

    Qsgd: Communication-efficient sgd via gradient quantization and encoding.Advances in neural information processing systems, 30, 2017

    Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan V ojnovic. Qsgd: Communication-efficient sgd via gradient quantization and encoding.Advances in neural information processing systems, 30, 2017

    2017

  2. [2]

    Kulkarni, and H

    Mohammad Mohammadi Amiri, Deniz G ¨und¨uz, Sanjeev R. Kulkarni, and H. Vincent Poor. Federated learning with quantized global model updates.CoRR, abs/2006.10672, 2020

    Pith/arXiv arXiv 2006

  3. [3]

    Post training 4- bit quantization of convolutional networks for rapid-deployment

    Ron Banner, Yury Nahshan, and Daniel Soudry. Post training 4- bit quantization of convolutional networks for rapid-deployment. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alch´e-Buc, Emily B. Fox, and Roman Garnett, editors,Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Syst...

    2019

  4. [4]

    signsgd with majority vote is communication efficient and fault tolerant

    Jeremy Bernstein, Jiawei Zhao, Kamyar Azizzadenesheli, and Anima Anandkumar. signsgd with majority vote is communication efficient and fault tolerant. In7th International Conference on Learning Repre- sentations, ICLR. OpenReview.net, 2019

    2019

  5. [5]

    Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, and Nicholas D

    Daniel J. Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, and Nicholas D. Lane. Flower: A friendly federated learning research framework.CoRR, abs/2007.14390, 2020

    Pith/arXiv arXiv 2007

  6. [6]

    Large-scale machine learning with stochastic gradient descent

    L ´eon Bottou. Large-scale machine learning with stochastic gradient descent. In19th International Conference on Computational Statistics, COMPSTAT, pages 177–186. Physica-Verlag, 2010

    2010

  7. [7]

    Adaptive federated dropout: Improving communication efficiency and generalization for federated learning

    Nader Bouacida, Jiahui Hou, Hui Zang, and Xin Liu. Adaptive federated dropout: Improving communication efficiency and generalization for federated learning. In2021 IEEE Conference on Computer Commu- nications Workshops, INFOCOM Workshops, pages 1–6. IEEE, 2021

    2021

  8. [8]

    PACT: parameterized clipping activation for quantized neural networks.CoRR, abs/1805.06085, 2018

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. PACT: parameterized clipping activation for quantized neural networks.CoRR, abs/1805.06085, 2018

    Pith/arXiv arXiv 2018

  9. [9]

    Bina- ryconnect: Training deep neural networks with binary weights during propagations

    Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Bina- ryconnect: Training deep neural networks with binary weights during propagations. InAdvances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, pages 3123–3131, 2015

    2015

  10. [10]

    Docofl: Downlink compression for cross-device federated learning

    Ron Dorfman, Shay Vargaftik, Yaniv Ben-Itzhak, and Kfir Yehuda Levy. Docofl: Downlink compression for cross-device federated learning. In International Conference on Machine Learning, ICML, volume 202, pages 8356–8388. PMLR, 2023

    2023

  11. [11]

    Esser, Jeffrey L

    Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha. Learned step size quantization. In8th International Conference on Learning Representations, ICLR. OpenReview.net, 2020

    2020

  12. [12]

    Deep sparse rectifier neural networks

    Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. InProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS, pages 315– 323, 2011

    2011

  13. [13]

    EF21- P and friends: Improved theoretical communication complexity for distributed optimization with bidirectional compression

    Kaja Gruntkowska, Alexander Tyurin, and Peter Richt ´arik. EF21- P and friends: Improved theoretical communication complexity for distributed optimization with bidirectional compression. InInternational Conference on Machine Learning, ICML, volume 202, pages 11761– 11807. PMLR, 2023. 11

    2023

  14. [14]

    Quantization robust federated learning for efficient inference on heterogeneous devices.Trans

    Kartik Gupta, Marios Fournarakis, Matthias Reisser, Christos Louizos, and Markus Nagel. Quantization robust federated learning for efficient inference on heterogeneous devices.Trans. Mach. Learn. Res., 2023

    2023

  15. [15]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In2015 IEEE International Conference on Computer Vision, ICCV, pages 1026–1034. IEEE Computer Society, 2015

    2015

  16. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 770–778. IEEE Computer Society, 2016

    2016

  17. [17]

    Neural networks for machine learning.Coursera video lectures, 2012

    Geoffrey Hinton. Neural networks for machine learning.Coursera video lectures, 2012

    2012

  18. [18]

    Robert H ¨onig, Yiren Zhao, and Robert D. Mullins. Dadaquant: Doubly- adaptive quantization for communication-efficient federated learning. In International Conference on Machine Learning, ICML, volume 162, pages 8852–8866. PMLR, 2022

    2022

  19. [19]

    Weinberger

    Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 2261–2269, 2017

    2017

  20. [20]

    Fedpara: Low-rank hadamard product for communication-efficient federated learning

    Nam Hyeon-Woo, Moon Ye-Bin, and Tae-Hyun Oh. Fedpara: Low-rank hadamard product for communication-efficient federated learning. In The Tenth International Conference on Learning Representations, ICLR. OpenReview.net, 2022

    2022

  21. [21]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. InProceed- ings of the 32nd International Conference on Machine Learning, ICML, pages 448–456, 2015

    2015

  22. [22]

    Sparse random networks for communication-efficient federated learning

    Berivan Isik, Francesco Pase, Deniz Gunduz, Tsachy Weissman, and Zorzi Michele. Sparse random networks for communication-efficient federated learning. InThe Eleventh International Conference on Learn- ing Representations, ICLR, 2023

    2023

  23. [23]

    Howard, Hartwig Adam, and Dmitry Kalenichenko

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew G. Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer- arithmetic-only inference. In2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 2704–2713. Computer Vision Foundation / IEEE Computer So...

    2018

  24. [24]

    Divyansh Jhunjhunwala, Advait Gadhikar, Gauri Joshi, and Yonina C. Eldar. Adaptive quantization of model updates for communication- efficient federated learning. InIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, pages 3110–3114. IEEE, 2021

    2021

  25. [25]

    Model pruning enables efficient federated learning on edge devices.CoRR, abs/1909.12326, 2019

    Yuang Jiang, Shiqiang Wang, Bong Jun Ko, Wei-Han Lee, and Leandros Tassiulas. Model pruning enables efficient federated learning on edge devices.CoRR, abs/1909.12326, 2019

    Pith/arXiv arXiv 1909

  26. [26]

    Quantizing deep convolutional networks for efficient inference: A whitepaper.CoRR, abs/1806.08342, 2018

    Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper.CoRR, abs/1806.08342, 2018

    Pith/arXiv arXiv 2018

  27. [27]

    Learning multiple layers of features from tiny images.Technical report, University of Toronto, 2009

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images.Technical report, University of Toronto, 2009

    2009

  28. [28]

    Tiny imagenet visual recognition challenge,

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge,

  29. [29]

    Fedmask: Joint computation and communication-efficient personalized federated learning via heterogeneous masking

    Ang Li, Jingwei Sun, Xiao Zeng, Mi Zhang, Hai Li, and Yiran Chen. Fedmask: Joint computation and communication-efficient personalized federated learning via heterogeneous masking. InSenSys ’21: The 19th ACM Conference on Embedded Networked Sensor Systems, pages 42–

  30. [30]

    H. Li, S. De, Z. Xu, C. Studer, H. Samet, and Goldstein T. Training quantized nets: A deeper understanding. InAdvances in Neural Infor- mation Processing Systems, 2017

    2017

  31. [31]

    Federated learning on non-iid data silos: An experimental study

    Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In38th IEEE International Conference on Data Engineering, ICDE, pages 965–978. IEEE, 2022

    2022

  32. [32]

    Masked random noise for communication-efficient federated learning

    Shiwei Li, Yingyi Cheng, Haozhao Wang, Xing Tang, Shijie Xu, Weihong Luo, Yuhua Li, Dugang Liu, Xiuqiang He, and Ruixuan Li. Masked random noise for communication-efficient federated learning. InACM International Conference on Multimedia (ACM MM), 2024

    2024

  33. [33]

    Adaptive low-precision training for embeddings in click-through rate prediction

    Shiwei Li, Huifeng Guo, Lu Hou, Wei Zhang, Xing Tang, Ruiming Tang, Rui Zhang, and Ruixuan Li. Adaptive low-precision training for embeddings in click-through rate prediction. InThirty-Seventh AAAI Conference on Artificial Intelligence (AAAI), Washington, DC, USA, 2023

    2023

  34. [34]

    The panaceas for improving low-rank decomposition in communication-efficient federated learning

    Shiwei Li, Xiandi Luo, Haozhao Wang, Xing Tang, Shijie Xu, Weihong Luo, Yuhua Li, Xiuqiang He, and Ruixuan Li. The panaceas for improving low-rank decomposition in communication-efficient federated learning. InInternational Conference on Machine Learning (ICML), 2025

    2025

  35. [35]

    Fedbat: Communication-efficient federated learning via learnable binarization

    Shiwei Li, Wenchao Xu, Haozhao Wang, Xing Tang, Yining Qi, Shijie Xu, Weihong Luo, Yuhua Li, Xiuqiang He, and Ruixuan Li. Fedbat: Communication-efficient federated learning via learnable binarization. InInternational Conference on Machine Learning (ICML), 2024

    2024

  36. [36]

    Towards efficient replay in federated incremental learning

    Yichen Li, Qunwei Li, Haozhao Wang, Ruixuan Li, Wenliang Zhong, and Guannan Zhang. Towards efficient replay in federated incremental learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12820–12829, 2024

    2024

  37. [37]

    Re- fed+: A better replay strategy for federated incremental learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Yichen Li, Haozhao Wang, Yining Qi, Wei Liu, and Ruixuan Li. Re- fed+: A better replay strategy for federated incremental learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  38. [38]

    Unleashing the power of continual learning on non-centralized devices: A survey.IEEE Communications Surveys & Tutorials, 2025

    Yichen Li, Haozhao Wang, Wenchao Xu, Tianzhe Xiao, Hong Liu, Minzhu Tu, Yuying Wang, Xin Yang, Rui Zhang, Shui Yu, et al. Unleashing the power of continual learning on non-centralized devices: A survey.IEEE Communications Surveys & Tutorials, 2025

    2025

  39. [39]

    FedSSI: Rehearsal-free continual federated learning with synergistic synaptic intelligence

    Yichen Li, Yuying Wang, Haozhao Wang, Yining Qi, Tianzhe Xiao, and Ruixuan Li. FedSSI: Rehearsal-free continual federated learning with synergistic synaptic intelligence. InForty-second International Conference on Machine Learning, 2025

    2025

  40. [40]

    Personalized federated domain-incremental learning based on adaptive knowledge matching

    Yichen Li, Wenchao Xu, Haozhao Wang, Yining Qi, Jingcai Guo, and Ruixuan Li. Personalized federated domain-incremental learning based on adaptive knowledge matching. InEuropean conference on computer vision, pages 127–144. Springer, 2024

    2024

  41. [41]

    Low precision local training is enough for federated learning

    Zhiwei Li, Yiqiu LI, Binbin Lin, Zhongming Jin, and Weizhong Zhang. Low precision local training is enough for federated learning. In Advances in Neural Information Processing Systems 38: Annual Con- ference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024

    2024

  42. [42]

    Communication-efficient learning of deep net- works from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Ag ¨uera y Arcas. Communication-efficient learning of deep net- works from decentralized data. InProceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS, volume 54, pages 1273–1282. PMLR, 2017

    2017

  43. [43]

    Up or down? adaptive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. InProceedings of the 37th International Conference on Machine Learning, ICML, pages 7197–7206. PMLR, 2020

    2020

  44. [44]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. InNIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011

    2011

  45. [45]

    Nguyen, Quoc Tran-Dinh, Dzung T

    Lam M. Nguyen, Quoc Tran-Dinh, Dzung T. Phan, Phuong Ha Nguyen, and Marten van Dijk. A unified convergence analysis for shuffling-type gradient methods.J. Mach. Learn. Res., 22:207:1–207:44, 2021

    2021

  46. [46]

    Fedaqt: Accurate quantized training with fed- erated learning

    Renkun Ni, Yonghui Xiao, Phoenix Meadowlark, Oleg Rybakov, Tom Goldstein, Ananda Theertha Suresh, Ignacio Lopez Moreno, Mingqing Chen, and Rajiv Mathews. Fedaqt: Accurate quantized training with fed- erated learning. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6100–

    2024

  47. [47]

    Cocofl: Communication- and computation-aware federated learning via partial NN freezing and quantization.Trans

    Kilian Pfeiffer, Martin Rapp, Ramin Khalili, and J ¨org Henkel. Cocofl: Communication- and computation-aware federated learning via partial NN freezing and quantization.Trans. Mach. Learn. Res., 2023

    2023

  48. [48]

    Xinchi Qiu, Javier Fern ´andez-Marqu´es, Pedro P. B. de Gusmao, Yan Gao, Titouan Parcollet, and Nicholas Donald Lane. Zerofl: Efficient on-device training for federated learning with local sparsity. InThe Tenth International Conference on Learning Representations, ICLR. OpenReview.net, 2022

    2022

  49. [49]

    Feddq: Communication-efficient federated learning with descending quantiza- tion

    Linping Qu, Shenghui Song, and Chi-Ying Tsui. Feddq: Communication-efficient federated learning with descending quantiza- tion. InIEEE Global Communications Conference, GLOBECOM, pages 281–286. IEEE, 2022

    2022

  50. [50]

    Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization

    Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jad- babaie, and Ramtin Pedarsani. Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization. InThe 23rd International Conference on Artificial Intelligence and Statistics, AISTATS, volume 108, pages 2021–2031. PMLR, 2020

    2021

  51. [51]

    Fetchsgd: Communication-efficient federated learning with sketching

    Daniel Rothchild, Ashwinee Panda, Enayat Ullah, Nikita Ivkin, Ion Stoica, Vladimir Braverman, Joseph Gonzalez, and Raman Arora. Fetchsgd: Communication-efficient federated learning with sketching. In Proceedings of the 37th International Conference on Machine Learning, ICML, volume 119, pages 8253–8265. PMLR, 2020

    2020

  52. [52]

    Vallapuram, Pengyuan Zhou, Young D

    Anish K. Vallapuram, Pengyuan Zhou, Young D. Kwon, Lik Hang Lee, Hengwei Xu, and Pan Hui. Hidenseek: Federated lottery ticket via server-side pruning and sign supermask.CoRR, abs/2206.04385, 2022. 12

    Pith/arXiv arXiv 2022

  53. [53]

    Error-compensated sparsification for communication-efficient decentral- ized training in edge environment.IEEE Trans

    Haozhao Wang, Song Guo, Zhihao Qu, Ruixuan Li, and Ziming Liu. Error-compensated sparsification for communication-efficient decentral- ized training in edge environment.IEEE Trans. Parallel Distributed Syst., 33(1):14–25, 2022

    2022

  54. [54]

    Dafkd: Domain-aware federated knowledge distillation

    Haozhao Wang, Yichen Li, Wenchao Xu, Ruixuan Li, Yufeng Zhan, and Zhigang Zeng. Dafkd: Domain-aware federated knowledge distillation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20412–20421, 2023

    2023

  55. [55]

    Fedcda: Federated learning with cross-rounds divergence- aware aggregation

    Haozhao Wang, Haoran Xu, Yichen Li, Yuan Xu, Ruixuan Li, and Tian- wei Zhang. Fedcda: Federated learning with cross-rounds divergence- aware aggregation. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024

    2024

  56. [56]

    Fednlr: Federated learning with neuron-wise learning rates

    Haozhao Wang, Peirong Zheng, Xingshuo Han, Wenchao Xu, Ruixuan Li, and Tianwei Zhang. Fednlr: Federated learning with neuron-wise learning rates. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024, pages 3069–3080. ACM, 2024

    2024

  57. [57]

    Decoupling general and personalized knowl- edge in federated learning via additive and low-rank decomposition

    Xinghao Wu, Xuefeng Liu, Jianwei Niu, Haolin Wang, Shaojie Tang, Guogang Zhu, and Hao Su. Decoupling general and personalized knowl- edge in federated learning via additive and low-rank decomposition. In Proceedings of the 32nd ACM International Conference on Multimedia, 2024

    2024

  58. [58]

    Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration

    Xinghao Wu, Xuefeng Liu, Jianwei Niu, Guogang Zhu, and Shaojie Tang. Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  59. [59]

    pfedgf: Enabling personalized federated learning via gradient fusion

    Xinghao Wu, Jianwei Niu, Xuefeng Liu, Tao Ren, Zhangmin Huang, and Zhetao Li. pfedgf: Enabling personalized federated learning via gradient fusion. In2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 639–649, 2022

    2022

  60. [60]

    The diversity bonus: Learning from dissimilar clients in personalized federated learning.IEEE Transactions on Neural Networks and Learning Systems, 2025

    Xinghao Wu, Jianwei Niu, Xuefeng Liu, Guogang Zhu, Shaojie Tang, Wanyu Lin, and Jiannong Cao. The diversity bonus: Learning from dissimilar clients in personalized federated learning.IEEE Transactions on Neural Networks and Learning Systems, 2025

    2025

  61. [61]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.CoRR, abs/1708.07747, 2017

    Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.CoRR, abs/1708.07747, 2017

    Pith/arXiv arXiv 2017

  62. [62]

    Greenewald, Trong Nghia Hoang, and Yasaman Khazaeni

    Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan H. Greenewald, Trong Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. InProceedings of the 36th International Conference on Machine Learning, ICML, volume 97, pages 7252–7261. PMLR, 2019. Shiwei Lireceived the bachelor’s degree from Bei- jing Universi...

    2019

  63. [2015]

    Stanford CS231n Course Project

  64. [2019]

    His research interests include decision making, deep learning, and optimization algorithms

    He worked at Ant Group, China, on machine learning algorithms for recommender systems from 2019 to 2025. His research interests include decision making, deep learning, and optimization algorithms. Dr. Li received the Syracuse University Graduate Fellowship Award in 2014 and the All University Doctoral Prize in 2018 by Syracuse University for superior achi...

    2019