Pith. sign in

REVIEW 2 major objections 4 minor 43 references

Client drift in federated learning lives mostly in low-frequency gradients; cutting those frequencies realigns updates and lifts accuracy under non-IID data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 21:04 UTC pith:E2U3DUTL

load-bearing objection Strong empirical FL regularizer that works; the “intrinsic spectral bias” framing is overstated because it depends on a particular 1-D flattening order. the 2 major comments →

arxiv 2607.04189 v1 pith:E2U3DUTL submitted 2026-07-05 cs.LG

SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity

Liyang Yuan , Yibo Yang , Dandan Guo , Peter Richtarik , Zhouchen Lin This is my paper
classification cs.LG
keywords Federated LearningNon-IID DataClient DriftSpectral AnalysisGradient FilteringFrequency DomainSpecGradFilter
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Federated learning fails when each client’s data distribution pulls the model in a different direction. This paper shows that those conflicting pulls are not spread evenly across the model’s gradient: they concentrate in the smooth, low-frequency parts of the gradient spectrum, while the high-frequency parts stay relatively consistent across clients. The authors therefore introduce SpecGradFilter, a simple operator that removes the low-frequency band of each client’s gradient before the local update step. The same principle can be realized either by an FFT high-pass mask or by cheap spatial detrending (Gaussian or average pooling). On CIFAR-10/100 and Tiny-ImageNet with severe Dirichlet heterogeneity the method substantially raises final accuracy and shortens the number of communication rounds needed to reach a target, without adding communication cost. The result reframes client drift as a spectral phenomenon that can be filtered rather than merely regularized in the spatial domain.

Core claim

Inter-client gradient divergence is predominantly concentrated in low-frequency components that encode client-specific distributional shifts, while high-frequency components that carry fine-grained shared features remain relatively consistent. Suppressing the low-frequency band during local SGD therefore reduces client drift and yields higher accuracy and faster convergence under non-IID data.

What carries the argument

SpecGradFilter: a spectral modulation operator Φ that high-pass-filters each layer’s flattened gradient (via rFFT mask with ratio r, or via spatial detrending) before every local update, thereby projecting out the drift-dominated low-frequency subspace.

Load-bearing premise

A fixed low-frequency cutoff applied after one particular one-dimensional flattening order of the weight tensors cleanly separates client-specific drift from useful signal for the tested networks and data partitions.

What would settle it

Re-run the main CIFAR-10/100 Dirichlet-α=0.1 experiments with the same r=0.05 filter but with random or channel-innermost tensor flattening; if accuracy collapses back to FedAvg levels, the claimed spectral separation fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper argues that client drift under non-IID federated data is not unstructured but exhibits a Spectral Bias of Drift: after 1-D flattening of layer-wise gradients, inter-client divergence (measured by mean and std of pairwise spectral distances Dist(m)/Std(m) in Eq. 4) concentrates in low-frequency bands that encode client-specific shifts, while high-frequency bands remain more consistent (Fig. 1, §III-B). Motivated by this and a pilot high-pass study (Fig. 2), it introduces SpecGradFilter, a plug-in operator Φ that suppresses the lowest-r fraction of Fourier coefficients (or spatial approximations via LAP-D/Gaussian detrend) inside each local SGD step (Eqs. 6–8, Alg. 1). Extensive experiments on CIFAR-10/100, Tiny-ImageNet, FEMNIST, BloodMNIST and Yahoo Answers, across ResNets, DenseNet, WideResNet and ViT, show large accuracy gains and faster convergence versus FedAvg, FedProx, SCAFFOLD, FedDyn and others, especially at α=0.1, with negligible communication cost and a basic non-convex convergence bound (Thm. IV.1).

Significance. If the spectral interpretation is robust, the work supplies a simple, architecture-agnostic regularizer that can be dropped into existing FL pipelines and that yields double-digit gains under severe heterogeneity (Table I: +19–21 points on CIFAR-10 α=0.1). The spatial detrend variants (Table III) and the consistent improvements when combined with multiple baselines and backbones (Table II, Fig. 6) are practically valuable, as is the demonstration that the same filter is mildly harmful in centralized SGD yet increasingly helpful as decentralization grows (Table V). Thorough ablations on filter type, ratio, client count, participation and feature-shift corruptions further strengthen the empirical case. The main conceptual contribution—an explicit frequency-domain view of client drift—would open a useful new research direction provided the claimed spectral concentration can be shown to be more than an artifact of a particular tensor layout.

major comments (2)
  1. [§III-B, Table IV] §III-B (Eqs. 3–4, Fig. 1) and §V-C (Table IV): The central claim of an intrinsic Spectral Bias of Drift is demonstrated exclusively under the default contiguous flattening order [Out,In,H,W] (0123). Table IV shows that only three channel-outermost orders retain large gains; channel-innermost orders, random permutation after flattening, and even a structure-preserving 2-D FFT collapse accuracy to FedAvg levels or below. No Dist(m)/Std(m) curves are supplied for any alternative layout, so it remains unclear whether low-frequency concentration is a genuine property of federated gradients or an artifact of how the authors chose to linearize the tensor. Because the abstract, introduction and methodology present SpecGradFilter as a general frequency-domain principle rather than a layout-specific regularizer, this gap is load-bearing: either report the spectral-distance statistics under the fai
  2. [§IV-B, Thm. IV.1, Table V] §IV-B–C and Thm. IV.1: The spectral-preconditioning view and the convergence bound treat P_spec = F^{-1} D F as a heterogeneity-aware projector whose residual bias is controlled by δ_high. The bound is formally correct under the stated assumptions, yet it does not explain why the same high-pass operator is beneficial only after multi-step local SGD on non-IID data and mildly harmful under centralized SGD (Table V). A short analysis or additional measurement linking the size of the low-frequency client-wise discrepancy to the number of local steps would make the theory support the empirical regime distinction rather than merely bound the filtered algorithm.
minor comments (4)
  1. [Abstract, §I] Abstract and §I: repeated phrasing issues (“significantly performs better performance”, “significantly severely hampers”) should be corrected for readability.
  2. [Fig. 1] Fig. 1 caption and §III-B: clarify that the four gradient snapshots are taken every 30 rounds; the exact communication-round indices would aid reproducibility.
  3. [§V-A] §V-A: the hyper-parameter search protocol is thorough, yet the final chosen values for all baselines are relegated to the supplement; a short summary table in the main text would help readers.
  4. [Table III] Table III and §V-C: the spatial detrend kernels (LAP-D, GD) are only sketched; giving the exact kernel sizes or σ values used would improve reproducibility of the O(d) variants.

Circularity Check

0 steps flagged

No circularity: spectral-bias observation is measured on unfiltered FedAvg gradients, method is then applied, and accuracy gains are external empirical outcomes.

full rationale

The paper's chain is observational then interventional, not definitional. In §III-B the authors compute Dist(m) and Std(m) (Eq. 4) by rFFT of ordinary client gradients under Dirichlet partitions, observe concentration in low-frequency bands (Fig. 1), and only afterwards introduce the high-pass operator Φ (Eq. 6–8). The pilot study (§III-C, Fig. 2) simply verifies that removing the high-Dist bands reduces layer-wise divergence—an immediate algebraic consequence of the measurement, not a circular prediction of accuracy. Final claims rest on held-out test accuracy (Tables I–X, Figs. 4–8) against external baselines; the filter ratio r=0.05 is a fixed hyper-parameter, not a fit that forces the reported numbers. Flattening-order sensitivity (Table IV) and the mild degradation under centralized SGD (Table V) are validity caveats about generality, not reductions of the claimed gains to the method’s own inputs. No self-citation supplies a uniqueness theorem or ansatz that the present results merely rename. The derivation is therefore self-contained against the paper’s own equations and external benchmarks.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 2 invented entities

The central claim rests on standard non-convex FL assumptions (smoothness, bounded variance, partial participation), the known spectral bias of neural nets, and two paper-specific modeling choices: that low-frequency energy after a particular flattening order is the dominant carrier of client drift, and that a fixed cutoff ratio r≈0.05 is sufficient. No new physical entities are postulated; the ‘Spectral Bias of Drift’ is an observed statistical pattern rather than an invented force or particle.

free parameters (3)
  • spectral filter ratio r = 0.05
    Default r=0.05 chosen by the authors; ablation shows robustness in [0.01,0.1] but the exact value is a free design choice that controls the strength of the claimed effect.
  • local learning rate and method-specific hyperparameters
    All baselines and SpecGradFilter were grid-searched; the reported gains therefore depend on the selected operating points listed in the supplement.
  • flattening order of convolutional kernels = 0123 (natural layout)
    Performance collapses under random or channel-innermost orders (Table IV); the natural [Out,In,H,W] layout is therefore an implicit free structural parameter of the method.
axioms (3)
  • domain assumption Local objectives are L-smooth and stochastic gradients have bounded variance; the global objective satisfies standard non-convex FL assumptions.
    Invoked for the O(1/√T) rate in Theorem IV.1; full statement deferred to supplementary Section II.
  • domain assumption Neural networks exhibit spectral bias, learning low-frequency functions before high-frequency ones.
    Cited from Rahaman et al. and Xu et al.; used to motivate why client-specific patterns appear first in low frequencies.
  • ad hoc to paper After 1-D flattening of layer gradients, client-specific distributional shifts concentrate in the lowest-frequency Fourier coefficients while task-relevant features remain in higher bands.
    Core empirical premise of §III-B and Fig. 1; not derived from first principles and shown to be sensitive to flattening order.
invented entities (2)
  • Spectral Bias of Drift independent evidence
    purpose: Name the observed concentration of inter-client gradient divergence in low-frequency bands under non-IID data.
    Introduced in abstract and §III-B as a novel structural property; independent evidence is the frequency-band distance plots themselves, which are falsifiable by re-running the same measurement on other models/datasets.
  • SpecGradFilter operator Φ (spectral or spatial high-pass) no independent evidence
    purpose: Concrete mechanism that suppresses the identified low-frequency drift components during local SGD steps.
    Defined by Eqs. 6–8 and the spatial detrending variants; its utility is demonstrated by the accuracy tables, but the operator itself is a design choice rather than a discovered natural object.

pith-pipeline@v1.1.0-grok45 · 25609 in / 3114 out tokens · 35708 ms · 2026-07-11T21:04:09.469084+00:00 · methodology

0 comments
read the original abstract

Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence. While existing approaches attempt to mitigate this drift through spatial-domain gradient correction or regularization, they overlook the intrinsic spectral structure of optimization signals. In this work, we revisit client drift from a novel frequency-domain perspective and uncover a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grained features remain relatively consistent. Motivated by this, we propose SpecGradFilter, a unified Spectral Gradient Filtering Framework that tames heterogeneity by suppressing discordant low-frequency signals. Crucially, we demonstrate that SpecGradFilter is a generalizable principle, effective not only via precise FFT-based truncation but also through spatial approximations like Gaussian detrending. Extensive experiments on benchmarks such as CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly performs better performance in highly Non-IID settings with negligible communication overhead, establishing a new paradigm for robust federated optimization.

Figures

Figures reproduced from arXiv: 2607.04189 by Dandan Guo, Liyang Yuan, Peter Richtarik, Yibo Yang, Zhouchen Lin.

Figure 1
Figure 1. Figure 1: Client gradient divergence across frequency bands under different heterogeneous distributions (Dirichlet hyperparameter α). Top: Mean inter-client distance showing drift magnitude. Bottom: Standard deviation showing drift instability. Results are obtained from ResNet-20 on CIFAR-10, with gradients collected every 30 rounds, for a total of four times. functions from low-frequency components to high-frequenc… view at source ↗
Figure 2
Figure 2. Figure 2: Layer-wise client gradient divergence before and after low-frequency [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed SpecGradFilter framework integrated with FedAvg. The figure illustrates the overall training pipeline including local spectral [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison between our method and FedAvg on CIFAR-10 ( [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Test accuracy comparisons of different federated learning methods across datasets and Dirichlet [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Test accuracy over communication rounds under Dirichlet ( [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Layer-wise CKA similarity among different methods on CIFAR-10 [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Accuracy improvement of our method over FedAvg across different [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 4 linked inside Pith

  1. [1]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017

    2017

  2. [2]

    Giant: Globally improved approximate newton method for dis- tributed optimization,

    S. Wang, F. Roosta, P. Xu, and M. W. Mahoney, “Giant: Globally improved approximate newton method for dis- tributed optimization,”Advances in neural information processing systems, vol. 31, 2018

    2018

  3. [3]

    Com- munication efficient distributed machine learning with the parameter server,

    M. Li, D. G. Andersen, A. Smola, and K. Yu, “Com- munication efficient distributed machine learning with the parameter server,”Advances in neural information processing systems, vol. 27, 2014

    2014

  4. [4]

    Firecaffe: Near-linear acceleration of deep neural network training on compute clusters,

    F. N. Iandola, M. W. Moskewicz, K. Ashraf, and K. Keutzer, “Firecaffe: Near-linear acceleration of deep neural network training on compute clusters,”2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2592–2600, 2015. [Online]. Available: https://api.semanticscholar.org/CorpusID:8480861

    2016

  5. [5]

    Co- boosting++: Coupled optimization of data and ensemble for one-shot federated learning,

    X. Yang, R. Dai, Y . Zhang, A. Li, T. Liu, and B. Han, “Co- boosting++: Coupled optimization of data and ensemble for one-shot federated learning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026

    2026

  6. [6]

    Tighter theory for local sgd on identical and heterogeneous data,

    A. Khaled, K. Mishchenko, and P. Richt ´arik, “Tighter theory for local sgd on identical and heterogeneous data,” inInternational Conference on Artificial Intelligence and Statistics, 2019. [Online]. Available: https://api. semanticscholar.org/CorpusID:210839089

    2019

  7. [7]

    Is local sgd better than minibatch sgd?

    B. Woodworth, K. K. Patel, S. U. Stich, Z. Dai, B. Bullins, H. B. McMahan, O. Shamir, and N. Srebro, “Is local sgd better than minibatch sgd?” inProceedings of the 37th International Conference on Machine Learning, ser. ICML’20. JMLR.org, 2020

    2020

  8. [8]

    Federated learning: Challenges, methods, and future directions,

    T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020

    2020

  9. [9]

    Fedskip: Combatting statistical heterogeneity with fed- erated skip aggregation,

    Z. Fan, Y . Wang, J. Yao, L. Lyu, Y . Zhang, and Q. Tian, “Fedskip: Combatting statistical heterogeneity with fed- erated skip aggregation,” in2022 IEEE International Conference on Data Mining (ICDM), 2022, pp. 131–140

    2022

  10. [10]

    Federated learning based on dynamic regularization,

    D. A. E. Acar, Y . Zhao, R. Matas, M. Mattina, P. What- mough, and V . Saligrama, “Federated learning based on dynamic regularization,” inInternational Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=B7v4QMR6Z9w

    2021

  11. [11]

    Model-contrastive federated learning,

    Q. Li, B. He, and D. Song, “Model-contrastive federated learning,” in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 10 708– 10 717

    2021

  12. [12]

    Towards the flatter landscape and better generalization in federated learning under client-level dif- ferential privacy,

    Y . Shi, K. Wei, L. Shen, Y . Liu, X. Wang, B. Yuan, and D. Tao, “Towards the flatter landscape and better generalization in federated learning under client-level dif- ferential privacy,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  13. [13]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,”Proceedings of Machine learning and systems, vol. 2, pp. 429–450, 2020

    2020

  14. [14]

    Scaffold: Stochastic controlled aver- aging for federated learning,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled aver- aging for federated learning,” inInternational conference on machine learning. PMLR, 2020, pp. 5132–5143

    2020

  15. [15]

    Feddisco: federated learning with discrepancy-aware col- laboration,

    R. Ye, M. Xu, J. Wang, C. Xu, S. Chen, and Y . Wang, “Feddisco: federated learning with discrepancy-aware col- laboration,” inProceedings of the 40th International Con- ference on Machine Learning, ser. ICML’23. JMLR.org, 2023

    2023

  16. [16]

    Fedawa: Adaptive optimization of aggregation weights in federated learning using client vectors,

    C. Shi, H. Zhao, B. Zhang, M. Zhou, D. Guo, and Y . Chang, “Fedawa: Adaptive optimization of aggregation weights in federated learning using client vectors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  17. [17]

    On the spectral bias of neural networks,

    N. Rahaman, A. Baratin, D. Arpit, F. Dr ¨axler, M. Lin, F. A. Hamprecht, Y . Bengio, and A. C. Courville, “On the spectral bias of neural networks,” inInternational Conference on Machine Learning,

  18. [18]

    Available: https://api.semanticscholar.org/ CorpusID:53012119

    [Online]. Available: https://api.semanticscholar.org/ CorpusID:53012119

  19. [19]

    Frequency principle: Fourier analysis sheds light on deep neural networks,

    Z.-Q. J. Xu, Y . Zhang, T. Luo, Y . Xiao, and Z. Ma, “Frequency principle: Fourier analysis sheds light on deep neural networks,”ArXiv, vol. abs/1901.06523,

    Pith/arXiv arXiv 1901

  20. [20]

    Available: https://api.semanticscholar.org/ CorpusID:58981616

    [Online]. Available: https://api.semanticscholar.org/ CorpusID:58981616

  21. [21]

    Fedpe: Adaptive model pruning-expanding for federated learning on mobile devices,

    L. Yi, X. Shi, N. Wang, J. Zhang, G. Wang, and X. Liu, “Fedpe: Adaptive model pruning-expanding for federated learning on mobile devices,”IEEE Transactions on Mobile Computing, vol. 23, no. 11, pp. 10 475–10 493, 2024

    2024

  22. [22]

    Tackling resource- constrained and data-heterogeneity in federated learning with double-weight sparse pack,

    Q. Yang, L. Chen, M. Xue, and S. Li, “Tackling resource- constrained and data-heterogeneity in federated learning with double-weight sparse pack,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 32, 2026, pp. 27 592–27 600. 14

    2026

  23. [23]

    Feddc: Federated learning with non-iid data via local drift decoupling and correction,

    L. Gao, H. Fu, L. Li, Y . Chen, M. Xu, and C.-Z. Xu, “Feddc: Federated learning with non-iid data via local drift decoupling and correction,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, June 2022, pp. 10 112–10 121

    2022

  24. [24]

    Certified robustness of joint embedding,

    J. Zhang, Z. Li, B. Li, J. Xu, S. Wu, S. Ding, and C. Wu, “Certified robustness of joint embedding,” inProceed- ings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesv ´ari, G. Niu, and S. Sabato, Eds., vol. 162. PMLR, 2022, pp. 26 311– 26 329

    2022

  25. [25]

    Revisiting weighted aggregation in federated learning with neural networks,

    Z. Li, T. Lin, X. Shang, and C. Wu, “Revisiting weighted aggregation in federated learning with neural networks,” inProceedings of the 40th International Conference on Machine Learning, ser. ICML’23. JMLR.org, 2023

    2023

  26. [26]

    Internal cross-layer gradients for extending homogeneity to heterogeneity in federated learning,

    Y .-H. Chan, R. Zhou, R. Zhao, Z. Jiang, and E. Ngai, “Internal cross-layer gradients for extending homogeneity to heterogeneity in federated learning,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 1983–2011

    2024

  27. [27]

    Fft-based gradient sparsification for the distributed training of deep neural networks,

    L. Wang, W. Wu, J. Zhang, H. Liu, G. Bosilca, M. Herlihy, and R. Fonseca, “Fft-based gradient sparsification for the distributed training of deep neural networks,” in Proceedings of the 29th International Symposium on High- Performance Parallel and Distributed Computing, 2020, pp. 113–124

    2020

  28. [28]

    High-energy concentration for federated learning in frequency domain,

    H. Shi, W. Xie, H. Ye, D. Li, J. Ma, Y . Li, and L. Fang, “High-energy concentration for federated learning in frequency domain,” 2025. [Online]. Available: https://arxiv.org/abs/2509.12630

    arXiv 2025

  29. [29]

    Fedft: Improving communication performance for federated learning with frequency space transformation,

    C. Palihawadana, N. Wiratunga, A. Wijekoon, and H. Kalutarage, “Fedft: Improving communication performance for federated learning with frequency space transformation,” 2024. [Online]. Available: https: //arxiv.org/abs/2409.05242

    Pith/arXiv arXiv 2024

  30. [30]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” University of Toronto, Toronto, Ontario, Tech. Rep. 0, 2009. [Online]. Available: https://www.cs.toronto.edu/ ∼kriz/ learning-features-2009-TR.pdf

    2009

  31. [31]

    Tiny imagenet visual recognition chal- lenge,

    S. U. CS231N, “Tiny imagenet visual recognition chal- lenge,” http://cs231n.stanford.edu/, 2015

    2015

  32. [32]

    Leaf: A benchmark for federated settings,

    S. Caldas, S. M. K. Duddu, P. Wu, T. Li, J. Kone ˇcn´y, H. B. McMahan, V . Smith, and A. Talwalkar, “Leaf: A benchmark for federated settings,” 2019. [Online]. Available: https://arxiv.org/abs/1812.01097

    Pith/arXiv arXiv 2019

  33. [33]

    Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification,

    J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfis- ter, and B. Ni, “Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification,” Scientific data, vol. 10, no. 1, p. 41, 2023

    2023

  34. [34]

    Fedcm: Federated learning with client-level momentum,

    J. Xu, S. Wang, L. Wang, and A. C.-C. Yao, “Fedcm: Federated learning with client-level momentum,” ArXiv, vol. abs/2106.10874, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:235490679

    Pith/arXiv arXiv 2021

  35. [35]

    Improving gen- eralization in federated learning by seeking flat minima,

    D. Caldarola, B. Caputo, and M. Ciccone, “Improving gen- eralization in federated learning by seeking flat minima,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 654–672

    2022

  36. [36]

    Generalized federated learning via sharpness aware minimization,

    Z. Qu, X. Li, R. Duan, Y . Liu, B. Tang, and Z. Lu, “Generalized federated learning via sharpness aware minimization,” inInternational conference on machine learning. PMLR, 2022, pp. 18 250–18 280

    2022

  37. [37]

    Fedlws: Federated learning with adaptive layer-wise weight shrink- ing,

    C. Shi, J. Li, H. Zhao, D. Guo, and Y . Chang, “Fedlws: Federated learning with adaptive layer-wise weight shrink- ing,” inInternational Conference on Learning Represen- tations (ICLR), 2025

    2025

  38. [38]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778

    2016

  39. [39]

    Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape,

    Y . Sun, L. Shen, S. Chen, L. Ding, and D. Tao, “Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 32 991–33 013

    2023

  40. [40]

    Locally estimated global perturbations are better than local perturbations for federated sharpness- aware minimization,

    Z. Fan, S. Hu, J. Yao, G. Niu, Y . Zhang, M. Sugiyama, and Y . Wang, “Locally estimated global perturbations are better than local perturbations for federated sharpness- aware minimization,” inInternational Conference on Machine Learning, 2024

    2024

  41. [41]

    Group normalization,

    Y . Wu and K. He, “Group normalization,” inComputer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIII. Berlin, Heidelberg: Springer-Verlag, 2018, p. 3–19. [Online]. Available: https://doi.org/10.1007/ 978-3-030-01261-8 1 Liyang Yuanis currently working toward the M.S. degree with the School of Arti...

    2018

  42. [42]

    His research interests lie in machine learning and computer vision

    He conducted research at JD Explore Academy and the University of Oxford. His research interests lie in machine learning and computer vision. He has published more than 40 publications in top- tier journals and conferences with more than 4000 citations. He was awarded the CAAI Wuwenjun AI Science and Technology Award, First Prize of Natural Science (5-th ...

    2021

  43. [43]

    15 Dandan Guois currently a Professor with the School of Artificial Intelligence, Jilin University, Changchun, China

    He has served as Area Chair for NeurIPS, ICML, and ICLR. 15 Dandan Guois currently a Professor with the School of Artificial Intelligence, Jilin University, Changchun, China. Since 2025, she has also been a Visiting Faculty at King Abdullah University of Science and Technology (KAUST), Saudi Arabia. She received the Ph.D. degree in electronic engineering ...

    2025