Pith. sign in

REVIEW 4 major objections 4 minor 65 references

Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FedGloSS claims that global sharpness, not local flatness, should be minimized in heterogeneous federated learning, and shows that server-side SAM with a one-round-delayed gradient achieves flatter minima at FedAvg-level communication cost.

desk verdict Solid FL paper: server-side SAM with a previous-gradient trick is genuinely new and well tested, but the main-text algorithm omits the ADMM terms that the appendix and experiments actually use. read the letter →

arxiv 2412.03752 v2 pith:WE62MRDT submitted 2024-12-04 cs.AI

Debora Caldarola , Pietro Cagnasso , Barbara Caputo , Marco Ciccone This is my paper · ORCID
classification cs.AI
keywords Federatedlearningsharpness-awareminimizationglobalvslocalflatnessserver-sideoptimizationADMMcommunicationefficiencydataheterogeneitylosslandscape
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that federated learning methods which seek flat minima on each client's local loss (FedSAM) miss the true goal: flatness in the server's global loss landscape. To fix that, it introduces FedGloSS, which applies Sharpness-Aware Minimization on the server using the previous round's pseudo-gradient as the perturbation direction, so no extra communication round is needed. An ADMM consistency term keeps client and server updates aligned so that this one-step-late approximation stays close to the true SAM perturbation. Across CIFAR and the large-scale Landmarks-User-160K benchmarks, the paper reports consistently flatter minima and accuracy gains over the best competitor of roughly 2.5 points on CIFAR and 3.4 points on Landmarks, at half the communication cost.

What carries the argument

The central mechanism is server-side Sharpness-Aware Minimization (SAM) with a delayed pseudo-gradient: the server perturbs the global model by $\rho\, \tilde{\Delta}^{t-1}/\|\tilde{\Delta}^{t-1}\|$ instead of computing a live ascent gradient, then descends along the resulting sharpness-aware direction. ADMM with per-client dual variables $\sigma_k$ is layered on top to align local and global updates, keeping the approximation error $\delta^t_\epsilon$ small across rounds. The paper argues that SAM's tendency toward small weight updates also prevents the parameter explosion that historically breaks ADMM-based federated methods.

What would settle it

Run FedGloSS on CIFAR-100 with $\alpha=0$ using stateless clients (dual variables reset to zero each round) and compare final accuracy and the maximum Hessian eigenvalue against the stateful version; the paper's mechanism predicts clear degradation in both. A second check is to measure $\delta^t_\epsilon$ from Eq. (6) under randomly changing client subsets: the claim implies the gap stays bounded with ADMM and grows without it.

Watch

Extended reading notes

Core claim

FedGloSS solves the global sharpness-aware objective $\min_{w} F(w) = \frac{1}{C}\sum_k \max_{\|\epsilon\|\le\rho} f_k(w+\epsilon)$ on the server, replacing the expensive SAM ascent direction with the pseudo-gradient from the previous round, $\tilde{\epsilon}^{t} = \rho\, \tilde{\Delta}^{t-1}/\|\tilde{\Delta}^{t-1}\|$. The authors claim this yields flatter global minima and better accuracy than both local-sharpness methods (FedSAM, FedSpeed, FedGamma) and the global-consistency method FedSMOO, while keeping FedAvg's per-round communication cost. The loss-landscape and Hessian-eigenvalue measurements in the paper support the claim that FedGloSS converges to the flattest global minima among all compared methods.

Load-bearing premise

The method depends on stateful clients that keep and return an ADMM dual variable across rounds; the paper itself calls this 'a key limitation of our approach.' If clients are stateless, drop in and out, or the dual variables drift, the alignment that keeps the previous pseudo-gradient close to the true SAM perturbation is lost, and the central approximation has no guarantee.

Editorial extensions

If this is right

  • Global flatness becomes optimizable at the same communication cost as FedAvg, removing the main practical barrier to global-sharpness methods.
  • Any client optimizer (SGD or SAM) can be used, so clients can avoid SAM's doubled local compute when needed.
  • The method reaches FedAvg-level accuracy about three to five times faster in bits exchanged, and matches the exact two-step server SAM (Naive FedGloSS) in final accuracy.
  • The combination of server-side SAM and ADMM avoids the parameter explosion observed in FedDyn and FedSMOO, keeping model norms bounded under extreme heterogeneity.
  • On the large-scale Landmarks-User-160K benchmark, FedGloSS is among the few methods that beat FedAvg, with the lowest bit cost of the top performers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the perturbation relies on the previous round's direction, the benefit likely shrinks when the client population changes rapidly or data is non-stationary; a direct measure of $\delta^t_\epsilon$ under such conditions would tell how much of the ADMM alignment is actually load-bearing.
  • The Hessian-eigenvalue evidence is correlational; a stronger test would compare FedGloSS and FedSMOO solutions of equal sharpness to see whether the accuracy gap persists independently of flatness.
  • The main-text update rule (Eq. 4) omits the ADMM term that appears in Algorithm 1, leaving the exact implemented update ambiguous; a clean reproduction would need to state which of the two is the deployed rule.
  • A stateless variant that moves the dual variable entirely to the server would be the natural test of whether the paper's gains survive the least restrictive cross-device setting.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes FedGloSS, a federated learning method that applies Sharpness-Aware Minimization on the server side, approximating the global sharpness perturbation with the pseudo-gradient from the previous round to avoid extra client-server communication. The method is combined with ADMM to align local and global updates, and the authors report extensive experiments on CIFAR-10, CIFAR-100, ResNet18, and Landmarks-User-160K, measuring accuracy, Hessian-based flatness, and communication cost. The central empirical claim is that FedGloSS reaches flatter global minima and higher accuracy than prior FL sharpness methods, including FedSMOO, while maintaining FedAvg-level communication cost.

Significance. If the method as described in Algorithm 1 is the intended contribution, the paper addresses a real gap: existing sharpness-aware FL methods either optimize local flatness only, which can be inconsistent with the global landscape, or they pay doubled communication cost as in FedSMOO. The experimental study is broad and includes multiple architectures, heterogeneity levels, a large-scale benchmark, standard deviations over seeds, Hessian eigenvalue flatness measurements, and a communication-cost analysis. The comparison with a 'NaiveFedGloSS' two-exchange baseline in Section 6.4.1 is a particularly useful control. The main weakness is a load-bearing mismatch between the main-text update rule and the implemented algorithm: the ADMM terms in Algorithm 1 are not present in Eq. (4), and the paper itself identifies the stateful-client ADMM machinery as a key limitation. Because the consistency mechanism that keeps the approximation error of Eq. (6) small is precisely the ADMM alignment, the paper as written does not establish that the communication-efficient server-side SAM idea works without that machinery.

major comments (4)
  1. [Section 5.2 and Algorithm 1 (Appendix A)] The paper should also clarify what happens when Algorithm 1 is used with client-side SGD: as printed, lines 12-15 always compute a sharpness-aware gradient, so the pseudocode does not actually implement the 'any CLIENT OPT' claim without a conditional branch that is not shown.
  2. [Section 5.2.1 and Table 2] This issue is load-bearing because the paper's communication-efficiency argument depends on FedGloSS being a drop-in server-side change, whereas Algorithm 1 changes the clients as well.
  3. [Eq. (6) and Figure 5] Without such an experiment or a revised narrative, the central claim 'global server-side SAM at FedAvg cost' is supported only for a method that is not defined in Section 5.2.
  4. [Section 5.2.1 and Section 6.4.2] The conclusion in Section 7 and the abstract describe the method without this caveat, so the stated scope in the abstract is stronger than what the stateful-client design can support.
minor comments (4)
  1. [Table 1] In Table 1, the class 'airplane' appears twice in the CIFAR-10 block; one of these rows should presumably be a different class, and the local client identity labels are inconsistent with the accompanying text.
  2. [Algorithm 1] Algorithm 1's text says it uses 'SGD or SAM as local optimizers, differently highlighted,' but the pseudocode as printed has no visible distinction between the two cases; lines 12-14 always compute a SAM perturbation and line 14 overwrites the gradient with the sharpness-aware gradient. The authors should add explicit conditionals, remove the claim, or clarify in a caption how the SGD variant is obtained.
  3. [Section 5.2.1, Eq. (7)] The ADMM augmented Lagrangian in Eq. (7) uses the notation σ_k^T (w^t - w_k^t) but the text below writes the subproblem with a negative sign; the sign convention is confusing and should be made consistent.
  4. [Notation] The name 'FedGloSS' is written in different typographical styles across the paper (FEDGLOSS in the abstract, FedGloSS in section headings, and FedGloss in some tables); a consistent style would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FedGloSS is evaluated against external benchmarks and its central server-side SAM approximation is derived from previous pseudo-gradients, not from the quantities it claims to predict.

full rationale

The central derivation chain is self-contained: FedGloSS defines its server-side perturbation as rho times the normalized previous pseudo-gradient (Sec. 5.2, steps 1-4), defines the approximation error delta_t^epsilon in Eq. (6), and then empirically evaluates accuracy, Hessian sharpness, and communication cost against external baselines (FedAvg, FedSAM, FedSMOO, etc.). The accuracy numbers in Tables 3-5 are not constructed from the method's own assumptions; they are benchmark results. The only self-citations, Refs. [4] and [5], are prior FedSAM and window-averaging works by overlapping authors, but they are used as baselines and background, not as the justification for the FedGloSS update or for the claim that server-side SAM yields flatter minima. The ADMM argument cites FedDyn [1] for convergence under heterogeneity, and Ref. [1] is not by the present authors; moreover the ADMM convergence claim is not what makes the accuracy results true. The measured decrease of delta_t^epsilon under ADMM (Fig. 5) is a self-consistency check, not a fitted target, and the method's final claims rest on external accuracy and flatness measurements. The mismatch between the main-text update (Eq. 4, which omits the ADMM dual term) and Algorithm 1 (which includes it) is a real reproducibility/correctness concern, but it is an ambiguity about which variant was actually evaluated, not a case where a prediction reduces by definition to its inputs. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' own prior work, and no known empirical pattern is merely relabeled. Under the hard rules, nothing here qualifies as circular.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The algorithm relies on two tuned radii, an ADMM penalty, and a scheduling horizon, all selected by grid search. The main ad hoc axiom is the previous-pseudo-gradient approximation of the SAM perturbation, which is justified only empirically. No new entities are postulated.

free parameters (4)
  • server-side SAM radius rho_s = grid-searched per dataset/model; e.g., 0.01-0.5 in Table 15
    Controls the perturbation magnitude for global sharpness. Tuned per setting.
  • local SAM radius rho = grid-searched; e.g., 0.01-0.2
    Used when SAM is the local optimizer; influences local flatness.
  • ADMM penalty beta = grid-searched; e.g., 1-100
    Controls the strength of the consistency term in Eq. (7).
  • rho scheduling length Ts = grid-searched; e.g., 0, 1000-15000
    Warm-up rounds for the local rho schedule in App. C.2.
assumptions (6)
  • standard math The SAM inner maximization is solved by a single normalized gradient ascent step (Eq. 5).
    Standard first-order approximation from Foret et al. [12]; used without error analysis.
  • domain assumption The server-side pseudo-gradient Delta-tilde^t is an estimate of the global gradient.
    Follows FedOpt [47]; assumed unbiased or usable despite partial client participation.
  • domain assumption ADMM converges under arbitrary heterogeneity and its dual variables remain bounded when combined with SAM.
    Cited from FedDyn [1] and argued empirically via Fig. 8; no theorem is proven in this paper.
  • domain assumption The maximum Hessian eigenvalue lambda_1 is a valid proxy for flatness and generalization.
    Inherited from prior flatness literature [4,12]; used as the main flatness metric.
  • ad hoc to paper The previous pseudo-gradient direction Delta-tilde^{t-1} approximates the current global sharpness direction.
    Core heuristic of FedGloSS; only empirical support via delta^t_epsilon in Eq. (6) and Fig. 5.
  • domain assumption The loss-landscape visualizations in 2D random directions represent the relevant geometry.
    Standard visualization technique from [31]; not a rigorous invariant.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning." pith.science (2026). https://pith.science/paper/WE62MRDT

@misc{pith2026241203752,
  author       = {Pith},
  title        = {Pith review of: Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WE62MRDT}},
  note         = {Machine review of arXiv:2412.03752}
}
read the original abstract

Federated learning (FL) enables collaborative model training with privacy preservation. Data heterogeneity across edge devices (clients) can cause models to converge to sharp minima, negatively impacting generalization and robustness. Recent approaches use client-side sharpness-aware minimization (SAM) to encourage flatter minima, but the discrepancy between local and global loss landscapes often undermines their effectiveness, as optimizing for local sharpness does not ensure global flatness. This work introduces FedGloSS (Federated Global Server-side Sharpness), a novel FL approach that prioritizes the optimization of global sharpness on the server, using SAM. To reduce communication overhead, FedGloSS cleverly approximates sharpness using the previous global gradient, eliminating the need for additional client communication. Our extensive evaluations demonstrate that FedGloSS consistently reaches flatter minima and better performance compared to state-of-the-art FL methods across various federated vision benchmarks.

Figures

Figures reproduced from arXiv: 2412.03752 by the authors.

Figure 1
Figure 1. Comparison of FEDAVG (solid) and FEDSAM (net) loss landscapes with varying degrees of data heterogeneity (α) on the CIFAR datasets. FEDSAM’s ef￾fectiveness in converging to global flat minima is highly influenced by the data heterogeneity, where higher heterogeneity (α → 0) leads to sharper minima, and the complexity of the task, e.g., higher sharpness for the more complex CIFAR100. This highlights the importance of… view at source ↗
Figure 2
Figure 2. Global vs. local perspective on FEDSAM. CIFAR100 α = 0 @ 20k rounds on CNN. Local models trained on one class, tested on the local (bot￾tom landscape) or global dataset (top landscape). Models trained with FEDSAM present significant differences between local and global behaviors [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of FEDGLOSS. The model w t is perturbed using ∆˜ t−1 w . The sharpness-aware direction (dashed) is used to compute w t+1 (solid), which lands in a flat region. Compared to FEDAVG. allowing any CLIENTOPT for local training, with compu￾tational costs varying based on the chosen optimizer. This differs from previous methods constrained to the more com￾putationally expensive SAM. In addition, differently fr… view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Global vs. local perspective of FEDGLOSS and FEDSMOO. Loss landscapes of clients models trained on one class, tested on the local (“Local loss”) or global dataset (“Global loss”). CIFAR100 α = 0 with SAM as local optimizer @ t = 20k, CNN. (a)-(b): Models trained with F…
Figure 5
Figure 5. Figure 5: Trend of the difference δ t ϵ (Eq. (6)), which decreases as ADMM is used and over training rounds. CIFAR datasets, CNN. 6. Experiments 6.1. Experimental Setting App. C details implementation and hyperparameter settings. Federated datasets. We leverage established FL be…
Figure 6
Figure 6. Figure 6: Loss landscapes of models trained with FEDGLOSS (net) vs. FEDSAM and FEDSMOO (solid) on CIFAR10/100. (a) - (c) - (e): The flatter regions reached by FEDGLOSS w.r.t. FEDSAM prove the effectiveness of optimizing for global flatness. (b) - (d) - (f): FEDGLOSS achieves fla…
Figure 7
Figure 7. Figure 7: Maximum Hessian eigenvalues (λ1), CNN. Values shown only if algo￾rithm converged. FEDGLOSS reaches the flattest global minima [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Trend of model parameters norm, ∥ w t ∥2, on SAM-based methods with ResNet18 on CIFAR datasets. SAM reduces the norm and the risk of parameters explosion, successfully enabling ADMM in heterogeneous FL. 6.3.4. Communication Efficiency with FedGloSS Communication cost i…
Figure 9
Figure 9. Figure 9: Accuracy gain vs. communication cost. CNN, CIFAR10 α = 0. The ADMM-based methods are usually faster, with FED￾GLOSS being the fastest with ResNet18 and MobileNetv2. While FEDSMOO is faster when using the CNN model, the transmitted bits double due to its increased commu…
Figure 10
Figure 10. Figure 10: Global vs. local perspective on FEDGLOSS. CIFAR100 α = 0 with SAM as local optimizer @ 20k rounds on CNN. (a) - (c): Local models trained on one class, tested on the local (“Local loss”) or global dataset (“Global loss”). Corresponding global perspective of local mode…
Figure 11
Figure 11. Figure 11: Global vs. local perspective on FEDSMOO. CIFAR100 α = 0 with SAM as local optimizer @ 20k rounds on CNN. (a) - (c): Local models trained on one class, tested on the local (“Local loss”) or global dataset (“Global loss”). Corresponding global perspective of local model…
Figure 12
Figure 12. Figure 12: Visualization of the loss landscapes of the CNN trained with [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Visualization of the loss landscapes of the CNN trained with [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Visualization of loss landscapes of the ResNet18 trained with [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: CIFAR10 with varying degrees of heterogeneity (α ∈ {0, 0.05}). Results of centralized runs (dashed lines) added as reference. Comparison of FEDGLOSS with state-of-the-art approaches, distinguished in SAM-based methods (a, c) and SGD-based ones (b, d). FEDGLOSS consist…
Figure 16
Figure 16. Figure 16: CIFAR100 with varying degrees of heterogeneity (α ∈ {0, 0.5}) with CNN. Results of centralized runs (dashed lines) added as reference. Comparison of FEDGLOSS with state-of-the-art approaches, distinguished in SAM-based methods (a, c) and SGD-based ones (b, d). FEDGLOS…
Figure 17
Figure 17. Figure 17: Accuracy trends with ResNet18 on CIFAR100 (left) and CIFAR10 (right). Comparison of FEDGLOSS with state-of-the-art ap￾proaches, distinguished in SAM-based methods (a, c) and SGD-based ones (b, d). FEDGLOSS consistently achieves the best performance, both in terms of f…
Figure 18
Figure 18. Figure 18: Accuracy trends of FEDGLOSS vs. NAIVEFEDGLOSS. The comparison includes the centralized upper bounds of SAM and SgAM (the adaptation of FEDGLOSS’ strategy to the centralized scenario). CNN on CIFAR10 and CIFAR100 with varying heterogeneity degree (α). NAIVEFEDGLOSS is …
Figure 19
Figure 19. Figure 19: Loss barriers resulting from interpolating NAIVEFEDGLOSS and FEDGLOSS’ models, which are found in the same basin. CIFAR datasets, CNN. Details on the computation in App. D.2. FEDGLOSS vs. NAIVEFEDGLOSS [PITH_FULL_IMAGE:figures/full_fig_p017_19.png]
Figure 20
Figure 20. Figure 20: analyzes the impact of ρs on the performance of the global model, both in terms of accuracy (Fig. 20a) and flatness of the solution (Fig. 20b). In details, Fig. 20a compares the accuracy of the global model trained on CIFAR100 when varying the model architecture (CNN …
Figure 21
Figure 21. Figure 21: CIFAR10 (left) and CIFAR100 (right) data distribution across clients with the heterogeneity levels tested in the experiments. On top of each chart we report the average number of classes seen by each client. Models. We trained a Convolutional Neural Network (CNN) insp…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 59 canonical work pages

  1. [1]

    Federated learning based on dynamic regular- ization

    Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regular- ization. ICLR, 2021. 1, 2, 3, 5, 6, 16, 18, 19

    work page 2021

  2. [2]

    Fed- erated learning for healthcare: Systematic review and archi- tecture proposal

    Rodolfo Stoffel Antunes, Cristiano Andr ´e da Costa, Arne K¨uderle, Imrana Abdullahi Yari, and Bj ¨orn Eskofier. Fed- erated learning for healthcare: Systematic review and archi- tecture proposal. ACM Transactions on Intelligent Systems and Technology (TIST), 13(4):1–23, 2022. 1

    work page 2022

  3. [3]

    Distributed optimization and sta- tistical learning via the alternating direction method of mul- tipliers

    Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and sta- tistical learning via the alternating direction method of mul- tipliers. Foundations and Trends® in Machine learning , 3 (1):1–122, 2011. 2, 5

    work page 2011

  4. [4]

    Im- proving generalization in federated learning by seeking flat minima

    Debora Caldarola, Barbara Caputo, and Marco Ciccone. Im- proving generalization in federated learning by seeking flat minima. In European Conference on Computer Vision, pages 654–672. Springer, 2022. 1, 2, 3, 4, 5, 6, 7, 16, 18, 19, 20

    work page 2022

  5. [5]

    Window-based model averaging improves generalization in heterogeneous federated learning

    Debora Caldarola, Barbara Caputo, and Marco Ciccone. Window-based model averaging improves generalization in heterogeneous federated learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2263–2271, 2023. 3, 19

    work page 2023

  6. [6]

    Leaf: A benchmark for federated set- tings

    Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Kone ˇcn`y, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated set- tings. Workshop on Data Privacy and Confidentiality, 2019. 6

    work page 2019

  7. [7]

    Fedgamma: Federated learn- ing with global sharpness-aware minimization

    Rong Dai, Xun Yang, Yan Sun, Li Shen, Xinmei Tian, Meng Wang, and Yongdong Zhang. Fedgamma: Federated learn- ing with global sharpness-aware minimization. IEEE Trans- actions on Neural Networks and Learning Systems, 2023. 1, 2, 5, 6

    work page 2023

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 19

    work page 2009

Show all 65 references
  1. [9]

    Effi- cient sharpness-aware minimization for improved training of neural networks

    Jiawei Du, Hanshu Yan, Jiashi Feng, Joey Tianyi Zhou, Lian- gli Zhen, Rick Siow Mong Goh, and Vincent YF Tan. Effi- cient sharpness-aware minimization for improved training of neural networks. ICLR, 2022. 3

    2022

  2. [10]

    Sharpness-aware training for free

    Jiawei Du, Daquan Zhou, Jiashi Feng, Vincent Tan, and Joey Tianyi Zhou. Sharpness-aware training for free. Advances in Neural Information Processing Systems , 35: 23439–23451, 2022. 3

    2022

  3. [11]

    Feddrive: Generalizing federated learning to seman- tic segmentation in autonomous driving

    Lidia Fantauzzo, Eros Fan `ı, Debora Caldarola, Antonio Tavera, Fabio Cermelli, Marco Ciccone, and Barbara Ca- puto. Feddrive: Generalizing federated learning to seman- tic segmentation in autonomous driving. In 2022 IEEE/RSJ International Conference on Intelligent Robots and ...

    2022

  4. [12]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. Int. Conf. Machine Learn., 2021. 1, 2, 3, 20

    2021

  5. [13]

    Feddc: Federated learning with non-iid data via local drift decoupling and correction

    Liang Gao, Huazhu Fu, Li Li, Yingwen Chen, Ming Xu, and Cheng-Zhong Xu. Feddc: Federated learning with non-iid data via local drift decoupling and correction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10112–10121, 2022. 3

    2022

  6. [14]

    Loss surfaces, mode connectivity, and fast ensembling of dnns

    Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in neural information processing systems, 31, 2018. 20

    2018

  7. [15]

    pytorch-hessian-eigenthings: efficient pytorch hessian eigendecomposition, 2018

    Noah Golmant, Zhewei Yao, Amir Gholami, Michael Ma- honey, and Joseph Gonzalez. pytorch-hessian-eigenthings: efficient pytorch hessian eigendecomposition, 2018. 20

    2018

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. corr abs/1512.03385 (2015), 2015. 6, 19

    2015 arXiv

  9. [17]

    Flat minima.Neu- ral computation, 9(1):1–42, 1997

    Sepp Hochreiter and J ¨urgen Schmidhuber. Flat minima.Neu- ral computation, 9(1):1–42, 1997. 1, 2

    1997

  10. [18]

    The non-iid data quagmire of decentralized ma- chine learning

    Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons. The non-iid data quagmire of decentralized ma- chine learning. In Int. Conf. Machine Learn. , pages 4387–

  11. [19]

    Mea- suring the effects of non-identical data distribution for fed- erated visual classification

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Mea- suring the effects of non-identical data distribution for fed- erated visual classification. Neurips Workshop on Federated Learning, 2019. 3, 6, 18

    2019

  12. [20]

    Feder- ated visual classification with real-world data distribution

    Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Feder- ated visual classification with real-world data distribution. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, pages 76–92. Springer, 2020. 1, 6, 16, 18, 19

    2020

  13. [21]

    Batch normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In International conference on machine learn- ing, pages 448–456. PMLR, 2015. 19

    2015

  14. [22]

    Averaging weights leads to wider optima and better generalization

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. Conference on Uncertainty in Artificial Intelligence, 2018. 3

    2018

  15. [23]

    Fantastic generalization mea- sures and where to find them

    Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization mea- sures and where to find them. ICLR, 2019. 1

    2019

  16. [24]

    Advances and open problems in federated learn- ing

    Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur´elien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cum- mings, et al. Advances and open problems in federated learn- ing. Foundations and Trends® in Machine Learning, ...

    2021

  17. [25]

    Mime: Mimicking centralized stochastic algorithms in federated learning

    Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh. Mime: Mimicking centralized stochastic algorithms in federated learning. Advances in Neural Information Processing Systems, 2020. 1, 3

    2020

  18. [26]

    Scaffold: Stochastic controlled averaging for fed- erated learning

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for fed- erated learning. In Int. Conf. Machine Learn., pages 5132–

  19. [27]

    On large- batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large- batch training for deep learning: Generalization gap and sharp minima. ICLR, 2017. 1, 2

    2017

  20. [28]

    Communication-efficient federated learning with accelera- tion of global momentum

    Geeho Kim, Jinkyu Kim, and Bohyung Han. Communication-efficient federated learning with accelera- tion of global momentum. arXiv preprint arXiv:2201.03172,

    arXiv

  21. [29]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. 6

    2009

  22. [30]

    Gradient-based learning applied to document recog- nition

    Yann LeCun, L ´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE , 86(11):2278–2324, 1998. 6, 19

    1998

  23. [31]

    Visualizing the loss landscape of neural nets

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Ad- vances in neural information processing systems , 31, 2018. 20

    2018

  24. [32]

    A review of applications in federated learning

    Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. A review of applications in federated learning. Computers & Industrial Engineering, 149:106854, 2020. 1, 7

    2020

  25. [33]

    Fed- erated learning on non-iid data silos: An experimental study

    Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Fed- erated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th International Conference on Data Engi- neering (ICDE), pages 965–978. IEEE, 2022. 7

    2022

  26. [34]

    Federated learning: Challenges, methods, and future directions

    Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine, 37(3):50–60,

  27. [35]

    Federated optimiza- tion in heterogeneous networks

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimiza- tion in heterogeneous networks. Proceedings of Machine learning and systems, 2:429–450, 2020. 1, 3, 6

    2020

  28. [36]

    On the convergence of fedavg on non-iid data

    Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. ICLR, 2020. 1, 2

    2020

  29. [37]

    Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous fre- quency space

    Quande Liu, Cheng Chen, Jing Qin, Qi Dou, and Pheng-Ann Heng. Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous fre- quency space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , page...

    2021

  30. [38]

    Towards efficient and scalable sharpness-aware minimization

    Yong Liu, Siqi Mai, Xiangning Chen, Cho-Jui Hsieh, and Yang You. Towards efficient and scalable sharpness-aware minimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12360– 12370, 2022. 3

    2022

  31. [39]

    Communication- efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics , pages 1273–1282. PMLR, 2017. 1, 2, 3, 6

    2017

  32. [40]

    Fed- seg: Class-heterogeneous federated learning for semantic segmentation

    Jiaxu Miao, Zongxin Yang, Leilei Fan, and Yi Yang. Fed- seg: Class-heterogeneous federated learning for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8042– 8052, 2023. 1, 2

    2023

  33. [41]

    A survey on federated learning applications in healthcare, finance, and data privacy/data security

    Theodora Nevrataki, Anastasia Iliadou, George Ntolkeras, Ioannis Sfakianakis, Lazaros Lazaridis, George Maraslidis, Nikolaos Asimopoulos, and George F Fragulis. A survey on federated learning applications in healthcare, finance, and data privacy/data security. In AIP Conferenc...

    2023

  34. [42]

    Dif- ferentially private sharpness-aware training

    Jinseong Park, Hoki Kim, Yujin Choi, and Jaewook Lee. Dif- ferentially private sharpness-aware training. Int. Conf. Ma- chine Learn., 2023. 3, 5

    2023

  35. [43]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zem- ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas Kopf, Edward Yang, Zachary DeVito, Mar- tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steine...

    2019

  36. [44]

    Relative flatness and gen- eralization, 2021

    Henning Petzka, Michael Kamp, Linara Adilova, Cristian Sminchisescu, and Mario Boley. Relative flatness and gen- eralization, 2021. 2

    2021

  37. [45]

    Generalized federated learning via sharpness aware minimization

    Zhe Qu, Xingyu Li, Rui Duan, Yao Liu, Bo Tang, and Zhuo Lu. Generalized federated learning via sharpness aware minimization. In Int. Conf. Machine Learn. , pages 18250– 18280. PMLR, 2022. 1, 2, 3, 4, 5, 6

    2022

  38. [46]

    Federated learning for medical applica- tions: A taxonomy, current trends, challenges, and future re- search directions

    Ashish Rauniyar, Desta Haileselassie Hagos, Debesh Jha, Jan Erik H ˚akeg˚ard, Ulas Bagci, Danda B Rawat, and Vladimir Vlassov. Federated learning for medical applica- tions: A taxonomy, current trends, challenges, and future re- search directions. IEEE Internet of Things Journ...

    2023

  39. [47]

    Adaptive federated optimization

    Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone ˇcn`y, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. ICLR, 2021. 1, 2, 3

    2021

  40. [48]

    FedJAX: Federated learning simulation with JAX

    Jae Hun Ro, Ananda Theertha Suresh, and Ke Wu. FedJAX: Federated learning simulation with JAX. arXiv preprint arXiv:2108.02117, 2021. 19

    2021 arXiv

  41. [49]

    An overview of gradient descent optimiza- tion algorithms

    Sebastian Ruder. An overview of gradient descent optimiza- tion algorithms. arXiv preprint arXiv:1609.04747, 2016. 3

    2016 arXiv

  42. [50]

    Mobilenetv2: Inverted residuals and linear bottlenecks

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zh- moginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 4510–4520, 2018. 6, 19

    2018

  43. [51]

    Learning across domains and devices: Style-driven source-free domain adaptation in clustered federated learning

    Donald Shenaj, Eros Fan `ı, Marco Toldo, Debora Caldarola, Antonio Tavera, Umberto Michieli, Marco Ciccone, Pietro Zanuttigh, and Barbara Caputo. Learning across domains and devices: Style-driven source-free domain adaptation in clustered federated learning. In Proceedings of ...

    2023

  44. [52]

    Make landscape flatter in differentially private federated learning

    Yifan Shi, Yingqi Liu, Kang Wei, Li Shen, Xueqian Wang, and Dacheng Tao. Make landscape flatter in differentially private federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 24552–24562, 2023. 2

    2023

  45. [53]

    Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape

    Yan Sun, Li Shen, Shixiang Chen, Liang Ding, and Dacheng Tao. Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape. Int. Conf. Machine Learn., 2023. 1, 2, 4, 5, 6, 16, 18

    2023

  46. [54]

    Fedspeed: Larger local interval, less commu- nication round, and higher generalization accuracy

    Yan Sun, Li Shen, Tiansheng Huang, Liang Ding, and Dacheng Tao. Fedspeed: Larger local interval, less commu- nication round, and higher generalization accuracy. ICLR,

  47. [55]

    On the importance of initialization and momentum in deep learning

    Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In Int. Conf. Machine Learn., pages 1139–

  48. [56]

    Adabest: Minimizing client drift in federated learning via adaptive bias estimation

    Farshid Varno, Marzie Saghayi, Laya Rafiee Sevyeri, Sharut Gupta, Stan Matwin, and Mohammad Havaei. Adabest: Minimizing client drift in federated learning via adaptive bias estimation. In European Conference on Computer Vi- sion, pages 710–726. Springer, 2022. 1, 2, 3, 4, 7

    2022

  49. [57]

    A survey on federated learning: chal- lenges and applications

    Jie Wen, Zhixia Zhang, Yang Lan, Zhihua Cui, Jianghui Cai, and Wensheng Zhang. A survey on federated learning: chal- lenges and applications. International Journal of Machine Learning and Cybernetics, 14(2):513–535, 2023. 1

    2023

  50. [58]

    Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval

    Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim. Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2575–2584, 2020. 6

    2020

  51. [59]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. In Pro- ceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. 19

    2018

  52. [60]

    Accelerated stochastic power iteration

    Peng Xu, Bryan He, Christopher De Sa, Ioannis Mitliagkas, and Chris Re. Accelerated stochastic power iteration. In In- ternational Conference on Artificial Intelligence and Statis- tics, pages 58–67. PMLR, 2018. 20

    2018

  53. [61]

    Communication-efficient heterogeneous federated learning with generalized heavy-ball momentum

    Riccardo Zaccone, Carlo Masone, and Marco Ciccone. Communication-efficient heterogeneous federated learning with generalized heavy-ball momentum. arXiv preprint arXiv:2311.18578, 2023. 3, 16, 18

    2023 arXiv

  54. [62]

    Federated learning for the internet of things: Applications, challenges, and op- portunities

    Tuo Zhang, Lei Gao, Chaoyang He, Mi Zhang, Bhaskar Kr- ishnamachari, and A Salman Avestimehr. Federated learning for the internet of things: Applications, challenges, and op- portunities. IEEE Internet of Things Magazine, 5(1):24–29,

  55. [65]

    Local loss

    selected in the last round, comparing the behavior of clients’ models trained with F EDGLOSS from the local and global perspectives. These results highlight the effectiveness of F EDGLOSS in achieving local models with consistent local-global behavior. Indeed, these models end...

  56. [2022]

    Algorithm Alg

    1 Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning Appendix A. Algorithm Alg. 1 summarizes FEDGLOSS, using as an example SGD or SAM as local optimizers, differently highlighted. The comment colors replicate those of Fig...

  57. [5143]

    1, 3, 5, 6, 16

    PMLR, 2020. 1, 3, 5, 6, 16

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.