{"id":"09d369d2-80d9-4074-b69a-7eb6c48751a5","arxiv_id":"2412.03752","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FedGloSS optimizes global sharpness on the server using the previous pseudo-gradient to approximate SAM's perturbation, achieving better accuracy and flatness in heterogeneous federated learning without extra communication.","lead":"A federated learning method, FedGloSS, applies sharpness-aware minimization on the server instead of only on clients, using the previous round's gradient to approximate the sharpness perturbation. It reports flatter minima and higher accuracy than prior federated SAM methods while keeping communication costs at the FedAvg level.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main-text FedGloSS update (Eq. 4, Sec. 5.2) omits the ADMM dual-variable term that Algorithm 1 includes; if the experiments use Algorithm 1, the defined method is not the evaluated method, leaving the central server-side SAM claim unsupported as stated.","rationale":"The central claim is that FedGloSS achieves global sharpness minimization at FedAvg-level communication cost. For that claim to be true, the algorithm whose results are reported must be the algorithm defined as server-side SAM with previous-gradient perturbation. I identified a mismatch: the main text's Eq. 4 and four-step description define an update without any ADMM terms, while Algorithm 1 and the experimental sections describe a method whose local objective, local dual updates, and server update all incorporate ADMM corrections. The two methods are not interchangeable. In particular, removing the ADMM terms removes the mechanism that the paper itself shows is necessary to keep δ_t^ε small (Eq. 6, Fig. 5), so the efficient approximation may not work at all without the stateful-client machinery. This is more load-bearing than the statelessness limitation per se: even in a fully stateful simulation, a reader implementing Section 5.2 would not run the method that produced Tables 3–5. The concrete test leverages the publicly available code, which is a real strength of the paper, and directly determines whether the reported gains come from the advertised server-side SAM or from the ADMM modification. If the ADMM-free version reproduces the results, the ambiguity is cosmetic and the paper can be accepted after a clarification. If not, the contribution must be redefined as an ADMM-based method with a SAM-style perturbation, and the communication-efficiency claim must be re-evaluated in that light. I found no other concern of comparable weight: the homogeneous-setting accuracy gap in App. B.4 is minor, the grid search without a validation split is common practice, and the absence of theory is not fatal given the breadth of empirical evidence. The paper's own 'key limitation' statement about stateful clients reinforces rather than resolves the ambiguity, because it confirms the ADMM machinery is integral.","tokens_in":31053,"tokens_out":8024,"duration_ms":75866,"concrete_test":"On the released code (github.com/pietrocagnasso/fedgloss), disable the ADMM machinery (remove σ_k updates at Alg. 1 line 19, the σ_k and (w_{k,i−1}−w_{k,0})/β terms in line 15, the global σ update in line 22, and the −βσ^{t+1} term in line 24), leaving only Section 5.2's four-step server-side SAM update. Reproduce the CIFAR-10 α=0 CNN experiment (same hyperparameters, 3 seeds) and compare final accuracy to Tab. 3's FedGloSS row (83.9±0.4) and FedSAM's 70.2±0.9. If the ADMM-free version does not match the reported FedGloSS accuracy, the main-text method is not what the experiments evaluated, so the central claim must be re-scoped or re-derived.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.2 defines FedGloSS by a four-step update: ε̃^t = ρ Δ̃^{t-1}/||Δ̃^{t-1}||, w̃^t = w^t + ε̃^t, collect client updates from w̃^t, and w^{t+1} = w^t − η_s Δ̃^t (Step 4). This is the advertised server-side SAM: the descent direction is the pseudo-gradient at the perturbed point. However, Section 5.2.1 immediately introduces ADMM, and Algorithm 1 (Appendix A) changes the algorithm materially. Client local updates become w_{k,i} ← w_{k,i−1} − η[g_{k,i} − σ_k + (w_{k,i−1} − w_{k,0})/β] (line 15), per-client dual variables σ_k are updated (line 19), and the server update is w^{t+1} = w^t − Δ̃^t − βσ^{t+1} (line 24). The two updates are not equivalent, and the main text's statement that FedGloSS 'allows any CLIENT OPT' is incompatible with line 15. Because the experimental results (e.g., Tab. 3, Fig. 5 'w/ ADMM', Tab. 8 'Global Consistency') align with the ADMM version, the evaluated method is not the method defined by Eq. 4. The ADMM terms are not a minor detail: Eq. 6 shows the approximation error δ_t^ε is controlled only when consecutive pseudo-gradients are directionally consistent, and Fig. 5 shows that consistency is achieved by ADMM. Without the ADMM terms, the previous-gradient approximation has no alignment mechanism, so the claimed 'global server-side SAM at FedAvg cost' could fail entirely. The paper itself flags stateful clients as 'a key limitation,' which further confirms the ADMM machinery is essential, not optional.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes FedGloSS, a federated learning method that applies Sharpness-Aware Minimization on the server side, approximating the global sharpness perturbation with the pseudo-gradient from the previous round to avoid extra client-server communication. The method is combined with ADMM to align local and global updates, and the authors report extensive experiments on CIFAR-10, CIFAR-100, ResNet18, and Landmarks-User-160K, measuring accuracy, Hessian-based flatness, and communication cost. The central empirical claim is that FedGloSS reaches flatter global minima and higher accuracy than prior FL sharpness methods, including FedSMOO, while maintaining FedAvg-level communication cost.","tokens_in":31405,"tokens_out":3688,"duration_ms":39770,"significance":"If the method as described in Algorithm 1 is the intended contribution, the paper addresses a real gap: existing sharpness-aware FL methods either optimize local flatness only, which can be inconsistent with the global landscape, or they pay doubled communication cost as in FedSMOO. The experimental study is broad and includes multiple architectures, heterogeneity levels, a large-scale benchmark, standard deviations over seeds, Hessian eigenvalue flatness measurements, and a communication-cost analysis. The comparison with a 'NaiveFedGloSS' two-exchange baseline in Section 6.4.1 is a particularly useful control. The main weakness is a load-bearing mismatch between the main-text update rule and the implemented algorithm: the ADMM terms in Algorithm 1 are not present in Eq. (4), and the paper itself identifies the stateful-client ADMM machinery as a key limitation. Because the consistency mechanism that keeps the approximation error of Eq. (6) small is precisely the ADMM alignment, the paper as written does not establish that the communication-efficient server-side SAM idea works without that machinery.","major_comments":[{"comment":"The paper should also clarify what happens when Algorithm 1 is used with client-side SGD: as printed, lines 12-15 always compute a sharpness-aware gradient, so the pseudocode does not actually implement the 'any CLIENT OPT' claim without a conditional branch that is not shown.","section":"Section 5.2 and Algorithm 1 (Appendix A)"},{"comment":"This issue is load-bearing because the paper's communication-efficiency argument depends on FedGloSS being a drop-in server-side change, whereas Algorithm 1 changes the clients as well.","section":"Section 5.2.1 and Table 2"},{"comment":"Without such an experiment or a revised narrative, the central claim 'global server-side SAM at FedAvg cost' is supported only for a method that is not defined in Section 5.2.","section":"Eq. (6) and Figure 5"},{"comment":"The conclusion in Section 7 and the abstract describe the method without this caveat, so the stated scope in the abstract is stronger than what the stateful-client design can support.","section":"Section 5.2.1 and Section 6.4.2"}],"minor_comments":[{"comment":"In Table 1, the class 'airplane' appears twice in the CIFAR-10 block; one of these rows should presumably be a different class, and the local client identity labels are inconsistent with the accompanying text.","section":"Table 1"},{"comment":"Algorithm 1's text says it uses 'SGD or SAM as local optimizers, differently highlighted,' but the pseudocode as printed has no visible distinction between the two cases; lines 12-14 always compute a SAM perturbation and line 14 overwrites the gradient with the sharpness-aware gradient. The authors should add explicit conditionals, remove the claim, or clarify in a caption how the SGD variant is obtained.","section":"Algorithm 1"},{"comment":"The ADMM augmented Lagrangian in Eq. (7) uses the notation σ_k^T (w^t - w_k^t) but the text below writes the subproblem with a negative sign; the sign convention is confusing and should be made consistent.","section":"Section 5.2.1, Eq. (7)"},{"comment":"The name 'FedGloSS' is written in different typographical styles across the paper (FEDGLOSS in the abstract, FedGloSS in section headings, and FedGloss in some tables); a consistent style would improve readability.","section":"Notation"}],"recommendation":"major_revision","confidential_remarks":"The main-text Algorithm mismatch is the key issue. The authors have a defensible empirical contribution, but they must either make the ADMM version the official method and adjust the 'any CLIENT OPT' and communication-efficiency claims, or provide a no-ADMM evaluation that supports the current Section 5.2 definition. I would also recommend asking for the stateful-client limitation to be reflected in the abstract and conclusion, since it directly affects the practicality claim in a cross-device FL venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FedGloSS is a solid, useful FL paper. The genuinely new piece is running SAM on the server and approximating the perturbation with the previous global pseudo-gradient, which buys global sharpness at FedAvg-level communication cost. The local-global sharpness inconsistency study (Figs. 1-2, Tab. 1) is a genuinely informative diagnostic and worth reading on its own. The empirical work is broad: CIFAR-10/100, ResNet18, and Landmarks-User-160K, with Hessian flatness metrics and standard deviations. The gains over FedSMOO of about 2.5 points on CIFAR and 3.4 on Landmarks at half the communication are credible, and the comparison against a two-round \"naive\" server-side SAM baseline is the right control.\n\nThe soft spots are real but not fatal. The main-text update in Sec. 5.2 (four steps ending with w^{t+1} = w^t - eta_s * tilde Delta^t) omits the ADMM dual-variable term that appears in Algorithm 1, line 24, and in the local update. The experiments clearly use the ADMM version, so the paper as written defines one thing and evaluates another. This needs fixing: either present the full ADMM-inclusive update in the main text, or say explicitly that Sec. 5.2 is the non-ADMM variant. It is not a hidden fatal flaw, because the paper openly credits ADMM for directional consistency and flags stateful clients as a key limitation, but the reader should not have to reverse-engineer the actual method from the appendix. Also, \"allows any CLIENT OPT\" is too strong: Algorithm 1 injects an ADMM correction term into the local update, so local optimizers are not used unmodified.\n\nTwo smaller issues: the abstract's \"consistently better\" is contradicted by the homogeneous setting in App. B.4, where FedSMOO edges out FedGloSS on CIFAR-10 accuracy; and hyperparameters are grid-searched without an explicit validation split, which is common but should be acknowledged more carefully.\n\nNone of this undermines the central claim. The evidence supports the combination of server-side SAM, previous-gradient approximation, and ADMM as a communication-efficient way to improve generalization under heterogeneity. The stateful-client requirement is a genuine deployment limitation, and the authors say so themselves. This paper deserves a serious referee: it is clearly written enough to review, the experiments are extensive, and the main idea is worth putting into the literature. My recommendation is to send it out, with the expectation that the authors reconcile the algorithm definitions, soften the \"any CLIENT OPT\" claim, and sharpen the abstract. I would cite it.","headline":"Solid FL paper: server-side SAM with a previous-gradient trick is genuinely new and well tested, but the main-text algorithm omits the ADMM terms that the appendix and experiments actually use.","tokens_in":738,"tokens_out":840,"would_cite":true,"duration_ms":36223,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedGloSS claims that global sharpness, not local flatness, should be minimized in heterogeneous federated learning, and shows that server-side SAM with a one-round-delayed gradient achieves flatter minima at FedAvg-level communication cost.","keywords":["Federated learning","sharpness-aware minimization","global vs local flatness","server-side optimization","ADMM","communication efficiency","data heterogeneity","loss landscape flatness"],"falsifier":"Run FedGloSS on CIFAR-100 with $\\alpha=0$ using stateless clients (dual variables reset to zero each round) and compare final accuracy and the maximum Hessian eigenvalue against the stateful version; the paper's mechanism predicts clear degradation in both. A second check is to measure $\\delta^t_\\epsilon$ from Eq. (6) under randomly changing client subsets: the claim implies the gap stays bounded with ADMM and grows without it.","tokens_in":30752,"feed_emoji":"📉","tokens_out":6318,"duration_ms":57209,"temperature":0.7,"pith_summary":"This paper claims that federated learning methods which seek flat minima on each client's local loss (FedSAM) miss the true goal: flatness in the server's global loss landscape. To fix that, it introduces FedGloSS, which applies Sharpness-Aware Minimization on the server using the previous round's pseudo-gradient as the perturbation direction, so no extra communication round is needed. An ADMM consistency term keeps client and server updates aligned so that this one-step-late approximation stays close to the true SAM perturbation. Across CIFAR and the large-scale Landmarks-User-160K benchmarks, the paper reports consistently flatter minima and accuracy gains over the best competitor of roughly 2.5 points on CIFAR and 3.4 points on Landmarks, at half the communication cost.","feed_headline":"Global sharpness beats local flatness in federated learning","feed_subtitle":"Server-side SAM with the previous round's gradient reaches flatter minima with no extra communication.","key_machinery":"The central mechanism is server-side Sharpness-Aware Minimization (SAM) with a delayed pseudo-gradient: the server perturbs the global model by $\\rho\\, \\tilde{\\Delta}^{t-1}/\\|\\tilde{\\Delta}^{t-1}\\|$ instead of computing a live ascent gradient, then descends along the resulting sharpness-aware direction. ADMM with per-client dual variables $\\sigma_k$ is layered on top to align local and global updates, keeping the approximation error $\\delta^t_\\epsilon$ small across rounds. The paper argues that SAM's tendency toward small weight updates also prevents the parameter explosion that historically breaks ADMM-based federated methods.","core_discovery":"FedGloSS solves the global sharpness-aware objective $\\min_{w} F(w) = \\frac{1}{C}\\sum_k \\max_{\\|\\epsilon\\|\\le\\rho} f_k(w+\\epsilon)$ on the server, replacing the expensive SAM ascent direction with the pseudo-gradient from the previous round, $\\tilde{\\epsilon}^{t} = \\rho\\, \\tilde{\\Delta}^{t-1}/\\|\\tilde{\\Delta}^{t-1}\\|$. The authors claim this yields flatter global minima and better accuracy than both local-sharpness methods (FedSAM, FedSpeed, FedGamma) and the global-consistency method FedSMOO, while keeping FedAvg's per-round communication cost. The loss-landscape and Hessian-eigenvalue measurements in the paper support the claim that FedGloSS converges to the flattest global minima among all compared methods.","pith_inferences":["Because the perturbation relies on the previous round's direction, the benefit likely shrinks when the client population changes rapidly or data is non-stationary; a direct measure of $\\delta^t_\\epsilon$ under such conditions would tell how much of the ADMM alignment is actually load-bearing.","The Hessian-eigenvalue evidence is correlational; a stronger test would compare FedGloSS and FedSMOO solutions of equal sharpness to see whether the accuracy gap persists independently of flatness.","The main-text update rule (Eq. 4) omits the ADMM term that appears in Algorithm 1, leaving the exact implemented update ambiguous; a clean reproduction would need to state which of the two is the deployed rule.","A stateless variant that moves the dual variable entirely to the server would be the natural test of whether the paper's gains survive the least restrictive cross-device setting."],"forward_implications":["Global flatness becomes optimizable at the same communication cost as FedAvg, removing the main practical barrier to global-sharpness methods.","Any client optimizer (SGD or SAM) can be used, so clients can avoid SAM's doubled local compute when needed.","The method reaches FedAvg-level accuracy about three to five times faster in bits exchanged, and matches the exact two-step server SAM (Naive FedGloSS) in final accuracy.","The combination of server-side SAM and ADMM avoids the parameter explosion observed in FedDyn and FedSMOO, keeping model norms bounded under extreme heterogeneity.","On the large-scale Landmarks-User-160K benchmark, FedGloSS is among the few methods that beat FedAvg, with the lowest bit cost of the top performers."],"supporting_citations":[{"why":"Defines the SAM min-max objective and the gradient-ascent perturbation formula that FedGloSS transplants to the server.","marker":"[12]"},{"why":"Supplies the FedSAM baseline that only optimizes local sharpness, the approach FedGloSS claims to improve upon.","marker":"[4]"},{"why":"Second FedSAM variant used as a baseline in the federated sharpness line.","marker":"[45]"},{"why":"FedSMOO, the strongest competitor targeting global consistency via ADMM; FedGloSS is benchmarked against it and claims higher accuracy at half the communication cost.","marker":"[53]"},{"why":"FedDyn supplies the ADMM dynamic-regularization scheme whose dual variables FedGloSS adapts, and which suffers parameter explosion without SAM.","marker":"[1]"},{"why":"Establishes the pseudo-gradient formulation and server-optimizer view of FedAvg that underlies server-side SAM.","marker":"[47]"},{"why":"DP-SAT provides the previous-gradient approximation of SAM's ascent step that FedGloSS transfers to the federated setting.","marker":"[42]"},{"why":"The ADMM distributed-optimization method behind the global-consistency mechanism.","marker":"[3]"},{"why":"Supplies the federated datasets and heterogeneity splits (CIFAR-10/100 and Landmarks-User-160K) used for evaluation.","marker":"[20]"}],"fun_headline_variants":["Global sharpness, zero extra comms","Server-side SAM, no added rounds","Reuse gradients to flatten globally","Flat minima without extra clients","FedGloSS: global flatness, FedAvg cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method depends on stateful clients that keep and return an ADMM dual variable across rounds; the paper itself calls this 'a key limitation of our approach.' If clients are stateless, drop in and out, or the dual variables drift, the alignment that keeps the previous pseudo-gradient close to the true SAM perturbation is lost, and the central approximation has no guarantee.","fun_headline_variants_meta":{"raw":{"variants":["Global sharpness, zero extra comms","Server-side SAM, no added rounds","Reuse gradients to flatten globally","Flat minima without extra clients","FedGloSS: global flatness, FedAvg cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1410,"prompt_tokens":875,"completion_tokens":535,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":471}},"tokens_in":491,"tokens_out":535,"duration_ms":5956,"temperature":1.0,"reasoning_tokens":471,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:07:39.251845+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FedGloSS on CIFAR-100 with $\\alpha=0$ using stateless clients (dual variables reset to zero each round) and compare final accuracy and the maximum Hessian eigenvalue against the stateful version; the paper's mechanism predicts clear degradation in both. A second check is to measure $\\delta^t_\\epsilon$ from Eq. (6) under randomly changing client subsets: the claim implies the gap stays bounded with ADMM and grows without it.","supporting_citations":[{"cited_title":"Sharpness-aware minimization for efficiently improving generalization","cited_arxiv_id":null,"evidence_quote":"Defines the SAM min-max objective and the gradient-ascent perturbation formula that FedGloSS transplants to the server."},{"cited_title":"Im- proving generalization in federated learning by seeking flat minima","cited_arxiv_id":null,"evidence_quote":"Supplies the FedSAM baseline that only optimizes local sharpness, the approach FedGloSS claims to improve upon."},{"cited_title":"Generalized federated learning via sharpness aware minimization","cited_arxiv_id":null,"evidence_quote":"Second FedSAM variant used as a baseline in the federated sharpness line."},{"cited_title":"Dynamic regularized sharpness aware minimization in federated learning: Approaching global consistency and smooth landscape","cited_arxiv_id":null,"evidence_quote":"FedSMOO, the strongest competitor targeting global consistency via ADMM; FedGloSS is benchmarked against it and claims higher accuracy at half the communication cost."},{"cited_title":"Federated learning based on dynamic regular- ization","cited_arxiv_id":null,"evidence_quote":"FedDyn supplies the ADMM dynamic-regularization scheme whose dual variables FedGloSS adapts, and which suffers parameter explosion without SAM."},{"cited_title":"Adaptive federated optimization","cited_arxiv_id":null,"evidence_quote":"Establishes the pseudo-gradient formulation and server-optimizer view of FedAvg that underlies server-side SAM."},{"cited_title":"Dif- ferentially private sharpness-aware training","cited_arxiv_id":null,"evidence_quote":"DP-SAT provides the previous-gradient approximation of SAM's ascent step that FedGloSS transfers to the federated setting."},{"cited_title":"Distributed optimization and sta- tistical learning via the alternating direction method of mul- tipliers","cited_arxiv_id":null,"evidence_quote":"The ADMM distributed-optimization method behind the global-consistency mechanism."},{"cited_title":"Feder- ated visual classification with real-world data distribution","cited_arxiv_id":null,"evidence_quote":"Supplies the federated datasets and heterogeneity splits (CIFAR-10/100 and Landmarks-User-160K) used for evaluation."}],"review_version":1}