Pith. sign in

REVIEW 4 major objections 4 minor 17 references

FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FedAPA claims that a server can generate personalized federated models by running gradient descent on aggregation weights, using the change in each client's uploaded parameters as a proxy for its local loss, and reports the best accuracy…

desk verdict The server-side aggregation-weight idea is new and sensible, but Eq. (7) has a load-bearing sign error that reverses the gradient direction, so the paper's central claim and convergence proof do not hold as written. read the letter →

arxiv 2502.07456 v2 pith:ODH2FXCD submitted 2025-02-11 cs.LG cs.CV

Yuxia Sun , Aoxiang Sun , Siyi Pan , Zhixiao Fu , Jingcai Guo This is my paper · ORCID
classification cs.LGcs.CV
keywords personalizedfederatedlearningserver-sideaggregationadaptiveweightsnon-IIDdataclientparameterdeltapartialmodelsharingconvergenceguaranteegradient-basedweightupdate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedAPA is a personalized federated learning method whose central proposal is that a server can learn how to blend clients' model parameters into personalized models by running gradient descent on the aggregation weights themselves. The paper argues that after a few local training steps, each client's uploaded feature-extractor parameters are close to that client's local optimum, so half the squared norm of the parameter change $\Delta\theta_i = \theta_i - \bar\theta_i$ is a valid proxy for the local loss; this turns aggregation-weight selection into an ordinary server-side optimization problem. The authors prove a one-round deviation bound for this update under Lipschitz smoothness, unbiased-gradient, and Lipschitz-continuity assumptions, and report that FedAPA beats ten personalized-FL baselines on accuracy in ten of twelve non-IID scenarios while keeping per-iteration compute close to FedAvg. If correct, FedAPA offers a way to get per-client personalization with almost no extra client-side cost, since the personalization logic is moved to the server.

What carries the argument

The central object is the per-client aggregation-weight vector $A_i=(a_{i1},...,a_{iM})^T$, used to build $\bar\theta_i=\sum_j a_{ij}\theta_j$. The load-bearing identity is $\nabla_{A_i}\bar\theta_i = \Theta$, the matrix of all uploaded client parameters, which makes Eq. (7) a cheap server-side gradient step whose direction is set by the client-parameter delta $\Delta\theta_i=\theta_i-\bar\theta_i$. The paper's proxy-loss argument—$\frac12\|\Delta\theta_i\|_2^2$ stands in for the local loss because $\theta_i$ is assumed close to $\theta_i^*$—is what connects this algebraic update to actual local objectives and underpins Theorem 1.

What would settle it

Train FedAPA on a non-IID benchmark with one local epoch per round and a large client learning rate (for example $\alpha=0.1$), and log both the proxy $\frac12\|\theta_i-\bar\theta_i\|_2^2$ and the true local loss $L_i$ on each client's held-out data. If the proxy decreases while true local loss increases, or accuracy falls sharply relative to the $E=2$, $\alpha=0.01$ setting, the assumption that $\theta_i$ approximates the local optimum fails and Eq. (7) is not minimizing what it claims to minimize.

Watch

Extended reading notes

Core claim

The paper's claim is that Eq. (7), $A_i \leftarrow A_i - \eta(\nabla_{A_i}\bar\theta_i)^T \Delta\theta_i$, is a valid server-side personalization rule. Because $\bar\theta_i=\sum_j a_{ij}\theta_j$, the gradient of the aggregated parameters with respect to the weight vector is easy to compute from the uploaded $\theta_j$, and the paper interprets $\Delta\theta_i$ as $-\nabla_{\bar\theta_i}L_i$: after local updates, $\theta_i$ approximates the local optimum $\theta_i^*$, so the distance from $\bar\theta_i$ to $\theta_i$ measures how far the current aggregate is from the client's target. Updating $A_i$ along this direction therefore decreases a proxy of the client's local loss, and the server can do this centrally for every client. The method shares only the feature extractor between server and clients, sets each client's self-weight to 0.5, clips and normalizes $A_i$, and proves convergence in Theorem 1 by combining a standard local-descent bound with a bound on the drift caused by the aggregation-weight update.

Load-bearing premise

The load-bearing premise is that after only a couple of local training epochs, each client's uploaded parameters are close enough to that client's true local optimum that the squared size of the parameter change faithfully measures how far the server's aggregate is from the client's loss minimum; if that closeness fails, the server's gradient step is not actually minimizing local loss.

Editorial extensions

If this is right

  • Personalization no longer requires client-side weight optimization: each client just does its normal local training and uploads parameters, while the server adjusts all aggregation weights from the uploaded deltas.
  • The per-round server work is a matrix-vector product and a small gradient step, so FedAPA can retain FedAvg-level communication (478 KB per iteration versus 484 KB for FedAvg on CIFAR-10) while improving accuracy.
  • Under the paper's three assumptions, each client's loss has a bounded one-round deviation; with suitable choices of local learning rate $\alpha$ and server learning rate $\eta$, the loss decreases monotonically per round.
  • FedAPA's accuracy advantage is largest in the practical Dirichlet-based non-IID settings, which the paper says better reflect realistic heterogeneous distributions; gains are smaller and sometimes negative in pathological settings.
  • Ablations attribute most of the gain to the adaptive aggregation rule (+28.25% on CIFAR-10), with partial model sharing adding +0.92%, and normalization being the most important post-processing step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: Eq. (7) is effectively a one-step implicit personalization step; a natural test the paper does not run is whether several server-side weight updates per round, or using deltas from more local epochs, improves or destabilizes convergence.
  • Editorial extension: because the update needs only parameter deltas, the same rule could, in principle, be applied under partial participation with heterogeneous architectures if clients align their feature-extractor dimensions, though FedAPA does not claim this.
  • Editorial extension: the convergence bound contains a drift term $2L_2\eta(t+1)M_{ax}^3$ growing with round index, which suggests the guarantee relies on bounded uploaded parameters and a carefully controlled or decaying server learning rate; the fixed $\eta=0.01$ in the experiments does not directly test that dependence.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes FedAPA, a personalized federated learning method in which the server maintains per-client aggregation weight vectors and updates them using a gradient computed from the difference between each client's locally-updated parameters and its aggregated personalized parameters. The method is presented as a server-side adaptive aggregation strategy that avoids client-side weight learning, claims a theoretical convergence guarantee, and reports accuracy gains over ten PFL baselines on FMNIST, CIFAR-10, and CIFAR-100 under two non-IID settings.

Significance. If the algorithm and convergence proof were correct, server-side gradient-based adaptation of aggregation weights would be a useful and efficient addition to the PFL toolbox, and the paper's emphasis on low computation and communication overhead is well motivated. However, the central gradient update in Eq. (7) has a sign error that reverses the stated optimization objective, and the convergence theorem contains a positive error term that grows with the round index, invalidating the claimed guarantee. The empirical comparison is further weakened by test-set tuning of the self-weight hyperparameter. The paper does not provide code or reproducible experiments, so the reported results cannot independently confirm which update direction was actually implemented. Given these load-bearing issues, the paper's main claims are not substantiated as written.

major comments (4)
  1. [Section 3.2, Eq. (7)] The update A_i ← A_i − η(∇_{A_i}¯θ_i)^T Δθ_i is gradient ascent, not descent, on the stated proxy loss. Since ¯θ_i = Θ A_i, the gradient of 1/2||θ_i − ¯θ_i||^2 with respect to A_i is −Θ^T(θ_i − ¯θ_i) = −Θ^T Δθ_i, so gradient descent would require A_i ← A_i + ηΘ^TΔθ_i. The text's own chain-rule argument also implies the opposite sign: if (∇_{A_i}¯θ_i)^T Δθ_i ≈ −∇_{A_i}L_i, then subtracting it increases L_i. This sign error negates the paper's central algorithmic claim that Eq. (7) minimizes each client's local loss.
  2. [Section 4, Theorem 1 (Eq. 13)] The bound contains the positive term 2L2η(t+1)Max^3, which grows linearly with the communication round t. A standard telescoping argument over rounds cannot yield convergence when the positive error term diverges. The sufficient conditions in Corollary 1 depend on t, and Eq. (29) requires η to be smaller than a quantity that shrinks with t; for any fixed η, the condition must fail for sufficiently large t. Thus the claimed convergence guarantee is not established.
  3. [Appendix, Lemma 3 step (d)] The proof uses ||Θ^{(t+1)}|| ≤ Max, but Θ is a d×M matrix whose column norms are bounded by Max; its spectral norm can be as large as √M Max. The inequality is invalid, and while replacing Max by √M Max would alter the constant, the fundamental O(t) growth of the error term would remain and still prevents convergence.
  4. [Section 5.6, Table 4] The self-weight μ is selected by sweeping over test accuracy on CIFAR-10, and the reported FedAPA result corresponds to the value with highest test accuracy (μ = 0.5). The paper does not describe a validation split, so this constitutes test-set tuning. This biases the comparison against the ten baselines, whose hyperparameters were not tuned on the same test set, and undermines the claim of superior accuracy in Table 1.
minor comments (4)
  1. [Section 1] The phrase 'novel personaliezed model based PFL approach' contains a typo: 'personaliezed' should be 'personalized'.
  2. [Section 3.2] The text refers to 'as detailed in subsubsection 3.2' but Section 3.2 is not a subsubsection; this cross-reference should be corrected.
  3. [Table 2] The column header 'Total Time (Practice)' appears to be a typo for 'Practical'.
  4. [Appendix A] The notation tE + 1/2 for the aggregation step is nonstandard and should be defined more clearly for readers unfamiliar with half-integer local iteration indices.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FedAPA's aggregation update and convergence proof are self-contained; the Eq. (7) sign concern is a correctness issue, not a circularity.

full rationale

I examined the derivation chain. Eq. (5) defines the personalized aggregate as a weighted sum, Eq. (6) defines the parameter delta, and Eq. (7) updates the aggregation weights using that delta. The paper justifies Eq. (7) by interpreting Delta_theta_i as a surrogate negative gradient of the local loss and applying the chain rule. This is an approximation relating the update to the local objective, not an identity that makes the claimed accuracy or convergence true by construction. The convergence proof in Lemma 2, Lemma 3, and Theorem 1 uses Eq. (7) algebraically through triangle-inequality bounds and does not assume that the proxy loss equals the local loss; the proof's conclusion is therefore not an input to its premises. No load-bearing self-citation appears in the manuscript: all cited competitors are external works, and the FedAPA contribution is not justified by an author-overlapping prior theorem. The only selection-on-evaluation issue is the self-weight mu sweep in Section 5.6, which tunes a hyperparameter on the benchmark and then reports that same benchmark in Table 1; this is a methodological leakage concern, not a circular derivation. The reviewer-flagged sign error in Eq. (7), where the stated update appears to be gradient ascent on the proxy loss, is a correctness or bug concern rather than a circularity, because the equation is not true by definition of the claimed result. Therefore I find no circular step under the requested definitions and assign score 0.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The algorithm itself introduces no new physical entities or external datasets. It rests on assumptions about local loss smoothness, the proxy interpretation of the parameter delta, boundedness of parameters, and the omission of post-processing from the proof. The free parameters are the server weight learning rate and the self-weight hyperparameter, with μ tuned on the evaluation benchmark.

free parameters (2)
  • self-weight μ = 0.5
    Selected by sweeping μ over {0, 0.3, 0.5, 0.7, 1} on CIFAR-10 practical non-IID (Table 4). The chosen value is then fixed for all experiments, and the headline CIFAR-10 result uses this tuned value.
  • server aggregation learning rate η = 0.01
    Set without sensitivity analysis or a decay schedule. The convergence conditions in Corollary 1 depend on η, but no validation of this choice is given.
assumptions (6)
  • domain assumption Assumption 1: local objectives are L1-Lipschitz smooth in the model parameters.
    Used in Lemma 1's quadratic bound (Appendix Eq. 16); typical but unverified for the LeNet-5 and cross-entropy combination used in experiments.
  • domain assumption Assumption 2: stochastic gradients are unbiased with bounded variance σ².
    Standard in SGD analysis; invoked in Lemma 1 to telescope local update steps.
  • domain assumption Assumption 3: local loss is L2-Lipschitz continuous with respect to the shared parameters θ.
    Invoked in Lemma 3 to bound the loss change after server-side aggregation.
  • ad hoc to paper θi after E local epochs approximates the local optimum θi^*.
    This is the proxy-loss justification for Eq. (7). With only E=2 local epochs and learning rate 0.01, no such proximity is established.
  • ad hoc to paper Client parameters remain bounded, so Max = max_{i,t} ||θi|| is finite.
    The proofs of Lemma 2 and Theorem 1 rely on Max; no boundedness assumption or weight-decay argument is provided.
  • ad hoc to paper The clipping, self-weight adjustment, and normalization post-processing steps do not invalidate the weight-update analysis.
    Theorem 1 and Lemma 2 analyze Eq. (7) without modeling these nonlinear post-processing steps, yet the deployed algorithm always applies them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data." pith.science (2026). https://pith.science/paper/ODH2FXCD

@misc{pith2026250207456,
  author       = {Pith},
  title        = {Pith review of: FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODH2FXCD}},
  note         = {Machine review of arXiv:2502.07456}
}
read the original abstract

Personalized federated learning (PFL) tailors models to clients' unique data distributions while preserving privacy. However, existing aggregation-weight-based PFL methods often struggle with heterogeneous data, facing challenges in accuracy, computational efficiency, and communication overhead. We propose FedAPA, a novel PFL method featuring a server-side, gradient-based adaptive aggregation strategy to generate personalized models, by updating aggregation weights based on gradients of client-parameter changes with respect to the aggregation weights in a centralized manner. FedAPA guarantees theoretical convergence and achieves superior accuracy and computational efficiency compared to 10 PFL competitors across three datasets, with competitive communication overhead.

Figures

Figures reproduced from arXiv: 2502.07456 by the authors.

Figure 1
Figure 1. Framework of FedAPA. ① The server generates person￾alized model ¯θi via aggregation according to weight vector Ai; ② Each client downloads its model ¯θi; ③ Local training on private data; ④ Each client upload its model θi; ⑤ The server computes the up￾date of model parameters △θi; ⑥ Updates Ai via gradient descent according to △θi; ⑦ Post-processes Ai via clipping, self-weight set￾ting, and normalization. Other PFL … view at source ↗
Figure 2
Figure 2. Convergence curves of FedAPA and 11 compared methods [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages

  1. [1]

    Dai, and Quoc V

    David Ha, Andrew M. Dai, and Quoc V. Le. Hypernetworks. CoRR , abs/1609.09106, 2016

    arXiv 2016

  2. [2]

    Personalized cross-silo federated learning on non-iid data

    Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, and Yong Zhang. Personalized cross-silo federated learning on non-iid data. Proceedings of the AAAI Conference on Artificial Intelligence , 35(9):7865--7873, May 2021

    work page 2021

  3. [3]

    Model-contrastive federated learning

    Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10713--10722, June 2021

    work page 2021

  4. [4]

    Fedphp: Federated personalization with inherited private models

    Xin-Chun Li, De-Chuan Zhan, Yunfeng Shao, Bingshuai Li, and Shaoming Song. Fedphp: Federated personalization with inherited private models. In Nuria Oliver, Fernando P \'e rez-Cruz, Stefan Kramer, Jesse Read, and Jose A. Lozano, editors, Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Research Track ,...

    work page 2021

  5. [5]

    Adapt to adaptation: Learning personalization for cross-silo federated learning

    Jun Luo and Shandong Wu. Adapt to adaptation: Learning personalization for cross-silo federated learning. In Lud De Raedt, editor, Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22 , pages 2166--2173. International Joint Conferences on Artificial Intelligence Organization, 7 2022. Main Track

    work page 2022

  6. [6]

    Layer-wised model aggregation for personalized federated learning

    Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Layer-wised model aggregation for personalized federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10092--10101, June 2022

    work page 2022

  7. [7]

    Communication-Efficient Learning of Deep Networks from Decentralized Data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data . In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics , volume 54 of Proceedings of Machine Learning Research , pages 1...

    work page 2017

  8. [8]

    Personalized federated learning using hypernetworks

    Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik. Personalized federated learning using hypernetworks. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 9489--9502. PMLR, 18--24 Jul 2021

    work page 2021

Show all 17 references
  1. [9]

    Fedproto: Federated prototype learning across heterogeneous clients

    Yue Tan, Guodong Long, LU LIU, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. Proceedings of the AAAI Conference on Artificial Intelligence , 36(8):8432--8440, Jun. 2022

    2022

  2. [10]

    Personalized federated learning for intelligent iot applications: A cloud-edge based framework

    Qiong Wu, Kaiwen He, and Xu Chen. Personalized federated learning for intelligent iot applications: A cloud-edge based framework. IEEE Open Journal of the Computer Society , 1:35--44, 2020

    2020

  3. [11]

    Personalized federated learning with feature alignment and classifier collaboration

    Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with feature alignment and classifier collaboration. arXiv preprint arXiv:2306.11867 , 2023

    2023 arXiv

  4. [12]

    Federated machine learning: Concept and applications

    Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. , 10(2), jan 2019

    2019

  5. [13]

    Fedgh: Heterogeneous federated learning with generalized global header

    Liping Yi, Gang Wang, Xiaoguang Liu, Zhuan Shi, and Han Yu. Fedgh: Heterogeneous federated learning with generalized global header. In Proceedings of the 31st ACM International Conference on Multimedia , MM '23, page 8686–8696, New York, NY, USA, 2023. Association for Computin...

    2023

  6. [14]

    B ayesian nonparametric federated learning of neural networks

    Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. B ayesian nonparametric federated learning of neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on ...

    2019

  7. [15]

    Fedala: Adaptive local aggregation for personalized federated learning

    Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. Proceedings of the AAAI Conference on Artificial Intelligence , 37(9):11237--11244, Jun. 2023

    2023

  8. [16]

    Eliminating domain bias for federated learning in representation space

    Jianqing Zhang, Yang Hua, Jian Cao, Hao Wang, Tao Song, Zhengui XUE, Ruhui Ma, and Haibing Guan. Eliminating domain bias for federated learning in representation space. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Informat...

    2024

  9. [17]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.