Pith. sign in

REVIEW 4 major objections 4 minor 84 references

FedAPM: Federated Learning via ADMM with Partial Model Personalization

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Partial model personalization can worsen client drift; FedAPM's ADMM update with explicit dual estimates removes the drift and converges globally from any starting point.

desk verdict A useful ADMM-based personalized FL algorithm with extensive KŁ theory, but the convergence guarantee rests on a coercivity assumption the experimental softmax models violate, and Theorem 1 as stated is not the theorem proved. read the letter →

arxiv 2506.04672 v1 pith:JDQCJRE5 submitted 2025-06-05 cs.LG

Shengkun Zhu , Feiteng Nie , Jinshan Zeng , Sheng Wang , Yuan Sun , Yuan Yao , Shangfeng Chen , Quanqing Xu
show 1 more author
Chuanhui Yang
This is my paper
classification cs.LG
keywords federatedlearningpartialmodelpersonalizationADMMclientdriftglobalconvergenceKurdyka-Łojasiewiczdataheterogeneitymultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning that lets each device keep part of the model private and shared ('partial personalization') has a known failure mode: the local copy of the shared part drifts away from the server's model. This paper claims that personalization can make that drift worse, not better, and proposes FedAPM, an ADMM-based algorithm that adds a dual variable per client to correct the drift. The paper proves that FedAPM converges from any starting point to a stationary point of the nonconvex objective, with constant, linear, or sublinear rates depending on the Kurdyka-Łojasiewicz exponent, under assumptions it argues are weaker than those used for existing partial-personalization methods. On four heterogeneous and multimodal datasets FedAPM reports average gains over the best baselines of 12.3% accuracy, 16.4% F1, and 18.0% AUC with fewer communication rounds. If correct, this makes partial personalization practical for non-i.i.d. and multimodal federated learning without sacrificing convergence guarantees.

What carries the argument

The engine is the augmented Lagrangian $L_i(\boldsymbol{v}_i,\boldsymbol{u}_i,\boldsymbol{\pi}_i,\boldsymbol{u}) = \alpha_i f_i(\boldsymbol{v}_i,\boldsymbol{u}_i)+\langle \boldsymbol{\pi}_i, \boldsymbol{u}_i-\boldsymbol{u}\rangle + \frac{\rho}{2}\|\boldsymbol{u}_i-\boldsymbol{u}\|^2$, with a proximal term $\frac{\sigma_i}{2}\|\boldsymbol{v}_i-\boldsymbol{v}_i^t\|^2$ added for the personalized block and an SGD-based approximate minimizer for the $\boldsymbol{u}_i$ block whose accuracy $\xi_i^{t+1}$ decays geometrically. The dual update $\boldsymbol{\pi}_i^{t+1} = \boldsymbol{\pi}_i^t + \rho(\boldsymbol{u}_i^{t+1}-\boldsymbol{u}^t)$ and the server aggregation $\boldsymbol{u}^{t+1} = \frac{1}{m}\sum_i(\boldsymbol{u}_i^{t+1}+\boldsymbol{\pi}_i^{t+1}/\rho)$ are what make the correction exact. The convergence argument tracks a Lyapunov function $\tilde{L} = L + \sum_i \frac{29}{\rho(1-\mu_i)}\xi_i^t$, which absorbs the approximation errors, and applies the Kurdyka-Łojasiewicz framework with desingularizing function $\phi(x) = \frac{\sqrt{c}}{1-\theta}x^{1-\theta}$ to get finite, linear, or sublinear rates depending on $\theta$.

What would settle it

Take a ReLU network with homogeneous activations, so scaling the weights by a factor $c>0$ leaves the predictions unchanged, and run FedAPM with the paper's prescribed $\rho$ and $\sigma$ settings. If the local-shared distance $\|\boldsymbol{u}_i-\boldsymbol{u}\|$ or the loss fails to converge to a stationary value, then the coercivity assumption (Assumption 3, Section 5.1) that underlies Lemma 12 and Theorem 2 is not satisfied by those models. A violation would not necessarily refute the empirical gains, but it would show that the formal convergence theorem does not apply to the architectures used in the experiments.

Watch

Extended reading notes

Core claim

The central claim is that client drift in partial-model personalization is not an unavoidable side effect but an artifact of treating the problem as an inexact penalty method. FedAPM instead solves $\min \sum_{i=1}^m \alpha_i f_i(\boldsymbol{v}_i,\boldsymbol{u}_i)$ with consensus constraints $\boldsymbol{u}_i = \boldsymbol{u}$ via ADMM: each client keeps a personalized part $\boldsymbol{v}_i$ and a local copy $\boldsymbol{u}_i$ of the shared model, updates a dual variable $\boldsymbol{\pi}_i$, and uploads $\boldsymbol{z}_i = \boldsymbol{u}_i + \boldsymbol{\pi}_i/\rho$; the server sets the next shared model to the average of the $\boldsymbol{z}_i$. The second-order proximal term $\frac{\rho}{2}\|\boldsymbol{u}_i-\boldsymbol{u}\|^2$ gives a fixed pull toward the shared model, while the first-order dual term $\langle \boldsymbol{\pi}_i, \boldsymbol{u}_i-\boldsymbol{u}\rangle$ supplies a compensating pull that the paper says prevents the local objective from dragging the shared model away. The paper's analysis shows that FedAlt and FedSim are special cases with the dual variable and penalty set to zero, i.e., inexact penalty methods; FedAPM is an exact penalty (augmented Lagrangian) method whose explicit multiplier estimates avoid the ill-conditioning of large $\rho$. Theorems 1-3 establish boundedness, convergence to a stationary point from arbitrary initialization, and Kurdyka-Łojasiewicz-based rates.

Load-bearing premise

Each client's expected loss function is assumed to be coercive, meaning the loss must grow without bound as the model parameters grow; over-parameterized neural networks with scale-invariant or bounded outputs usually violate this, so the paper's convergence guarantee may not cover the models used in its own experiments.

Editorial extensions

If this is right

  • With FedAPM, partial personalization no longer trades away the shared model: the same training loop can serve clients with different private architectures while keeping a common representation.
  • Convergence is guaranteed from any initialization for nonconvex losses, so practitioners do not need special starting points or convex surrogates.
  • The required assumptions are milder than FedAlt/FedSim's (no bounded variance or partial gradient diversity), widening the class of federated problems with formal guarantees.
  • Communication rounds shrink: to reach a fixed loss, FedAPM needs fewer uploads, because the dual correction prevents local updates from cancelling each other.
  • The penalty parameter $\rho$ can stay small (the paper's experiments favour $\rho$ around 0.01), avoiding the ill-conditioning that makes large-penalty methods unstable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the coercivity assumption (Assumption 3) is the load-bearing premise; if it fails for the very networks used in the experiments, Theorem 2's boundedness step would not apply. A testable fix is to add weight decay or a bounded-domain projection, and check whether the stated rates persist.
  • Beyond the paper: the same dual-correction recipe could be applied to FedAvg with no personalization, introducing a per-client multiplier on the consensus constraint to reduce drift in a way that FedProx's proximal term approximates only inexactly.
  • Beyond the paper: the gap $\|\boldsymbol{u}_i-\boldsymbol{u}\|$ plotted in the paper is a cheap health metric. Monitoring it could tune $\rho$ online: if the gap plateaus above zero, increase the dual correction; if it oscillates, lower $\rho$.
  • Beyond the paper: because the rates depend on the desingularizing exponent $\theta$, measuring $\theta$ from the loss landscape of a real federated task would tell practitioners which rate (linear vs sublinear) they should expect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes FedAPM, a federated learning algorithm for partial model personalization based on ADMM. The authors introduce auxiliary local models with a consensus constraint, incorporate first- and second-order proximal terms into the augmented Lagrangian, and provide a convergence analysis in the Kurdyka-Łojasiewicz framework, claiming global convergence to stationary points with constant, linear, and sublinear rates under assumptions they describe as mild. They also report experiments on CIFAR10, CrisisMMD, KU-HAR, and Crema-D, claiming faster convergence and average improvements of 12.3% accuracy, 16.4% F1, and 18.0% AUC over state-of-the-art baselines.

Significance. The paper addresses a timely problem, client drift in partial model personalization, and the ADMM-plus-proximal formulation is a reasonable and nontrivial contribution. The analysis is extensive and follows the standard KŁ-based framework of Attouch, Bolte, and Svaiter and of Wang, Yin, and Zeng, and the authors release open-source code for the experiments. If the coercivity issue is resolved or the assumptions are aligned with the actual training objectives, FedAPM would provide a useful convergence guarantee for a practical FL framework. However, the current manuscript has a load-bearing mismatch between the stated assumptions and the proof, and the coercivity assumption is not satisfied by the networks used in the experiments, so the headline convergence claim is not yet supported for the reported setting.

major comments (4)
  1. [Section 5.2, Theorem 1(a) and Appendix A.5, Theorem 4(a)] The theorem is stated under Assumption 2 only, but the proof of boundedness explicitly invokes 'the coercive property of f_i' after Lemma 12(b). Without coercivity, an upper bound on L˜ and f does not imply boundedness of the parameters, since a bounded loss is compatible with parameter sequences escaping to infinity. This is an internal inconsistency: either Theorem 1 must also assume Assumption 3, or a boundedness proof under smoothness alone must be supplied. As written, the theorem statement is not justified by the proof.
  2. [Section 5.1, Assumption 3; Lemma 12(b); Theorem 2] The coercivity assumption (Assumption 3) is not satisfied by the models used in the experiments described in Section 6 and Appendix B.2. Those classifiers end in a softmax cross-entropy layer, and adding a constant vector c1 to the final-layer logits leaves the cross-entropy loss unchanged: f_i(v,u; b+c1) = f_i(v,u; b) for every c. Consequently, ||(v,u)|| can diverge to infinity along this direction while f_i remains constant, contradicting coercivity. Since Lemma 12(b) uses coercivity to bound (V^t,u^t), and Theorem 2 relies on that boundedness for its global convergence guarantee, the theorem does not cover the networks actually trained in the paper. The abstract's claim of global convergence 'under mild assumptions' is therefore not supported for the reported experimental regime; this limitation should be stated explicitly or the theory should be extended, for example by adding a coercive regularizer.
  3. [Section 5.3, Theorem 3 vs Appendix A.7, Theorem 6] The assumption sets are inconsistent. The main text states Theorem 3 under Assumptions 1, 2, and 3, but the appendix restatement (Theorem 6) says 'under Assumptions 1 and 2' and the proof uses the lower-boundedness of L˜ from Lemma 12(b), which depends on Assumption 3. The rate results therefore inherit the same coercivity gap as Theorem 2, and the list of required assumptions should be made consistent and accurate.
  4. [Table 3, CrisisMMD row; Abstract and Section 6.2.1] The empirical headline is overstated. In Table 3, FedAPM's top-1 accuracy on CrisisMMD (0.357 ± 0.028) is below FedAvg (0.374), FedAlt (0.364), FedSim (0.364), and FedProx (0.380). This contradicts the unqualified statements in the Abstract and Section 6.2.1 that FedAPM outperforms the other methods. The claimed 12.3% average accuracy improvement should be recomputed and reported with a clear definition, and the superiority claim should be qualified to the metrics and datasets where it actually holds.
minor comments (4)
  1. [Section 4.4] The statement that FedAPM reduces to FedAlt and FedSim when the dual variables and rho are set to zero is not well-defined, because Algorithm 1 uses z_i^{t+1} = u_i^{t+1} + (1/rho) pi_i^{t+1} and the ADMM formulation assumes rho > 0. This should be phrased as a limiting or parameter-identification statement rather than a literal reduction.
  2. [Section 5.2, Lemma 2 vs Appendix A.4, Lemma 11] The definition of the error term in Lemma 2 is inconsistent with the appendix. The main text defines tilde(Xi)^{t+1} := Xi^{t+1} - Xi^t, which is non-positive because each xi_i is non-increasing, while the proof in Appendix A.4 uses the positive quantity sum_i (xi_i^t - xi_i^{t+1}). Please make the notation uniform and correct.
  3. [Appendix A.4, Lemma 11] The lemma statement bounds dist(0, ∂L(P^t)) but the proof bounds components of ∂L˜(P^t). The notation should be aligned to avoid confusion about which Lyapunov function's subdifferential is being used.
  4. [Appendix B.2.4, Crema-D classifier] In the first Conv1D layer description, 'output channels = 80' appears before 'output channels = 32'; this is likely a typo for 'input channels = 80' and should be corrected.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the convergence analysis is a self-contained derivation from explicit assumptions, with only incidental self-citations and a non-circular correctness gap about coercivity.

full rationale

The derivation chain is self-contained: Theorem 2 is proved from Assumptions 1-3 via Lemma 1 (sufficient descent), Lemma 2 (relative error), Lemma 12 (boundedness and lower boundedness), and Proposition 1 (KŁ property), all proved in the appendix using standard external results such as Attouch-Bolte-Svaiter [6], Bolte-Daniilidis-Lewis [9], and real-algebraic geometry [8, 33, 58]. The convergence-rate results are conditional statements in the KŁ framework, not fits to the experimental data. The self-citations to [67, 73, 74, 81] cite co-authored prior work for the general ADMM/KŁ convergence framework and proximal stabilization, but the present proof does not reduce to those citations: it supplies its own sufficient-descent and relative-error estimates with the inexact accuracy term, and the KŁ framework itself is independently established in [6]. The experimental claims are benchmark comparisons after validation-set hyperparameter tuning, not predictions of fitted quantities. One genuine defect is that the statement of Theorem 1(a) assumes only Assumption 2, while its proof (Appendix A.5, Theorem 4) invokes Assumption 3, coercivity, to conclude boundedness; this is an unstated-assumption/correctness gap for the softmax/ReLU classifiers used in the experiments, but it is not a circular reduction: coercivity is not defined in terms of the algorithm's boundedness, and the theorem's conclusion does not feed back into the hypotheses used to derive it. Thus the paper exhibits no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The analysis rests on standard smoothness, coercivity, and geometric assumptions on the loss functions, plus the chosen ADMM hyperparameters. No new physical entities or model components are introduced.

free parameters (3)
  • penalty parameter rho = 0.001, 0.01, 0.02, 0.05, 0.1
    Grid-searched per dataset; controls the ADMM constraint enforcement and appears in every convergence inequality.
  • proximal coefficient sigma_i = not specified
    Required to satisfy sigma_i >= alpha_i L_v and rho <= 15 sigma_i / 8; exact per-client values are not reported.
  • accuracy decay rate mu_i = not specified
    Chosen in (0,1); drives the geometric decay of the local subproblem accuracy xi_i and enters the Lyapunov function constants.
assumptions (4)
  • domain assumption Expected losses f_i are proper lower semicontinuous, nonnegative, and either real analytic or semialgebraic (Assumption 1)
    Used to establish the Kurdyka-Lojasiewicz property in Proposition 1. Many practical neural network losses satisfy this, but non-analytic components and non-semialgebraic terms are excluded.
  • domain assumption Each f_i is continuously differentiable with Lipschitz gradients in both arguments (Assumption 2)
    Invoked in the sufficient descent and relative error lemmas. The constants L_u, L_v, L_uv, and L_vu are assumed known, which is often unrealistic in deep learning.
  • domain assumption Each expected loss f_i is coercive (Assumption 3)
    Used to prove boundedness of the sequence P^t in Lemma 12 and Theorem 2. Coercivity is not generally satisfied by over-parameterized networks with scale-invariant outputs, making this the most brittle premise.
  • domain assumption The local subproblem for u_i is solved to a xi_i-approximate stationary condition by SGD in a finite number of iterations (Definition 2)
    The algorithm assumes such approximate solutions exist and that xi_i decays geometrically, but no iteration budget or stochastic complexity analysis is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedAPM: Federated Learning via ADMM with Partial Model Personalization." pith.science (2026). https://pith.science/paper/JDQCJRE5

@misc{pith2026250604672,
  author       = {Pith},
  title        = {Pith review of: FedAPM: Federated Learning via ADMM with Partial Model Personalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JDQCJRE5}},
  note         = {Machine review of arXiv:2506.04672}
}
read the original abstract

In federated learning (FL), the assumption that datasets from different devices are independent and identically distributed (i.i.d.) often does not hold due to user differences, and the presence of various data modalities across clients makes using a single model impractical. Personalizing certain parts of the model can effectively address these issues by allowing those parts to differ across clients, while the remaining parts serve as a shared model. However, we found that partial model personalization may exacerbate client drift (each client's local model diverges from the shared model), thereby reducing the effectiveness and efficiency of FL algorithms. We propose an FL framework based on the alternating direction method of multipliers (ADMM), referred to as FedAPM, to mitigate client drift. We construct the augmented Lagrangian function by incorporating first-order and second-order proximal terms into the objective, with the second-order term providing fixed correction and the first-order term offering compensatory correction between the local and shared models. Our analysis demonstrates that FedAPM, by using explicit estimates of the Lagrange multiplier, is more stable and efficient in terms of convergence compared to other FL frameworks. We establish the global convergence of FedAPM training from arbitrary initial points to a stationary point, achieving three types of rates: constant, linear, and sublinear, under mild assumptions. We conduct experiments using four heterogeneous and multimodal datasets with different metrics to validate the performance of FedAPM. Specifically, FedAPM achieves faster and more accurate convergence, outperforming the SOTA methods with average improvements of 12.3% in test accuracy, 16.4% in F1 score, and 18.0% in AUC while requiring fewer communication rounds.

Figures

Figures reproduced from arXiv: 2506.04672 by the authors.

Figure 1
Figure 1. Client drift in FedAvg [48] and FedAlt/FedSim [52] is illustrated using two clients with three local update steps. Partial model personalization reduces the gap between 𝒖𝑖 and 𝒖 ∗ 𝑖 , but increases the gap between 𝒖𝑖 and 𝒖 ∗ . This may cause the shared model 𝒖 to drift further from 𝒖 ∗ , as 𝒖 = (𝒖1 +𝒖2)/2. Our main contribution in this paper is to address the issue that partial model personalization can exacerbate c… view at source ↗
Figure 2
Figure 2. Three examples of partial model personalization [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A running example of FedAPM. Different clients may possess heterogeneous or multimodal data. Each client updates its local parameters using local data and uploads 𝒛𝑖 to the server, which then updates and broadcasts 𝒖 to all clients. each client optimizes its own objective rather than the global one due to differing local objectives, which could hinder convergence or even lead to divergence. While increasing 𝜌 can mi… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison of training loss across various methods. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The distance between the local and the shared mod [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: illustrates the variation in the training loss of FedAPM with different numbers of selected clients, where the fraction of selected clients is {0.1, 0.2, 0.3, 0.5}. It can be observed that as the number of selected clients increases, the training loss decreases more ra…
Figure 8
Figure 8. Figure 8: Comparison of training loss across various methods. [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]
Figure 9
Figure 9. Figure 9: Convergence v.s. penalty parameter. 0 100 200 300 Communication Rounds 25 50 75 100 Training Loss CIFAR10 0 100 200 300 Communication Rounds 0.0 0.5 1.0 1.5 Training Loss CrisisMMD 0 100 200 300 Communication Rounds 0.5 1.0 1.5 2.0 Training Loss KU-HAR 0 100 200 300 Co…
Figure 10
Figure 10. Figure 10: Convergence v.s. client selection. B.5 Complete Results B.5.1 Comparison of Multiple Methods [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 71 canonical work pages

  1. [1]

    California Consumer Privacy Act

    2018. California Consumer Privacy Act. https://en.wikipedia.org/wiki/California_ Consumer_Privacy_Act

    2018

  2. [2]

    General Data Protection Regulation

    2018. General Data Protection Regulation. https://en.wikipedia.org/wiki/ General_Data_Protection_Regulation

    2018

  3. [3]

    Firoj Alam, Ferda Ofli, and Muhammad Imran. 2018. CrisisMMD: Multimodal Twitter Datasets from Natural Disasters. In ICWSM. 465–473

    2018

  4. [4]

    Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. 2019. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818 (2019)

    arXiv 2019

  5. [5]

    Hédy Attouch and Jérôme Bolte. 2009. On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116, 1-2 (2009), 5–16

    2009

  6. [6]

    Hédy Attouch, Jérôme Bolte, and Benar Fux Svaiter. 2013. Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward- backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137, 1-2 (2013), 91–129

    work page 2013

  7. [7]

    Courville, and Pascal Vincent

    Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. 2013. Representation Learning: A Review and New Perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 8 (2013), 1798–1828

    work page 2013

  8. [8]

    Jacek Bochnak, Michel Coste, and Marie-Françoise Roy. 1998. Real algebraic geometry

    work page 1998

Show all 84 references
  1. [9]

    Jérôme Bolte, Aris Daniilidis, and Adrian Lewis. 2007. The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17, 4 (2007), 1205–1223

    2007

  2. [10]

    Kallista A. Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konečný, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards Federated Learning at ...

    2019

  3. [11]

    Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122

    2011

  4. [12]

    Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečn`y, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. 2018. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097 (2018)

    2018 arXiv

  5. [13]

    Cooper, Michael K

    Houwei Cao, David G. Cooper, Michael K. Keutmann, Ruben C. Gur, Ani Nenkova, and Ragini Verma. 2014. CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset. IEEE Trans. Affect. Comput. 5, 4 (2014), 377–390

    2014

  6. [14]

    Hong-You Chen and Wei-Lun Chao. 2022. On Bridging Generic and Personalized Federated Learning for Image Classification. In ICLR

    2022

  7. [15]

    Jiayi Chen and Aidong Zhang. 2022. FedMSplit: Correlation-Adaptive Federated Multi-Task Learning across Multimodal Split Networks. In KDD. 87–96

    2022

  8. [16]

    Jiayi Chen and Aidong Zhang. 2024. FedMBridge: Bridgeable Multimodal Feder- ated Learning. In ICML

    2024

  9. [17]

    Li-Wei Chen and Alexander Rudnicky. 2023. Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition. In ICASSP. 1–5

    2023

  10. [18]

    Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP. 1724–1734

    2014

  11. [19]

    Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. 2021. Exploiting Shared Representations for Personalized Federated Learning. In ICML, Vol. 139. 2089–2099

    2021

  12. [20]

    Ana Garcia del Molino and Michael Gygli. 2018. PHD-GIFs: Personalized High- light Detection for Automatic GIF Creation. In ACM MM. 600–608

    2018

  13. [21]

    James Demmel. 1997. Applied Numerical Linear Algebra. SIAM

    1997

  14. [22]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  15. [23]

    Dinh, Nguyen Hoang Tran, and Tuan Dung Nguyen

    Canh T. Dinh, Nguyen Hoang Tran, and Tuan Dung Nguyen. 2020. Personalized Federated Learning with Moreau Envelopes. In NeurIPS. 21394–21405

    2020

  16. [24]

    Tiantian Feng, Digbalay Bose, Tuo Zhang, Rajat Hebbar, Anil Ramakrishna, Rahul Gupta, Mi Zhang, Salman Avestimehr, and Shrikanth Narayanan. 2023. FedMultimodal: A Benchmark for Multimodal Federated Learning. InKDD. 4035– 4045

    2023

  17. [25]

    Yonghai Gong, Yichuan Li, and Nikolaos M. Freris. 2022. FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity. In ICDE. 2575–2587

    2022

  18. [26]

    Ian Goodfellow. 2016. Deep learning

    2016

  19. [27]

    Filip Hanzely, Boxin Zhao, and Mladen Kolar. 2021. Personalized federated learning: A unified framework and universal optimization techniques. arXiv preprint arXiv:2102.09743 (2021)

    2021 arXiv

  20. [28]

    Andrew G Howard. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

    2017 arXiv

  21. [29]

    Peter Kairouz, Ziyu Liu, and Thomas Steinke. 2021. The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation. In ICML, Vol. 139. 5201–5212

    2021

  22. [30]

    Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista A

    Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista A. Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett,...

    2021

  23. [31]

    Heejoo Kang, Minsoo Kim, Bumsuk Lee, and Hongseok Kim. 2024. FedAND: Federated Learning Exploiting Consensus ADMM by Nulling Drift. IEEE Trans. Ind. Informatics 20, 7 (2024), 9837–9849

    2024

  24. [32]

    Reddi, Se- bastian U

    Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Se- bastian U. Stich, and Ananda Theertha Suresh. 2020. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. In ICML, Vol. 119. 5132–5143

    2020

  25. [33]

    Steven G Krantz and Harold R Parks. 2002. A primer of real analytic functions

    2002

  26. [34]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  27. [35]

    Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, and Jingren Zhou. 2024. FederatedScope- LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning. In KDD. ACM, 5260–5271

    2024

  28. [36]

    Krzysztof Kurdyka. 1998. On gradients of functions definable in o-minimal structures. In Annales de l’institut Fourier, Vol. 48. 769–783

    1998

  29. [37]

    Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton. 2015. Deep learning. Nat. 521, 7553 (2015), 436–444

    2015

  30. [38]

    Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. 2022. Federated Learning on Non-IID Data Silos: An Experimental Study. In ICDE. 965–978

    2022

  31. [39]

    Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. 2021. Ditto: Fair and Robust Federated Learning Through Personalization. In ICML, Vol. 139. 6357–6368

    2021

  32. [40]

    Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Process. Mag. 37, 3 (2020), 50–60

    2020

  33. [41]

    Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated Optimization in Heterogeneous Networks. In MLSys

    2020

  34. [42]

    Paul Pu Liang, Terrance Liu, Ziyin Liu, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020. Think Locally, Act Globally: Federated Learning with Local and Global Representations. In NeurIPS

    2020

  35. [43]

    Shiyun Lin, Yuze Han, Xiang Li, and Zhihua Zhang. 2022. Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness. InNeurIPS. 30471–30485

    2022

  36. [44]

    Yang Liu, Tao Fan, Tianjian Chen, Qian Xu, and Qiang Yang. 2021. FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection. J. Mach. Learn. Res. 22 (2021), 226:1–226:6

    2021

  37. [45]

    Stanis Lojasiewicz. 1965. Ensembles semi-analytiques. Institut des Hautes Etudes Scientifiques (1965)

    1965

  38. [46]

    Mi Luo, Fei Chen, Dapeng Hu, Yifan Zhang, Jian Liang, and Jiashi Feng. 2021. No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data. In NeurIPS. 5972–5984

    2021

  39. [47]

    Jordan, Peter Richtárik, and Martin Takác

    Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter Richtárik, and Martin Takác. 2015. Adding vs. Averaging in Distributed Primal-Dual Optimiza- tion. In ICML, Vol. 37. 1973–1982

    2015

  40. [48]

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep net- works from decentralized data. In AISTATS. 1273–1282

    2017

  41. [49]

    Tomás Mikolov and Geoffrey Zweig. 2012. Context dependent recurrent neural network language model. In SLT. 234–239

    2012

  42. [50]

    Boris S Mordukhovich. 2006. Variational analysis and generalized differentiation I: Basic Theory

    2006

  43. [51]

    Jorge Nocedal and Stephen J. Wright. 1999. Numerical Optimization. Springer

    1999

  44. [52]

    Rabbat, Maziar Sanjabi, and Lin Xiao

    Krishna Pillutla, Kshitiz Malik, Abdelrahman Mohamed, Michael G. Rabbat, Maziar Sanjabi, and Lin Xiao. 2022. Federated Learning with Partial Model Personalization. In ICML. 17716–17758

    2022

  45. [53]

    Lin, Radomír Mech, and David J

    Jian Ren, Xiaohui Shen, Zhe L. Lin, Radomír Mech, and David J. Foran. 2017. Personalized Image Aesthetics. In ICCV. 638–647

    2017

  46. [54]

    R Tyrrell Rockafellar and Roger J-B Wets. 1998. Variational analysis

    1998

  47. [55]

    Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. 2020. Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data. IEEE Trans. Neural Networks Learn. Syst. 31, 9 (2020), 3400–3413

    2020

  48. [56]

    Shai Shalev-Shwartz and Tong Zhang. 2013. Stochastic dual coordinate ascent methods for regularized loss. J. Mach. Learn. Res. 14, 1 (2013), 567–599. FedAPM: Federated Learning via ADMM with Partial Model Personalization KDD ’25, August 3–7, 2025, Toronto, ON, Canada

    2013

  49. [57]

    Shai Shalev-Shwartz and Tong Zhang. 2016. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. Math. Program. 155, 1-2 (2016), 105–145

    2016

  50. [58]

    Masahiro Shiota. 1997. Geometry of subanalytic and semialgebraic sets

    1997

  51. [59]

    Niloy Sikder and Abdullah Al Nahid. 2021. KU-HAR: An open dataset for hetero- geneous human activity recognition. Pattern Recognit. Lett. 146 (2021), 46–54

    2021

  52. [60]

    Karan Singhal, Hakim Sidahmed, Zachary Garrett, Shanshan Wu, John Rush, and Sushant Prakash. 2021. Federated Reconstruction: Partially Local Federated Learning. In NeurIPS. 11220–11232

    2021

  53. [61]

    Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. 2017. Federated Multi-Task Learning. In NeurIPS. 4424–4434

    2017

  54. [62]

    Jordan, and Martin Jaggi

    Virginia Smith, Simone Forte, Chenxin Ma, Martin Takác, Michael I. Jordan, and Martin Jaggi. 2017. CoCoA: A General Framework for Communication-Efficient Distributed Optimization. J. Mach. Learn. Res. 18 (2017), 230:1–230:49

    2017

  55. [63]

    Guangyu Sun, Matías Mendieta, Jun Luo, Shandong Wu, and Chen Chen. 2023. FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning. In ICCV. 4965–4975

    2023

  56. [64]

    Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020. Mobilebert: a compact task-agnostic bert for resource-limited devices. arXiv preprint arXiv:2004.02984 (2020)

    2020 arXiv

  57. [65]

    Chen Wang, Jialin Qiao, Xiangdong Huang, Shaoxu Song, Haonan Hou, Tian Jiang, Lei Rui, Jianmin Wang, and Jiaguang Sun. 2023. Apache IoTDB: A Time Series Database for IoT Applications. Proc. ACM Manag. Data 1, 2 (2023), 195:1– 195:27

    2023

  58. [66]

    Han Wang, Siddartha Marella, and James Anderson. 2022. FedADMM: A federated primal-dual algorithm allowing partial participation. In CDC. IEEE, 287–294

    2022

  59. [67]

    Yu Wang, Wotao Yin, and Jinshan Zeng. 2019. Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78, 1 (2019), 29–63

    2019

  60. [68]

    Liang Xiao, Xiaoyue Wan, Xiaozhen Lu, Yanyong Zhang, and Di Wu. 2018. IoT Security Techniques Based on Machine Learning: How Do IoT Devices Use AI to Enhance Security? IEEE Signal Process. Mag. 35, 5 (2018), 41–49

    2018

  61. [69]

    Yangyang Xu and Wotao Yin. 2013. A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion. SIAM J. Imaging Sci. 6, 3 (2013), 1758–1789

    2013

  62. [70]

    Piyush Yadav, Dhaval Salwala, Felipe Arruda Pontes, Praneet Dhingra, and Ed- ward Curry. 2021. Query-Driven Video Event Processing for the Internet of Multimedia Things. Proc. VLDB Endow. 14, 12 (2021), 2847–2850

    2021

  63. [71]

    Yuen, and Dacheng Tao

    Mang Ye, Xiuwen Fang, Bo Du, Pong C. Yuen, and Dacheng Tao. 2024. Hetero- geneous Federated Learning: State-of-the-art and Research Challenges. ACM Comput. Surv. 56, 3 (2024), 79:1–79:44

    2024

  64. [72]

    Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin Du, Yanfeng Wang, and Siheng Chen. 2024. OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning. InKDD. ACM, 6137–6147

    2024

  65. [73]

    Jinshan Zeng, Tim Tsz-Kit Lau, Shaobo Lin, and Yuan Yao. 2019. Global Conver- gence of Block Coordinate Descent in Deep Learning. InICML, Vol. 97. 7313–7323

    2019

  66. [74]

    Jinshan Zeng, Shao-Bo Lin, Yuan Yao, and Ding-Xuan Zhou. 2021. On ADMM in Deep Learning: Convergence and Saturation-Avoidance. J. Mach. Learn. Res. 22 (2021), 199:1–199:67

    2021

  67. [75]

    Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. 2023. FedALA: Adaptive Local Aggregation for Personalized Federated Learning. In AAAI. 11237–11244

    2023

  68. [76]

    Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. 2023. FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy. In KDD. 3249–3261

    2023

  69. [77]

    Xinwei Zhang, Mingyi Hong, Sairaj Dhople, Wotao Yin, and Yang Liu. 2021. FedPD: A federated learning framework with adaptivity to non-iid data. IEEE Trans. Signal Process. 69 (2021), 6055–6070

    2021

  70. [78]

    Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chan- dra. 2018. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018)

    2018 arXiv

  71. [79]

    Shenglong Zhou and Geoffrey Ye Li. 2023. Federated Learning Via Inexact ADMM. IEEE Trans. Pattern Anal. Mach. Intell. 45, 8 (2023), 9699–9708

    2023

  72. [80]

    Shenglong Zhou and Geoffrey Ye Li. 2023. FedGiA: An Efficient Hybrid Algorithm for Federated Learning. IEEE Trans. Signal Process. 71 (2023), 1493–1508

    2023

  73. [81]

    Shengkun Zhu, Jinshan Zeng, Sheng Wang, Yuan Sun, Xiaodong Li, Yuan Yao, and Zhiyong Peng. 2024. On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness. arXiv preprint arXiv:2407.16397 (2024). KDD ’25, August 3–7, 2025, Toronto, ON, Canada Sheng...

    2024 arXiv

  74. [82]

    We first estimate Δ𝑡

  75. [83]

    (40) where (40) follows from (8)

    According to the Lagarangian function (4), we have Δ𝑡 1 =L( 𝑽 𝑡+1, 𝑼 𝑡+1, 𝚷𝑡+1, 𝒖𝑡+1)−L( 𝑽 𝑡+1, 𝑼 𝑡+1, 𝚷𝑡+1, 𝒖𝑡) = 𝑚∑︁ 𝑖=1 𝛼𝑖𝑓𝑖(𝒗𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 )+⟨ 𝝅𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 − 𝒖𝑡+1⟩+ 𝜌 2∥𝒖𝑡+1 𝑖 − 𝒖𝑡+1∥2− 𝑚∑︁ 𝑖=1 𝛼𝑖𝑓𝑖(𝒗𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 )+⟨ 𝝅𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 − 𝒖𝑡⟩+ 𝜌 2∥𝒖𝑡+1 𝑖 − 𝒖𝑡∥2 ! (37) = ...

  76. [84]

    where (43) follows from (12), and (44) follows from Lemma 5

    Based on the Lagrangian function (4), we have Δ𝑡 2 =L( 𝑽 𝑡+1, 𝑼 𝑡+1, 𝚷𝑡+1, 𝒖𝑡)−L( 𝑽 𝑡+1, 𝑼 𝑡+1, 𝚷𝑡, 𝒖𝑡) = 𝑚∑︁ 𝑖=1 𝛼𝑖𝑓𝑖(𝒗𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 )+⟨ 𝝅𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 − 𝒖𝑡⟩+ 𝜌 2∥𝒖𝑡+1 𝑖 − 𝒖𝑡∥2− 𝑚∑︁ 𝑖=1 𝛼𝑖𝑓𝑖(𝒗𝑡+1 𝑖 , 𝒖𝑡+1 𝑖 )+⟨ 𝝅𝑡 𝑖, 𝒖𝑡+1 𝑖 − 𝒖𝑡⟩+ 𝜌 2∥𝒖𝑡+1 𝑖 − 𝒖𝑡∥2 ! (41) = 𝑚∑︁ 𝑖=1 ⟨𝝅𝑡+1 𝑖 ...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.