REVIEW 3 major objections 5 minor 1 cited by
PE-MA: Parameter-Efficient Co-Evolution of Multi-Agent Systems
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that PE-MA reaches an asymptotically optimal convergence rate of $O(1/\sqrt{NK})$ while transmitting only small adapters.
desk verdict Useful dual-adapter idea with a detailed proof attempt, but the central update in Eq. (12) breaks the doubly stochastic consensus assumption, so the advertised O(1/sqrt(NK)) rate is not proven for the algorithm as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dual-adapter module: a local personalized adapter $v_i$ that is never transmitted, and a shared adapter $w_i$ that is exchanged among neighbors using the symmetric doubly stochastic matrix $P$ ($P^\top = P$, $P\mathbf{1} = \mathbf{1}$). The argument proceeds by bounding the composite quantity $M(t) := \frac{1}{N}\sum_{i=1}^N \|w_i(t) - \bar w(t)\|^2 + \|\nabla_w L(\bar w(t),\{v_i(t+1)\})\|^2 + \frac{\alpha\tau}{\beta}\|\nabla_v L(\bar w(t),\{v_i(t)\})\|^2$, where $\bar w$ is the average shared adapter. The two-timescale learning rates ($\eta_w$ and $\eta_v$) ensure that local personalization acts as fast noise, while a consensus lemma bounds the mixing error of $P$ in terms of its smallest positive entry. This combination converts the multi-agent objective into a standard descent-plus-noise recursion.
What would settle it
Compute $(1/N)\sum_i \|w_i(t) - (1/N)\sum_j w_j(t)\|^2$ over training with the update (12) on a ring topology; convergence of this quantity to zero at the rate predicted by Lemma A.8 would confirm the proof, while a non-vanishing plateau would falsify it.
Extended reading notes
Core claim
The paper claims that PE-MA, with learning rates $\eta_w = \sqrt{N/K}$ and $\eta_v = 1/(\tau \sqrt{K})$, satisfies a bound on the averaged composite $M(t)$—the per-agent consensus error of the shared adapters plus the squared norms of the global partial gradients for $w$ and $v$—that yields a convergence rate of $O(1/\sqrt{NK} + 1/(K\sqrt{N}) + 1/(\tau\sqrt{NK}))$, which it calls asymptotically optimal. The proof (Theorem V.5 and Corollary V.7) treats the personalized adapters as a two-timescale disturbance whose contribution is absorbed by smaller learning rates, and it controls the shared-adapter consensus error via a doubly stochastic mixing matrix $P$. The experiments aim to demonstrate the same trade-off in practice: PE-MA outperforms full-model independent training and Decentralized SGD variants that share input, output, or adapter layers, under fully connected, Erdős–Rényi, and ring topologies.
Load-bearing premise
The proof assumes the shared-adapter update is the symmetric doubly stochastic consensus $w_i(t+1)=\sum_j P_{ij} w_j(t+1/2)$ with $P$ symmetric and $P\mathbf{1}=\mathbf{1}$, while the algorithm text in Eq. (12) writes $w_i(t+1)=(1/|\mathcal{N}(i)|)\sum_j P_{ij} w_j(t+1/2)$, an extra normalization the analysis never addresses, and the rate collapses if that discrepancy is resolved in favor of Eq. (12).
Editorial extensions
If this is right
- PE-MA converges at $O(1/\sqrt{NK})$, the same leading rate as decentralized SGD, so the communication bottleneck does not come at the cost of asymptotic performance.
- The shared adapter alone is sufficient for global coordination: only $w_i$ is averaged, and the personalized adapters $v_i$ remain local, so privacy-sensitive personalization is preserved by construction.
- The rate holds under any connected communication topology for which a symmetric doubly stochastic matrix $P$ exists, including fully connected, Erdős–Rényi, and ring graphs.
- Experimentally, PE-MA improves accuracy by 2–5% over baselines while reducing communication parameters by roughly 77–87% and trainable parameters by over 70%.
- The two-timescale learning-rate schedule ($\eta_w=\sqrt{N/K}$, $\eta_v=1/(\tau\sqrt{K})$) is a concrete recipe practitioners can adopt directly.
Reading between the lines
- The unresolved mismatch between Eq. (12)'s $1/|\mathcal{N}(i)|$ factor and the proof's row-stochastic update suggests a direct test: re-running the analysis for a row-normalized consensus matrix would either produce a different rate or reveal that the implementation must drop the factor to match the theorem.
- The same dual-adapter split should transfer to other parameter-efficient modules (LoRA, prefix prompts, visual-prompt tokens), because the convergence argument relies only on Lipschitz gradients and bounded variance, not on the adapter's internal structure.
- The fixed mixing coefficient $\mu$ could become a per-agent adaptive variable; the ablation results on data-poor agents hint that $\mu$ should decrease when local data is scarce, which the current theorem does not capture.
- If extended to large language models, the framework would let a swarm of agents co-fine-tune shared adapters over a frozen backbone with communication savings of the same order, though non-convexity and transformer-specific gradient noise would need empirical confirmation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PE-MA, a decentralized multi-agent learning framework with a dual-adapter architecture: each agent trains a localized personalized adapter and a shared adapter that is exchanged with neighbors. The central theoretical claim is an asymptotically optimal convergence rate of O(1/sqrt(NK)) under Lipschitz, unbiased-gradient, bounded-variance, and bounded-heterogeneity assumptions. The paper also reports experiments on Office-Home, Office-Caltech10, and DomainNet showing accuracy gains and reduced communication/training parameter counts compared with several decentralized SGD baselines.
Significance. If the convergence-rate claim is correct, the paper makes a useful contribution to parameter-efficient decentralized learning: it shows that a communication-efficient dual-adapter scheme can match the standard decentralized SGD rate. The empirical study is reasonable and the reported communication savings are substantial. However, the theoretical result is the paper's main claim, and the proof currently applies to a different update rule than the one stated in Section IV, so the contribution is not yet established as written.
major comments (3)
- [§IV, Eq. (12) vs. Appendix A, Lemmas A.3/A.8] Equation (12) defines the communication update as w_i(t+1) = (1/|N(i)|) Σ_{j∈N(i)} P_ij w_j(t+1/2), which corresponds to the operator Q = D^{-1}P. On a general graph this Q is not doubly stochastic even when P is symmetric and P1=1. The convergence proof, however, analyzes the unnormalized update w_i(t+1)=Σ_j P_ij w_j(t+1/2): Lemma A.3(a1) relies on the identity ¯w(t+1)−¯w(t) = −(η_w/N)Σ_i g_w(wi(t), vi(t+1)), which holds only when the consensus matrix preserves the average; Lemma A.7 bounds |1/N − P^k(i,j)| under the assumption that P is doubly stochastic; and Lemma A.8's entire consensus-error bound is built on those two facts. With Eq. (12), the average is not preserved, so the identity in Lemma A.3(a1) fails and Lemma A.7 cannot be applied to Q^k. Algorithm 1 line 20 and Section III Eq. (1) use the unnormalized update, so the paper contains two inconsistent definitions of the same step. As submitted, the theorem is not proven for the algorithm described in Section IV.
- [§V, Theorem V.5] Theorem V.5 states learning-rate constraints in terms of α and β, but neither α nor β is defined anywhere, and the displayed bound in Eq. (13) is written in terms of η_w and η_v. Corollary V.7 silently switches to η_v = 1/(τ√K) and η_w = √(N/K). The appendix proof uses conditions such as max(Lη_w, η_vτL(1+36τ²)) ≤ 1 and η_w ≤ min(1/L, N L²/(2L²+2), (1−q)/(3√2 C L N)), none of which are related to the α and β inequalities in the theorem statement. The theorem's hypotheses must be restated with consistent parameters that match the proof.
- [Appendix A, Lemma A.8 (Eq. (32)) and Eq. (34)] The proof assumes away the initial consensus error: step (e2) says 'W.l.o.g., we assume that the initial term wi(0), ∀i is small enough and can be neglected,' and Eq. (34) later uses Σ_i E∥wi(0)−¯w(0)∥² = 0 as if it were an established fact. This is not a harmless 'without loss of generality' unless the initialization guarantees all wi(0) are identical. If Algorithm 1's single initial adapter w0 indeed means all agents start from the same w0, the term is exactly zero and this should be stated explicitly; otherwise the bound in Lemma A.8 and the final rate in Theorem V.5 acquire an extra term that is not accounted for. The paper needs to state the initialization assumption precisely and verify it against Algorithm 1.
minor comments (5)
- [§I and §V (Corollary V.7)] The abstract and conclusion state the rate as O(1/√(NK)), while Corollary V.7 gives O(1/√(NK) + 1/(K√N) + 1/(τ√(NK))). Please state explicitly the conditions under which the latter two terms are dominated by the first, and avoid the unqualified label 'asymptotically optimal' unless a matching lower bound is provided or cited.
- [Appendix A, Lemma A.5] The bound in Eq. (25), with coefficients (18τ²−15τ−3)σ² and (18τ²−18τ)E∥gv(·)∥², is obtained by invoking 'Lemma 3 in [38]' without stating the lemma or its hypotheses. Since the final rate depends on these constants, the lemma statement or a short derivation should be included in the appendix.
- [§IV, Eq. (12)] Please reconcile Eq. (12) with Algorithm 1 line 20 and Section III Eq. (1). If the intended update is the unnormalized one, remove the 1/|N(i)| factor from Eq. (12); if the normalized update is intended, the convergence analysis must be reworked for a row-stochastic consensus matrix.
- [Table I] The abstract's claim of '2%–5%' accuracy improvement is stronger than the table shows; for example, on Office-Caltech10 under the ER topology PE-MA achieves 97.68% versus 97.18% for DSGD_SIM (Input layer), an improvement of about 0.5%. Please qualify the reported gains or provide per-dataset ranges.
- [Throughout] The rendered text contains several garbled formulas (e.g., 'O( 1√ N K)' in the abstract) and inconsistent capitalization in the index terms ('dual-Adapters'). A thorough pass to fix LaTeX and typography is needed.
Circularity Check
No circularity: the convergence bound is derived from stated assumptions using standard external lemmas; the proof does not assume its target rate, and no fitted quantity is relabeled as a prediction.
full rationale
The paper's central theoretical claim, the O(1/sqrt(NK)) convergence rate of PE-MA, is derived rather than assumed. Theorem V.5 is proved from Assumptions V.1 through V.4 using Lemmas A.1 through A.8, each of which is established in the appendix from standard smoothness, unbiasedness, bounded-variance, and bounded-heterogeneity inequalities. The proof invokes external results, notably Lemma A.7 from Nedic and Ozdaglar [39] and Lemma 3 from Reddi et al. [38], which do not depend on the present paper and do not presuppose the target convergence rate. Corollary V.7 obtains the claimed rate by substituting explicit learning rates (eta_v = 1/(tau sqrt(K)), eta_w = sqrt(N/K)) into the general bound in Theorem V.5; the rate is a consequence of the bound, not an input to it. No fitted parameter is later presented as a prediction. The experimental mixing coefficient mu is tuned on validation data, but this is ordinary hyperparameter selection rather than logical circularity. The paper contains no self-citations that carry any load-bearing argument. There is a notable correctness gap: Eq. (12) includes a 1/|N(i)| normalization that is not present in Algorithm 1 or in the consensus proof, so the update analyzed in Lemma A.8 is not exactly the update as written in Section IV. This is a rigor or consistency problem, not a circularity problem, because the missing reconciliation does not make the proof equivalent to its inputs. Accordingly, no circular steps are identified and the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Mixing coefficient mu =
not reported for final runs; swept in ablation
- Learning rates eta_w and eta_v =
initial 0.01 with per-3-epoch decay by 0.1 in experiments
- Local update epochs tau =
not stated for experiments
- Adapter rank and bottleneck dimension =
not reported
assumptions (6)
- domain assumption L-Lipschitz continuous gradients (Assumption V.1)
- domain assumption Unbiased local gradient estimators (Assumption V.2)
- domain assumption Bounded gradient variance (Assumption V.3)
- domain assumption Bounded global variability of shared-adapter gradients (Assumption V.4)
- domain assumption P is symmetric doubly stochastic with P^T = P and P1 = 1
- ad hoc to paper Initial consensus error is zero
Cite this review
Pith. "Pith review of PE-MA: Parameter-Efficient Co-Evolution of Multi-Agent Systems." pith.science (2026). https://pith.science/paper/XYDWOHJ4
@misc{pith2026250611803,
author = {Pith},
title = {Pith review of: PE-MA: Parameter-Efficient Co-Evolution of Multi-Agent Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/XYDWOHJ4}},
note = {Machine review of arXiv:2506.11803}
}
read the original abstract
Multi-Agent Systems have recently emerged as a promising paradigm for collaborative reasoning and solving complex tasks. However, the design of collaborative learning algorithms in multi-agent systems faces several challenges, including high communication overhead and insufficient agent-level personalization. In this paper, we propose PE-MA (Parameter-Efficient Multi-Agent Co-Evolution), a novel collaboration framework that supports efficient, scalable, and personalized co-evolution in multi-agent systems. In PE-MA, each agent maintains a lightweight personalized adapter to support agent-specific behavior, while a shared adapter is collaboratively optimized across neighboring agents. This design balances global coordination with local adaptation under heterogeneous environments. We achieve an asymptotically optimal convergence rate of O( 1/(NK)^(1/2) ), where N is the number of agents and K the local update steps.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
In-the-flow RL of a planner in a modular tool-using agent system yields large gains, allowing a 7B model to outperform GPT-4o across several reasoning benchmarks.
Reference graph
Works this paper leans on
-
[1]
Generative agents: Interactive simulacra of human behavior,
J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” in Proceedings of the 36th annual acm symposium on user interface software and technology, 2023, pp. 1–22
2023
-
[2]
Metagpt: Meta programming for multi-agent col- laborative framework,
S. Hong, X. Zheng, J. Chen, Y . Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al., “Metagpt: Meta programming for multi-agent col- laborative framework,” arXiv preprint arXiv:2308.00352, vol. 3, no. 4, p. 6, 2023
arXiv 2023
-
[3]
V oyager: An open- ended embodied agent with large language models,
G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open- ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291 , 2023
arXiv 2023
-
[4]
Coordinated multi-robot exploration,
W. Burgard, M. Moors, C. Stachniss, and F. E. Schneider, “Coordinated multi-robot exploration,” IEEE Transac- tions on robotics , vol. 21, no. 3, pp. 376–386, 2005
work page 2005
-
[5]
Inner monologue: Embodied reasoning through planning with language models,
W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Flo- rence, A. Zeng, J. Tompson, I. Mordatch, Y . Cheb- otar et al. , “Inner monologue: Embodied reasoning through planning with language models,” arXiv preprint arXiv:2207.05608, 2022
arXiv 2022
-
[6]
Lamm: Language- assisted multi-modal instruction-tuning dataset, frame- work, and benchmark,
Z. Yin, J. Wang, J. Cao, Z. Shi, D. Liu, M. Li, X. Huang, Z. Wang, L. Sheng, L. Bai et al. , “Lamm: Language- assisted multi-modal instruction-tuning dataset, frame- work, and benchmark,” Advances in Neural Information Processing Systems, vol. 36, pp. 26 650–26 685, 2023
work page 2023
-
[7]
Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,
Y . Shen, K. Song, X. Tan, D. Li, W. Lu, and Y . Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,” Advances in Neural Information Processing Systems, vol. 36, pp. 38 154–38 180, 2023
work page 2023
-
[8]
Llm multi-agent systems: Challenges and open problems,
S. Han, Q. Zhang, Y . Yao, W. Jin, Z. Xu, and C. He, “Llm multi-agent systems: Challenges and open problems,” arXiv preprint arXiv:2402.03578 , 2024
arXiv 2024
Show all 40 references
-
[9]
Large language models empowered agent- based modeling and simulation: A survey and perspec- 10 tives,
C. Gao, X. Lan, N. Li, Y . Yuan, J. Ding, Z. Zhou, F. Xu, and Y . Li, “Large language models empowered agent- based modeling and simulation: A survey and perspec- 10 tives,” Humanities and Social Sciences Communications , vol. 11, no. 1, pp. 1–24, 2024
2024
-
[10]
Learning multiagent communication with backpropagation,
S. Sukhbaatar, R. Fergus et al. , “Learning multiagent communication with backpropagation,” Advances in neu- ral information processing systems , vol. 29, 2016
2016
-
[11]
Learning to communicate with deep multi-agent reinforcement learning,
J. Foerster, I. A. Assael, N. De Freitas, and S. White- son, “Learning to communicate with deep multi-agent reinforcement learning,” Advances in neural information processing systems, vol. 29, 2016
2016
-
[12]
Tarmac: Targeted multi-agent communication,
A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, and J. Pineau, “Tarmac: Targeted multi-agent communication,” in International Conference on machine learning. PMLR, 2019, pp. 1538–1546
2019
-
[13]
Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication,
R. Xu, H. Xiang, X. Xia, X. Han, J. Li, and J. Ma, “Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication,” in 2022 International Conference on Robotics and Au- tomation (ICRA). IEEE, 2022, pp. 2583–2589
2022
-
[14]
Learning distilled collaboration graph for multi-agent perception,
Y . Li, S. Ren, P. Wu, S. Chen, C. Feng, and W. Zhang, “Learning distilled collaboration graph for multi-agent perception,” Advances in Neural Information Processing Systems, vol. 34, pp. 29 541–29 552, 2021
2021
-
[15]
Mkd-cooper: cooperative 3d object detection for autonomous driving via multi-teacher knowledge distillation,
Z. Li, H. Liang, H. Wang, M. Zhao, J. Wang, and X. Zheng, “Mkd-cooper: cooperative 3d object detection for autonomous driving via multi-teacher knowledge distillation,” IEEE Transactions on Intelligent Vehicles , vol. 9, no. 1, pp. 1490–1500, 2023
2023
-
[16]
Lora: Low-rank adaptation of large language models,
E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations (ICLR) , 2021
2021
-
[17]
Parameter-efficient transfer learning for nlp,
N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Mor- rone, Q. de Laroussilhe, A. Gesmundo, A. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning (ICML), 2019, pp. 2790–2799
2019
-
[18]
Emp: Edge-assisted multi-vehicle per- ception,
X. Zhang, A. Zhang, J. Sun, X. Zhu, Y . E. Guo, F. Qian, and Z. M. Mao, “Emp: Edge-assisted multi-vehicle per- ception,” in Proceedings of the 27th Annual International Conference on Mobile Computing and Networking, 2021, pp. 545–558
2021
-
[19]
Edgecooper: Network-aware cooperative li- dar perception for enhanced vehicular awareness,
G. Luo, C. Shao, N. Cheng, H. Zhou, H. Zhang, Q. Yuan, and J. Li, “Edgecooper: Network-aware cooperative li- dar perception for enhanced vehicular awareness,” IEEE Journal on Selected Areas in Communications , vol. 42, no. 1, pp. 207–222, 2023
2023
-
[20]
Cooperative perception and localization for cooperative driving,
A. Miller, K. Rim, P. Chopra, P. Kelkar, and M. Likhachev, “Cooperative perception and localization for cooperative driving,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2020, pp. 1256–1262
2020
-
[21]
Prefix-tuning: Optimizing continuous prompts for generation,
X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190, 2021
2021 arXiv
-
[22]
The power of scale for parameter-efficient prompt tuning,
B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Pro- ceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2021, pp. 3045–3059
2021
-
[23]
Learning to prompt for vision-language models,
K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision-language models,” in International Journal of Computer Vision (IJCV) , vol. 130, 2022, pp. 2337–2348
2022
-
[24]
Visual prompt tuning,
M. Jia, L. Tang, B.-C. Wang, J. Ainslie, Q. V . Le, Y . Song, Y . Cui, T. A. Nguyen, and D. Zhou, “Visual prompt tuning,” in European Conference on Computer Vision (ECCV), 2022
2022
-
[25]
Adaptive parameter-efficient fine-tuning for vision transformers,
R. Zhang, Z. Liu, X. Ma, Z. Ma, and T. Lu, “Adaptive parameter-efficient fine-tuning for vision transformers,” in Advances in Neural Information Processing Systems (NeurIPS), 2022
2022
-
[26]
Com- pacter: Efficient low-rank hypercomplex adapter layers,
R. K. Mahabadi, J. Henderson, and S. Ruder, “Com- pacter: Efficient low-rank hypercomplex adapter layers,” in Advances in Neural Information Processing Systems (NeurIPS), 2021
2021
-
[27]
V . S. Borkar and V . S. Borkar,Stochastic approximation: a dynamical systems viewpoint . Springer, 2008, vol. 9
2008
-
[28]
Deprl: Achieving linear convergence speedup in personalized decentralized learning with shared representations,
G. Xiong, G. Yan, S. Wang, and J. Li, “Deprl: Achieving linear convergence speedup in personalized decentralized learning with shared representations,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 14, 2024, pp. 16 103–16 111
2024
-
[29]
Stochastic gradient push for distributed deep learn- ing,
M. Assran, N. Loizou, N. Ballas, and M. Rabbat, “Stochastic gradient push for distributed deep learn- ing,” in International Conference on Machine Learning . PMLR, 2019, pp. 344–353
2019
-
[30]
Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,
X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[31]
Deep hashing network for unsupervised do- main adaptation,
H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Pan- chanathan, “Deep hashing network for unsupervised do- main adaptation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 5018–5027
2017
-
[32]
Geodesic flow kernel for unsupervised domain adaptation,
B. Gong, Y . Shi, F. Sha, and K. Grauman, “Geodesic flow kernel for unsupervised domain adaptation,” in 2012 IEEE conference on computer vision and pattern recognition. IEEE, 2012, pp. 2066–2073
2012
-
[33]
Moment matching for multi-source domain adaptation,
X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” in Proceedings of the IEEE/CVF interna- tional conference on computer vision , 2019, pp. 1406– 1415
2019
-
[34]
Think locally, act globally: Fed- erated learning with local and global representations,
P. P. Liang, T. Liu, L. Ziyin, R. Salakhutdinov, and L. philippe Morency, “Think locally, act globally: Fed- erated learning with local and global representations,” ArXiv, vol. abs/2001.01523, 2020
2001 arXiv
-
[35]
Exploiting shared representations for personalized fed- erated learning,
L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting shared representations for personalized fed- erated learning,” in International conference on machine learning. PMLR, 2021, pp. 2089–2099
2021
-
[36]
Automatic differentiation in pytorch,
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” inNIPS- 11 W, 2017
2017
-
[37]
Optimization methods for large-scale machine learning,
L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale machine learning,” Siam Review, vol. 60, no. 2, pp. 223–311, 2018
2018
-
[38]
Adaptive federated optimization,
S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Kone ˇcn`y, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” arXiv preprint arXiv:2003.00295, 2020
2003 arXiv
-
[39]
Distributed Subgradient Methods for Multi-Agent Optimization,
A. Nedic and A. Ozdaglar, “Distributed Subgradient Methods for Multi-Agent Optimization,” IEEE Transac- tions on Automatic Control , vol. 54, no. 1, pp. 48–61, 2009. 1 APPENDIX A CONVERGENCE We have parameters of shared adapter and personalized adapter. According to the conclu...
2009
-
[40]
(33) According to Lemma A.8, we have the following inequality K−1X k=0 NX i=1 E∥wi(t) − ¯w(t)∥2 ≤ NX i=1 E∥wi(0) − ¯w(0)∥2 + K−1X k=1 18N 2η2 wC 2L2 1 − q k−1X r=0 qk−r NX i=1 E∥wi(r) − ¯w(r)∥2 + 18η2 wC 2N 3 1 − q k−1X r=0 qk−rE∥∇wL( ¯w(r), {vi(r + 1)}N i=1)∥2 + 6η2 wC 2N 3 (...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.