{"id":"725ac3db-9c97-4a72-8010-d2b02286a538","arxiv_id":"2506.11803","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A decentralized multi-agent training method with a shared and a private adapter claims O(1/sqrt(NK)) convergence and big communication savings, but the evidence is incomplete.","lead":"PE-MA is a multi-agent learning scheme in which each agent trains a private personal adapter and shares a separate small adapter with neighbors. It claims a near-optimal convergence rate and large communication savings, but the paper has several unresolved inconsistencies.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section IV Eq. (12) uses a 1/|N(i)| normalization that makes the consensus matrix row-scaled; the proof in Lemma A.8 applies only to doubly stochastic P, so the O(1/sqrt(NK)) rate is not proven for the update as written.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing issue: Eq. (12) is not the consensus update analyzed in the proof. This is not a matter of disagreeing with the community consensus or of missing experiments; it is an internal inconsistency between the algorithmic definition and the theoretical object. The proof of Lemma A.8 relies on P^k and on the doubly stochastic property P1 = 1 and 1^T P = 1^T. With Eq. (12), the effective matrix is D^{-1}P, which fails the second property in general, and the average update identity used throughout Lemma A.3 and Lemma A.8 is false. Since the paper's central claim is that PE-MA provably attains O(1/sqrt(NK)), the proof must cover the update that the paper tells a reader to implement. It does not. I credit the paper for Algorithm 1 being self-consistent with Eq. (1), so the issue may be a fixable typo, but as submitted the theorem does not apply to Section IV's update. The reader's REJECT verdict remains appropriate; a revised version that reconciles Eq. (12) with the pseudocode, defines alpha and beta in Theorem V.5, and re-verifies Lemma A.8 for the corrected update would change that assessment.","tokens_in":26001,"tokens_out":10997,"duration_ms":110408,"concrete_test":"Run a 4-agent ring simulation of the two candidate updates. Use P = (1/3)(I + adjacency matrix), which is symmetric and satisfies P1 = 1, and initialize w_i to distinct scalars. Iterate both Eq. (12) and Algorithm 1 line 20 for 200 rounds with no gradient updates. If the Eq. (12) trajectory does not converge to the uniform average (for this P, it shrinks toward zero because each row of Q sums to 1/3), while Algorithm 1 converges to the mean, then the Section IV update is not the doubly stochastic consensus used in Lemma A.8. Reporting the final max_i |w_i - mean| for both updates settles which update the convergence theorem actually covers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III Eq. (1) and Algorithm 1 line 20 define the shared-adapter aggregation as w_i(t+1) = sum_{j in N(i)} P_ij w_j(t+1/2) with P symmetric and P1 = 1. Section IV Eq. (12) instead writes w_i(t+1) = (1/|N(i)|) sum_{j in N(i)} P_ij w_j(t+1/2). These updates differ whenever |N(i)| is not already absorbed into P. With Eq. (12), the consensus operator is Q = D^{-1}P, where D_ii = |N(i)|. Q is generally not doubly stochastic: 1^T D^{-1} P is not 1^T unless the graph is regular and P has constant row weights, so Q does not preserve the average of the shared adapters. Consequently the proof's identity wbar(t+1) - wbar(t) = -(eta_w/N) sum_i g_w(w_i(t), v_i(t+1)), used in Lemma A.3(a1), does not hold; Lemma A.7's bound on |1/N - P^k(i,j)| is inapplicable to Q^k; and Lemma A.8's consensus-error bound collapses. The paper never reconciles Eq. (12) with the pseudocode or the proof. The theorem statement also leaves alpha and beta undefined, and Corollary V.7 silently switches to eta_w and eta_v. As submitted, the object analyzed in the proof is not the update described in Section IV, so the asymptotic optimality claim is unsupported for the method as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PE-MA, a decentralized multi-agent learning framework with a dual-adapter architecture: each agent trains a localized personalized adapter and a shared adapter that is exchanged with neighbors. The central theoretical claim is an asymptotically optimal convergence rate of O(1/sqrt(NK)) under Lipschitz, unbiased-gradient, bounded-variance, and bounded-heterogeneity assumptions. The paper also reports experiments on Office-Home, Office-Caltech10, and DomainNet showing accuracy gains and reduced communication/training parameter counts compared with several decentralized SGD baselines.","tokens_in":26352,"tokens_out":10214,"duration_ms":91876,"significance":"If the convergence-rate claim is correct, the paper makes a useful contribution to parameter-efficient decentralized learning: it shows that a communication-efficient dual-adapter scheme can match the standard decentralized SGD rate. The empirical study is reasonable and the reported communication savings are substantial. However, the theoretical result is the paper's main claim, and the proof currently applies to a different update rule than the one stated in Section IV, so the contribution is not yet established as written.","major_comments":[{"comment":"Equation (12) defines the communication update as w_i(t+1) = (1/|N(i)|) Σ_{j∈N(i)} P_ij w_j(t+1/2), which corresponds to the operator Q = D^{-1}P. On a general graph this Q is not doubly stochastic even when P is symmetric and P1=1. The convergence proof, however, analyzes the unnormalized update w_i(t+1)=Σ_j P_ij w_j(t+1/2): Lemma A.3(a1) relies on the identity ¯w(t+1)−¯w(t) = −(η_w/N)Σ_i g_w(wi(t), vi(t+1)), which holds only when the consensus matrix preserves the average; Lemma A.7 bounds |1/N − P^k(i,j)| under the assumption that P is doubly stochastic; and Lemma A.8's entire consensus-error bound is built on those two facts. With Eq. (12), the average is not preserved, so the identity in Lemma A.3(a1) fails and Lemma A.7 cannot be applied to Q^k. Algorithm 1 line 20 and Section III Eq. (1) use the unnormalized update, so the paper contains two inconsistent definitions of the same step. As submitted, the theorem is not proven for the algorithm described in Section IV.","section":"§IV, Eq. (12) vs. Appendix A, Lemmas A.3/A.8"},{"comment":"Theorem V.5 states learning-rate constraints in terms of α and β, but neither α nor β is defined anywhere, and the displayed bound in Eq. (13) is written in terms of η_w and η_v. Corollary V.7 silently switches to η_v = 1/(τ√K) and η_w = √(N/K). The appendix proof uses conditions such as max(Lη_w, η_vτL(1+36τ²)) ≤ 1 and η_w ≤ min(1/L, N L²/(2L²+2), (1−q)/(3√2 C L N)), none of which are related to the α and β inequalities in the theorem statement. The theorem's hypotheses must be restated with consistent parameters that match the proof.","section":"§V, Theorem V.5"},{"comment":"The proof assumes away the initial consensus error: step (e2) says 'W.l.o.g., we assume that the initial term wi(0), ∀i is small enough and can be neglected,' and Eq. (34) later uses Σ_i E∥wi(0)−¯w(0)∥² = 0 as if it were an established fact. This is not a harmless 'without loss of generality' unless the initialization guarantees all wi(0) are identical. If Algorithm 1's single initial adapter w0 indeed means all agents start from the same w0, the term is exactly zero and this should be stated explicitly; otherwise the bound in Lemma A.8 and the final rate in Theorem V.5 acquire an extra term that is not accounted for. The paper needs to state the initialization assumption precisely and verify it against Algorithm 1.","section":"Appendix A, Lemma A.8 (Eq. (32)) and Eq. (34)"}],"minor_comments":[{"comment":"The abstract and conclusion state the rate as O(1/√(NK)), while Corollary V.7 gives O(1/√(NK) + 1/(K√N) + 1/(τ√(NK))). Please state explicitly the conditions under which the latter two terms are dominated by the first, and avoid the unqualified label 'asymptotically optimal' unless a matching lower bound is provided or cited.","section":"§I and §V (Corollary V.7)"},{"comment":"The bound in Eq. (25), with coefficients (18τ²−15τ−3)σ² and (18τ²−18τ)E∥gv(·)∥², is obtained by invoking 'Lemma 3 in [38]' without stating the lemma or its hypotheses. Since the final rate depends on these constants, the lemma statement or a short derivation should be included in the appendix.","section":"Appendix A, Lemma A.5"},{"comment":"Please reconcile Eq. (12) with Algorithm 1 line 20 and Section III Eq. (1). If the intended update is the unnormalized one, remove the 1/|N(i)| factor from Eq. (12); if the normalized update is intended, the convergence analysis must be reworked for a row-stochastic consensus matrix.","section":"§IV, Eq. (12)"},{"comment":"The abstract's claim of '2%–5%' accuracy improvement is stronger than the table shows; for example, on Office-Caltech10 under the ER topology PE-MA achieves 97.68% versus 97.18% for DSGD_SIM (Input layer), an improvement of about 0.5%. Please qualify the reported gains or provide per-dataset ranges.","section":"Table I"},{"comment":"The rendered text contains several garbled formulas (e.g., 'O( 1√ N K)' in the abstract) and inconsistent capitalization in the index terms ('dual-Adapters'). A thorough pass to fix LaTeX and typography is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's central theorem is not proven for the algorithm as described in Section IV, but the pseudocode and Section III point to the unnormalized update, so the inconsistency may be fixable by correcting Eq. (12) and the theorem statement. However, the burden of proof is on the authors to reconcile the written algorithm with the analysis; if they cannot, the paper should be rejected. I recommend major revision with a request for a revised proof that matches a single, clearly specified update rule."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the dual-adapter architecture is a sensible combination of FedSim/FedAlt-style representation separation with PEFT adapters, and the proof is long and mostly standard. But the paper has a load-bearing inconsistency: the shared-adapter update in Section IV Eq. (12) has an extra 1/|N(i)| normalization that breaks the doubly stochastic consensus matrix, while Algorithm 1 and the proof use the unnormalized weighted sum. The convergence analysis applies to the latter, not to the update as written in Eq. (12). That means the headline O(1/sqrt(NK)) rate is not proven for the method as described.\n\nWhat is genuinely new: the dual-adapter design itself—shared adapter for consensus, private adapter for personalization—plus the first convergence bound for that specific combination. The communication and trainable-parameter numbers are attractive: transmitting only a small shared adapter is a real practical win, and the experiments show consistent 2–5% accuracy gains over the baselines across three datasets and three topologies. The proof borrows standard machinery (L-smoothness, bounded variance, consensus error via mixing matrix) and the lemmas are laid out in enough detail to follow.\n\nWhere it falls short: the Eq. (12) issue is the big one, and the theorem statement itself is sloppy—α and β are undefined, and Corollary V.7 silently switches to η_w and η_v. The 'asymptotically optimal' label is asserted without any lower bound, so it should be softened to 'matches the best known rate.' Experiments have no error bars, no code, and the mixing coefficient μ is tuned per dataset to produce the reported peaks; the 'dual-randomness aggregation' mentioned in Section VI is never specified. These are fixable, but as submitted they leave the empirical claims under-supported.\n\nBottom line: this is a paper with a good idea and a fixable proof, but the central theorem currently does not cover the algorithm as written. It deserves a serious referee—the issues are exactly the kind a good review could catch and the authors could repair—but it should not be accepted in its current form.","headline":"Useful dual-adapter idea with a detailed proof attempt, but the central update in Eq. (12) breaks the doubly stochastic consensus assumption, so the advertised O(1/sqrt(NK)) rate is not proven for the algorithm as written.","tokens_in":26873,"tokens_out":3272,"would_cite":false,"duration_ms":32221,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","68W15","90C26"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that PE-MA reaches an asymptotically optimal convergence rate of $O(1/\\sqrt{NK})$ while transmitting only small adapters.","keywords":["multi-agent systems","co-evolution","parameter-efficient fine-tuning","dual adapters","decentralized SGD","consensus","personalization","convergence rate"],"falsifier":"Compute $(1/N)\\sum_i \\|w_i(t) - (1/N)\\sum_j w_j(t)\\|^2$ over training with the update (12) on a ring topology; convergence of this quantity to zero at the rate predicted by Lemma A.8 would confirm the proof, while a non-vanishing plateau would falsify it.","tokens_in":25809,"feed_emoji":"🧩","tokens_out":8688,"duration_ms":80813,"temperature":0.7,"pith_summary":"PE-MA is a decentralized multi-agent training framework built on a dual-adapter architecture: each agent keeps a small personalized adapter for its own data and a shared adapter that is averaged with neighbors through a doubly stochastic communication matrix. The paper's central claim is that this scheme converges at an asymptotically optimal rate of $O(1/\\sqrt{NK})$, where $N$ is the number of agents and $K$ the number of communication rounds, matching the best known decentralized SGD rates while communicating only lightweight adapters. On three cross-domain image classification benchmarks, PE-MA reports 2–5% higher accuracy than independent training and decentralized SGD baselines, with about 77–87% lower communication and training costs. If the convergence proof holds, the framework shows that personalization and global coordination can be separated cleanly without sacrificing either.","feed_headline":"Dual adapters match optimal decentralized SGD rate","feed_subtitle":"PE-MA co-evolves personal and shared adapters, cutting communication by ~87% while keeping provable convergence.","key_machinery":"The load-bearing object is the dual-adapter module: a local personalized adapter $v_i$ that is never transmitted, and a shared adapter $w_i$ that is exchanged among neighbors using the symmetric doubly stochastic matrix $P$ ($P^\\top = P$, $P\\mathbf{1} = \\mathbf{1}$). The argument proceeds by bounding the composite quantity $M(t) := \\frac{1}{N}\\sum_{i=1}^N \\|w_i(t) - \\bar w(t)\\|^2 + \\|\\nabla_w L(\\bar w(t),\\{v_i(t+1)\\})\\|^2 + \\frac{\\alpha\\tau}{\\beta}\\|\\nabla_v L(\\bar w(t),\\{v_i(t)\\})\\|^2$, where $\\bar w$ is the average shared adapter. The two-timescale learning rates ($\\eta_w$ and $\\eta_v$) ensure that local personalization acts as fast noise, while a consensus lemma bounds the mixing error of $P$ in terms of its smallest positive entry. This combination converts the multi-agent objective into a standard descent-plus-noise recursion.","core_discovery":"The paper claims that PE-MA, with learning rates $\\eta_w = \\sqrt{N/K}$ and $\\eta_v = 1/(\\tau \\sqrt{K})$, satisfies a bound on the averaged composite $M(t)$—the per-agent consensus error of the shared adapters plus the squared norms of the global partial gradients for $w$ and $v$—that yields a convergence rate of $O(1/\\sqrt{NK} + 1/(K\\sqrt{N}) + 1/(\\tau\\sqrt{NK}))$, which it calls asymptotically optimal. The proof (Theorem V.5 and Corollary V.7) treats the personalized adapters as a two-timescale disturbance whose contribution is absorbed by smaller learning rates, and it controls the shared-adapter consensus error via a doubly stochastic mixing matrix $P$. The experiments aim to demonstrate the same trade-off in practice: PE-MA outperforms full-model independent training and Decentralized SGD variants that share input, output, or adapter layers, under fully connected, Erdős–Rényi, and ring topologies.","pith_inferences":["The unresolved mismatch between Eq. (12)'s $1/|\\mathcal{N}(i)|$ factor and the proof's row-stochastic update suggests a direct test: re-running the analysis for a row-normalized consensus matrix would either produce a different rate or reveal that the implementation must drop the factor to match the theorem.","The same dual-adapter split should transfer to other parameter-efficient modules (LoRA, prefix prompts, visual-prompt tokens), because the convergence argument relies only on Lipschitz gradients and bounded variance, not on the adapter's internal structure.","The fixed mixing coefficient $\\mu$ could become a per-agent adaptive variable; the ablation results on data-poor agents hint that $\\mu$ should decrease when local data is scarce, which the current theorem does not capture.","If extended to large language models, the framework would let a swarm of agents co-fine-tune shared adapters over a frozen backbone with communication savings of the same order, though non-convexity and transformer-specific gradient noise would need empirical confirmation."],"forward_implications":["PE-MA converges at $O(1/\\sqrt{NK})$, the same leading rate as decentralized SGD, so the communication bottleneck does not come at the cost of asymptotic performance.","The shared adapter alone is sufficient for global coordination: only $w_i$ is averaged, and the personalized adapters $v_i$ remain local, so privacy-sensitive personalization is preserved by construction.","The rate holds under any connected communication topology for which a symmetric doubly stochastic matrix $P$ exists, including fully connected, Erdős–Rényi, and ring graphs.","Experimentally, PE-MA improves accuracy by 2–5% over baselines while reducing communication parameters by roughly 77–87% and trainable parameters by over 70%.","The two-timescale learning-rate schedule ($\\eta_w=\\sqrt{N/K}$, $\\eta_v=1/(\\tau\\sqrt{K})$) is a concrete recipe practitioners can adopt directly."],"supporting_citations":[{"why":"Supplies the two-timescale stochastic-approximation framework that motivates the weighted partial-gradient loss.","marker":"[27]"},{"why":"Gives the personalized decentralized shared-representation setting and the weighted-gradient formulation that PE-MA's proof follows.","marker":"[28]"},{"why":"Provides the geometric-series inequality used to sum the consensus-error recursion over rounds.","marker":"[29]"},{"why":"Establishes the decentralized SGD rate that the paper compares against as the optimal baseline.","marker":"[30]"},{"why":"Its Lemma 3 bounds the accumulation of local update errors in the personalized adapter, which the proof cites directly.","marker":"[38]"},{"why":"Supplies the doubly stochastic matrix mixing bound (Lemma A.7) that controls the convergence of the shared adapters to their average.","marker":"[39]"},{"why":"Defines the FedSim-style concurrent-update baseline used in the experimental comparison.","marker":"[34]"},{"why":"Defines the FedAlt-style alternating-update baseline used in the experimental comparison.","marker":"[35]"}],"fun_headline_variants":["Personalized adapters shrink multi-agent comms by 87%","Shared and personal adapters hit optimal decentralized rate","Parameter-efficient co-evolution with personalized adapters","Co-evolution via shared and personal adapters, optimal rate","Dual adapters slash communication, keep optimal convergence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes the shared-adapter update is the symmetric doubly stochastic consensus $w_i(t+1)=\\sum_j P_{ij} w_j(t+1/2)$ with $P$ symmetric and $P\\mathbf{1}=\\mathbf{1}$, while the algorithm text in Eq. (12) writes $w_i(t+1)=(1/|\\mathcal{N}(i)|)\\sum_j P_{ij} w_j(t+1/2)$, an extra normalization the analysis never addresses, and the rate collapses if that discrepancy is resolved in favor of Eq. (12).","fun_headline_variants_meta":{"raw":{"variants":["Personalized adapters shrink multi-agent comms by 87%","Shared and personal adapters hit optimal decentralized rate","Parameter-efficient co-evolution with personalized adapters","Co-evolution via shared and personal adapters, optimal rate","Dual adapters slash communication, keep optimal convergence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001223,"raw_usage":{"total_tokens":5005,"prompt_tokens":897,"completion_tokens":4108,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":4029}},"tokens_in":513,"tokens_out":4108,"duration_ms":29959,"temperature":1.0,"reasoning_tokens":4029,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:04:26.594718+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute $(1/N)\\sum_i \\|w_i(t) - (1/N)\\sum_j w_j(t)\\|^2$ over training with the update (12) on a ring topology; convergence of this quantity to zero at the rate predicted by Lemma A.8 would confirm the proof, while a non-vanishing plateau would falsify it.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the two-timescale stochastic-approximation framework that motivates the weighted partial-gradient loss."},{"cited_title":"Deprl: Achieving linear convergence speedup in personalized decentralized learning with shared representations,","cited_arxiv_id":null,"evidence_quote":"Gives the personalized decentralized shared-representation setting and the weighted-gradient formulation that PE-MA's proof follows."},{"cited_title":"Stochastic gradient push for distributed deep learn- ing,","cited_arxiv_id":null,"evidence_quote":"Provides the geometric-series inequality used to sum the consensus-error recursion over rounds."},{"cited_title":"Distributed Subgradient Methods for Multi-Agent Optimization,","cited_arxiv_id":null,"evidence_quote":"Supplies the doubly stochastic matrix mixing bound (Lemma A.7) that controls the convergence of the shared adapters to their average."},{"cited_title":"Exploiting shared representations for personalized fed- erated learning,","cited_arxiv_id":null,"evidence_quote":"Defines the FedAlt-style alternating-update baseline used in the experimental comparison."}],"review_version":1}