{"id":"bd23ca42-f6fe-493b-b0a3-ebf2be918c5f","arxiv_id":"2505.07575","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"KARULA constrains pairwise model differences proportionally to a Wasserstein-based measure of client data dissimilarity and converges at O(1/K) to an epsilon-stationary point under standard smoothness assumptions.","lead":"This paper introduces KARULA, a method for personalized federated learning that keeps client models close only when their data distributions are similar, using a Wasserstein-distance-based measure of similarity. The authors prove convergence to a neighborhood of a stationary point and report gains over standard baselines on synthetic and MNIST-based federated tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Problem (1) and the feasible set K used by Algorithm 1 are inconsistent: one bounds squared model distances, the other bounds norms, so the convergence guarantee targets a different problem.","rationale":"The reader's weakest_assumption focuses on the embedding Phi(D_i) approximating the 1-Wasserstein distance. That is a legitimate concern about the quality of the data-driven dissimilarity parameters, but it is an approximation issue that is partly acknowledged by the paper's use of 'surrogate' and 'approximates'. A more fundamental and directly checkable flaw is the mismatch between the problem formulation and the algorithm's actual feasible set. Problem (1) constrains squared model distances, while Algorithm 1 and the convergence analysis constrain the distances themselves. This mismatch means that even a perfect Wasserstein surrogate would not make Algorithm 1 solve Problem (1). The convergence guarantee in Theorem 4.3 is for a different constraint set, so the paper's core theoretical claim about solving Problem (1) is not established as written. This is fixable by clarifying which constraint is intended and adjusting the scaling of D_ij and the proof, which is why a conditional acceptance remains appropriate rather than outright rejection. The verdict is therefore unchanged, but the justification differs from the reader's primary concern.","tokens_in":14541,"tokens_out":7150,"duration_ms":64105,"concrete_test":"Run a two-client, one-dimensional toy instance with D_12 = 2 and t = 1. Take theta_1 = 1.5, theta_2 = 0. This point is feasible for the norm constraint (||theta_1 - theta_2|| = 1.5 <= 2) but infeasible for the squared constraint (1.5^2 = 2.25 > 2), demonstrating that the feasible sets in Sec. 3.1 and Sec. 3.3 differ. Then check whether the projection definition and Lemma A.2 in Appendix A.2 are stated for the norm constraint or the squared constraint; if the intended problem is (1), re-derive Theorem 4.3 with the squared constraint and verify whether the same convergence constants and delta-projection argument hold, or restate Problem (1) with the norm constraint and rescale D_ij accordingly so that Proposition 3.4's squared-distance bound is respected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that KARULA solves the constrained personalized FL problem stated in Problem (1). However, Problem (1) (Sec. 3.1) defines constraints as ||theta_i - theta_j||_2^2 <= t D_ij, whereas the algorithm's feasible set K (Sec. 3.3) and Theorem 4.3 use ||theta_i - theta_j||_2 <= t D_ij. These feasible sets are not equivalent unless D_ij is rescaled inconsistently. For example, with D_ij = 2 and t = 1, a pair with ||theta_i - theta_j|| = 1.5 is feasible for the norm constraint but violates the squared constraint (2.25 > 2). Moreover, Proposition 3.4 bounds the squared model distance by (2 L_X / gamma) W_1, so the theory justifies a bound on the squared distance, not on the norm. As written, Theorem 4.3 proves convergence to a stationary point of a constrained problem that differs from the problem the paper claims to solve. This is not a merely cosmetic issue: the constraint mismatch changes the feasible set, the projection, and the meaning of the 'dissimilarity constraints' that are the paper's main contribution. The authors must clarify which formulation is intended and adjust the dissimilarity parameters, the problem statement, and the theory consistently.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KARULA, a personalized federated learning method that constrains pairwise differences between client models by data-driven dissimilarity parameters derived from a linear-embedding approximation of the 1-Wasserstein distance. The authors present an inexact projected stochastic gradient algorithm with variance reduction for partial client participation, prove an O(1/K) convergence bound to an epsilon-stationary point for smooth possibly nonconvex losses, and report experiments on synthetic ridge regression and federated MNIST showing improved accuracy over Local, FedAvg, and IFCA.","tokens_in":14816,"tokens_out":6826,"duration_ms":56215,"significance":"If the algorithm is made consistent with the stated problem and the Wasserstein-embedding approximation is justified, the method would be a useful peer-based PFL contribution: it avoids clustering assumptions, has a communication-light dissimilarity computation, and the convergence analysis explicitly accounts for stochastic gradients and inexact projections. The empirical results are plausible and support the method on the tested tasks.","major_comments":[{"comment":"The optimization problem is stated with constraints ||theta_i - theta_j||_2^2 <= t D_ij, while the feasible set actually used by Algorithm 1 and Theorem 4.3 is {theta : ||theta_i - theta_j||_2 <= t D_ij}. These feasible sets differ: for D_ij=2 and t=1, a pair with ||theta_i - theta_j||=1.5 satisfies the norm constraint but violates the squared one. Since Proposition 3.4 only bounds the squared distance, the theory justifies the squared formulation, whereas the algorithm and convergence theorem prove a result for a different problem. The formulation must be made consistent.","section":"Sec. 3.1 Eq. (1) and Sec. 3.3 set K"},{"comment":"The claim that ||Phi(D_i)-Phi(D_j)||_1 approximates W_1(mu_i, mu_j) is asserted with citations but without a statement of the approximation error, the conditions on the reference distribution mu_0 and sample size N_0, or a proof. Because these D_ij values directly determine the feasible set, a poor approximation would invalidate the transfer of the Wasserstein-based bound in Eq. (2) to the constraints actually used in the algorithm.","section":"Sec. 3.2"},{"comment":"The proof as written does not yield the stated bound. Substituting eta = 3/(8L) into the final displayed inequality 2/(eta(1-2*eta*L)) gives 64L/3, not 8L/3, and the proof uses an undefined factor alpha in the definition of the gradient mapping. The telescoping steps need to be rechecked and the constants corrected.","section":"Sec. 4, Theorem 4.3 and Appendix A.3"},{"comment":"The delta-inexact projection is required to return feasible points (theta in K), but the computation is said to be described in supplemental material that is not included, and the Limitations section concedes that feasibility is not always guaranteed. The theorem therefore may not apply to the implemented projection; the authors should either provide the projection algorithm and prove its feasibility guarantee, or adjust the theory to allow infeasible iterates.","section":"Sec. 3.3 and Limitations"}],"minor_comments":[{"comment":"The sentence listing final test accuracies ends with '83.9 +/- 2.60%, respectively' without identifying which strategy this number belongs to (presumably IFCA).","section":"Sec. 5.2"},{"comment":"'centerally constrained' should be 'centrally constrained'.","section":"Introduction"},{"comment":"The numeric dissimilarity values are shown without units or an explicit explanation of their scale; a brief caption note would help.","section":"Fig. 3"},{"comment":"Please report the number of random trials or seeds used to compute the mean and the +/- 2*SE intervals.","section":"Sec. 5.1"}],"recommendation":"major_revision","confidential_remarks":"The constraint mismatch is the main obstacle; if the authors fix it and supply the missing projection details, resubmission would be appropriate. The embedding approximation should also be supported by at least a numerical validation of the surrogate against the true W1 distance. The empirical study is otherwise reasonable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2505.07575. First, KARULA is a genuinely new idea: it uses a linear-embedding surrogate for the 1-Wasserstein distance between client data distributions to set pairwise model-dissimilarity constraints in personalized FL. That's a clean way to handle overlapping client relationships that clustering can't capture, and the empirical results on synthetic ridge regression and federated MNIST are solid. Second, there are two load-bearing inconsistencies that need fixing before the theory can be trusted.\n\nThe first is that Problem (1) constrains the squared distance, ||θ_i−θ_j||_2^2 ≤ tD_ij, but the feasible set K in Algorithm 1 and the convergence analysis use the un-squared norm, ||θ_i−θ_j||_2 ≤ tD_ij. Those are different feasible sets unless D_ij is rescaled, and the paper doesn't say that. Worse, Proposition 3.4 gives a bound on the squared distance in terms of W_1, so the theory actually supports the squared constraint. As written, Theorem 4.3 proves convergence to a stationary point of a different problem than the one the paper claims to solve. This is not cosmetic; it changes the projection and the meaning of the dissimilarity constraints.\n\nThe second is that the proof of Theorem 4.3 has an algebraic slip. Somewhere between inequality (10) and the Lyapunov recursion, a factor of η/2 appears to be dropped from the ||G_η||^2 term. The final constant 8L/3 doesn't follow from the displayed recurrence for the chosen step size. I spent a while trying to reproduce it and couldn't; the authors need to check the algebra.\n\nThere are smaller gaps: the projection subroutine is promised in the supplement, not described; the choice of reference data set D_0 and N_0 for the Wasserstein embedding is not discussed, even though the approximation quality depends on it; and the proof of Prop 3.4 uses μ where the assumption says γ. These are minor.\n\nThe core idea is worth taking seriously, and the experiments are appropriately reported. The paper deserves a serious referee, but it's not ready as is.","headline":"KARULA has a novel and sensible way to set pairwise model constraints from Wasserstein-based distributional similarity, but two load-bearing internal inconsistencies (constraint mismatch and a proof algebra slip) need fixing before the theory holds.","tokens_in":15325,"tokens_out":4782,"would_cite":false,"duration_ms":37007,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces KARULA, a personalized federated learning method in which each client's model is constrained by data-driven Wasserstein-distance surrogates, and proves an $O(1/K)$ convergence rate to an $\\epsilon$-stationary point…","keywords":["personalized federated learning","statistical heterogeneity","1-Wasserstein distance","model dissimilarity constraints","linear embedding","variance reduction","inexact projection","nonconvex optimization"],"falsifier":"Run the synthetic ridge regression experiment with ground-truth model parameters, replacing the embedding surrogate with the exact empirical 1-Wasserstein distance computed between client data sets; if KARULA's estimation error does not improve or becomes worse than with the surrogate, the Wasserstein bound is not the operative mechanism behind the reported gains. Alternatively, compare $\\|\\Phi(D_i)-\\Phi(D_j)\\|_1$ with exact $W_1$ on distributions with known distance while varying the reference distribution and sample size $N_0$; if the surrogate's ranking disagrees with true $W_1$, the constraints are not distributionally grounded.","tokens_in":14295,"feed_emoji":"🤝","tokens_out":8454,"duration_ms":71479,"temperature":0.7,"pith_summary":"This paper introduces KARULA, a personalized federated learning strategy in which the models of different clients are allowed to differ only up to bounds set by how dissimilar their data distributions are. Dissimilarity is measured by a linear-embedding surrogate for the 1-Wasserstein distance, so no raw client data are exchanged. The paper proves that, under standard smoothness and bounded-variance assumptions, the proposed inexact projected stochastic gradient algorithm reaches an $\\epsilon$-stationary point at rate $O(1/K)$, with no assumption bounding client heterogeneity. On synthetic ridge regression and federated MNIST, KARULA reports lower estimation error and higher accuracy than local training, FedAvg, and IFCA. If these claims hold, the method offers a data-driven way to set personalization strength instead of choosing a fixed regularization by hand.","feed_headline":"Federated models personalize under Wasserstein dissimilarity caps","feed_subtitle":"Clients with similar data learn similar models; the algorithm provably converges at rate $O(1/K)$.","key_machinery":"The load-bearing object is the linear embedding $\\Phi(D_i) = (N_0 \\pi_i^\\star D_i - D_0)/\\sqrt{N_0}$, where $\\pi_i^\\star$ is the optimal transport plan between the reference empirical distribution and client $i$'s empirical distribution, and $D_0$ is the reference data matrix; its $\\ell^1$ pairwise distance approximates $W_1(\\hat{\\mu}_i, \\hat{\\mu}_j)$. This embedding lets each client compute one transport plan instead of $n(n-1)/2$ plans, and transmits only the reference set and embeddings. The optimization machinery is an inexact projected stochastic gradient algorithm: a SAGA-style variance-reduced server update removes the bias of partial participation, and a $\\delta$-inexact projection onto the constraint set $\\mathcal{K}=\\{\\theta: \\|\\theta_i-\\theta_j\\|_2 \\le t D_{ij}\\}$ keeps the iterates feasible while contributing a controlled $\\delta$ term to the final neighborhood.","core_discovery":"The central claim is that constraining personalized model differences by pairwise distributional distances is both principled and algorithmically tractable. Under quadratic functional growth and data-Lipschitz losses, the paper proves that the ideal models satisfy $\\|\\theta_i^\\star-\\theta_j^\\star\\|_2^2 \\le (2L_X/\\gamma) W_1(\\mu_i,\\mu_j)$, so distributional nearness justifies model nearness. In the federated setting it replaces the exact Wasserstein distance with $\\|\\Phi(D_i)-\\Phi(D_j)\\|_1$, where $\\Phi$ is a linear embedding computed from a single optimal transport plan between each client's empirical data and a reference data set. The resulting constrained problem is solved by an inexact projected stochastic gradient method with variance reduction and partial participation; Theorem 4.3 bounds the minimum squared gradient mapping by $O(1/K)$ plus $\\epsilon = 4\\sigma^2(s+1)/s + 2\\delta$.","pith_inferences":["Editorial inference: An immediate testable extension is to replace the fixed reference distribution with a learnable or client-adaptive reference; the theory's dependence on reference choice and reference sample size is unspecified, so the best reference may be distribution-selection-sensitive.","Editorial inference: The same constraint machinery could be applied to other dissimilarity surrogates, such as optimal-transport distances between feature representations rather than raw data, keeping the convergence result intact as long as the embedding defines a metric.","Editorial inference: The $\\ell^1$ embedding inherits the slow $O(N^{-1/d})$ empirical-Wasserstein convergence, so KARULA's advantage over local models should be largest when per-client sample sizes are comparable to feature dimension and should shrink in very high dimensions.","Editorial inference: The variance-reduced inexact projection step is stated generically enough that it could be reused by other federated algorithms whose feasible sets are convex."],"forward_implications":["Setting $t=0$ recovers a single global model and $t\\to\\infty$ gives fully local models, so KARULA continuously interpolates between the two extremes using constraints that are fully data-driven.","Because the convergence analysis does not bound client heterogeneity, the $O(1/K)$ rate holds for arbitrary distributional differences among clients, provided the loss is smooth and stochastic gradients have bounded variance.","The projection error $\\delta$ appears additively in the stationarity neighborhood, giving a concrete trade-off: looser projections save per-round computation at the price of a larger final error.","The pairwise dissimilarity structure is preserved during training, so models can share information across overlapping subgroups without forcing a hard cluster assignment.","FedAvg and IFCA can be seen as special cases or degenerate limits of the constraint set, which helps explain the reported empirical advantage when client distributions overlap in complex ways."],"supporting_citations":[{"why":"Supplies Kantorovich-Rubinstein duality, used to bound loss differences by $W_1$ in the proof of Proposition 3.4.","marker":"Villani (2008)"},{"why":"Establishes that the 1-Wasserstein distance between client distributions quantifies the effect of heterogeneity on federated learning, motivating the dissimilarity measure.","marker":"Zhao et al. (2018)"},{"why":"Supplies the $O(N^{-1/d})$ convergence rate for empirical Wasserstein distances used to argue embeddings beat direct model-distance estimation in low-sample regimes.","marker":"Wang et al. (2021)"},{"why":"Introduces the Wasserstein embedding of data sets that KARULA adapts into the linear embedding $\\Phi$.","marker":"Kolouri et al. (2020)"},{"why":"Extends Wasserstein task embedding to measure task similarities, supporting the choice of the embedding surrogate.","marker":"Liu et al. (2025)"},{"why":"Contributes FEDVARP, the partial-participation variance reduction technique that Algorithm 1 builds on.","marker":"Jhunjhunwala et al. (2022)"},{"why":"Supplies the proximal stochastic variance-reduction framework from which the convergence analysis is adapted.","marker":"Reddi et al. (2016)"},{"why":"Provides the IFCA baseline and the cluster-count experimental setup used in the federated MNIST comparison.","marker":"Ghosh et al. (2020)"}],"fun_headline_variants":["Federated learning ties model likeness to data distance","Personalized FL: Wasserstein-constrained model updates","KARULA: Federated learning with provable personalized convergence","Model dissimilarity caps from Wasserstein distances in FL","Bounding personalized models by distributional nearness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire scheme depends on treating the linear data embedding computed against a single reference data set as a faithful stand-in for the true 1-Wasserstein distance between client distributions; if that approximation is poor, the theoretical bound that motivates the constraints no longer governs the models the algorithm actually produces.","fun_headline_variants_meta":{"raw":{"variants":["Federated learning ties model likeness to data distance","Personalized FL: Wasserstein-constrained model updates","KARULA: Federated learning with provable personalized convergence","Model dissimilarity caps from Wasserstein distances in FL","Bounding personalized models by distributional nearness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1683,"prompt_tokens":873,"completion_tokens":810,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":733}},"tokens_in":489,"tokens_out":810,"duration_ms":7260,"temperature":1.0,"reasoning_tokens":733,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:15:00.412835+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the synthetic ridge regression experiment with ground-truth model parameters, replacing the embedding surrogate with the exact empirical 1-Wasserstein distance computed between client data sets; if KARULA's estimation error does not improve or becomes worse than with the surrogate, the Wasserstein bound is not the operative mechanism behind the reported gains. Alternatively, compare $\\|\\Phi(D_i)-\\Phi(D_j)\\|_1$ with exact $W_1$ on distributions with known distance while varying the reference distribution and sample size $N_0$; if the surrogate's ranking disagrees with true $W_1$, the constraints are not distributionally grounded.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Kantorovich-Rubinstein duality, used to bound loss differences by $W_1$ in the proof of Proposition 3.4."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the $O(N^{-1/d})$ convergence rate for empirical Wasserstein distances used to argue embeddings beat direct model-distance estimation in low-sample regimes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Extends Wasserstein task embedding to measure task similarities, supporting the choice of the embedding surrogate."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes FEDVARP, the partial-participation variance reduction technique that Algorithm 1 builds on."},{"cited_title":"J., Sra, S., Poczos, B., and Smola, A","cited_arxiv_id":null,"evidence_quote":"Supplies the proximal stochastic variance-reduction framework from which the convergence analysis is adapted."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the IFCA baseline and the cluster-count experimental setup used in the federated MNIST comparison."}],"review_version":1}