REVIEW 4 major objections 5 minor 31 references
Joint Graph Estimation and Signal Restoration for Robust Federated Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Jointly learning a client graph while restoring noisy model parameters gives federated learning a 2–5% accuracy edge in biased, noisy settings.
desk verdict A sensible joint graph-learning and signal-restoration formulation whose published algorithm has wrong gradients, so the experiments cannot be reproduced from the text. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the difference-of-convex decomposition of the biconvex smoothness coupling. In the joint objective, the term $\operatorname{tr}(W D(\Psi))$ is convex in $W$ for fixed $\Psi$ and convex in $\Psi$ for fixed $W$, but not jointly convex. The identity $\operatorname{tr}(W D(\Psi)) = \tfrac{1}{2}\|W + D(\Psi)\|_F^2 - \tfrac{1}{2}(\|W\|_F^2 + \|D(\Psi)\|_F^2)$ rewrites it as a convex term minus a convex term, giving a DC program with components $f$, $g$, and $h$. The proximal DC algorithm then alternates a gradient step on the difference and a proximal step on $g$, producing a local minimizer with finite-iteration convergence guarantees. The graph Laplacian smoothness assumption provides the semantic content: it is what lets the learned $W$ act as a low-pass filter that restores missing or noisy parameter entries from their neighbors.
What would settle it
Run the method on a synthetic federated task with known client clusters and randomly permute client identities each round so the parameters are not smooth on any fixed graph; if the accuracy margin over FedAvg persists, the graph-smoothness assumption is not what produces the gain.
Extended reading notes
Core claim
The paper's central claim is that graph-based aggregation in federated learning should not treat the inter-client graph as a fixed input. Under Assumption 1, the aggregated parameter matrix $\Psi$ is assumed to vary smoothly over an unknown client graph, so the server should minimize a Laplacian smoothness term $\operatorname{tr}(\Psi^\top L\Psi)$ while simultaneously reconstructing $\Psi$ from the noisy, partially masked received matrix $\tilde X_{\mathrm{local}}$. The authors fold the smooth graph-learning objective and the graph-filtered aggregation into one biconvex problem, then decompose the biconvex term $\operatorname{tr}(W D(\Psi))$ as a difference of two convex functions using the identity $\|W + D(\Psi)\|_F^2 - (\|W\|_F^2 + \|D(\Psi)\|_F^2)$. Solving that DC program with the proximal DC algorithm yields a sequence that converges to a local optimum. The reported result is that this joint estimator beats two-step graph-learning-then-aggregation and the FedAvg, FedProx, CFL, FedAMP, and pFedGraph baselines on MNIST and CIFAR-10 at noise levels 0.1 and 0.2, with the largest gap being 95.17% versus 93.12% on MNIST at noise level 0.1.
Load-bearing premise
The load-bearing premise is that the true model parameters from different clients differ smoothly from one another according to an unknown similarity graph; if the noisy updates do not have that smooth structure, jointly learning the graph and restoring the parameters could amplify the noise instead of removing it.
Editorial extensions
If this is right
- Under biased data distributions and Gaussian transmission noise at levels 0.1 and 0.2, joint graph estimation and restoration yields higher average test accuracy than all five compared baselines on both MNIST and CIFAR-10.
- On MNIST at noise level 0.1 the proposed method reaches 95.17% accuracy versus 93.12% for the best baseline FedAMP, while the two-step alternating solver reaches only 88.99%, showing that joint optimization is the source of the gain.
- With missing rates from 0 to 0.1, the method keeps accuracy within 10% of its noise-free level on both datasets, indicating it can compensate for absent parameters using the learned graph.
- Because the graph-filtered aggregation specializes to cluster-wise and adjacency-wise graph aggregation in limiting cases, the proposed objective extends existing graph-based federated aggregations rather than replacing them.
Reading between the lines
- Inference: the per-round learned inter-client graph could be tracked across communication rounds as a privacy-preserving diagnostic for client drift or clustering, since it is derived only from transmitted parameters, not raw data.
- Inference: the same DC decomposition applies to any bilinear smoothness term between a signal and a graph, so the method should transfer to restoring gradients, activations, or sensor-network signals under missing entries, not just federated model parameters.
- Inference: because the DC formulation has a per-round cost that grows with the number of client pairs, practical large-scale deployment would likely need a sparse or block approximation of the learned graph; the paper does not address this scaling.
- Inference: a natural test of the smoothness assumption would be to compare the learned graph with a ground-truth similarity graph computed from clean parameters before adding noise, reporting edge-recovery quality alongside accuracy.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a robust federated-learning aggregation method that jointly estimates an inter-client graph and restores noisy/missing model parameters during the global model update. The aggregation problem is formulated as a difference-of-convex (DC) optimization, solved by a proximal DC algorithm (PDCA), and evaluated on MNIST and CIFAR-10 with five published baselines. The central claims are that the joint formulation improves classification accuracy by 2–5% under biased distributions and transmission noise, and that the DC/PDCA formulation provides convergence guarantees.
Significance. If the algorithmic issues are repaired, the paper offers a clean and useful idea: unifying graph learning and signal restoration inside the FL aggregation step, with a DC decomposition that is algebraically sound in Eq. (13). The reported accuracy gains against five published baselines are concrete and are evaluated on external datasets, so the empirical claims are not circular. The paper also includes a two-step baseline, which helps isolate the benefit of joint optimization. Counterbalancing these strengths, the manuscript does not provide code, proof sketches, or sensitivity analyses, and the experimental setup is small (20 clients, 30 rounds, simple CNN). The main concern is that the printed gradient formulas in Section 4.2 do not match the stated objective, which undermines the reproducibility of the algorithm and the convergence claim as written.
major comments (4)
- [Section 4.2, Algorithm 1 and Eq. (15)] The printed gradients ∇Ψf and ∇Ψh are not the derivatives of f and h defined in (15). For K=2, α=1, w=1, and Ψ=(0,1), the true gradient of α||w+Tvec(D(Ψ))||² is (-8,8), whereas the printed formula αD*(vec^{-1}(T^T(w+Tvec(D(Ψ)))))Ψ evaluates to (-4,0). Even if the argument of D* were symmetrized, the result would be (-4,4), still off by a factor of 2. The h-gradient has the same structural problem: the true derivative of α||Tvec(D(Ψ))||² is (-4,4), but the printed expression gives (-2,0). Consequently, the vector u-s used in Algorithm 1 is not the gradient of f-h, the update is not a PDCA step for (14), and the claimed convergence guarantee is unsupported. Please correct the gradient formulas (the correct forms involve 4α(diag(V)-V)Ψ with V=W+D(Ψ) for f, and 4α(diag(D)-D)Ψ for h) and re-verify the experiments.
- [Section 4.1, Eq. (15b)] The mapping from (11) to (14) is not an exact equivalence as claimed. In (11), the regularizer is γ||W||₁ over a symmetric, zero-diagonal W, which equals 2γ||w||₁ for w=upper(W). Equation (15b) instead uses γ||w||₁. This changes the objective by a factor of 2 unless γ is implicitly redefined; the paper does not state this. Please correct the factor or explicitly absorb it into the hyperparameter.
- [Section 4.2, footnote 2] The convergence claim is not supported. The footnote states that the PDCA conditions are satisfied but omits the details, and the paper does not identify which conditions are needed for the proximal DC algorithm or verify them for the specific f, g, and h in (15). Moreover, because the gradients in Algorithm 1 are incorrect, the update is not a valid PDCA iteration even if the abstract conditions were met. Please either state and verify the conditions or substantially weaken the convergence statement in the Introduction.
- [Section 3.2 and Section 5] Assumption 1—that the noisy/missing client updates vary smoothly on the inter-client graph—is the modeling premise of the entire aggregation objective, but it is never validated. No experiment reports the quality of the learned graph (e.g., edge recovery against a known graph, smoothness of the restored Ψ, or behavior of the learned graph as noise increases). Without such evidence, the robustness claim rests on an unexamined assumption, although this is distinct from the accuracy comparison in Table 1.
minor comments (5)
- [Section 3.3] The definition of the Euclidean distance matrix contains an erroneous summation symbol: [D(X)]mn should be ||x_m - x_n||², not Σ_{m,n}||x_m - x_n||².
- [Algorithm 1 and Section 5.1] The initialization of w(r+1,0) is said to use cosine similarity among the row entries of the received parameters, but the optimization objective (11) is built on squared Euclidean distances. Please specify the exact initialization used, since different initial graphs can affect the local minimum reached.
- [Table 1 and Fig. 4] The paper reports 10 independent runs but does not report standard deviations or error bars. The 2–5% accuracy differences in Table 1 would be more convincing with variance information, and Fig. 4 would be stronger if at least FedAvg and FedAMP were included for comparison.
- [Section 5.1] The hyperparameters α=0.05 and μ=β=γ=ρ=1.0 are described as chosen experimentally, but no sensitivity analysis is provided. It is therefore unclear whether the reported gains are robust to hyperparameter changes or are a result of fine-tuning.
- [References] Reference [23] is cited for CIFAR-10 but appears to point to the AlexNet paper; the usual citation is Krizhevsky's 'Learning Multiple Layers of Features from Tiny Images' (2009).
Circularity Check
No significant circularity: the proposed joint graph estimation and signal restoration method is evaluated on external benchmarks with manually set hyperparameters, and its derivation rests on algebraic identities and external references.
full rationale
The paper's central claim is a robustness improvement in federated aggregation, tested on MNIST and CIFAR-10 against five published baselines plus a two-step variant. No prediction is defined through the method's own fitted values, and hyperparameters such as alpha = 0.05 and mu = beta = gamma = rho = 1.0 are set experimentally rather than fitted to the test outcome. The joint objective in Eq. (11) is a biconvex formulation, and the DC decomposition in Eq. (13) is an algebraic identity rather than an assumption that smuggles in the target result. Assumption 1 is an explicit modeling premise drawn from an external reference [11], and the convergence guarantee is cited to an external PDCA result [12]; neither is a self-citation chain. The derivation of Eq. (6) as a generalized graph-based aggregation is shown by explicit limiting and Taylor-expansion arguments, not by renaming a known result. The review's noted concern that Algorithm 1's printed gradient formulas may not match Eq. (14) is a correctness or reproducibility issue, not circularity, because it does not make the experimental result equivalent to an input by construction. Overall, the derivation chain is self-contained with respect to the paper's stated benchmarks, so no circular step is present.
Assumptions & free parameters
free parameters (5)
- alpha =
0.05
- mu =
1.0
- beta =
1.0
- gamma =
1.0
- rho =
1.0
assumptions (3)
- domain assumption Client model parameters vary smoothly on the inter-client graph
- domain assumption Received parameters follow the noise model m_k times x_k plus Gaussian noise
- ad hoc to paper The DC decomposition and PDCA convergence conditions from the cited literature hold for the proposed f, g, h
Cite this review
Pith. "Pith review of Joint Graph Estimation and Signal Restoration for Robust Federated Learning." pith.science (2026). https://pith.science/paper/ZA7V4243
@misc{pith2026250511648,
author = {Pith},
title = {Pith review of: Joint Graph Estimation and Signal Restoration for Robust Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZA7V4243}},
note = {Machine review of arXiv:2505.11648}
}
abstract
We propose a robust aggregation method for model parameters in federated learning (FL) under noisy communications. FL is a distributed machine learning paradigm in which a central server aggregates local model parameters from multiple clients. These parameters are often noisy and/or have missing values during data collection, training, and communication between the clients and server. This may cause a considerable drop in model accuracy. To address this issue, we learn a graph that represents pairwise relationships between model parameters of the clients during aggregation. We realize it with a joint problem of graph learning and signal (i.e., model parameters) restoration. The problem is formulated as a difference-of-convex (DC) optimization, which is efficiently solved via a proximal DC algorithm. Experimental results on MNIST and CIFAR-10 datasets show that the proposed method outperforms existing approaches by up to $2$--$5\%$ in classification accuracy under biased data distributions and noisy conditions.
Reference graph
Works this paper leans on
-
[1]
Joint Graph Estimation and Signal Restoration for Robust Federated Learning
INTRODUCTION Federated learning (FL) is an emerging paradigm in machine learn- ing [1, 2]. It enables collaborative training of neural network(s) by distributing partial models across local servers (referred to as clients). These locally trained models are periodically updated by transmitting model parameters from the clients to the central server, follow...
work page Pith review arXiv 2025
-
[2]
We first briefly introduce the basic form of FL, then the GFL is reviewed
RELA TED WORKS In this section, we briefly review the literature on FL. We first briefly introduce the basic form of FL, then the GFL is reviewed. We illus- trate these FL approaches in Fig. 1. 2.1. Standard FL In standard FL, the central server aims to make a common global model for all clients by aggregating the local model parameters sent from the clie...
-
[3]
First, we formalize the GFL framework
PRELIMINARIES In this section, we introduce the building blocks used in the proposed FL. First, we formalize the GFL framework. Second, we introduce a graph-based aggregation method based on the graph signal smooth- ness assumption. Third, we describe a graph learning method. 3.1. Framework of Graph-based Federated Learning Let us consider GFL with K clie...
-
[4]
(3) Global model update The server aggregates the locally updated parameters by minimizing (2) w.r.t. Ψ, i.e., Ψ(r+1) = arg min Ψ µ 2∥Ψ− X(r+1) local ∥2 Z +R(Ψ), (4) where X(r+1) local = [ x(r+1) 1,local,..., x(r+1) K,local]⊤ are the parameters received by the server at the current round. As observed, GFL is an alternative optimization approach: The clien...
-
[5]
JOINT GRAPH ESTIMA TION AND SIGNAL RESTORA TION FOR FL In this section, we present a graph-based aggregation method for GFL based on JGESR. We first formulate a graph-based aggregation in the global model update as a DC optimization problem and then solve it using PDCA. We illustrate the overview of the proposed FL in Fig. 2. 4.1. Problem Formulation Many...
-
[6]
EXPERIMENTS In this section, we evaluate the performance of the proposed method for image classification using two benchmark datasets. 5.1. Experimental setup Datasets: In the experiments, we use the following two benchmark datasets: 2Our method satisfies the conditions: We omit their details due to space limitation. Algorithm 1: Proposed GFL Input: M, Z,...
-
[7]
CONCLUSION In this paper, we propose a robust aggregation method for GFL un- der noisy communications. First, we assume that the model param- eters, aggregated in the global model update, smoothly vary on the inter-client graph. Based on the assumption, we formulate the graph- based aggregation as a DC optimization problem so that it simulta- neously opti...
-
[8]
Advances and open problems in federated learn- ing,
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cum- mings, et al., “Advances and open problems in federated learn- ing,” Found. Trends Mach. Learn., vol. 14, no. 1–2, pp. 1–210, 2021
work page 2021
Show all 31 references
-
[9]
Communication-efficient learning of deep networks from decentralized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in AISTATS. PMLR, 2017, pp. 1273– 1282
2017
-
[10]
Fed- erated learning for smart healthcare: A survey,
D. C. Nguyen, Q.-V . Pham, P. N. Pathirana, M. Ding, A. Seneviratne, Z. Lin, O. Dobre, and W.-J. Hwang, “Fed- erated learning for smart healthcare: A survey,” ACM Comput. Surv., vol. 55, no. 3, pp. 1–37, 2022
2022
-
[11]
Federated learning: Opportunities and chal- lenges,
P. M. Mammen, “Federated learning: Opportunities and chal- lenges,” arXiv preprint arXiv:2101.05428, 2021
2021 arXiv
-
[12]
Robust federated learning with noisy communication,
F. Ang, L. Chen, N. Zhao, Y . Chen, W. Wang, and F. R. Yu, “Robust federated learning with noisy communication,” IEEE Trans. Commun., vol. 68, no. 6, pp. 3452–3464, 2020
2020
-
[13]
Over- coming noisy and irrelevant data in federated learning,
T. Tuor, S. Wang, B. J. Ko, C. Liu, and K. K. Leung, “Over- coming noisy and irrelevant data in federated learning,” in 25th Int. Conf. Pattern Recogn. (ICPR). IEEE, 2021, pp. 5020– 5027
2021
-
[14]
Federated learning of deep networks using model averaging,
H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,” arXiv preprint arXiv:1602.05629 , vol. 2, no. 2, pp. 15–18, 2016
2016 arXiv
-
[15]
Personalized cross-silo federated learning on non- iid data,
Y . Huang, L. Chu, Z. Zhou, L. Wang, J. Liu, J. Pei, and Y . Zhang, “Personalized cross-silo federated learning on non- iid data,” in AAAI Conf. Artif. Intell., 2021, vol. 35, pp. 7865– 7873
2021
-
[16]
Personalized fed- erated learning with inferred collaboration graphs,
R. Ye, Z. Ni, F. Wu, S. Chen, and Y . Wang, “Personalized fed- erated learning with inferred collaboration graphs,” in ICML. PMLR, 2023, pp. 39801–39817
2023
-
[17]
An ef- ficient framework for clustered federated learning,
A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An ef- ficient framework for clustered federated learning,” NeurIPS, vol. 33, pp. 19586–19597, 2020
2020
-
[18]
The emerging field of signal processing on graphs: Extending high-dimensional data analysis to net- works and other irregular domains,
D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to net- works and other irregular domains,” IEEE Signal Proc. Mag., vol. 30, no. 3, pp. 83–98, 2013
2013
-
[19]
Dc formulations and algorithms for sparse optimization problems,
J. y. Gotoh, A. Takeda, and K. Tono, “Dc formulations and algorithms for sparse optimization problems,”Math. Program., vol. 169, pp. 141–176, 2018
2018
-
[20]
Federated optimization in heterogeneous networks,
T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” Proc. Mach. Learn. Syst., vol. 2, pp. 429–450, 2020
2020
-
[21]
SCAFFOLD: Stochastic controlled averag- ing for federated learning,
S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “SCAFFOLD: Stochastic controlled averag- ing for federated learning,” in Proc. Int. Conf. Mach. Learn. (ICML). PMLR, 2020, pp. 5132–5143
2020
-
[22]
Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints,
F. Sattler, K.-R. M ¨uller, and W. Samek, “Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 8, pp. 3710–3722, 2020
2020
-
[23]
Federated learning with personalization layers,
M. G. Arivazhagan, V . Aggarwal, A. K. Singh, and S. Choud- hary, “Federated learning with personalization layers,” arXiv preprint arXiv:1912.00818, 2019
1912 arXiv
-
[24]
Fed- erated multi-task learning,
V . Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Fed- erated multi-task learning,” Advances in neural information processing systems, vol. 30, 2017
2017
-
[25]
How to learn a graph from smooth signals,
V . Kalofolias, “How to learn a graph from smooth signals,” in AISTATS. PMLR, 2016, pp. 920–929
2016
-
[26]
Convex optimization,
S. Boyd, “Convex optimization,” Cambridge Univ. Press , 2004
2004
-
[27]
Federated learning over wireless fading channels,
M. M. Amiri and D. G¨und¨uz, “Federated learning over wireless fading channels,” IEEE Trans. Wireless Commun., vol. 19, no. 5, pp. 3546–3557, 2020
2020
-
[28]
Federated learning via over-the-air computation,
K. Yang, T. Jiang, Y . Shi, and Z. Ding, “Federated learning via over-the-air computation,”IEEE Trans. Wireless Commun., vol. 19, no. 3, pp. 2022–2035, 2020
2022
-
[29]
Gradient- based learning applied to document recognition,
Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient- based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, 1998
1998
-
[30]
Im- agenet classification with deep convolutional neural networks,
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Im- agenet classification with deep convolutional neural networks,” Adv. Neural Inf. Process. Syst., vol. 25, 2012
2012
-
[31]
Learning laplacian matrix in smooth graph signal representa- tions,
X. Dong, D. Thanou, P. Frossard, and P. Vandergheynst, “Learning laplacian matrix in smooth graph signal representa- tions,” IEEE Trans. Signal Process., vol. 64, no. 23, pp. 6160– 6173, 2016
2016
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.