{"id":"63697dcd-31ba-40ae-a412-1f83676bc60a","arxiv_id":"2412.16144","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"FedGAT uses a truncated Chebyshev polynomial approximation of the GAT attention score to enable federated GAT training with a single pre-communication round.","lead":"This paper introduces FedGAT, an algorithm that trains Graph Attention Networks in a federated setting while sending node information between clients only once before training. It approximates attention scores with a polynomial so that per-round updates can be computed without exchanging node features.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Multi-layer FedGAT requires per-round cross-client embedding exchange, so the one-round claim fails for the 2-layer GATs used in the experiments.","rationale":"The central claim of the paper is that FedGAT trains a GAT in a federated setting with only one pre-training communication round while approximately matching centralized GAT accuracy. The reader's weakest assumption identified the multi-layer extension as the point where this claim breaks: for L>1, regular GAT updates require cross-client embeddings that are functions of the current trainable parameters, so they change every round and cannot be pre-communicated. I independently traced Algorithm 2 and confirmed that the l=2 attention score for a cross-client edge requires the other client's first-layer embedding at the current parameter state. The paper's response is to assume clients can view such embeddings, which is not free communication and is never counted in the overhead analysis. The reported experiments use two-layer GATs, so the empirical validation is not a test of the one-round multi-layer method. This is a correctness risk in the central contribution, not a stylistic issue. The single-layer algebraic construction may be salvageable, and the Chebyshev-domain issue is additional, but the multi-layer communication gap is the most load-bearing concern because it invalidates the headline claim for standard GAT configurations. The reader's verdict of REJECT remains appropriate.","tokens_in":23406,"tokens_out":4159,"duration_ms":39984,"concrete_test":"Run a strict communication-budget version of Algorithm 2 on Cora with the Appendix C 2-layer GAT: after the pre-training round, forbid all inter-client messages, so for l=2 each client can use only the cross-client first-layer embeddings that were made available during the pre-training round, computed with the initial W^{(0)}. Compare test accuracy to Table 1's FedGAT entry (0.80 ± 0.005, non-iid). If accuracy drops materially, the multi-layer one-round claim fails. As a complementary check, instrument Algorithm 2 and count the number of inter-client messages required to supply h_j^{(1)} to each cross-client neighbor for every global round t=1..T; report whether that count is 1 or Theta(T * |E_cross|).","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4's multi-layer extension assumes 'nodes are permitted to view the embeddings of any other node generated after the first GAT layer, including nodes on other clients.' This assumption is load-bearing because Algorithm 2 uses the regular GAT update, Eqs. (1)-(3), for every layer l>1. For a cross-client edge (i,j), the l=2 attention score e_{ij}^{(2)} = exp(psi(a1^T W h_i^{(1)} + a2^T W h_j^{(1)})) requires h_j^{(1)}, which is computed at client C_k using the current global parameters W^{(t)} and changes every training round. Providing that embedding to client C_i is an inter-client communication event; it cannot be folded into the single pre-training round unless W is frozen or the embeddings are stale. The paper neither specifies a protocol for this sharing nor includes it in Theorem 1's communication analysis. Since the experiments (Appendix C) use a 2-layer GAT, the headline claim of 'only one pre-training communication round' is unsupported for exactly the setting evaluated. The single-layer construction may be coherent, but it does not extend to L>1 without additional per-round communication.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedGAT, a federated training algorithm for Graph Attention Networks (GATs) on graphs partitioned across clients. The method approximates the attention scores exp(ψ(x_ij)) by a truncated Chebyshev series, re-expresses the result as a power series in x_ij, and pre-computes parameter-independent aggregate matrices (M1, M2, K1, K2) during a single pre-training communication round. These aggregates then allow clients to compute approximate GAT updates without exchanging node features during training. The paper provides a communication overhead analysis, approximation error bounds across layers, a heuristic privacy analysis, and experiments on Cora, Citeseer, and Pubmed showing accuracy close to centralized GAT and above FedGCN and DistGAT.","tokens_in":23477,"tokens_out":5857,"duration_ms":52872,"significance":"The single-layer construction is original and addresses a real obstacle in federated GAT training: attention scores depend on features at both endpoints of cross-client edges and change every training round. If the one-communication-round claim were fully supported, the paper would be a meaningful advance, since existing methods either drop cross-client edges or incur per-round communication. The paper includes hand-written proofs for the approximation error and a communication complexity theorem, and the empirical results are promising. However, the multi-layer extension is not compatible with the one-round claim as written, and the Chebyshev error bounds are applied outside their stated domain. These issues concern the paper's central contributions and need to be resolved before the claims can be accepted.","major_comments":[{"comment":"The one-round communication claim is unsupported for multi-layer GATs. The paper assumes 'nodes are permitted to view the embeddings of any other node generated after the first GAT layer, including nodes on other clients,' but for L>1 the regular GAT update (Eqs. 1-3) used in Algorithm 2 requires, for each cross-client edge (i,j), the current embedding h_j^{(l-1)}, which is a function of the current global parameters and changes every training round. No protocol for sharing these embeddings is specified, and Theorem 1 counts only the pre-training communication. Since Appendix C evaluates a 2-layer GAT, the experiments exercise exactly the setting in which the one-round claim is not established.","section":"Section 4, 'FedGAT for Multiple GAT Layers'; Algorithm 2; Theorem 1"},{"comment":"The Chebyshev approximation and its error bound are stated for a function on [-1,1], but the argument x_ij = b_1^T h_i + b_2^T h_j is a learned, potentially unbounded scalar that depends on the current parameters and features. The paper does not restrict x_ij to [-1,1], does not rescale or clamp it, and does not re-derive the Chebyshev coefficients or error bounds for the actual range of x_ij. Consequently, Theorems 2-5 do not apply to the algorithm as implemented, and the 'provable bounds' claim is not supported.","section":"Section 4, Eq. (5); Section 5, Theorem 2"},{"comment":"The privacy argument is informal and does not match the strong 'privacy-preserving' claim in the title and abstract. The analysis shows that certain products of K1_i and K2_i recover only aggregate sums of features, but it does not define an adversary model, does not account for auxiliary information or repeated queries, and does not quantify the leakage through the M2_i(s) matrices when the random U_j masks are unknown to the observer. Section 7 correctly identifies 'theoretical privacy guarantees' as future work, which conflicts with the unqualified privacy claim made earlier in the paper.","section":"Section 5, 'Privacy Analysis of FedGAT'"},{"comment":"The error propagation results assume a Lipschitz, monotone activation ψ (Assumption 4), but Theorem 2 requires f to be k-times differentiable with f^{(k)} of bounded variation. Standard GAT activations such as LeakyReLU are not differentiable at zero, so the composition exp(ψ(x)) does not satisfy the differentiability condition needed for the stated Chebyshev convergence rate. The paper should either state which activation functions the error bounds actually cover or derive bounds that do not rely on higher-order differentiability.","section":"Section 5, Theorems 3-5; Appendix E"}],"minor_comments":[{"comment":"In the definition of M2_i(s), the symbol 'A_j' appears instead of the earlier-defined 'U_j'; this makes the algorithm ambiguous.","section":"Appendix A, Algorithm 1"},{"comment":"The text says 'FedCGN (Yao et al. 2023a)' but the correct name is FedGCN; please correct this typo.","section":"Section 6, 'Methods Compared'"},{"comment":"The expression 'b2 + hj(s)' should presumably read 'b2(s) + hj(s)' or 'b2(s)hj(s)'; as written, it is unclear.","section":"Section 4, after Eq. (14)"},{"comment":"In the chain of inequalities, the term 'ˆα_ij' appears inside a sum over k; this should likely be 'ˆα_ik' or similar, and the derivation should be cleaned up.","section":"Appendix E, proof of Claim 2"},{"comment":"The symbol B is used for both the maximum node degree in Section 5 and the background appendix title; this is a minor notational collision that could confuse readers.","section":"Section 5, 'Communication Overhead'"},{"comment":"The conclusion states FedGAT is 'one of the first' algorithms while the introduction says 'to the best of our knowledge, this is the first work'; please make these claims consistent.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's own conclusion admits that FedGAT is 'not optimized to handle GATs with several layers (more than 2),' which is in tension with the abstract's unqualified 'requires only one pre-training communication round' and with the 2-layer experiments. The multi-layer communication issue and the Chebyshev domain issue are both fixable in principle, but they require either scoping the claims to single-layer GATs or adding an explicit per-round embedding exchange protocol and redoing the communication analysis. If the authors choose the former, the experiments would need to be redone for the scoped setting; if the latter, the headline contribution changes substantially. I recommend major revision rather than rejection because the single-layer idea is sound and the empirical results are promising."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the core idea—approximate the GAT attention score with a polynomial so all cross-client feature exchange collapses into a single pre-training round—is genuinely new, and the single-layer algebra is coherent. The experiments are also believable: on Cora/Citeseer/Pubmed, FedGAT tracks centralised GAT accuracy and beats FedGCN. That is real value.\n\nThe soft spots are not minor. The one-round claim breaks for multi-layer GATs. Section 4's extension assumes nodes can see each other's post-layer-1 embeddings across clients. Those embeddings depend on the current weights, so they change every round; sharing them is per-round communication. The paper never specifies the protocol or counts the cost. Since the main experiments use a 2-layer GAT, the headline claim fails for exactly the setting evaluated. That alone is a load-bearing flaw.\n\nSecond, the provable error bounds don't follow as stated. Eq. (5) expands exp(psi(x_ij)) as a Chebyshev series on R, but Theorem 2 (the quoted bound) is for [-1,1]. Without a rescaling argument or a bound on the range of x_ij under the norm assumptions, Theorems 3-5 rest on a misapplied theorem. This is fixable but needs to be done.\n\nThird, the privacy framing oversells. In the actual algorithm the server receives every raw feature vector in plaintext; homomorphic encryption is mentioned as future work. The client-side aggregates may resist straightforward inversion, but that is a heuristic claim, not a guarantee. Also, the communication bound is self-contradictory: Theorem 1 says O(K B^L d B^3), while the appendix proof concludes O(K B^L d B^2).\n\nWhat is salvageable: the single-layer construction is a legitimate extension of FedGCN's one-round scheme to attention-based models, and the empirical study is reasonable. Multi-layer can likely be fixed by either accepting per-round sharing for L>1 or restricting the method to one-layer GATs, and the domain issue by rescaling.\n\nThis paper deserves a serious referee, but not because it is right as written—it is a solid idea with three correctable defects. A reading group would get good value from dissecting the multi-layer gap.","headline":"Clever single-layer trick, but the one-round claim collapses for the multi-layer GATs actually used in the experiments.","tokens_in":24186,"tokens_out":4318,"would_cite":true,"duration_ms":35661,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedGAT claims that the attention scores of a Graph Attention Network can be replaced by a precomputable Chebyshev polynomial expansion, so federated GAT training needs only one pre-training communication round.","keywords":["federated learning","graph attention networks","node classification","Chebyshev approximation","communication efficiency","privacy-preserving graph learning","cross-client edges","semi-supervised learning"],"falsifier":"On a small graph, run the pre-training round and train a single-layer FedGAT, comparing the approximate attention coefficients to the centralized GAT. If the maximum error does not shrink as the Chebyshev degree $p$ increases, the error-bound theorem fails. Separately, try training a 3-layer FedGAT while enforcing strict client isolation, so no client ever receives another client's post-first-layer embeddings; if the updates cannot be computed or accuracy drops to the edge-dropping baseline, the one-round claim does not hold for more than one layer.","tokens_in":23054,"feed_emoji":"🤝","tokens_out":8990,"duration_ms":75341,"temperature":0.7,"pith_summary":"The paper proposes a federated training method for Graph Attention Networks (GATs) on graphs split across clients. In a GAT, updating a node requires attention scores to cross-client neighbors, and those scores depend on learnable parameters, so they change every training round; naive sharing is costly and leaks features. FedGAT expands each attention score in a Chebyshev polynomial and rewrites the update so all feature-dependent terms become parameter-independent matrices that can be computed once and shared before training. The result is a single pre-training communication round, with error bounds showing the approximation can be made arbitrarily tight, and experiments on Cora, Citeseer, and Pubmed show accuracy close to a centralized GAT across client counts and data splits.","feed_headline":"Federated GATs need only one pre-training round","feed_subtitle":"Chebyshev-approximated attention scores match centralized GAT accuracy on citation networks.","key_machinery":"The load-bearing device is a truncated Chebyshev series for the attention score. The paper defines $x_{ij} = b_1^T h_i + b_2^T h_j$ and expands $\\exp(\\psi(x_{ij}))$ as $\\sum_{n=0}^p q_n x_{ij}^n$. It then builds, for each node $i$, idempotent matrices $U_j$ from a set of orthonormal vectors, so that the weighted sum $D_i = \\sum_{j \\in N_i} x_{ij} U_j$ obeys $D_i^n = \\sum_{j \\in N_i} x_{ij}^n U_j$. With auxiliary vectors $K_{1i}$ and $K_{2i}$, this turns the graph-neighbourhood sums $E_i^{(n)}$ and $F_i^{(n)}$ into matrix products $K^T D^n K$, and the matrices inside $D_i$ are separated into parameter-independent pieces $M_{1i}(s)$ and $M_{2i}(s)$ that can be precomputed and transmitted once.","core_discovery":"The central claim is that a GAT update can be approximated in a federated setting without dropping cross-client edges and without per-round feature exchange. The paper writes the attention score $e_{ij} = \\exp(\\psi(x_{ij}))$ as a truncated power series in $x_{ij}$, and then shows the sums $E_i^{(n)}$ and $F_i^{(n)}$ needed for the update can be recovered from four quantities ($M_{1i}(s)$, $M_{2i}(s)$, $K_{1i}$, $K_{2i}$) that depend only on fixed input features and a private orthonormal construction. Because those quantities do not depend on the learnable parameters, they are shared exactly once before training; after that, clients compute approximate GAT updates locally and only exchange model parameters. Theorems bound the error in attention coefficients and embeddings, showing the error shrinks with polynomial degree and propagates across layers at a rate controlled by the Lipschitz constants of the activations.","pith_inferences":["The single-round trick likely extends beyond GATs to any attention layer whose score is exp of a bilinear form over fixed input features, so the same precomputation idea could apply to first layers of transformers in federated settings.","For more than one layer, the paper's assumption that post-first-layer embeddings are freely visible is a second implicit communication round (or a trusted shared memory); a complete protocol would need to specify how those embeddings are exchanged before the one-round claim covers multilayer GATs.","The aggregate-only privacy guarantee is heuristic, not cryptographic; pairing the pre-training exchange with secure aggregation or homomorphic encryption, which the paper names as a future direction, would convert it into a computational guarantee.","Because FedGAT keeps all cross-client edges, its robustness to data heterogeneity is better explained by edge retention than by the federated averaging scheme; an ablation that randomly removes cross-client edges could separate the two effects."],"forward_implications":["Federated GAT training becomes communication-feasible: feature-derived information is exchanged in a single pre-training round, and all later rounds exchange only model parameters.","Cross-client edges can be retained, so accuracy does not degrade with the number of clients or with non-iid label distributions the way edge-dropping baselines do.","The approximation error is tunable: increasing the Chebyshev degree decreases attention-score error, and the propagation bounds say the final embedding error stays controlled for the shallow GATs used in practice.","Privacy is preserved in aggregate form: the shared objects reveal only neighbourhood sums, and the algorithm drops a cross-client neighbor when it would be the only one, avoiding direct feature recovery.","Computational and communication costs grow with the maximum node degree ($O(K B^L d B^2)$ communication), making the method suited to sparse graphs; an appendix variant lowers the per-node cost at the price of weaker privacy for special features."],"supporting_citations":[{"why":"Defines the GAT update equations and attention mechanism that FedGAT approximates.","marker":"Veličković et al. 2018"},{"why":"Supplies the Chebyshev-series truncation error bound used to prove FedGAT's approximation error can be made small.","marker":"Trefethen 2019"},{"why":"FedGCN demonstrates one-round pre-training communication of cross-client information and is the baseline FedGAT extends to attention scores.","marker":"Yao et al. 2023a"},{"why":"FedAvg is the parameter-aggregation algorithm used in the experiments and the default assumption for training rounds.","marker":"McMahan et al. 2017"},{"why":"Defines GCNs, the simpler convolution baseline and the motivation for why attention requires additional information.","marker":"Kipf and Welling 2017"},{"why":"Provides the federated GNN setting that drops cross-client edges, used as the DistGAT baseline.","marker":"He et al. 2021"},{"why":"FedSage+ is the alternative that drops cross-client edges and learns missing neighbors, contrasted with FedGAT's direct retention.","marker":"Zhang et al. 2021b"},{"why":"Establishes the Chebyshev approximation background for using Chebyshev polynomials in function approximation.","marker":"Trefethen 1981"}],"fun_headline_variants":["FedGAT: one pre-training round for near-central GAT accuracy","FedGAT matches centralized GAT with one pre-training round","One shared round gives GAT accuracy in federated setting","FedGAT: no per-round exchange, near-central accuracy","Privacy-preserving GAT approximation with one-shot feature share"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For more than one GAT layer, the method assumes that embeddings produced after the first layer can be viewed by nodes on other clients, and the paper gives no protocol or communication cost for sharing those changing embeddings.","fun_headline_variants_meta":{"raw":{"variants":["FedGAT: one pre-training round for near-central GAT accuracy","FedGAT matches centralized GAT with one pre-training round","One shared round gives GAT accuracy in federated setting","FedGAT: no per-round exchange, near-central accuracy","Privacy-preserving GAT approximation with one-shot feature share"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001942,"raw_usage":{"total_tokens":7637,"prompt_tokens":1029,"completion_tokens":6608,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":6522}},"tokens_in":645,"tokens_out":6608,"duration_ms":40221,"temperature":1.0,"reasoning_tokens":6522,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:46:12.247020+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small graph, run the pre-training round and train a single-layer FedGAT, comparing the approximate attention coefficients to the centralized GAT. If the maximum error does not shrink as the Chebyshev degree $p$ increases, the error-bound theorem fails. Separately, try training a 3-layer FedGAT while enforcing strict client isolation, so no client ever receives another client's post-first-layer embeddings; if the updates cannot be computed or accuracy drops to the edge-dropping baseline, the one-round claim does not hold for more than one layer.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Chebyshev-series truncation error bound used to prove FedGAT's approximation error can be made small."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the Chebyshev approximation background for using Chebyshev polynomials in function approximation."}],"review_version":1}