{"id":"9dd3afb7-3612-4ee7-aafe-cf7730685738","arxiv_id":"2501.11706","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A federated Transformer training method that transmits k-means centroids instead of weights, but whose convergence proof is flawed and privacy claim is unsupported.","lead":"Trustformer is a federated learning method that trains a Transformer for translation by having each client cluster its own layer weights and send only the cluster centroids to a central server, cutting communication compared with sending full weights. The paper claims this also prevents privacy leakage, but the proof of convergence to the FedAvg baseline is flawed and the privacy guarantee is not formally established.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Privacy claim contradicted at the tested operating point: with β=0.9, N_oc≈0.9r, so most centroids are singleton clusters equal to raw weight rows; no formal privacy guarantee is provided.","rationale":"The reader identified a real flaw in the FedAvg-convergence proof (Lemma 2), but that theorem is asymptotic and trivially true at N_oc=r; the actual utility claim at β=0.9 is empirical. The more damaging issue is that the tested configuration makes the privacy claim self-contradictory: Equation (3) and the paper's own statement that N_oc=r reduces to FedAvg imply that β=0.9 is nearly the no-privacy baseline. At this setting, most centroids are singleton clusters and hence exact copies of weight rows, so the server receives nearly the full model. The paper's security section offers only qualitative assertions, and the SGX layer is simulated, not a hardware enclave. Because privacy is the paper's primary motivation, this concern is load-bearing and strengthens the rejection. I therefore keep the reader's REJECT verdict unchanged; the paper would need a formal privacy analysis and evaluation at regimes where centroids actually aggregate many weights to become acceptable.","tokens_in":22208,"tokens_out":5358,"duration_ms":58837,"concrete_test":"Reproduce the β=0.9 run from Section 7 and record the transmitted centroid sets C_local for each client and layer. Compute (1) the fraction of centroid rows that are identical to a row of W_local (cluster size 1) and (2) the reconstruction error ||W_local − Ĉ||_F where Ĉ maps each weight row to its transmitted centroid. If the singleton fraction exceeds ~50% or the reconstruction error is below 5%, the 'no privacy leakage' claim is falsified at the tested operating point.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.3 fixes N_oc=⌊r·β⌋. At the only configuration reported for utility (β=0.9, Figures 4–6), N_oc=0.9r, so the average cluster contains r/N_oc≈1.11 weight vectors. k-means on such a setting necessarily produces many singleton clusters; a singleton cluster's centroid is the original weight row itself. Thus the server receives ~90% of the model weights in near-raw form. This directly contradicts the privacy rationale in Section 5.1 that centroids 'due to generalization, do not reveal much information about the weights' and the Section 8 claim that the server 'cannot infer any useful information.' No formal privacy definition (DP, semantic security, or an adversary model) is given; the only concrete protection is a simulated SGX layer (Section 7.1.4), which is not a real TEE and is acknowledged in Section 8 to be vulnerable to side-channel attacks. Since the paper's stated motivation is to prevent privacy leakage, and the tested configuration leaks almost exactly the weights, the central contribution is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Trustformer, a federated learning scheme for training Transformers from scratch. After local training, each client applies k-means clustering to the rows of each layer's weight matrix, sends the resulting centroids to a central server, the server averages the centroids across clients, and each client reconstructs an approximation of the FedAvg global model by adding the difference between global and local centroids to every point in each cluster. The paper claims that this approach provides no privacy leakage, reduces communication overhead, and maintains translation quality comparable to DP-FedAvg, DP-FedSAM, and DP-BLUR-LUS baselines. The central theoretical result, Theorem 1, states that the reconstructed models converge to the FedAvg global model as the number of clusters N_oc approaches the number of parameters r. Experiments on Russian-to-English translation with three clients report loss curves, BLEU/METEOR/BERT F1 scores, communication sizes, and training times.","tokens_in":22414,"tokens_out":7828,"duration_ms":71297,"significance":"The method is straightforward and the experimental setup is clearly described, with code made available, multiple evaluation metrics, and explicit reporting of communication volume and timing. If the central claims were valid, the scheme would be an interesting lightweight baseline for compressed federated learning. However, the theoretical result is not established for the practical regime N_oc < r, and the privacy claim is contradicted at the operating point used for the utility comparisons (β = 0.9). The claimed significance therefore rests on unsupported assertions rather than on the evidence presented.","major_comments":[{"comment":"The proof of Theorem 1 depends on Lemma 2, which asserts that the average of the updated parameters across clients equals the FedAvg global model for every parameter position l. The cancellation in the proof, cavg,j - (1/n)Σ_i ci,j = 0, is valid only if all clients assign a given parameter index l to the same cluster index j. The assumptions in §4.3 (same clustering algorithm, same β, same seed) do not guarantee that k-means, run independently on each client's non-IID local weights, produces aligned cluster assignments. Without aligned assignments, the average of the per-client centroid corrections does not cancel, and Lemma 2 is false. The only part of the theorem that is rigorously true is the exact equality at N_oc = r (Corollary 2), which is tautological because singleton clusters make the centroids equal to the raw parameters. Corollary 1 also asserts that the bounds B and C decrease with N_oc, but B and C are assumed constants in Lemmas 3 and 4, with no formal dependence on N_oc established. Consequently, Theorem 1 does not support the paper's claims of convergence to FedAvg for finite N_oc.","section":"§6 and Appendix, Theorem 1 and Lemma 2"},{"comment":"The privacy guarantee is contradicted at the configuration that provides utility close to the baseline. With β = 0.9, Eq. (3) gives N_oc = floor(0.9 r), so the average cluster contains r/N_oc ≈ 1.11 parameter rows; most clusters are singletons whose centroids are exactly the raw weight rows. The server thus receives roughly 90% of the model parameters essentially unchanged. This directly contradicts §5.1's claim that centroids, 'due to generalization, do not reveal much information about the weights,' and §8's claim that an untrusted server 'cannot infer any useful information.' No formal privacy definition (differential privacy, semantic security, or an explicit adversary model) is given anywhere in the manuscript. The only concrete mechanism, described in §7.1.4, is a simulation of Intel SGX using a fixed key and AES cryptography, and §8 itself acknowledges side-channel attacks such as page-fault attacks. The central privacy motivation is therefore unsupported at the tested operating point.","section":"§5.3, Eq. (3), §7.3.2 Table 3, §8"},{"comment":"The statement that Figure 4 'experimentally underscores the correctness of Theorem 1' is not justified. The figure shows that training loss moves toward the FedAvg baseline as β increases; this is expected for any compression scheme that smoothly decreases the amount of compression. It does not test the specific claim of Lemma 2, namely that the average of the reconstructed models across clients equals the FedAvg model for intermediate N_oc, nor does it provide any measure of the distance between the reconstructed and FedAvg models. With only three clients, ten training epochs, and no repeated runs or confidence intervals, the experiment cannot substantiate a convergence theorem.","section":"§6 and §7.3.1, Figure 4"},{"comment":"The abstract states that the method 'significantly reduces the model size transmitted.' At the operating point where utility is close to the baseline (β = 0.9, Figures 4 and 6), Table 3 shows a reduction from 782,003 KB to 702,608 KB, i.e., only about 10%. The much larger reductions (e.g., β = 0.1 gives 78,019 KB) correspond to configurations with degraded utility, as Figure 4 indicates. While Section 6 explicitly describes the trade-off between N_oc and approximation error, the framing in the abstract and conclusion overstates the communication benefit at the configuration that supports the accuracy claim.","section":"Abstract and §7.3.2, Table 3"}],"minor_comments":[{"comment":"The notation 'N oc' should be typeset consistently as N_oc with a proper subscript; the subscript is inconsistently rendered across the manuscript.","section":"§4.3 and throughout"},{"comment":"In the DP-FedAvg row, the total time (664 s) does not match the sum of training time (563 s) and aggregation time (11 s); either an additional component is missing or this is a typo.","section":"Table 4"},{"comment":"DP-BLUR-LUS is cited as [11] in Section 3 but as [10] in Section 7.3.1 and Figure 5; the reference numbering should be consistent.","section":"§3 and §7"},{"comment":"The abstract says SGX is 'leveraged,' but Section 7.1.4 states that Intel SGX was simulated with fixed-key AES and not run on actual SGX hardware; this discrepancy should be stated clearly in the abstract and introduction.","section":"Abstract and §7.1.4"},{"comment":"The text says that at β = 0.9 all clients have the same loss value, but the figures do not report numeric values or confidence intervals; presenting the actual values and standard deviations would improve clarity.","section":"§7.3.1"}],"recommendation":"reject","confidential_remarks":"The paper's stated novelty is the privacy guarantee combined with reduced communication. Both elements are load-bearing and unsupported: the formal proof is invalid for N_oc < r, and the operating point with acceptable utility transmits almost raw weights. The method is essentially k-means weight quantization applied to federated learning, and the manuscript does not position itself against quantization-based communication compression. In my view, rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The concrete idea—cluster each layer's weights locally with k-means, send only centroids, average centroids on the server, then reconstruct the global model by shifting local weights according to the centroid differences—is clean and easy to implement. It's not revolutionary; it sits naturally in the line of quantization/compression work in FL, but the combination is new as far as I know. The authors ship code and data, and the WMT19 experiments with BLEU/METEOR/BERT F1 show that at beta=0.9 the utility is close to FedAvg. The communication table is honest and useful.\n\nThe soft spots are the load-bearing parts. Theorem 1's proof relies on Lemma 2, which claims the average of updated parameters equals FedAvg for every parameter position. That cancellation only works if every client assigns the same parameter index to the same cluster label. With non-IID data and separate k-means runs, the assignments differ, so the equality breaks. What remains is the trivial N_oc=r case, which just recovers FedAvg. So the formal convergence claim does not cover the regime the method actually uses.\n\nThe privacy claim is worse. At the only utility-tested setting, beta=0.9, N_oc=0.9r, so most clusters are singletons and the centroids are essentially raw weight rows. The server receives about 90% of the model in near-raw form. That directly contradicts the abstract's 'no privacy leakage' and Section 8's 'server cannot infer any useful information.' There is no formal privacy definition, SGX is simulated with fixed-key AES, and the authors concede side-channel attacks. So the privacy argument is not just weak; it's misleading at the operating point they emphasize.\n\nMethodologically, only 3 clients, one translation dataset, and no error bars or variance across runs. The claim of 'significantly reducing' communication at beta=0.9 is 10%, which is modest.\n\nWho's this for: people working on weight compression in FL might find the reconstruction trick worth a look. But as a privacy-preserving framework, the current write-up overclaims.\n\nMy recommendation: send it to peer review. The compression scheme is concrete and reproducible, and a serious referee could push the authors to either prove convergence properly, adopt a real privacy guarantee like DP, or substantially reduce their claims. It needs major revision, but it's not a desk-reject.","headline":"A clean compression trick with reproducible experiments, but the convergence proof is trivial in the general case and the privacy claim is contradicted at the tested operating point.","tokens_in":22956,"tokens_out":2425,"would_cite":false,"duration_ms":24668,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a federated Transformer can be trained from scratch without ever sharing full model weights: each client clusters each layer's weights with k-means, sends only the centroids, and reconstructs a close approximation…","keywords":["Trustformer","federated learning","Transformer","k-means clustering","privacy-preserving machine learning","communication efficiency","Intel SGX","machine translation"],"falsifier":"Run Trustformer and FedAvg on the same deliberately heterogeneous client data (for example, each client gets a different topic or genre), set β to 0.5, and compare each client's reconstructed weights with the FedAvg average after each round; if the per-parameter difference systematically exceeds the bound in the paper's Lemma 5 or the translation scores fall below the DP-FedSAM baseline, then the claimed convergence to FedAvg does not hold for non-IID data.","tokens_in":21981,"feed_emoji":"🔐","tokens_out":12324,"duration_ms":123797,"temperature":0.7,"pith_summary":"The paper is trying to show that privacy-preserving federated learning for Transformers does not have to choose between leaking information and huge communication bills. It proposes Trustformer, in which each client trains a Transformer locally, runs k-means on each layer's weight matrix, and sends only the centroids to the server. The server averages the centroids and returns them, and each client uses the difference between global and local centroids, plus the cluster labels it kept, to reconstruct a close approximation of the global FedAvg model. The paper claims this provides no privacy leakage, cuts per-round data transfer to as little as about 10% of FedAvg's volume at its lowest tested clustering ratio, and on a Russian-to-English translation benchmark maintains translation quality comparable to privacy-preserving baselines. If true, this would make from-scratch federated training of large Transformers more practical in settings where privacy and bandwidth are both constrained.","feed_headline":"Federated Transformer trains on centroids, not weights","feed_subtitle":"Clients send only layer centroids; the server never sees full weights, and traffic drops to about 10% of standard.","key_machinery":"The load-bearing mechanism is centroid-mediated model reconstruction: after local training, each client runs k-means on every layer's weight matrix, keeps the cluster labels private, and transmits only the N_oc centroids; the server averages the centroids into global centroids; each client then moves every weight by the difference between the global and its local centroid for that weight's cluster. This difference-vector update is what lets clients approximate the FedAvg global model without the server ever seeing the weight matrices, with Theorem 1 bounding the approximation error by the cluster count.","core_discovery":"The central claim is that Transformer training can be federated by clustering weights instead of sending them. Each client trains locally, clusters each layer's weight matrix, and transmits only the N_oc centroids; the server averages the centroids and sends back global centroids; each client then shifts its own weights by the centroid differences and obtains a model that, by Theorem 1, converges to the FedAvg global model as N_oc approaches the number of parameters. The authors further claim that this scheme provides no privacy leakage under their threat model, with centroids exchanged inside Intel SGX enclaves, and that it preserves translation quality while substantially reducing communication overhead.","pith_inferences":["An implication the authors leave implicit is that the privacy guarantee is not uniform in β: as β approaches 1 the transmitted centroids approach raw weights, so \"no privacy leakage\" is only meaningful for small clustering ratios unless a formal privacy bound is added.","The authors restrict their claim to training from scratch; the same centroid-exchange mechanism could plausibly extend to federated fine-tuning of pre-trained language models, but that extension is untested.","Because k-means runs independently on each client, non-IID client data can yield mismatched cluster assignments across clients; a natural next experiment is measuring how much reconstruction error grows on skewed data splits.","The paper's security discussion notes two caveats: SGX was simulated with a fixed key and AES rather than real enclave hardware, and page-fault attacks on SGX are acknowledged as out of scope."],"forward_implications":["Per-round communication scales with the clustering ratio β; at β = 0.1 the reported data per round is about 10% of FedAvg's volume, and even at β = 0.9 the traffic is still about 10% lower than FedAvg.","At the limit N_oc = r, the update rule reproduces FedAvg exactly, making Trustformer a tunable generalization of FedAvg rather than a fundamentally different optimizer.","Because the server only averages centroids and never receives a complete weight matrix, it does not hold the final global model, reducing what an attacker gains from a server breach.","In the reported Russian-to-English experiments, the reconstructed model's loss overlaps FedAvg's at β = 0.9, and its translation quality is comparable to DP-FedSAM on BLEU and METEOR while achieving the highest BERT F1 among the compared methods."],"supporting_citations":[{"why":"Defines FedAvg, the baseline global model that Trustformer's reconstruction aims to match and the main communication-cost comparison.","marker":"[33]"},{"why":"Documents gradient-leakage attacks on exchanged updates, motivating the decision to transmit centroids instead of weights or gradients.","marker":"[18]"},{"why":"Shows federated Transformers are vulnerable to reconstruction attacks, the privacy threat Trustformer is designed to counter.","marker":"[17]"},{"why":"DP-FedAvg is a differentially private baseline whose utility and communication cost Trustformer compares against.","marker":"[34]"},{"why":"DP-FedSAM is a state-of-the-art private federated learning baseline used for loss, translation quality, and model-size comparisons.","marker":"[40]"},{"why":"DP-BLUR-LUS is a sparsification-based private baseline used for utility and communication comparisons.","marker":"[11]"},{"why":"BLEU is the primary metric supporting the paper's claim that Trustformer maintains comparable translation quality.","marker":"[37]"},{"why":"METEOR is the second translation-quality metric supporting the comparable-utility claim.","marker":"[13]"},{"why":"BERTScore (BERT F1) is the semantic-similarity metric on which Trustformer reports the best result among compared methods.","marker":"[50]"}],"fun_headline_variants":["Federated Transformers trained on centroids, not weights","Centroids instead of weights slash federated Transformer traffic","Private federated Transformers via k-means centroid sharing","SGX-secured centroid sharing for federated Transformers","Transformer federation slashed to 10% comm via centroids"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof in the appendix that clients arrive at exactly the FedAvg model assumes the same weight positions fall into the same clusters on every client, which is only guaranteed when the number of clusters equals the number of parameters; with different local data the clusters can differ, so the equality is approximate.","fun_headline_variants_meta":{"raw":{"variants":["Federated Transformers trained on centroids, not weights","Centroids instead of weights slash federated Transformer traffic","Private federated Transformers via k-means centroid sharing","SGX-secured centroid sharing for federated Transformers","Transformer federation slashed to 10% comm via centroids"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1421,"prompt_tokens":902,"completion_tokens":519,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":436}},"tokens_in":518,"tokens_out":519,"duration_ms":5950,"temperature":1.0,"reasoning_tokens":436,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:57:10.361849+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Trustformer and FedAvg on the same deliberately heterogeneous client data (for example, each client gets a different topic or genre), set β to 0.5, and compare each client's reconstructed weights with the FedAvg average after each round; if the per-parameter difference systematically exceeds the bound in the paper's Lemma 5 or the translation scores fall below the DP-FedSAM baseline, then the claimed convergence to FedAvg does not hold for non-IID data.","supporting_citations":[{"cited_title":"B., M OORE , E., R AMAGE , D., H AMPSON , S., AND Y ARCAS , B","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the baseline global model that Trustformer's reconstruction aims to match and the main communication-cost comparison."},{"cited_title":"Gradient leakage attacks in federated learning","cited_arxiv_id":null,"evidence_quote":"Documents gradient-leakage attacks on exchanged updates, motivating the decision to transmit centroids instead of weights or gradients."},{"cited_title":"B., R AMAGE , D., T ALWAR, K., AND ZHANG , L","cited_arxiv_id":null,"evidence_quote":"DP-FedAvg is a differentially private baseline whose utility and communication cost Trustformer compares against."},{"cited_title":"Make landscape flatter in differentially private federated learning","cited_arxiv_id":null,"evidence_quote":"DP-FedSAM is a state-of-the-art private federated learning baseline used for loss, translation quality, and model-size comparisons."},{"cited_title":"S., AND CHENG , J","cited_arxiv_id":null,"evidence_quote":"DP-BLUR-LUS is a sparsification-based private baseline used for utility and communication comparisons."},{"cited_title":"Meteor universal: Language spe- cific translation evaluation for any target language","cited_arxiv_id":null,"evidence_quote":"METEOR is the second translation-quality metric supporting the comparable-utility claim."}],"review_version":1}