{"id":"036b812b-c7e5-4ce0-96b2-1f8f558b3102","arxiv_id":"2412.01079","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FedBS improves federated EEG motor-imagery classification by keeping batch-normalization local and using sharpness-aware minimization, outperforming six FL baselines and typically centralized training.","lead":"A federated learning method for brain-computer interfaces, called FedBS, keeps each user's batch-normalization parameters local and adds sharpness-aware training. On three motor-imagery EEG datasets it beats six federated baselines and usually matches or exceeds centralized training, though several gaps are not statistically significant.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FedBS's test-time dependence on batch statistics (test batch size 8, Section IV-H) leaves no specified single-trial inference path, undermining the real-time BCI applicability claimed in the abstract.","rationale":"The reader's weakest_assumption identified the test-batch-size issue, and this is indeed the most load-bearing concern. The central empirical claim is about a method for privacy-preserving BCIs, and BCI use is inherently real-time. The paper's own Figure 6 shows accuracy varies with test batch size, yet no single-trial inference method is provided, so the method cannot be deployed as described for its intended application. Other concerns, such as the fairness of the centralized-training comparison and the equating of FL with privacy, are secondary: they affect interpretation but do not invalidate the core FL comparisons, which remain internally consistent. Because the reported results are valid under the stated batch protocol and the concern is about a missing capability rather than an internal contradiction, the conditional verdict is appropriate. A concrete test with batch size 1 would determine whether the gap is easily closed or requires substantive changes.","tokens_in":20532,"tokens_out":4914,"duration_ms":46719,"concrete_test":"Run the released code with a modified test protocol using test batch size 1 on MI1, MI2, and MI3 with EEGNet, comparing against the batch-size-8 results in Fig. 6. If accuracy drops substantially or no prediction can be produced for the first trial, the paper must specify a single-trial/streaming inference procedure and revise the BCI-applicability claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper evaluates FedBS with a fixed test batch size of 8 (Section IV-C) and computes batch-specific BN statistics at inference (Section III-C, Eqs. 1-2). Section IV-H and Fig. 6 explicitly show that accuracy depends on test batch size, converging only at size 4. In an online BCI, trials arrive one at a time; a single-trial batch has undefined variance, and the paper gives no procedure for single-trial or streaming inference (e.g., using running statistics). The abstract's claim that FedBS 'improves the BCI decoding accuracy' is therefore only supported for offline batch classification, not the real-time use case that motivates the work. This is a gap in the method itself, not merely an omitted implementation detail, because the model's predictions at test time depend on statistics of the entire test batch, which are unavailable when a single trial must be classified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes FedBS, a federated learning algorithm for EEG motor-imagery classification that combines local batch-specific batch normalization (BN) with sharpness-aware minimization (SAM). In FedBS, each client's BN parameters remain local and are not uploaded to the server, while the server aggregates the remaining parameters; at inference time, BN statistics are computed from a test batch. The authors evaluate FedBS against centralized training and six FL baselines (FedAvg, FedProx, SCAFFOLD, MOON, FedFA, GA) on three public MI datasets with three deep-learning backbones, report cross-subject accuracies, perform paired t-tests with Benjamini-Hochberg correction, and include ablation studies. The code is released. The central claim is that FedBS outperforms all compared FL approaches and usually also outperforms centralized training while preserving data privacy.","tokens_in":20719,"tokens_out":7855,"duration_ms":67483,"significance":"The core idea is a plausible and simple combination of two existing techniques (local BN and SAM) applied to FL for EEG, and the paper offers a broad experimental comparison on public benchmarks with three architectures, ablations, and publicly available code. If the claims are fully supported, FedBS would be a useful baseline for privacy-preserving FL in BCI research. However, the current evaluation is offline only and depends on a nonstandard test-time batch, the comparison with baselines is confounded by the test-time normalization protocol, and the statistical support contains a serious pseudoreplication issue, so the significance of the contribution as stated is not yet established.","major_comments":[{"comment":"The method has no defined single-trial inference path. Equations (1) and (2) compute BN statistics from a test batch; Section IV-C fixes the test batch size to 8, and Fig. 6 shows accuracy depends on batch size and converges only at size 4. For an online BCI, trials arrive one at a time and a batch of size 1 has undefined variance. The paper gives no procedure for streaming inference (e.g., using running statistics from a calibration set), so the abstract's claim that FedBS improves BCI decoding accuracy is only supported for offline batch classification. Please add and evaluate a single-trial inference protocol, or revise the claims to clearly state the offline setup.","section":"Section III-C, IV-C, IV-H"},{"comment":"The comparison between FedBS and the baselines is not controlled: FedBS uses batch-specific BN statistics at test time (Eqs. 1-2), whereas the baselines and CT are described as using standard batch normalization, which normally employs running statistics accumulated during training. The accuracy advantage of FedBS could therefore come from the test-time adaptation rather than from the federated training procedure. To support the central claim, the authors should evaluate all methods under the same inference-time normalization protocol, e.g., by also applying batch-specific statistics to the baselines, or by using running statistics for FedBS.","section":"Section III-C and IV-C"},{"comment":"The reported degrees of freedom (53, 83, 71) imply that each of the six random-seed repeats per subject was treated as an independent observation. Since accuracies from the same test subject across repeats are correlated, this is pseudoreplication that inflates the sample size and makes the p-values anti-conservative. The significance tests should be performed on subject-level averaged accuracies (df = n_subjects - 1) or with a mixed-effects model that accounts for subject as a random factor, and the conclusions should be adjusted accordingly.","section":"Appendix A, Tables X-XII"},{"comment":"The abstract's claim that FedBS outperformed centralized training is not supported for all settings: in Table VII, FedBS vs CT is not significant for ShallowConvNet on MI2 (p=0.313) and MI3 (p=0.206), and only borderline on MI1 (p=0.050); for DeepConvNet on MI2, p=0.219. The unqualified statement about outperforming six FL approaches also ignores several non-significant comparisons (e.g., ShallowConvNet on MI3 vs FedAvg, p=0.059, and vs FedProx, p=0.082). Please qualify the claims to reflect the statistical results.","section":"Abstract and Section IV-D, Table VII"}],"minor_comments":[{"comment":"The 'BN' and 'SAM' columns contain garbled symbols (e.g., '/enc-37 /enc-33'), which appear to be check/cross marks that failed to render; please replace them with proper glyphs and ensure the table is readable.","section":"Table VIII"},{"comment":"There are inconsistent spellings: 'DeepConvnet' in Table III should be 'DeepConvNet', and 'FedFa' in Section IV-B should be 'FedFA'.","section":"Tables III and Section IV-B"},{"comment":"Please define the client selection weight P explicitly and state its value in the main experiments; Section IV-C says half of the clients are selected (rounded down), but the mapping to P is not stated, and the expression max(P·K, 1) needs a rounding convention.","section":"Algorithm 1 and Section IV-C"},{"comment":"Please add axis labels to Fig. 6 and describe how 'converged at 4' was determined (e.g., the threshold on accuracy difference from larger batch sizes).","section":"Fig. 6"},{"comment":"The numerical summary 'FedBS outperformed CT by 1.97%' should be accompanied by the qualification that several per-model comparisons were not statistically significant, as detailed in Table VII.","section":"Section IV-D"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope and the proposed method is simple and reasonably motivated. The main concerns are methodological: the test-time batch-size dependence and confounded comparison with baselines, plus the pseudoreplication in the significance tests. These are fixable with additional experiments and re-analysis, so I recommend major revision rather than rejection. No concerns about citation practices; self-citations appear in the background and are appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on FL for EEG. The paper combines two known ideas—local batch normalization and sharpness-aware minimization—into one federated pipeline (FedBS) and tests it on three public MI datasets with three backbones. That combination is genuinely new for this setting, and the empirical work is solid: public data, multiple models, leave-one-subject-out, paired t-tests with FDR correction, ablation studies, and code on GitHub. The consistent accuracy gains over FedAvg and other FL baselines are believable, and the ablation shows both components contribute.\n\nThe soft spots are real but not fatal. The abstract's 'remarkably outperformed centralized training' oversells it: in several cells CT is within noise (e.g., ShallowConvNet on MI2 p=0.31, MI3 p=0.21; DeepConvNet on MI2 p=0.22). The paper's own tables show FedBS is usually better but not uniformly. A more measured claim would help.\n\nThe bigger issue, which the stress-test note gets right, is that FedBS at test time computes BN statistics over a batch of size 8. Section IV-H shows accuracy converges around batch size 4 and drops with smaller batches. For real-time BCI, trials arrive one at a time and a batch of one has undefined variance; the paper gives no single-trial inference procedure (e.g., running statistics or population estimates). So the real-time framing in the intro/abstract is not supported by the method as specified. That's a gap in the method, not just a missing implementation detail.\n\nThe privacy claim is also loose. FL keeps raw data on clients, but the paper does not discuss gradients or model updates leaking information, nor does it add differential privacy or encryption. Equating FL with privacy protection is common in the literature but needs at least a threat model and a caveat.\n\nThe citation pattern looks fine; self-citations are background material and not load-bearing.\n\nOverall: solid, incremental empirical contribution with one load-bearing assumption that needs to be addressed. If you review it, I'd push the authors to either specify a single-trial/streaming inference mode for FedBS or soften the real-time claim, and to tone down the centralized-training comparison. It deserves a serious referee; the experiments are careful enough that the results will be useful as a baseline.","headline":"Solid empirical FL-for-EEG paper with a real test-time batch dependency and an overclaimed centralized comparison; worth reviewing, needs revision.","tokens_in":21195,"tokens_out":2039,"would_cite":true,"duration_ms":17378,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated EEG motor-imagery classification can beat centralized training while keeping raw brain data on users' devices.","keywords":["brain-computer interface","motor imagery classification","federated learning","EEG privacy","batch normalization","sharpness-aware minimization","cross-subject generalization"],"falsifier":"Evaluate FedBS in streaming mode with one trial at a time, so no batch exists for computing BN statistics; if accuracy collapses to chance level, the empirical advantage over centralized training is contingent on batch availability. A concrete check would be reproducing the three-dataset experiments with a test batch size of 1, or showing that a causal running average of past trials preserves the reported performance.","tokens_in":20353,"feed_emoji":"🧠","tokens_out":9031,"duration_ms":75702,"temperature":0.7,"pith_summary":"This paper proposes FedBS, a federated learning method for EEG motor imagery classification that trains a global classifier across many users without moving raw EEG data to a server. FedBS combines local batch-specific batch normalization with a sharpness-aware minimization optimizer, so each client's batch-normalization layer stays private while the server obtains a complete, deployable model. In experiments on three public datasets with three deep-learning backbones, FedBS outperformed six existing federated approaches and, in the large majority of dataset–model combinations, also scored above centralized training that pools all data with no privacy protection. If these results are correct, they indicate that privacy-preserving collaboration among BCI users need not sacrifice accuracy and may even improve cross-subject generalization.","feed_headline":"Federated EEG model beats pooled-data training in 3 benchmarks","feed_subtitle":"Batch-specific normalization plus flat-minima search lets BCI users train jointly without sharing raw brain data.","key_machinery":"The central mechanism is the combination of two named ingredients. First, local batch-specific batch normalization: each client trains its own BN affine parameters (\\gamma, \\$\\beta$), which are never sent to the server, and only the non-BN parameters are aggregated and redistributed; instead of global running statistics, FedBS recomputes the BN mean \\mu_B and variance \\$sigma_B^{2}$ from each batch at both training and test time (Eqs. (1)-(2)), aligning feature distributions across subjects and letting the server model adapt to a new test subject. Second, the SAM optimizer: during local updates, each client first takes a gradient-ascent step of size \\rho to locate a nearby point of higher loss and then descends from there (Eq. (8)), steering the model toward a flat minimum; when such models are averaged, the global model generalizes better. Together these components reduce feature shift among clients and improve cross-subject decoding accuracy.","core_discovery":"In the paper's own terms, the discovery is that a federated classifier can be simultaneously private and more accurate than pooled-data training by keeping batch-normalization parameters local and computing batch-specific statistics at test time, while each client trains with the SAM optimizer. FedBS gives the server a complete model: the server aggregates all parameters but distributes only the non-BN weights, so each client's BN layer remains private; at inference the BN mean and variance are recomputed from the current test batch, adapting the model to a new subject's distribution. The experiments report consistent gains over FedAvg, FedProx, SCAFFOLD, MOON, FedFA, and GA across EEGNet, DeepConvNet, and ShallowConvNet, with statistically significant improvements in most comparisons and with ablation studies confirming that both components contribute.","pith_inferences":["A natural extension would be to replace the required test batch with a short causal buffer of the new subject's own past trials, enabling single-trial real-time BCI operation; this would make FedBS directly usable in streaming settings.","The same recipe—client-private normalization with batch-specific statistics plus flat-minima optimization—could transfer to other federated medical or wearable domains where each device has its own distribution and the server must serve new users.","The result that privacy-preserving federated training can beat centralized pooling is surprising under the usual assumption that more data always helps; it suggests that pooled training may suffer from inter-subject distribution shift that batch-specific normalization naturally counteracts, which deserves a dedicated matched-hyperparameter study.","The ablation shows each component helps separately, so a finer parsing of the contribution of normalization versus flatness could let practitioners adopt only the component their infrastructure supports."],"forward_implications":["If FedBS is correct, a hospital or BCI vendor can train a deployment-ready classifier across many users' data without collecting raw EEG, and that classifier should classify a new user at least as well as one trained on centrally pooled data.","The gains are not tied to a single network: FedBS worked with EEGNet, DeepConvNet, and ShallowConvNet, so it can be inserted into existing EEG deep-learning pipelines.","Unlike FedBN, FedBS leaves the server with a complete, immediately usable classifier rather than a personalized per-client model, which is what enables cross-subject testing.","Test-time batch size is a practical design parameter: accuracy rises with batch size and levels off around four trials per test batch, so the server needs only a small batch of trials from the new subject."],"supporting_citations":[{"why":"Defines FedAvg, the base federated learning algorithm whose batch-normalization handling FedBS modifies and the primary baseline compared.","marker":"[7]"},{"why":"Introduces local batch normalization for federated learning (FedBN); FedBS extends it by keeping BN parameters private and adding batch-specific statistics.","marker":"[8]"},{"why":"Introduces sharpness-aware minimization (SAM), the optimizer used in local training to seek flat minima.","marker":"[10]"},{"why":"FedProx, a state-of-the-art federated baseline that adds a proximal term to handle data heterogeneity.","marker":"[21]"},{"why":"SCAFFOLD, a state-of-the-art federated baseline that uses control variables to correct client drift.","marker":"[22]"},{"why":"MI1 dataset (BCI Competition IV 2a), one of the three public motor-imagery datasets used for evaluation.","marker":"[31]"},{"why":"MI2 dataset (BNCI 2014-002), the second evaluation dataset.","marker":"[32]"},{"why":"MI3 dataset (BNCI 2015-001), the third evaluation dataset.","marker":"[33]"},{"why":"EEGNet, one of the three backbone deep-learning models tested.","marker":"[38]"},{"why":"DeepConvNet and ShallowConvNet, the other two backbone models tested.","marker":"[39]"}],"fun_headline_variants":["Federated EEG model beats centralized training privately","Private FedBS: batch-specific norm beats 6 federated methods","Federated BCI: local norm + flat minima outperform pooling","Privacy-preserving EEG nets surpass centralized accuracy","FedBS: federated EEG with sharpness-aware gains over all baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the server holds a batch of test trials from the new user so it can compute batch-specific batch-normalization statistics at inference; the paper gives no single-trial classification procedure, so in a real-time BCI where trials arrive one at a time this assumption must be handled separately.","fun_headline_variants_meta":{"raw":{"variants":["Federated EEG model beats centralized training privately","Private FedBS: batch-specific norm beats 6 federated methods","Federated BCI: local norm + flat minima outperform pooling","Privacy-preserving EEG nets surpass centralized accuracy","FedBS: federated EEG with sharpness-aware gains over all baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1388,"prompt_tokens":874,"completion_tokens":514,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":430}},"tokens_in":490,"tokens_out":514,"duration_ms":4798,"temperature":1.0,"reasoning_tokens":430,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:42:47.726990+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate FedBS in streaming mode with one trial at a time, so no batch exists for computing BN statistics; if accuracy collapses to chance level, the empirical advantage over centralized training is contingent on batch availability. A concrete check would be reproducing the three-dataset experiments with a test batch size of 1, or showing that a causal running average of past trials preserves the reported performance.","supporting_citations":[{"cited_title":"Communication-Efficient Learning of Deep Networks from De central- ized Data,","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the base federated learning algorithm whose batch-normalization handling FedBS modifies and the primary baseline compared."},{"cited_title":"FedBN: Fede rated Learning on Non-IID Features via Local Batch Normalization ,","cited_arxiv_id":null,"evidence_quote":"Introduces local batch normalization for federated learning (FedBN); FedBS extends it by keeping BN parameters private and adding batch-specific statistics."},{"cited_title":"Shar pness-aware Minimization for Efficiently Improving Generalization,","cited_arxiv_id":null,"evidence_quote":"Introduces sharpness-aware minimization (SAM), the optimizer used in local training to seek flat minima."},{"cited_title":"Federated Optimization in Heterogeneous Networks,","cited_arxiv_id":null,"evidence_quote":"FedProx, a state-of-the-art federated baseline that adds a proximal term to handle data heterogeneity."},{"cited_title":"Scaffold: Stochastic controlled averaging for fe derated learn- ing,","cited_arxiv_id":null,"evidence_quote":"SCAFFOLD, a state-of-the-art federated baseline that uses control variables to correct client drift."},{"cited_title":"Review of the BCI Competition IV,","cited_arxiv_id":null,"evidence_quote":"MI1 dataset (BCI Competition IV 2a), one of the three public motor-imagery datasets used for evaluation."},{"cited_title":"Autocalibration and Recurrent Adaptation: Towards a Plug and Play Online ERD-BCI,","cited_arxiv_id":null,"evidence_quote":"MI3 dataset (BNCI 2015-001), the third evaluation dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DeepConvNet and ShallowConvNet, the other two backbone models tested."}],"review_version":1}