REVIEW 2 major objections 5 minor 35 references
Federated Learning for Epileptic Seizure Prediction Across Heterogeneous EEG Datasets
T0 review · 2 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper claims that a fixed-size random subset per client, aggregated by simple averaging, removes most of the site bias that standard federated averaging exhibits on heterogeneous EEG.
desk verdict Useful benchmark with a real fairness finding, but the split-vs-augmentation ordering is unresolved and puts the headline numbers on hold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is Random Subset Aggregation, a modified FedAvg in which each client draws a fixed-size subset $\tilde{D}_k$ of size $M$ without replacement at every local epoch, computes its update on that subset, and only then are client models averaged equally by the server. Because all clients see exactly $M$ samples per epoch, dataset size no longer controls influence during aggregation; $M$ is treated as a tunable parameter, and the reported peak occurs at $M = 10{,}000$, with smaller $M$ underbalancing and larger $M$ regressing toward the biased aggregate. Supporting machinery is a privacy-preserving global normalization that uses zero-sum masking to compute the global mean and
What would settle it
Run the pipeline twice, once with sliding-window augmentation after the split and once before it, holding everything else fixed; if the before-split version keeps Helsinki near 81.7% and NCH near 68.7%, the claim survives, while a drop toward 50% would show leakage drives the gains. Inspecting the released training code for the order of augmentation and splitting settles it directly.
Extended reading notes
Core claim
Across four heterogeneous EEG datasets treated as separate hospitals, models trained on one site fail on the others, and weighted FedAvg favors the largest dataset, leaving Helsinki and NCH near chance. The central claim: Random Subset Aggregation—each client samples $M$ records without replacement per local epoch, then the server averages equally—restores balance. Helsinki accuracy rises from 50.8% to 81.7%, NCH from 50.6% to 68.7%, Siena to 76.6%, and pooled accuracy reaches 80.0%, within a point of the 81.2% centralized non-private model. The paper frames this as the first federated seizure-prediction study across these four datasets, using a single common EEG channel (F3-C3) and the ligh
Load-bearing premise
The reported accuracies rest on the assumption that the sliding-window augmentation is applied only after the 80/10/10 split, so no overlapping two-second window from the same raw recording appears in both training and test sets; if the overlap runs across the split, the same signal can leak into both sets and inflate every accuracy.
Editorial extensions
If this is right
- A federated model with balanced client sampling reaches 80.0% pooled and 77.1% macro accuracy—within about one point of the centralized model on the pooled test set.
- Under-represented hospitals move from near-chance performance under FedAvg to 81.7% accuracy on Helsinki and 68.7% on NCH, so FL can produce a usable global model for small or atypical sites.
- Balancing also helps the second-largest site: Siena reaches 76.6% accuracy, slightly above the centralized model's 75.8%, at the cost of a modest drop on the largest site, CHB-MIT.
- The per-client subset size matters: macro accuracy peaks at $M = 10{,}000$ and declines at both smaller and larger $M$, so equal contribution alone is not sufficient.
- Combined with weight-only exchange and secure global normalization, the approach supports a privacy-preserving cross-hospital training pipeline with no raw EEG leaving any site.
Reading between the lines
- The same fixed-size-subset rule should transfer to other imbalanced federated medical tasks, such as imaging or ICU monitoring, where one or two large sites typically dominate; a direct test is comparing macro-accuracy against weighted FedAvg on those tasks.
- The paper leaves open how to choose $M$ without leaking the smallest dataset's size; a natural extension is adaptive or noisy per-client sampling rates that preserve equal influence without revealing $M$.
- The dome-shaped accuracy curve in $M$ suggests a bias-variance tradeoff—needing enough samples to learn each site's distribution without letting large sites reassert dominance—which could be formalized as a data-informed hyperparameter choice.
- If the reported numbers survive a leakage-free re-analysis, the practical implication is that multi-hospital seizure-prediction consortia can collaborate with weak privacy infrastructure (masking plus weight exchange) and still approach the accuracy of a centralized model.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies federated learning (FL) for epileptic seizure prediction across four heterogeneous public EEG datasets (Siena, CHB-MIT, Helsinki, NCH), simulating distinct hospitals. The authors propose Random Subset Aggregation, in which each client trains on a fixed-size random subset of its local data per round, and aggregate by simple averaging. They compare this with weighted/unweighted FedAvg and with a centralized, non-private model. They report that the proposed method substantially improves accuracy on under-represented sites (Helsinki 81.7%, NCH 68.7%) and achieves a macro-average accuracy of 77.1% and pooled accuracy of 80.0%, while reducing the skew of standard FedAvg. The paper also describes a privacy-preserving global normalization protocol and a full sweep over the subset-size parameter M.
Significance. If the reported numbers are clean, the central finding is practically useful: a simple, communication-inexpensive sampling rule can mitigate the non-IID bias of FedAvg in cross-hospital seizure prediction without personalization or additional regularization. The paper's strengths include a clearly stated problem setup, public datasets, a released code repository, reporting of accuracy/F1/AUROC, and a full M-sweep that shows the qualitative improvement on under-represented sites is not an artifact of a single parameter value. However, the headline quantitative claims rest on a methodological detail that the manuscript does not specify: whether the sliding-window augmentation is applied before or after the train/validation/test split. If augmentation precedes splitting, near-duplicate overlapping windows can leak into the test set and inflate all reported accuracies, including the claimed superiority over FedAvg. This must be resolved before the paper's empirical contribution can be accepted.
major comments (2)
- [§III-B, §V] The manuscript never states whether the sliding-window augmentation described in Section III-B is applied before or after the 80/10/10 split described in Section V. Table II reports substantial overlaps (95% for CHB-MIT, 75% for NCH, 62% for Helsinki, 91% for Siena). If the split is applied after augmentation—or if the split is at the segment level rather than at the patient/recording level—then overlapping (or adjacent) windows from the same raw EEG recording can appear in both training and test sets, allowing the model to memorize near-duplicates. This is load-bearing for the abstract's headline claims (81.7% Helsinki, 68.7% NCH, 77.1% macro-average, 80.0% pooled) and for the comparison against FedAvg baselines. Please state the ordering explicitly, verify it in the code, and if necessary re-run all experiments with a patient/recording-level split applied before augmentation, and re-re
- [§V-D, Table VI] The headline subset size M=10,000 is selected as the 'best performing subset size' from the test metrics (Figure 4, Tables VI and VII), and the paper does not describe what the 10% validation split is used for. The full M-sweep mitigates this concern because all evaluated M>1,000 improve Helsinki/NCH over FedAvg, but the specific claims '77.1% macro-average' and 'best performing subset size M=10k' are partly self-selected on the test set. Please select M using the validation set (or describe early stopping and hyperparameter choices), or explicitly frame the M-sweep as exploratory and the reported peak as a test-set observation rather than a model-selection result.
minor comments (5)
- [§V-B vs §V-C] The centralized model is called an 'ideal upper bound' in Section V-B, but the proposed method later exceeds it on NCH (68.7% vs 66.4%) and Siena (76.6% vs 75.8%). Please reconcile this terminology or rephrase the centralized model as a reference baseline rather than an upper bound.
- [§III-C] The modifications to TinySleepNet are described only vaguely ('kernel size, number of filters, removing dropout layers'). Please provide the exact architecture (layer types, kernel sizes, filter counts, stride/pooling, activation) or a precise pointer to the code, so the experiments are reproducible without inspecting an external repository.
- [Table II] The 'Window Overlap' column is formatted inconsistently: CHB-MIT shows '60 - 95%' and Siena shows '7 - 91%', while Helsinki and NCH show single percentages. It is unclear whether the first number is a preictal window length in minutes or something else. Clarify the header, units, and how the per-patient stride is computed.
- [§V] The text says 'All results are reported over 5 runs of 80-10-10 train-validation-test splits' but never states how the validation split is used. Please specify whether validation is used for early stopping, hyperparameter selection, or not at all.
- [Fig. 4] The bars in Figure 4 are annotated with mean values but no error bars are shown, despite the text reporting standard deviations. Consider adding error bars, especially for Helsinki and NCH F1-scores where the standard deviations in Tables IV and V are very large (up to 20.0).
Circularity Check
No significant circularity: the empirical FL comparison rests on external datasets, an algorithmically defined aggregation rule, and a fully disclosed M sweep.
full rationale
The paper contains no derivation chain that reduces to its own inputs. The central claim is an empirical comparison of federated aggregation methods on four public EEG datasets, not a derived theoretical result. Random Subset Aggregation is defined algorithmically in Section IV-C and Algorithm 1 as fixed-size per-client random sampling followed by equal averaging; this definition is independent of any outcome. The best subset size M=10,000 is selected from a reported performance sweep, and the paper explicitly states 'we treat M as a tunable parameter' and presents the full M sweep in Section V-D (Figure 5, Tables VI and VII). Because the qualitative improvement on underrepresented sites holds across the sweep (e.g., at M=1,000, Helsinki accuracy is 64.3% vs 50.8% for weighted FedAvg, and NCH is 57.9% vs 50.6%), the headline comparison is not forced by construction. No load-bearing self-citations are present; references to prior work such as TinySleepNet, FedSampling, and EEG windowing studies are external context for architecture, preprocessing, and related methods. The possible ambiguity about whether sliding-window augmentation is applied before or after the 80/10/10 split (Section III-B vs Section V) is a data-leakage/correctness risk, not a circularity of derivation, and therefore does not affect this score.
Assumptions & free parameters
free parameters (3)
- M (subset size) =
10,000
- E (local epochs per round) =
1
- Per-patient augmentation stride =
dataset-specific overlaps: CHB-MIT 95%, Helsinki 62%, Siena 91%, NCH 75%
assumptions (4)
- domain assumption Secure aggregation requires synchronous participation of all clients with no dropouts; zero-sum masks cancel only if all clients contribute.
- domain assumption Seizure annotations across CHB-MIT, Helsinki, Siena, and NCH are reliable and mutually consistent enough for binary preictal/interictal labeling.
- domain assumption A single EEG channel F3-C3, reconstructed differently per dataset, preserves the discriminative signal for seizure prediction.
- domain assumption The 80/10/10 split is applied before data augmentation, so no overlapping augmented windows appear in both train and test.
Cite this review
Pith. "Pith review of Federated Learning for Epileptic Seizure Prediction Across Heterogeneous EEG Datasets." pith.science (2026). https://pith.science/paper/VWIZ3YLK
@misc{pith2026250808159,
author = {Pith},
title = {Pith review of: Federated Learning for Epileptic Seizure Prediction Across Heterogeneous EEG Datasets},
year = {2026},
howpublished = {\url{https://pith.science/paper/VWIZ3YLK}},
note = {Machine review of arXiv:2508.08159}
}
read the original abstract
Developing accurate and generalizable epileptic seizure prediction models from electroencephalography (EEG) data across multiple clinical sites is hindered by patient privacy regulations and significant data heterogeneity (non-IID characteristics). Federated Learning (FL) offers a privacy-preserving framework for collaborative training, but standard aggregation methods like Federated Averaging (FedAvg) can be biased by dominant datasets in heterogeneous settings. This paper investigates FL for seizure prediction using a single EEG channel across four diverse public datasets (Siena, CHB-MIT, Helsinki, NCH), representing distinct patient populations (adult, pediatric, neonate) and recording conditions. We implement privacy-preserving global normalization and propose a Random Subset Aggregation strategy, where each client trains on a fixed-size random subset of its data per round, ensuring equal contribution during aggregation. Our results show that locally trained models fail to generalize across sites, and standard weighted FedAvg yields highly skewed performance (e.g., 89.0% accuracy on CHB-MIT but only 50.8% on Helsinki and 50.6% on NCH). In contrast, Random Subset Aggregation significantly improves performance on under-represented clients (accuracy increases to 81.7% on Helsinki and 68.7% on NCH) and achieves a superior macro-average accuracy of 77.1% and pooled accuracy of 80.0% across all sites, demonstrating a more robust and fair global model. This work highlights the potential of balanced FL approaches for building effective and generalizable seizure prediction systems in realistic, heterogeneous multi-hospital environments while respecting data privacy.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[35]
Fedsampling: a bet- ter sampling strategy for federated learning,
T. Qi, F. Wu, L. Lyu, Y . Huang, and X. Xie, “Fedsampling: a bet- ter sampling strategy for federated learning,” in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence , pp. 4154–4162, 2023
work page 2023
- [1]
-
[2]
EPILEPSIAE: A European epilepsy database,
I. Obeid, P. D. Kellmeyer, L. Kuhlmann, J.-D. Lemarechal, C. Abdal- lah, F. B. Ali, et al. , “EPILEPSIAE: A European epilepsy database,” Computers in Biology and Medicine , vol. 101, pp. 129–134, Oct. 2018
work page 2018
-
[3]
Domain adaptation for eeg-based, cross-subject epileptic seizure pre- diction,
I. Jemal, L. Abou-Abbas, K. Henni, A. Mitiche, and N. Mezghani, “Domain adaptation for eeg-based, cross-subject epileptic seizure pre- diction,” Frontiers in Neuroinformatics, vol. 18, p. 1303380, 2024
work page 2024
-
[4]
Epileptic seizure detection using machine learning methods,
S. Usman, M. Usman, M. A. Usman, S. M. Adnan, M. S. Alkatheiri, and M. Al Khasawneh, “Epileptic seizure detection using machine learning methods,” Computational Intelligence and Neuroscience , vol. 2022, p. 8322732, mar 2022
work page 2022
-
[5]
Health insurance portability and accountability act of 1996
United States Congress, “Health insurance portability and accountability act of 1996.” Public Law 104-191, Aug. 1996. Accessed on 2025-04-26
work page 1996
-
[6]
European Union, “Regulation (EU) 2016/679 of the European Parlia- ment and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation).” Official Journal of the European Union L 119/1, Apr. 20...
work page 2016
-
[7]
Privacy-preserving edge fed- erated learning for intelligent mobile-health systems,
A. Aminifar, M. Shokri, and A. Aminifar, “Privacy-preserving edge fed- erated learning for intelligent mobile-health systems,” Future Generation Computer Systems, vol. 161, pp. 625–637, 2024
work page 2024
Show all 35 references
-
[8]
Communication-efficient learning of deep networks from decen- tralized data,
H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Ag ¨uera y Arcas, “Communication-efficient learning of deep networks from decen- tralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) , vol. 54, pp. 1273–1...
2017
-
[9]
The future of digital health with federated learning,
N. Rieke, J. Hancox, W. Li, F. Milletar `ı, H. F. Roth, S. Albarqouni, et al., “The future of digital health with federated learning,” npj Digital Medicine, vol. 3, p. 119, Sept. 2020
2020
-
[10]
Advances and open problems in federated learning,
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, et al. , “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1– 210, 2021
2021
-
[11]
Federated learning: Challenges, methods, and future directions,
T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020
2020
-
[12]
Federated learning on non-iid data: A survey,
H. Zhu, J. Xu, S. Liu, and Y . Jin, “Federated learning on non-iid data: A survey,” Neurocomputing, vol. 465, pp. 371–390, 2021
2021
-
[13]
CHB-MIT scalp EEG database (version 1.0. 0),
J. Guttag, “CHB-MIT scalp EEG database (version 1.0. 0),” PhysioNet, 2010
2010
-
[14]
A. H. Shoeb, Application of machine learning to epileptic seizure onset detection and treatment . PhD thesis, Massachusetts Institute of Technology, 2009
2009
-
[15]
A dataset of neonatal eeg recordings with seizure annotations,
N. J. Stevenson, K. Tapani, L. Lauronen, and S. Vanhatalo, “A dataset of neonatal eeg recordings with seizure annotations,” Scientific Data , vol. 6, no. 1, pp. 1–8, 2019
2019
-
[16]
Siena scalp EEG database,
P. Detti, “Siena scalp EEG database,” PhysioNet. doi , vol. 10, p. 493, 2020
2020
-
[17]
Eeg synchroniza- tion analysis for seizure prediction: A study on data of noninvasive recordings,
P. Detti, G. Vatti, and G. Zabalo Manrique de Lara, “Eeg synchroniza- tion analysis for seizure prediction: A study on data of noninvasive recordings,” Processes, vol. 8, no. 7, 2020
2020
-
[18]
A large collection of real-world pediatric sleep studies,
H. Lee, B. Li, S. DeForte, M. L. Splaingard, Y . Huang, Y . Chi, et al., “A large collection of real-world pediatric sleep studies,” Scientific Data, vol. 9, no. 1, p. 421, 2022
2022
-
[19]
NCH Sleep DataBank: A large collection of real-world pediatric sleep studies with longitudinal clinical data (version 3.1.0)
H. Lee, B. Li, Y . Huang, Y . Chi, and S. Lin, “NCH Sleep DataBank: A large collection of real-world pediatric sleep studies with longitudinal clinical data (version 3.1.0).” PhysioNet, 2021. RRID:SCR 007345. doi:10.13026/p2rp-sg37
2021 doi
-
[20]
Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals,
A. L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley, “Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals,” circulation, vol. 1...
2000
-
[21]
TinySleepNet: An Efficient Deep Learning Model for Sleep Stage Scoring based on Raw Single-Channel EEG,
A. Supratak and Y . Guo, “TinySleepNet: An Efficient Deep Learning Model for Sleep Stage Scoring based on Raw Single-Channel EEG,” in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) , pp. 641–644, 2020
2020
-
[22]
Patient-specific preictal pattern-aware epileptic seizure prediction with federated learning,
R. Saemaldahr and M. Ilyas, “Patient-specific preictal pattern-aware epileptic seizure prediction with federated learning,” Sensors, vol. 23, no. 14, 2023
2023
-
[23]
Focal onset seizure prediction using convolutional networks,
H. Khan, L. Marcuse, M. Fields, K. Swann, and B. Yener, “Focal onset seizure prediction using convolutional networks,” IEEE Transactions on Biomedical Engineering, vol. 65, no. 9, pp. 2109–2118, 2018
2018
-
[24]
Geometric deep learning for subject independent epileptic seizure prediction using scalp EEG signals,
T. Dissanayake, T. Fernando, S. Denman, S. Sridharan, and C. Fookes, “Geometric deep learning for subject independent epileptic seizure prediction using scalp EEG signals,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 2, pp. 527–538, 2021
2021
-
[25]
Interpretable EEG seizure prediction using a multiobjective evolutionary algorithm,
M. Pinto, T. Coelho, A. Leal, F. Lopes, A. Dourado, P. Martins, et al. , “Interpretable EEG seizure prediction using a multiobjective evolutionary algorithm,” Scientific Reports, vol. 12, no. 1, p. 4420, 2022
2022
-
[26]
Epilepsy seizure prediction on eeg using common spatial pattern and convolutional neural network,
Y . Zhang, Y . Guo, P. Yang, W. Chen, and B. Lo, “Epilepsy seizure prediction on eeg using common spatial pattern and convolutional neural network,” IEEE Journal of Biomedical and Health Informatics , vol. 24, no. 2, pp. 465–474, 2020
2020
-
[27]
Abood and S
W. Abood and S. Bandyopadhyay, Postictal Seizure State . StatPearls Publishing, Treasure Island (FL), 2023. Updated July 10, 2023
2023
-
[28]
Online Seizure Prediction System: A Novel Probabilistic Approach for Efficient Pre- diction of Epileptic Seizure with iEEG Signal,
B. Abbaszadeh, C. A. Teixeira, and M. C. Yagoub, “Online Seizure Prediction System: A Novel Probabilistic Approach for Efficient Pre- diction of Epileptic Seizure with iEEG Signal,” The Open Biomedical Engineering Journal, vol. 16, no. 1, 2022
2022
-
[29]
Optimum window size and overlap for robust probabilistic prediction of seizures with ieeg,
B. Abbaszadeh and M. Yagoub, “Optimum window size and overlap for robust probabilistic prediction of seizures with ieeg,” in 2019 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) , pp. 1–5, 2019
2019
-
[30]
A deep learning framework for epileptic seizure detection based on neonatal EEG signals. Sci. Rep. 12, 13010 (2022)
A. Gramacki and J. Gramacki, “A deep learning framework for epileptic seizure detection based on neonatal EEG signals. Sci. Rep. 12, 13010 (2022).”
2022
-
[31]
Scaffold: Stochastic controlled averaging for federated learn- ing,
S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learn- ing,” in International Conference on Machine Learning, pp. 5132–5143, PMLR, 2020
2020
-
[32]
Practical secure aggregation for federated learning on user-held data,
K. Bonawitz, V . Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, et al. , “Practical secure aggregation for federated learning on user-held data,” arXiv preprint arXiv:1611.04482 , 2016
2016 arXiv
-
[33]
Federated learning for healthcare: Systematic review and architecture proposal,
R. S. Antunes, C. Andr ´e da Costa, A. K ¨uderle, I. A. Yari, and B. Eskofier, “Federated learning for healthcare: Systematic review and architecture proposal,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 4, pp. 1–23, 2022
2022
-
[34]
Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,
J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in International Conference on Machine Learning, pp. 6028–6039, PMLR, 2020
2020
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.