Pith. sign in

REVIEW 4 major objections 2 minor 14 references

PySeizure: A single machine learning classifier framework to detect seizures in diverse datasets

T0 review · 4 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single ML framework detects seizures across two EEG datasets

desk verdict Useful framework, honest limitations, but the headline overstates transfer: one direction is near chance, and the leakage question is unresolved. read the letter →

arxiv 2508.07253 v1 pith:SNWZGVBI submitted 2025-08-10 cs.LG eess.SPq-bio.NC

Bartlomiej Chybowski , Shima Abdullateef , Hollan Haule , Alfredo Gonzalez-Sulser , Javier Escudero This is my paper
classification cs.LGeess.SPq-bio.NC
keywords seizuredetectionEEGmachinelearningmajorityvotingcross-datasetgeneralizationautomatedpreprocessingCHB-MITTUSZ
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces PySeizure, an open-source machine-learning framework intended to detect epileptic seizures from EEG across different clinical datasets without per-dataset retuning. The claim is that one pipeline—automated preprocessing to standardize montage and sampling rate plus per-second majority voting by multiple models—achieves strong within-dataset performance (AUC 0.904±0.059 on CHB-MIT, 0.864±0.060 on TUSZ) and partial cross-dataset transfer (AUC 0.615 from CHB-MIT to TUSZ, 0.762 in reverse). If this holds, a single trained system could be deployed across hospitals with different EEG setups, reducing the manual workload of seizure review while leaving final decisions to experts.

What carries the argument

Two components carry the argument. First, an automated preprocessing pipeline standardizes raw EEG across datasets—aligning electrode montages and resampling signals so that different clinical acquisitions become comparable. Second, a majority-voting scheme in which multiple independently trained models each score every second of EEG and the final label is decided by a vote; this dampens individual model errors and yields per-second seizure predictions that can be lightly post-processed for a mild gain (within-dataset AUC 0.913 and 0.867 after post-processing).

What would settle it

Run the released framework on a third EEG dataset (e.g., Siena or a local clinical recording) using the same automated preprocessing, and check whether AUC stays near the reported 0.86–0.90 range; alternatively, re-run the CHB-MIT and TUSZ experiments with strictly patient-disjoint cross-validation and compare the resulting AUCs.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single machine-learning framework, without dataset-specific tuning, can be trained to detect seizures in EEG recordings from different clinical sources. On each dataset the framework learns its own models from standardized, automatically preprocessed data; at inference time, multiple models score each second of EEG and a majority vote produces the final decision. Within-dataset AUCs reach 0.904 (CHB-MIT) and 0.864 (TUSZ), and cross-dataset AUCs are 0.615 (trained CHB-MIT, tested TUSZ) and 0.762 (trained TUSZ, tested CHB-MIT). The authors interpret these results as evidence that a dataset-agnostic, reproducible pipeline for seizure detection is achievable,

Load-bearing premise

The load-bearing premise is that the reported train/test splits separate patients, so the within-dataset and cross-dataset AUCs are not inflated by segments from the same patient appearing in both training and testing.

Editorial extensions

If this is right

  • A single training pipeline suffices for multiple EEG datasets, reducing the need for dataset-specific feature engineering and manual tuning.
  • Models trained on one hospital's EEG data give usable baseline performance at a new site without retuning, with direction-dependent transfer quality.
  • Per-second majority voting combined with automated preprocessing yields stable predictions that can be further improved by light post-processing.
  • The open-source release makes the framework reproducible and a candidate base for prospective clinical validation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cross-dataset asymmetry (TUSZ→CHB-MIT better than CHB-MIT→TUSZ) may reflect differences in dataset size, seizure morphology, or recording protocols; it hints that transfer quality is direction-dependent and worth characterizing per pair of sites.
  • If patient-disjoint evaluation confirms the numbers, a practical deployment path is training on a large pooled corpus and then fine-tuning on a few labeled hours from a target hospital, rather than training from scratch.
  • The majority-voting design could be stress-tested by ablating the number of models or using disagreement rates as an uncertainty signal for a human reviewer—a test the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 2 minor

Summary. The submitted manuscript, as provided, consists of an abstract for a paper titled 'PySeizure: A single machine learning classifier framework to detect seizures in diverse datasets' and a full text that is a different paper on egocentric 4D point-cloud question answering. The abstract claims a generalizable seizure-detection framework, with within-dataset AUCs (0.904±0.059 on CHB-MIT, 0.864±0.060 on TUSZ) and cross-dataset transfer results (0.615±0.039 CHB-MIT→TUSZ, 0.762±0.175 TUSZ→CHB-MIT). Because the full text does not correspond to the abstract, none of the methods, data partitions, hyperparameters, or reproducibility details for PySeizure are available for review. The abstract-level evidence is insufficient to support the central claims as written.

Significance. If the claims were substantiated, a single framework with automated preprocessing and majority voting that works across CHB-MIT and TUSZ could be a useful contribution to clinical seizure-detection research. The abstract's reporting of both transfer directions with standard deviations is more informative than selective reporting, and the idea of a standardized preprocessing pipeline is sensible. However, the mismatch between the abstract and the full text makes the submission unverifiable as a scientific manuscript. The apparent transfer results, especially CHB-MIT→TUSZ AUC 0.615, also contradict the 'strong generalisation' wording. As submitted, the paper cannot establish its significance.

major comments (4)
  1. [Full text (entire document)] The body of the manuscript is not the paper described in the abstract. It is 'Understanding Dynamic Scenes in Egocentric 4D Point Clouds' by different authors, with no mention of PySeizure, EEG, CHB-MIT, TUSZ, or seizure detection. The abstract's claims about the automated pre-processing pipeline, majority voting, model training, and reproducibility therefore have no accompanying evidence. This is a load-bearing error: the manuscript cannot be scientifically assessed as submitted.
  2. [Abstract, evaluation description] The sentence 'We train, tune, and evaluate models within each dataset' does not state whether training and test partitions are patient-disjoint. If EEG segments from the same patient appear in both training and testing, the reported within-dataset AUCs (0.904±0.059 and 0.864±0.060) would be optimistically biased through memorization of patient-specific baseline rhythms. The central 'robust and generalisable' claim depends on this protocol detail; it must be specified and justified.
  3. [Abstract, cross-dataset results] The claimed 'strong generalisation across datasets' is not supported by the abstract's own transfer numbers. CHB-MIT→TUSZ AUC 0.615±0.039 is barely above chance, and the reverse direction TUSZ→CHB-MIT AUC 0.762±0.175 has very high variance. If these are the headline cross-dataset results, the conclusion should be reframed as partial transferability with substantial domain shift, not strong generalisation.
  4. [Abstract, reproducibility statement] The claim 'By making our methodology fully reproducible' is unverifiable: the full text contains no repository link, no implementation details for PySeizure, and no description of the preprocessing, model architecture, or voting mechanism. Reproducibility is explicitly promised in the abstract but not delivered in the provided manuscript.
minor comments (2)
  1. [Abstract, metrics] The notation 'AUC 0.904+/-0.059' should specify the cross-validation scheme and the number of folds or runs; the current notation conflates variability across folds with variability across independent runs.
  2. [Abstract, terminology] The phrases 'without any post-processing' and 'mild post-processing' are not defined; if this terminology is retained, the exact operations and hyperparameters need to be stated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: cross-dataset transfer is genuinely out-of-sample; no fitted quantity is renamed as a prediction.

full rationale

The abstract and available text contain no derivation chain in which an output is defined in terms of an input, no fitted parameter is relabeled as a prediction, and no load-bearing self-citation. The within-dataset AUCs (0.904/0.864) are reported as tuned performance on each dataset, not as predictions derived from first principles; the cross-dataset AUCs (0.615/0.762) are the actual transfer predictions, and they are genuinely out-of-sample because training and testing occur on different datasets. The possible absence of patient-disjoint splitting is a data-leakage/validity concern, not a circularity of the kind this pass flags; no quote in the manuscript shows that the training set was constructed from the test labels or that the evaluation score is algebraically identical to a fitted objective. The provided full-text body belongs to a different paper (arXiv:2508.07251v3, egocentric 4D point-cloud QA) and contains no load-bearing claims about PySeizure, so it neither supplies nor refutes circularity evidence. Score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

Abstract-only review. The framework introduces no new physical or conceptual entity; it composes standard ML components (preprocessing, per-second classification, voting ensemble). The free parameters are standard ML hyperparameters, none of which are reported in the abstract. The axioms are domain assumptions about EEG annotation granularity, the safety of the standardization step, and the representativeness of two public datasets.

free parameters (3)
  • Ensemble size (number of models in the majority vote)
    The abstract says multiple models vote per second but does not report how many or how they were chosen; ensemble size is a tunable hyperparameter that affects the voting outcome.
  • Within-dataset classifier hyperparameters
    The abstract states 'We train, tune, and evaluate models within each dataset', so hyperparameters were tuned per dataset; no values are disclosed in the abstract.
  • Post-processing parameters (e.g., threshold, smoothing)
    A 'mild post-processing' improves AUC to 0.913/0.867 within datasets and 0.619/0.768 across datasets, but the abstract does not say which parameters were adjusted or whether they were tuned on the test data.
assumptions (4)
  • domain assumption Per-second EEG segments are the correct granularity for seizure detection and the public datasets' segment-level labels are reliable.
    The framework scores 'each second of EEG', which presumes seizures manifest at one-second resolution and that CHB-MIT and TUSZ annotations at that granularity are trustworthy.
  • domain assumption Automated standardization of recordings (montage, sampling rate, referencing) preserves seizure-relevant signal while removing dataset differences.
    The preprocessing pipeline is the stated mechanism enabling cross-dataset transfer; if standardization discards seizure-relevant features, the transfer AUCs understate real-world performance.
  • domain assumption CHB-MIT and TUSZ are representative of 'diverse clinical settings' sufficient to support a dataset-agnostic claim.
    The abstract generalizes from two public datasets, one predominantly pediatric and from a single center, to 'varied clinical datasets'; this is a thin empirical basis for the heading claim.
  • standard math Majority voting of independent models improves robustness without introducing systematic bias.
    Standard ensemble assumption: model errors must be at least partially independent for voting to help. Not stated or tested in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PySeizure: A single machine learning classifier framework to detect seizures in diverse datasets." pith.science (2026). https://pith.science/paper/SNWZGVBI

@misc{pith2026250807253,
  author       = {Pith},
  title        = {Pith review of: PySeizure: A single machine learning classifier framework to detect seizures in diverse datasets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SNWZGVBI}},
  note         = {Machine review of arXiv:2508.07253}
}
read the original abstract

Reliable seizure detection is critical for diagnosing and managing epilepsy, yet clinical workflows remain dependent on time-consuming manual EEG interpretation. While machine learning has shown promise, existing approaches often rely on dataset-specific optimisations, limiting their real-world applicability and reproducibility. Here, we introduce an innovative, open-source machine-learning framework that enables robust and generalisable seizure detection across varied clinical datasets. We evaluate our approach on two publicly available EEG datasets that differ in patient populations and electrode configurations. To enhance robustness, the framework incorporates an automated pre-processing pipeline to standardise data and a majority voting mechanism, in which multiple models independently assess each second of EEG before reaching a final decision. We train, tune, and evaluate models within each dataset, assessing their cross-dataset transferability. Our models achieve high within-dataset performance (AUC 0.904+/-0.059 for CHB-MIT and 0.864+/-0.060 for TUSZ) and demonstrate strong generalisation across datasets despite differences in EEG setups and populations (AUC 0.615+/-0.039 for models trained on CHB-MIT and tested on TUSZ and 0.762+/-0.175 in the reverse case) without any post-processing. Furthermore, a mild post-processing improved the within-dataset results to 0.913+/-0.064 and 0.867+/-0.058 and cross-dataset results to 0.619+/-0.036 and 0.768+/-0.172. These results underscore the potential of, and essential considerations for, deploying our framework in diverse clinical settings. By making our methodology fully reproducible, we provide a foundation for advancing clinically viable, dataset-agnostic seizure detection systems. This approach has the potential for widespread adoption, complementing rather than replacing expert interpretation, and accelerating clinical integration.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

  1. [4]

    Ma, X.; Yong, S.; Zheng, Z.; Li, Q.; Liang, Y .; Zhu, S.-C.; and Huang, S

    SpatialCoT: Ad- vancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning.arXiv preprint arXiv:2501.10074. Ma, X.; Yong, S.; Zheng, Z.; Li, Q.; Liang, Y .; Zhu, S.-C.; and Huang, S

    arXiv

  2. [5]

    Radford, A.; Kim, J

    GPT4Scene: Understand 3D Scenes from Videos with Vision- Language Models.arXiv:2501.01428. Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agar- wal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    arXiv

  3. [8]

    Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scene

    Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scene.arXiv preprint arXiv:2503.15019. Xiong, H.; Zhuge, Y .; Zhu, J.; Zhang, L.; and Lu, H

    work page Pith review arXiv

  4. [9]

    Yang, J.; Cen, J.; Peng, W.; Liu, F., Shuai amd Hong; Li, X.; Zhou, K.; Chen, Q.; and Liu, Z

    3UR- LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding.arXiv preprint arXiv:2501.07819. Yang, J.; Cen, J.; Peng, W.; Liu, F., Shuai amd Hong; Li, X.; Zhou, K.; Chen, Q.; and Liu, Z

    arXiv

  5. [10]

    ADL4D: Towards A Contextually Rich Dataset for 4D Activities of Daily Living

    Adl4d: Towards a contextually rich dataset for 4d activities of daily living.arXiv preprint arXiv:2402.17758. Zeng, S.; Chang, X.; Xie, M.; Liu, X.; Bai, Y .; Pan, Z.; Xu, M.; and Wei, X

    work page Pith review arXiv

  6. [11]

    Zheng, D.; Huang, S.; Li, Y .; and Wang, L

    FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving.arXiv preprint arXiv:2505.17685. Zheng, D.; Huang, S.; Li, Y .; and Wang, L

    arXiv

  7. [12]

    Zheng, D.; Huang, S.; and Wang, L

    Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geome- try Priors.arXiv e-prints, arXiv:2505.24625. Zheng, D.; Huang, S.; and Wang, L

    arXiv

  8. [13]

    arXiv preprint arXiv:2505.12253

    LLaV A-4D: Embedding Spa- tioTemporal Prompt into LMMs for 4D Scene Understanding. arXiv preprint arXiv:2505.12253. Zhou, S.; Xiao, J.; Li, Q.; Li, Y .; Yang, X.; Guo, D.; Wang, M.; Chua, T.-S.; and Yao, A

    arXiv

Show all 14 references
  1. [14]

    LLaV A- 3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness.arXiv preprint arXiv:2409.18125

    arXiv

  2. [2020]

    InCom- puter Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX

    Scanrefer: 3d object localization in rgb-d scans using natural language. InCom- puter Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX

    2020

  3. [2021]

    Grauman, K.; Westbury, A.; Byrne, E.; Chavis, Z.; Furnari, A.; Girdhar, R.; Hamburger, J.; Jiang, H.; Liu, M.; Liu, X.; Martin, M.; Nagarajan, T.; Radosavovic, I.; Ramakrishnan, S

    Johnson-Lindenstrauss lemma, linear and nonlinear random pro- jections, random Fourier features, and random kitchen sinks: Tuto- rial and survey.arXiv preprint arXiv:2108.04172. Grauman, K.; Westbury, A.; Byrne, E.; Chavis, Z.; Furnari, A.; Girdhar, R.; Hamburger, J.; Jiang, H...

    arXiv

  4. [2023]

    Wu, S.; Fei, H.; Yang, J.; Li, X.; Li, J.; Zhang, H.; and Chua, T.-s

    Chat-3d: Data-efficiently tuning large language model for universal dialogue of 3d scenes.arXiv preprint arXiv:2308.08769. Wu, S.; Fei, H.; Yang, J.; Li, X.; Li, J.; Zhang, H.; and Chua, T.-s

    arXiv

  5. [2024]

    Wang, T.; Mao, X.; Zhu, C.; Xu, R.; Lyu, R.; Li, P.; Chen, X.; Zhang, W.; Chen, K.; Xue, T.; Liu, X.; Lu, C.; Lin, D.; and Pang, J. 2024a. EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI. InProceedings of the IEEE/CVF Conference on Computer Vision ...

    arXiv

  6. [2025]

    5-vl technical report.arXiv preprint arXiv:2502.13923

    Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923. Banerjee, P.; Shkodrani, S.; Moulon, P.; Hampali, S.; Han, S.; Zhang, F.; Zhang, L.; Fountain, J.; Miller, E.; Basol, S.; New- combe, R.; Wang, R.; Engel, J. J.; and Hodan, T

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.