{"id":"835ebb93-6e71-4d47-bb59-0acb37c3f165","arxiv_id":"2501.10342","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A hybrid 1D-CNN with multi-head attention and wavelet preprocessing reportedly reaches 99.83% accuracy on the UCI epilepsy dataset, but the result is undermined by a leaky data split and internally inconsistent metrics.","lead":"This paper reports a deep learning model that combines wavelet preprocessing, 1D convolutional layers, and multi-head attention to classify one-second EEG clips as seizure or non-seizure, claiming 99.83% accuracy on the UCI epilepsy dataset. The evaluation is compromised by a likely random segment-level split, which allows the model to memorize subject-specific patterns, and by metric values that contradict the reported confusion matrix.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5.1's unstated random 80/20 split over 11,500 segments, with 23 correlated one-second windows per subject (Section 3.1.1), is likely subject leakage; the 99.83% SOTA claim is not established without a subject-wise split.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the train/test split is over segments rather than subjects. The dataset construction in Section 3.1.1 makes the 23 windows per subject contiguous slices of one recording, so a random segment-level split creates near-duplicates between train and test. This is a correctness risk under standard subject-independent evaluation of EEG models, and it directly undermines the stated state-of-the-art accuracy. One nuance: the accuracy itself is consistent with the confusion matrix (2296/2300 = 99.83%), so the reader's broader statement about 'internally inconsistent metrics' is only partially correct. However, Table 1's F1 and CSI values are indeed incompatible with the matrix, adding a secondary evidentiary problem. The central conclusion remains unchanged: the paper's headline result is not established without subject-wise splitting, and the lack of code or split details prevents verification. Therefore I would keep the reader's REJECT verdict.","tokens_in":11166,"tokens_out":3835,"duration_ms":38220,"concrete_test":"Obtain the authors' split code or, failing that, reimplement the model from Section 4.1 and re-run with a subject-wise GroupKFold (or an 80/20 split where all 23 segments of each subject stay in the same fold). If the test accuracy drops materially below 99.83% (e.g., below the 99.47% of the closest comparator), the SOTA claim fails. Independently recompute F1 and CSI from the reported confusion matrix to confirm the inconsistency.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the 99.83% accuracy on the UCI Epileptic Seizure Dataset. Section 3.1.1 constructs each subject's 23.6-second recording into 23 contiguous one-second windows, yielding 11,500 segments. Section 5.1 says only 'allocating 80% of the dataset for training and the remaining 20% for testing' with no subject-wise separation. A random split therefore places adjacent windows from the same subject in both train and test; these windows come from the same recording and are highly correlated. A 1D-CNN with multi-head attention can memorize subject-specific amplitude or baseline patterns and appear to generalize. The 99.83% figure and the Table 2 'outperforming all existing models' comparison depend entirely on this split assumption. The confusion matrix itself is compatible with the accuracy (2296/2300), though Table 1's F1 and CSI values (0.9990) are not compatible with the matrix (F1 ≈ 0.9957, CSI ≈ 0.9914), which further erodes confidence in the reported evidence. Without code, seeds, or a group-wise split, the central claim is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a hybrid deep learning model for binary epileptic seizure detection from EEG, combining a single-level Daubechies-1 wavelet transform, three 1D convolutional layers with batch normalization and max pooling, a four-head attention mechanism with a skip connection, global average pooling, and two dense layers with dropout and L2 regularization. The authors evaluate on the UCI Epileptic Seizure Dataset, binarizing the five original classes into seizure versus non-seizure, and report 99.83% test accuracy, 0.995 MCC, 0.999 F1, and 0.999 CSI, claiming to outperform all existing models on this dataset. The paper includes a confusion matrix, accuracy/loss curves, and a comparison table of prior results.","tokens_in":11430,"tokens_out":5432,"duration_ms":46447,"significance":"If the reported performance were obtained under a subject-independent evaluation protocol, the architecture would be a useful reference point for EEG-based seizure detection, and the systematic description of regularization and attention integration is a strength. However, the current evidence does not support the central claim: the random segment-level split creates likely subject leakage, and the reported F1/CSI values are inconsistent with the confusion matrix. The paper provides no code, random seed, or data split details, which limits reproducibility. The work is therefore best viewed as an architecture proposal requiring re-validation.","major_comments":[{"comment":"The dataset construction in Section 3.1.1 creates 23 contiguous one-second segments from each subject's single 23.6-second recording, yielding 11,500 samples, and Section 5.1 describes only an 80/20 split of the dataset without any subject-wise separation. Because segments from the same subject are highly correlated and non-independent, a random split over all segments is very likely to place segments from the same subject in both training and test sets, allowing the model to memorize subject-specific characteristics and inflate the reported 99.83% accuracy. The central state-of-the-art claim in the Abstract and Table 2 is therefore not established. The authors must re-evaluate with a subject-wise (group-wise) split, such as splitting by subject IDs or leave-one-subject-out, and report the resulting performance.","section":"§3.1.1 and §5.1"},{"comment":"Table 1 reports F1 = 0.9990 and CSI = 0.9990, but the confusion matrix in Figure 10 (TP=462, FP=1, FN=3, TN=1834) yields F1 = 2·462/(2·462+1+3) = 0.9957 and CSI = 462/(462+1+3) = 0.9914, while the test accuracy of 99.83% (2296/2300) is consistent with the matrix. This inconsistency means at least one of the reported results is incorrect and undermines confidence in the reliability of the reported metrics. The authors should recompute and present a consistent set of metrics.","section":"§5.1, Table 1 and Figure 10"},{"comment":"Table 2 compares the proposed model with prior methods reporting accuracies on the same dataset, but the manuscript does not establish that the cited results were obtained under the same evaluation protocol (e.g., random segment split versus subject-wise split, cross-validation scheme, preprocessing steps such as wavelet transform and feature scaling). Without this information, the claim that the proposed model outperforms all existing models is not justified; differences in accuracy may reflect differences in evaluation protocol rather than model quality.","section":"§5.2, Table 2"},{"comment":"The reproducibility of the reported results is limited: Section 5.1 gives no random seed, no stratification details, and no code or implementation link, and Section 4.2 says only that a single-level db1 wavelet transform is applied without specifying whether the approximation coefficients, detail coefficients, or a reconstructed signal are used as the (178,1) input to the network. These missing details prevent a reader from reproducing the 99.83% accuracy result.","section":"§4.2 and §5.1"}],"minor_comments":[{"comment":"References [8] and [10] are identical, and references [29] and [30] are identical; duplicate citations should be merged or renumbered.","section":"References"},{"comment":"The headers in Table 1 and Table 2 contain spacing artifacts, \"V alue\" and \"Publication Y ear\", which should be corrected to \"Value\" and \"Publication Year\".","section":"Tables 1 and 2"},{"comment":"The statement that the wavelet transform \"retains both high- and low-frequency data\" is not enough to determine the model input; please specify how the wavelet coefficients are processed and how the resulting representation matches the declared input shape (178,1).","section":"§4.2"},{"comment":"The phrase \"miss-classifications\" should be \"misclassifications\", and in the caption of Figure 2, \"preporcessing\" should be \"preprocessing\".","section":"§5.1"},{"comment":"Figure 8 is described as a \"heat-map for weighting process\", but the figure as printed does not show an interpretable heat-map; please replace it with a clear visualization of the attention weights or remove it.","section":"Figure 8"},{"comment":"The sentence \"with total of 178 + 1 (y-output) column\" should use the plural \"columns\" and clarify that each of the 11,500 rows is a segment with 178 features and one label.","section":"§3.1.1"}],"recommendation":"major_revision","confidential_remarks":"The two load-bearing issues (subject leakage and metric inconsistency) are significant but fixable through re-analysis. If the authors cannot provide a subject-wise evaluation that still yields competitive results, the paper should be rejected at that point. The current manuscript is not acceptable in its present form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's what you need to know: the 99.83% accuracy is almost certainly an artifact of data leakage. The UCI dataset gives each of 500 subjects 23 contiguous one-second segments from the same 23.6s recording, but Section 5.1 says only 'allocating 80% ... and 20%' with no subject-wise separation. A random split over 11,500 segments puts near-duplicate windows from the same subject in both train and test, so the model can memorize subject-level baseline characteristics. The outcome is a number that doesn't reflect generalization.\n\nTo give credit where it's due: the architecture is a straightforward but sensible stack – db1 wavelet, three Conv1D blocks with batch norm and pooling, a 4-head attention layer with a skip connection, global average pooling, and two dense layers with dropout and L2. The text describes it well enough to reproduce. The comparison table lists the main recent work on this dataset.\n\nThe soft spots are serious. First, the split issue alone kills the SOTA claim; without a subject-wise split, the reported 99.83% is meaningless. Second, Table 1 says F1=0.9990 and CSI=0.9990, but the confusion matrix in Figure 10 (TP=462, FP=1, FN=3) gives F1≈0.9957 and CSI≈0.9914. That's a direct arithmetic inconsistency. Third, the gain over the next best model (99.47%) is 0.36% on a saturated benchmark, so even if the number held it would be a marginal improvement. Fourth, reference [26] is retracted and cited without caveat, which is sloppy.\n\nWho gets value from this? A practitioner looking for a simple CNN+attention template might find it useful, but anyone comparing methods should not trust the headline. As it stands, this is not ready for peer review; the authors would need to redo the evaluation with a subject-wise split, correct the reported metrics, and check their citations. I'd desk-reject it now and invite a resubmission if those issues are addressed.","headline":"The 99.83% SOTA claim collapses under a subject-wise split; the reported metrics contradict the confusion matrix, so this incremental CNN+attention paper is not ready for review.","tokens_in":12032,"tokens_out":4292,"would_cite":false,"duration_ms":40091,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid wavelet + 1D-CNN + multi-head attention model classifies one-second EEG windows as seizure or non-seizure with 99.83% accuracy, beating all compared models on a benchmark dataset.","keywords":["epileptic seizure detection","EEG signal classification","1D convolutional neural network","multi-head attention","wavelet transform","deep learning","seizure detection","binary classification"],"falsifier":"Run the identical pipeline but split by subject: train only on segments from a subset of the 500 subjects and test only on held-out subjects' segments, then compare the resulting accuracy with the reported 99.83% random-split result.","tokens_in":10935,"feed_emoji":"🧠","tokens_out":9554,"duration_ms":88985,"temperature":0.7,"pith_summary":"Epilepsy affects roughly 50 million people, and automatic seizure detection from EEG is hard because the signals are noisy, non-stationary, and vary across people. The paper proposes a hybrid model—single-level wavelet denoising followed by three 1D convolutional layers and a four-head attention layer with a skip connection—that classifies one-second EEG windows as seizure or non-seizure. On the benchmark dataset of 11,500 such windows, the authors report 99.83% test accuracy, an MCC of 0.9950, and an F1 score of 0.9990. Their claim is that this combination outperforms all compared models on the same dataset, including LSTM and Bayesian hybrids.","feed_headline":"EEG model reports 99.83 percent seizure-detection accuracy","feed_subtitle":"A hybrid of wavelet denoising, convolutional layers, and attention beats earlier EEG classifiers on one benchmark.","key_machinery":"The mechanism that carries the argument is a three-stage pipeline: a single-level wavelet transform using the db1 wavelet smooths noise while preserving high and low frequencies; three 1D convolutional blocks (32, 64, and 128 filters with kernel sizes 7, 5, and 3, each followed by batch normalization and max pooling) extract local temporal patterns; and a multi-head attention layer with four heads and key dimension 32 lets the model weigh different time positions simultaneously, with a skip connection adding the last convolutional output to the attention output. Global average pooling, two dense layers with dropout of 0.5, L2 regularization at 0.001, early stopping, and learning-rate reduction are used to control overfitting. The final sigmoid neuron maps the representation to a seizure versus non-seizure decision.","core_discovery":"On its own terms, the paper's central claim is that attention, not recurrence, is the better complement to a convolutional stack for EEG seizure classification. The input is a 178-point one-second window; the first three layers grow filter counts 32 to 64 to 128 while shrinking kernel sizes 7 to 5 to 3, and the final convolutional output is joined by a skip connection to a four-head attention layer with key dimension 32. The authors report that on the 2,300 test windows the model makes four errors—one false positive and three false negatives—which gives the 99.83% figure. They present this as the highest accuracy on the benchmark compared with the prior models in their comparison table.","pith_inferences":["A subject-wise split is the natural extension: each of the 500 subjects contributes 23 contiguous one-second windows, so a random window split may let the model learn subject-specific baselines; a held-out-subject test would likely produce a lower accuracy than the reported 99.83%.","The attention heat-map suggests the model could double as a localizer: the per-time-step weights might indicate when within a segment epileptiform activity begins, a use the paper does not develop.","The same wavelet + 1D-CNN + multi-head attention recipe transfers directly to other one-dimensional biomedical time series, such as ECG arrhythmia or EMG event detection, where local waveform shape and long-range timing both matter."],"forward_implications":["One second of a single EEG channel is enough for the seizure/non-seizure decision, so the model could run in low-latency monitoring settings that flag abnormal windows as they arrive.","The attention weights provide a per-window importance map, giving a clinician a view of which moments within a segment drove the decision rather than a bare binary label.","On the paper's comparison table, the model would rank above the 99.47% Bayesian Conv1D-LSTM and the 99.3% Conv1D-LSTM, the two closest prior results.","Because the design uses no recurrent layer, training and inference avoid the sequential bottlenecks of LSTM-based hybrids."],"supporting_citations":[{"why":"Supplies the benchmark EEG dataset of 500 subjects and 11,500 one-second segments that defines the classification task.","marker":"[24]"},{"why":"Foundational adaptive wavelet filtering method that motivates the wavelet preprocessing step.","marker":"[16]"},{"why":"Demonstrates that attention mechanisms improve EEG-based recognition, the rationale for the multi-head attention layer.","marker":"[29]"},{"why":"LSTM baseline on the same dataset that the proposed model claims to outperform.","marker":"[36]"},{"why":"Conv1D+LSTM baseline at 99.3% accuracy that the comparison table uses as a benchmark.","marker":"[37]"},{"why":"Hybrid CNN+GRU model at 99.01% that the comparison table uses as a benchmark.","marker":"[38]"},{"why":"KNN+PCA baseline at 99% accuracy that the comparison table uses as a benchmark.","marker":"[42]"},{"why":"Closest baseline, a Bayesian Conv1D-LSTM at 99.47%, which the 99.83% figure is meant to beat.","marker":"[43]"}],"fun_headline_variants":["Attention achieves 99.83% in EEG seizure classification","EEG seizure model reports 99.83% accuracy via attention","Attention mechanism reaches 99.83% on EEG seizure task","Attention over recurrence yields 99.83% EEG seizure accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 23 one-second segments taken from the same person's recording are independent enough that a random split into training and test sets gives an honest measure of accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Attention achieves 99.83% in EEG seizure classification","EEG seizure model reports 99.83% accuracy via attention","Attention mechanism reaches 99.83% on EEG seizure task","Attention over recurrence yields 99.83% EEG seizure accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001532,"raw_usage":{"total_tokens":6069,"prompt_tokens":821,"completion_tokens":5248,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":437,"completion_tokens_details":{"reasoning_tokens":5177}},"tokens_in":437,"tokens_out":5248,"duration_ms":33037,"temperature":1.0,"reasoning_tokens":5177,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:10:58.463081+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical pipeline but split by subject: train only on segments from a subset of the 500 subjects and test only on held-out subjects' segments, then compare the resulting accuracy with the reported 99.83% random-split result.","supporting_citations":[{"cited_title":"In: 2024 11th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO), pp","cited_arxiv_id":null,"evidence_quote":"Closest baseline, a Bayesian Conv1D-LSTM at 99.47%, which the 99.83% figure is meant to beat."},{"cited_title":"Physical Review E 64(6), 061907 (2001)","cited_arxiv_id":null,"evidence_quote":"Supplies the benchmark EEG dataset of 500 subjects and 11,500 one-second segments that defines the classification task."},{"cited_title":"IEEE Transactions on signal processing 44(9), 2163–2171 (1996)","cited_arxiv_id":null,"evidence_quote":"Foundational adaptive wavelet filtering method that motivates the wavelet preprocessing step."},{"cited_title":"Journal of Engineering and Applied Science 71(1), 21 (2024)","cited_arxiv_id":null,"evidence_quote":"LSTM baseline on the same dataset that the proposed model claims to outperform."},{"cited_title":"Neural Computing and Applications 36(6), 2835–2852 (2024)","cited_arxiv_id":null,"evidence_quote":"Conv1D+LSTM baseline at 99.3% accuracy that the comparison table uses as a benchmark."},{"cited_title":"Brain Informatics 11(1), 21 (2024)","cited_arxiv_id":null,"evidence_quote":"Hybrid CNN+GRU model at 99.01% that the comparison table uses as a benchmark."},{"cited_title":"Journal of Investigations on Engineering and Technology 4(2), 47–60 (2021)","cited_arxiv_id":null,"evidence_quote":"KNN+PCA baseline at 99% accuracy that the comparison table uses as a benchmark."}],"review_version":1}