{"id":"5d4aa7e0-8a5f-4e9b-9695-459e68778a8f","arxiv_id":"2411.16165","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"MST-ECoGNet uses Modified S Transform time-frequency features and a compact spatial-filter network to decode visual categories from macaque ECoG, reporting accuracy gains over a TCN baseline with a much smaller model.","lead":"This paper proposes a compact neural network, MST-ECoGNet, that combines a Modified S Transform with a small convolutional classifier to decode which of six image categories a macaque is viewing from electrocorticography (ECoG) signals. The authors report higher accuracy and a much smaller model than a previous deep learning baseline, along with exploratory findings about where visual information lives in ECoG signals.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Frequency-range selection in Sec. IV-A appears to use the same cross-validation folds as the final accuracy report, so the claimed gains over [5] may be inflated by test-set selection; no separate validation split is described.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the per-subject frequency range was chosen by model performance without a separate validation split, so the reported accuracies are suspect. This is indeed the most load-bearing issue for the central claim because the frequency range directly sets the input size and is selected differently per subject. If the selection had been done properly on a held-out set, the accuracy comparison could have been meaningful; but as written, the protocol description in Sec. IV-A combined with the 5-fold CV description in Sec. III.C gives no indication of such a hold-out. The paper does have some independent support: the model is simple, parameter counts are plausible, and the use of MST features is physically interpretable, but those do not resolve the evaluation flaw. The inconsistencies noted by the reader (e.g., abstract vs. Section III.E reporting 10.82% vs 1.08%) are additional signs of careless reporting but are secondary to the selection leakage. Therefore, the verdict of REJECT remains appropriate; the paper should be rejected until the evaluation is redone with nested CV or another clearly held-out model-selection procedure.","tokens_in":12165,"tokens_out":2568,"duration_ms":28096,"concrete_test":"Reproduce MST-ECoGNet with nested cross-validation: for each of the 5 outer test folds, select the frequency range using only the remaining 4 folds (inner cross-validation), then evaluate on the outer fold. Report the outer-fold average accuracy for each subject. If the resulting accuracies drop to within 1–2 points of the baseline [5] (36.8% MonC, 27.59% MonJ), the claimed improvement is largely a selection artifact. Additionally, require the authors to disclose whether the original selection reused test folds; if it did, the corrected numbers are the only valid evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MST-ECoGNet outperforms the baseline [5] by 6.63–16.63 percentage points while using ~10% of the parameters. That claim depends on the accuracy numbers in Table II, which in turn depend on the per-subject frequency ranges (MonJ 0–52 Hz, MonC 0–38 Hz) chosen in Sec. IV-A. The paper states that the frequency range was selected 'based on the performance of the network' after training on reduced datasets, but it never states that this selection used partitions separate from the 5-fold test partitions described in Sec. III.C. If the same 5-fold partitions were reused across candidate frequency ranges and the best range was then reported with the same folds, the final accuracy is the result of model selection on the test set. With tens of candidate ranges (up to 128 Hz at 1 Hz resolution, with unknown step size), the expected optimistic bias is material. The frequency range determines the input size F, so this is not a nuisance detail; it directly alters the model input and is exactly the kind of hyperparameter that must be tuned on a validation split. Without a held-out selection procedure, the reported accuracy advantage over [5] cannot be attributed to the architecture or the MST features. The baseline comparison is also weaker because [5]'s numbers are taken from a previous paper without evidence of matched preprocessing, splits, or evaluation, but the frequency-selection leakage is the single load-bearing flaw: it alone can explain the headline improvement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes MST-ECoGNet, a lightweight CNN decoder for six-class visual decoding from ECoG signals. It first applies the Modified S Transform to 128-channel ECoG segments, producing a complex three-dimensional time-frequency-spatial feature volume; separate encoders process the real and imaginary parts with a spatial-filter convolutional layer, and the concatenated features are classified by a fully connected layer. Experiments are reported on two macaque subjects with five-fold cross-validation. The paper claims accuracy improvements over baseline [5] of 6.63 percentage points for MonJ and 16.63 points for MonC while reducing parameter count to about 9-11% of [5], and it reports exploratory analyses on frequency-range selection, spatial/temporal/frequency feature importance, and real-imaginary complementarity, including a 50 ms lag in informative ECoG content after stimulus onset.","tokens_in":12372,"tokens_out":5699,"duration_ms":50322,"significance":"The proposed pipeline is attractive for BCI because it is simple, physically interpretable, and potentially much smaller than the prior model; the exploratory frequency/time importance analyses are useful hypotheses for neuroscience. The paper also makes the comparison falsifiable by specifying the dataset and a concrete baseline. However, the central accuracy claim is not reliably established: the per-subject frequency range is selected on the basis of model performance without a described held-out validation, the baseline numbers are quoted rather than re-run under matched conditions, and several reported values are internally inconsistent. These issues directly affect the headline comparison, so the strength of the contribution cannot be assessed until they are resolved.","major_comments":[{"comment":"The per-subject frequency range is chosen by training the network on reduced data and selecting the range 'based on the performance of the network' (Sec. IV-A), but the paper never states that this selection used validation folds separate from the 5-fold test partitions described in Sec. III-C. Because the frequency range determines the input dimension F and is a hyperparameter, selecting it on the same folds used for the reported accuracies in Table II causes optimistic bias in the headline 6.63% and 16.63% improvements over [5]. A nested validation scheme, or an explicit statement that a separate validation split was used, is required before the accuracy comparison can be accepted.","section":"Sec. IV-A and Table II"},{"comment":"The arithmetic of the reported improvements is inconsistent. Table II reports MonC accuracy 54.15±1.15%, which is 17.35 percentage points above the 36.8±0.51% baseline, not 16.63%; MonJ improves by 8.39 points, not 6.63. Section III-E states MonC accuracy 53.43%, differing from Table II's 54.15%. Also, the MonJ model-size reduction is described in Sec. III-E as 'down to 1.08%' whereas 0.0488/0.451 = 10.82%. The central quantitative claims must be corrected and made consistent.","section":"Sec. III-E, Table II, and Abstract"},{"comment":"The comparison with [5] uses accuracy numbers quoted from the prior paper without re-running that model under the same preprocessing, cross-validation folds, and evaluation protocol used for MST-ECoGNet. Unless the implementation and protocol are matched, the reported gains and the parameter-count comparison could reflect differences in the experimental setup rather than the proposed architecture. The authors should re-run [5] on the same preprocessed inputs and folds, or explicitly document that the published numbers were obtained under an identical protocol.","section":"Sec. III-E and Table II"}],"minor_comments":[{"comment":"There are typographical errors: 'nn.Cov3d' should be 'nn.Conv3d', 'Cantenate' should be 'Concatenate', 'trails' should be 'trials', and 'wad' should be 'was'; these should be corrected.","section":"Throughout"},{"comment":"The frequency screening grid is not specified (range, step size, and number of candidate ranges); please state it so that the selection procedure is reproducible.","section":"Sec. IV-A"},{"comment":"The MST parameters a=5, b=50, c=74 are cited to [40], but the manuscript should state whether these were fixed a priori or tuned on data.","section":"Sec. II-A"},{"comment":"The column header 'Params / M' is ambiguous; specify whether the values are in millions and include the exact parameter counts used in the comparison.","section":"Table II"},{"comment":"The assignments in Algorithm 1 are not typeset cleanly and the notation ti=0 versus ti+=1 is hard to follow; please reformat the pseudocode.","section":"Algorithm 1"},{"comment":"Figures 5, 6, and 7 do not report error bars or the number of folds; adding this information would help assess the reliability of the exploratory findings.","section":"Figures 5-9"},{"comment":"The sentence 'first, we used the 3D Feature data obtained by the MST method to train the corresponding model' should clarify that the temporal-importance evaluation is done on the already-trained model, as Algorithm 1 states.","section":"Sec. IV-D"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is more of an applied BCI paper than a mathematical or numerical analysis contribution; for a math.NA venue the methodological novelty (MST plus a simple CNN) is modest. The refereeing should focus on whether the empirical validation can be made sound. If the authors cannot supply a separate validation split for the frequency-range selection, and if the baseline cannot be re-run under matched conditions, the paper should be rejected. The inconsistent accuracy figures also need to be resolved before any revision can be evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The stress-test note holds up: Sec. IV-A selects the per-subject frequency range by training the network on reduced datasets, and the only evaluation described anywhere is the 5-fold CV of Sec. III.C. There is no separate validation split. That makes the frequency range a hyperparameter tuned on the test folds, and with up to 128 candidate ranges the reported gains over [5] are very likely inflated. This is the load-bearing flaw, and it lands.\n\nThat said, the paper is not empty. Combining MST time-frequency features with a compact EEGNet-style spatial-filter CNN is a sensible, explainable design. The empirical observations—low-frequency dominance, a spatial pattern that the spatial filter picks up, complementary real/imaginary information, and a roughly 50 ms latency from stimulus to informative ECoG—are worth a follow-up, even though they rest on two monkeys and a masking procedure without statistical validation.\n\nThe soft spots beyond the leakage: the baseline [5] is quoted without evidence of matched preprocessing, splits, or evaluation; and the manuscript has internal inconsistencies (abstract vs Table II improvement numbers, Section III.E says MonJ is 1.08% of the baseline size rather than 10.82%). These don't change the main problem; they make it harder to trust the manuscript as it stands.\n\nWho is this for? BCI and visual-neuroscience readers who want an accessible, explainable baseline and some candidate phenomena to test on larger datasets. I would not cite it in its current form, but it deserves referee time rather than a desk reject: the data are real, the approach is transparent, and the evaluation can be redone with a proper validation split and a re-run baseline. Send it out with a major-revision request.","headline":"Frequency-range selection on test folds is the load-bearing flaw; the compact architecture and exploratory findings still warrant a second look.","tokens_in":12972,"tokens_out":3873,"would_cite":false,"duration_ms":34981,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A compact convolutional net with MST time-frequency features decodes visual categories from ECoG signals more accurately than a baseline model ten times its size.","keywords":["ECoG decoding","visual information","Modified S Transform","time-frequency analysis","spatial filter","brain-computer interface","explainable deep learning","macaque visual cortex"],"falsifier":"Retrain MST-ECoGNet and the baseline with the frequency band for each subject selected on the training folds only (or fixed in advance to e.g. 0-60 Hz), then evaluate on held-out test folds; if the accuracy gap over [5] collapses, the reported advantage is an artifact of test-time band selection rather than the architecture.","tokens_in":11875,"feed_emoji":"🧠","tokens_out":9339,"duration_ms":80496,"temperature":0.7,"pith_summary":"The paper proposes MST-ECoGNet, a decoder that turns electrocorticogram (ECoG) recordings of macaque visual cortex into a six-way classification of viewed images (building, body part, face, fruit, insect, tool). It aims to show that a small, interpretable pipeline—Modified S Transform time-frequency analysis followed by a single spatial-filter convolution—extracts the visual information that larger black-box models miss. The reported result is higher accuracy than the baseline model at roughly one tenth of the parameters: 54.15% vs 36.8% for one monkey and 35.98% vs 27.59% for the other. The paper also claims this architecture reveals where visual information lives: low-frequency bands, spatial patterns across electrodes, and complementary real/imaginary parts of the complex transform, with a roughly 50 ms delay after stimulus onset. These findings matter for building lightweight, explainable brain-computer interfaces and for guiding future electrode design.","feed_headline":"Tiny ECoG net beats 10x-larger decoder on visual categories","feed_subtitle":"Frequency and spatial filtering lift accuracy up to 16 points at one-tenth the parameters.","key_machinery":"The load-bearing object is the Modified S Transform (MST), a time-frequency transform whose window width adapts to frequency through an arctangent function; it converts each 128-electrode ECoG trial into a 3D complex feature cube indexed by frequency, time, and space. The classifier side is minimal: a batch-normalized ELU network whose first conv3d kernel (1 x 128 x 1) acts as a spatial filter over electrodes, followed by average pooling and two small convolutions, with identical real and imaginary encoders running in parallel and concatenating before a fully connected layer. MST supplies the frequency localization; the spatial filter supplies the channel selection; the parallel encoders supply the complementary real/imaginary information. Together they make the pipeline inspectable at each stage.","core_discovery":"The central claim is that visual information in ECoG signals is recoverable from the complex time-frequency representation produced by the Modified S Transform, and that a deliberately simple network can exploit it. The model feeds the real and imaginary parts of the MST output through two parallel encoders whose core is a single conv3d layer acting as a spatial filter across the 128 electrodes; the encoded vectors are concatenated and classified by one fully connected layer. On the macaque ECoG dataset, the paper reports six-class accuracies of 54.15±1.15% (MonC) and 35.98±1.15% (MonJ) against baseline accuracies of 36.8±0.51% and 27.59±0.73%, with 0.0396M and 0.0488M parameters versus 0.451M. Ablation experiments attribute the gain to three properties: low-frequency dominance (optimal bands 0-38 Hz and 0-52 Hz), a spatial pattern selected by the spatial filter, and complementarity between real and imaginary coefficients, which the parallel encoders combine. A temporal-importance test finds decodable visual information appearing about 50 ms after stimulus onset.","pith_inferences":["A natural test the paper does not run is whether the per-subject frequency band can be selected inside each training fold; if the advantage persists under that protocol, the method would be cross-validated rather than tuned to the test set.","The spatial filter's learned weights could be visualized as a 128-electrode map; comparing that map across monkeys might reveal whether the temporal-cortex spatial pattern is a general visual-code property or an individual implant artifact.","If the 50 ms onset delay is a neural signature rather than an artifact of the transform window, then the same ablation protocol applied to other modalities (e.g., EEG or spiking data) should reproduce a comparable delay.","Because the paper uses only two subjects and six categories, a direct extension would test whether the real/imaginary parallel encoder helps on multi-subject pooled data and on finer-grained visual categories."],"forward_implications":["If the reported accuracy holds, the same time-frequency plus spatial-filter recipe can be applied to other high-density ECoG arrays, yielding lightweight decoders for BCI use.","The per-subject optimal bands (0-52 Hz, 0-38 Hz) imply that future recording systems can concentrate sampling or filtering in low frequencies without losing decodable visual information.","The superiority of the spatial filter suggests that electrode array geometry and density, not just signal processing, can be optimized to capture the visual pattern.","The real/imaginary complementarity result implies that treating complex time-frequency coefficients as magnitude-only discards usable information, so neural decoders should keep phase-related structure.","The observed 50 ms lag between stimulus onset and decodable signal provides a concrete timing anchor for studying visual processing in inferior temporal cortex."],"supporting_citations":[{"why":"Provides the ECoG visual dataset, the baseline model, and the comparison accuracies and parameter counts that all performance claims are measured against.","marker":"[5]"},{"why":"Defines the Modified S Transform with adjustable window that the feature extractor is built on.","marker":"[40]"},{"why":"Supplies the compact convolutional architecture (EEGNet) whose spatial-filter design the classifier borrows.","marker":"[23]"},{"why":"Establishes that separate frequency bands in ECoG encode visual features, motivating the time-frequency approach.","marker":"[6]"}],"fun_headline_variants":["Explainable ECoG decoder gains 16 points with 90% fewer parameters","Real and imaginary ECoG features combine for 16% accuracy boost","MST-ECoGNet: small, explainable, decodes visual info 50ms after stimulus","Explainable ECoG decoder: 90% less params, 16 points more accuracy","ECoG visual decoding: 50ms after onset, small net explains it all"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The per-subject frequency range was chosen by training the model on progressively truncated bands and keeping the one with the best accuracy, and the paper never states that this selection used a separate validation split rather than the same test folds reported in Table II.","fun_headline_variants_meta":{"raw":{"variants":["Explainable ECoG decoder gains 16 points with 90% fewer parameters","Real and imaginary ECoG features combine for 16% accuracy boost","MST-ECoGNet: small, explainable, decodes visual info 50ms after stimulus","Explainable ECoG decoder: 90% less params, 16 points more accuracy","ECoG visual decoding: 50ms after onset, small net explains it all"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001271,"raw_usage":{"total_tokens":5288,"prompt_tokens":1124,"completion_tokens":4164,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":740,"completion_tokens_details":{"reasoning_tokens":4051}},"tokens_in":740,"tokens_out":4164,"duration_ms":27668,"temperature":1.0,"reasoning_tokens":4051,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:27:38.209554+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain MST-ECoGNet and the baseline with the frequency band for each subject selected on the training folds only (or fixed in advance to e.g. 0-60 Hz), then evaluate on held-out test folds; if the accuracy gap over [5] collapses, the reported advantage is an artifact of test-time band selection rather than the architecture.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ECoG visual dataset, the baseline model, and the comparison accuracies and parameter counts that all performance claims are measured against."},{"cited_title":"and Hsieh, M.-H., ``Classification of mental task from eeg data using neural networks based on particle swarm optimization,'' Neurocomputing, vol","cited_arxiv_id":null,"evidence_quote":"Defines the Modified S Transform with adjustable window that the feature extractor is built on."}],"review_version":1}