{"id":"28b5a081-f291-4973-9989-9477de40ad3b","arxiv_id":"1908.05743","paper_version":5,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"This paper shows that with small vocabularies, EEG signals can drive speech recognition models, and that speech spectra can be reconstructed from EEG with LSTM and GAN models.","lead":"The authors report that EEG brain recordings can be used for continuous speech recognition on a small set of English sentences, and that speech features (MFCCs) can be decoded from EEG using LSTM and generative models. The study suggests EEG-based speech decoding is possible, but the results are preliminary and come with significant methodological caveats about data splits and baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EEG-to-spectrum decoding results depend on an unspecified 90/10 split; because all subjects repeat the same nine sentences, random splits can place identical or temporally adjacent utterances in both train and test, inflating the low RMSE/MCD.","rationale":"The reader's weakest assumption is the same one I would flag: the decoding experiments in Sections 6.1 and 6.2 do not state whether the 90/10 split is subject-independent or utterance-independent. Since every subject produced or listened to the same 9 or 30 USC-TIMIT sentences and repeated each three times, a random split can place same-sentence, same-speaker, or even temporally adjacent data in both training and test. At 100 Hz feature sampling, frame-level leakage is especially severe because of autocorrelation. This is not a minor methodological detail; it is the difference between 'decode speech spectrum from EEG' and 'memorize the training corpus.' The paper's own text supports this concern: Section 10 says the databases are only planned for release, so the exact split cannot be checked. The ASR experiments in Section 8 also leave the CTC/attention split ambiguous, though the RNN-transducer part explicitly separates subjects. I am not claiming the results are fabricated; only that the manuscript, as written, does not rule out the most likely alternative explanation. A revised paper with explicit subject- and utterance-independent splits and released split scripts could rehabilitate the claim. The title 'state-of-the-art' is not supported by any comparison, but that is secondary once the leakage issue is addressed. Therefore the reader's REJECT verdict is appropriate.","tokens_in":10615,"tokens_out":5792,"duration_ms":61341,"concrete_test":"Request the exact train/test split for the Section 6.1 LSTM regression and re-run Table 8 (feature set 1) under three controls: (1) leave-one-subject-out; (2) leave-one-utterance-out so no sentence appears in both train and test; (3) temporal holdout, training on the first 90% of each trial and testing on the last 10%. If any control raises normalized RMSE from 0.029 to above roughly 0.15, or average MCD from 1.4 to above 5, the reported decoding is explained by train/test overlap rather than by EEG-to-spectrum generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that EEG can be decoded into speech spectrum (MFCC), as evidenced by LSTM regression reaching normalized RMSE ~0.029 and MCD ~1.4 (Tables 8-10) and spoken MFCC normalized RMSE 0.126 (Table 11). The load-bearing condition for this claim is that the test set measures generalization to unseen EEG-MFCC pairs. Section 6.1 states only '90 % of the data was used to train the model and remaining 10 % was used as test set,' and Section 6.2 repeats this; no split unit (frame, trial, sentence, or subject) is specified. Databases A, B, and C use the same first 9 or 30 USC-TIMIT sentences, and each subject repeats each sentence three times. With a random 90/10 frame-level split, consecutive 100 Hz EEG/MFCC frames can straddle the train/test boundary; because EEG and MFCC are strongly autocorrelated, this is near-duplicate testing. Even a trial-level random split leaks, since the same sentence from the same subject appears in both sets, and for listen EEG the same audio stimulus is reused. The ASR results in Section 8 have a parallel ambiguity: CTC and attention models use an 80/10/10 split without subject independence, while only the RNN-transducer section explicitly separates subjects. The paper also provides no baseline (e.g., predicting the mean MFCC) to calibrate the reported low RMSE, and Section 10 says the databases are only planned for release, so the split cannot currently be audited. If the split is actually subject- and utterance-independent, the concern is resolved; the text as written does not establish that.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims two main results: continuous noisy speech recognition from EEG using end-to-end ASR models (CTC, attention, RNN transducer) across three databases and three EEG feature sets, and decoding of speech spectrum (MFCC) from EEG using LSTM regression, GAN, and WGAN models. The authors report low WER/CER for small sentence corpora (e.g., 0% WER for 3 sentences in some settings) and very low normalized RMSE and MCD for MFCC prediction (e.g., normalized RMSE 0.0291 in Table 8). The paper concludes that EEG can support both recognition and synthesis of speech over a limited vocabulary.","tokens_in":10974,"tokens_out":3407,"duration_ms":32159,"significance":"If the claims are valid, this would be a notable step toward non-invasive EEG-based speech communication, with implications for assistive technology and brain-computer interfaces. The paper is commendable for exploring multiple experimental conditions (spoken, listen, concatenated), three feature sets, and several model families, and for providing preliminary synthesis results. However, the central feasibility claim is not currently supported by the evidence as presented: the decoding experiments lack a specified train/test split unit, no baselines or significance tests are provided, and the ASR experiments have similar split ambiguities. These issues are load-bearing because they affect whether the reported metrics reflect true generalization. Therefore, the current significance is conditional on a substantially improved validation.","major_comments":[{"comment":"The train/test split for the MFCC decoding experiments is not specified at the level of frames, trials, sentences, or subjects. In Sections 6.1 and 6.2 the paper states only that 90% of the data is used for training and 10% for testing, without defining the unit of the split. Since each subject repeats the same nine (or thirty) sentences, a random frame-level split can place nearly identical EEG and MFCC frames in both training and test sets. Given the strong autocorrelation of these signals, this would inflate the decoding metrics and undermine the claim of spectrum decoding. Please provide the split unit and results for a subject-independent and utterance-independent split.","section":"Sections 6.1 and 6.2, Tables 8-11"},{"comment":"The ASR experiments for CTC and attention models use an 80/10/10 train/validation/test split, but the split unit is not stated. Only the RNN transducer section (Section 8) explicitly separates subjects. If the CTC/attention splits are random at the utterance or frame level, then the same sentence spoken or listened to by the same subject can appear in both training and test sets, artificially lowering WER and CER. The paper must clarify the split unit and provide results for a subject-independent split, or justify why the current split is appropriate.","section":"Section 8, Tables 1-7"},{"comment":"No baselines or statistical significance tests are reported anywhere. For the decoding experiments, there is no comparison to a trivial predictor such as the mean MFCC or the previous-frame MFCC, so the low normalized RMSE and MCD values in Tables 8-11 cannot be evaluated. For the ASR experiments, there is no chance-level WER/CER or confidence intervals, making it impossible to judge whether the results are meaningful. The authors should add baselines and significance tests, or explicitly justify their absence.","section":"Tables 8-11 and Tables 1-7"},{"comment":"The paper claims 'state-of-the-art speech recognition using EEG' in the title and abstract, but it does not compare the reported WER/CER with the prior results in references [1] and [2] under matched conditions. The only stated comparison is internal among the authors' own feature sets and models. Without a direct comparison to existing EEG-based ASR results, the 'state-of-the-art' claim is unsupported. Please provide quantitative comparisons with the same evaluation protocols.","section":"Section 8 and Tables 1-7"}],"minor_comments":[{"comment":"The procedure for choosing the KPCA dimension using the explained-variance plot from PCA is conceptually questionable, since KPCA does not share the same explained-variance interpretation as PCA. Please clarify the justification or use a more standard model-selection approach (e.g., validation-set performance).","section":"Section 5"},{"comment":"The text says 'we used three evaluation metrics: RMSE, Normalized RMSE and Mel cepstral distortion (MCD)' but does not define how the average values in Tables 8-11 are computed across test set samples. Please specify the averaging procedure.","section":"Section 6"},{"comment":"The paper mentions that the databases are planned for release, but they are not currently available. Given that the split ambiguity is a central issue, providing data or detailed subject/utterance indices for the splits would greatly help verification.","section":"Section 10"},{"comment":"Several figures (Figures 3-5, 6-7, 8-10, etc.) are not referenced in the main text, making it unclear where the reader should look for supporting evidence. Please add appropriate references.","section":"Throughout"},{"comment":"The feature extraction text states 'for this set, EEG feature dimension was 31(channels) × 5 or 155' and similar lines, but the notation is inconsistent with the later statement that final dimensions include delta and delta-delta features. Please clarify the dimensions before and after taking differentials.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and timely problem, and the authors have attempted a substantial experimental effort. However, the unspecified data-splitting procedure is a serious flaw that affects both the ASR and decoding claims. If the split is not subject- and utterance-independent, the reported results are essentially uninterpretable. I recommend that the editor require a clear description of the split unit and, if necessary, a re-analysis with a strictly held-out split. Without that, the paper would not meet the standards of the journal. The lack of baselines and significance tests is also a concern, but it is more easily fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — read the decoding section if you read anything. The ASR part is an incremental extension of the authors' own prior work on a nine-sentence corpus, and the error rates are high. The genuinely new piece is decoding listen and spoken MFCC spectra from EEG using LSTM regression and GANs, with reported normalized RMSEs around 0.03. Those numbers look too good to be real, and the paper as written does not let you check.\n\nCredit where due: the paper extends EEG-based ASR to listening and listen+spoken conditions, compares three feature sets, and honestly reports that the simple LSTM beats the GAN and WGAN, and that the RNN transducer underperformed. The feature extraction is described in enough detail to reproduce if you had the data. The authors do say the databases are only planned for release, not yet available.\n\nThe soft spots are load-bearing. The 90/10 split for the decoding experiments is described only as '90% of the data used to train and remaining 10% used as test'; no unit is specified. Since each subject repeats the same nine or thirty sentences, a frame-level or even trial-level random split puts near-duplicate EEG/MFCC pairs in both training and test, which trivially yields low RMSE and MCD. The same ambiguity affects the CTC and attention ASR splits (80/10/10 with no subject separation); only the RNN-transducer section explicitly separates subjects. There are no baselines like predicting the mean MFCC or the most common sentence, and no confidence intervals or significance tests anywhere. The 'state-of-the-art' claim has no comparisons.\n\nThe stress-test note is on target. I don't think the authors are being deceptive; the split is just not auditable. The fix is straightforward and requires no new data: re-run with subject-independent or utterance-independent splits, state the split unit, add trivial baselines, and report per-condition error bars. If the results survive, they would be worth taking seriously. If not, the decoding claim collapses to a memorization artifact.\n\nWho is this for? Researchers building EEG-to-speech BCIs, and anyone thinking about how to evaluate small-vocabulary neural decoding. It is a useful cautionary example, so I would bring it to a reading group.\n\nRecommendation: send to peer review, but with an explicit request for the split unit, subject-independent results, and baselines. The core idea deserves a serious referee's time; the version in front of us does not yet support the headline decoding numbers.","headline":"Interesting preliminary work on EEG-based speech recognition and MFCC decoding, but the headline decoding numbers cannot be trusted until the train/test split and baselines are clarified.","tokens_in":717,"tokens_out":1572,"would_cite":false,"duration_ms":27423,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper reports that EEG recorded while people speak or listen in noise can drive continuous end-to-end speech recognition on a small English vocabulary, and that an LSTM regression model decodes speech spectrum features (MFCCs) from…","keywords":["EEG speech recognition","end-to-end ASR","CTC","attention model","RNN transducer","MFCC decoding","LSTM regression","speech synthesis from EEG"],"falsifier":"Re-run the MFCC-decoding experiments with a leave-one-subject-out or leave-one-sentence-out split: if the LSTM regression's normalized RMSE and MCD rise to the level of the GAN/WGAN results (or higher), the low reported errors are artifacts of train/test overlap rather than evidence of generalizable speech-spectrum decoding from EEG.","tokens_in":10422,"feed_emoji":"🧠","tokens_out":7921,"duration_ms":68225,"temperature":0.7,"pith_summary":"The paper tries to show that non-invasive EEG signals carry enough information about spoken and heard speech to support both recognition and spectrum reconstruction on a small English vocabulary. Using end-to-end ASR models (CTC, attention, RNN transducer), the authors report continuous noisy speech recognition for three to nine sentences, with the attention model reaching 0% word error rate on the three-sentence spoken corpus. They also train LSTM-regression, GAN, and WGAN models to map EEG to MFCC features, and report that LSTM regression gives the lowest RMSE and mel cepstral distortion on held-out test data. If these results hold, EEG could become a non-invasive input for speech-restoration interfaces, avoiding the surgery required by ECoG-based decoders.","feed_headline":"EEG transcribes nine spoken sentences and rebuilds speech spectra","feed_subtitle":"Attention models hit 0% word error on nine spoken sentences; LSTM regression recovers the speech spectrum from EEG.","key_machinery":"The machinery is a pipeline from EEG to text or spectrum. Three EEG feature sets are extracted at 100 Hz — the set used in earlier work (RMS, zero-crossing rate, moving-window average, kurtosis, power spectral entropy), a short-time-Fourier/wavelet-entropy set, and a neuroscience-style band-entropy/Hurst/fractal-dimension set — then reduced with kernel PCA and augmented with delta and delta-delta features. Recognition runs through three end-to-end ASR architectures: CTC, attention-based encoder-decoder, and RNN transducer. Spectrum decoding runs through a two-layer LSTM regression model and through GAN/WGAN generators that take real EEG and output MFCC vectors; the LSTM regression model carries the argument because it produces the reported low test errors, while the adversarial models are presented as a less stable alternative.","core_discovery":"The central claim is that EEG recorded in parallel with speech (and with listening) can be mapped directly to text and to acoustic features. For recognition, the paper uses character-level CTC and RNN-transducer models and a word-level attention encoder-decoder, all trained on EEG feature sets after KPCA dimensionality reduction; on the smallest corpus of three sentences the attention model's test WER is 0%, rising as vocabulary grows, and the CTC model yields CER values from about 27% to 73% depending on condition and corpus size. For spectrum decoding, an LSTM regression model predicts 13-dimensional MFCC sequences from EEG, with normalized RMSE around 0.027-0.126 and MCD around 1.3-5.7, while GAN and WGAN models perform substantially worse and are harder to train. The paper frames these as feasibility results: EEG supports continuous noisy speech recognition across different experimental conditions and provides preliminary, not production-ready, evidence for speech synthesis from EEG.","pith_inferences":["The MFCC-decoding numbers should be read with caution until a subject-independent split is tested: because every subject repeats the same sentences, a random 90/10 split can place repetitions of the same sentence in both training and test, which would make the low RMSE/MCD values partly a memorization result.","A stronger test of the recognition claim would use held-out sentences (leave-one-sentence-out) as well as held-out speakers; the reported 0% WER on three sentences likely depends on the closed vocabulary of nine sentences.","If the decoding pipeline does transfer across subjects, the natural next comparison is to the invasive ECoG decoder that inspired this work, since EEG's non-invasiveness would be its main practical advantage.","The observation that a simple LSTM regression beats adversarial models suggests that for EEG-to-spectrum problems, reconstruction loss may be the right objective and GAN training instability is a cost without evident benefit in this data regime."],"forward_implications":["If the recognition results generalize beyond the small corpus, EEG could be used as a non-invasive input modality for voice-activated systems and speech-restoration devices, avoiding ECoG surgery.","The reported performance ranking — attention and CTC outperform RNN transducer, and LSTM regression outperforms GAN/WGAN — gives future EEG-based systems a concrete architecture baseline to start from.","Since error rates rise with vocabulary size on all models, larger multi-speaker EEG corpora and external language models would be needed before real-world continuous EEG speech recognition is practical.","The predicted MFCC features can in principle be converted to audible speech with Griffin-Lim reconstruction, making a complete silent-speech interface from EEG a testable next step."],"supporting_citations":[{"why":"Supplies the original EEG features and the no-speech/noisy-speech ASR baseline that this paper extends.","marker":"[1]"},{"why":"Establishes continuous EEG-based ASR and provides the attention mechanism and feature pipeline reused here.","marker":"[2]"},{"why":"Demonstrates neural decoding and synthesis of spoken sentences from ECoG, motivating the EEG-to-MFCC decoding experiments.","marker":"[3]"},{"why":"Provides the LSTM architecture used in the regression, GAN, and WGAN models.","marker":"[8]"},{"why":"Defines the GAN training objective used for generating MFCC features from EEG.","marker":"[9]"},{"why":"Introduces the Wasserstein loss used as a stabilized alternative to the GAN objective.","marker":"[10]"},{"why":"Provides the CTC loss and end-to-end training procedure used by the CTC ASR model.","marker":"[12]"},{"why":"Defines the RNN transducer sequence-transduction model used as the third ASR architecture.","marker":"[16]"},{"why":"Supplies the USC-TIMIT sentences used to construct all three EEG databases.","marker":"[21]"},{"why":"Provides the Griffin-Lim algorithm that converts predicted MFCC features to interpretable audio.","marker":"[24]"}],"fun_headline_variants":["EEG speech recognition: 0% WER on small set, spectrum decoded too","EEG to text: 0% WER on tiny set; spectrum recovery via LSTM","EEG transcribes speech, reconstructs spectra with LSTM","EEG-based ASR: 0% WER on small set, spectrum reconstruction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The speech-spectrum decoding results depend on the assumption that the 90/10 train/test split keeps the same utterance from the same speaker out of the training set; the paper does not state that the split is subject-independent.","fun_headline_variants_meta":{"raw":{"variants":["EEG speech recognition: 0% WER on small set, spectrum decoded too","EEG to text: 0% WER on tiny set; spectrum recovery via LSTM","EEG transcribes speech, reconstructs spectra with LSTM","EEG-based ASR: 0% WER on small set, spectrum reconstruction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001014,"raw_usage":{"total_tokens":4231,"prompt_tokens":842,"completion_tokens":3389,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":3301}},"tokens_in":458,"tokens_out":3389,"duration_ms":23693,"temperature":1.0,"reasoning_tokens":3301,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:20:23.710199+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the MFCC-decoding experiments with a leave-one-subject-out or leave-one-sentence-out split: if the LSTM regression's normalized RMSE and MCD rise to the level of the GAN/WGAN results (or higher), the low reported errors are artifacts of train/test overlap rather than evidence of generalizable speech-spectrum decoding from EEG.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the original EEG features and the no-speech/noisy-speech ASR baseline that this paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes continuous EEG-based ASR and provides the attention mechanism and feature pipeline reused here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates neural decoding and synthesis of spoken sentences from ECoG, motivating the EEG-to-MFCC decoding experiments."},{"cited_title":"Schmidhuber","cited_arxiv_id":null,"evidence_quote":"Provides the LSTM architecture used in the regression, GAN, and WGAN models."},{"cited_title":"Pouget-Abadie, M","cited_arxiv_id":null,"evidence_quote":"Defines the GAN training objective used for generating MFCC features from EEG."},{"cited_title":"Chintala, L","cited_arxiv_id":null,"evidence_quote":"Introduces the Wasserstein loss used as a stabilized alternative to the GAN objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CTC loss and end-to-end training procedure used by the CTC ASR model."},{"cited_title":"Toutios, V","cited_arxiv_id":null,"evidence_quote":"Supplies the USC-TIMIT sentences used to construct all three EEG databases."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Griffin-Lim algorithm that converts predicted MFCC features to interpretable audio."}],"review_version":1}