{"id":"13dc7deb-81c9-4fda-8897-9590060b88b7","arxiv_id":"2506.00381","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage model aligns intracranial EEG signals with text embeddings and reconstructs the semantic content of perceived speech from as little as 30 minutes of neural data.","lead":"Researchers trained a system that translates brain recordings of people listening to speech into approximate text, needing only 30 minutes of data per person. The system aligns brain signals with language embeddings and then uses a text-generating model to write out the decoded meaning, potentially easing brain-computer interface data requirements.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 'outperforms baseline' rests on sentence-level t-tests that ignore subject/story clustering; per-subject or mixed-effects analysis is needed before the claim is credible.","rationale":"The reader's weakest_assumption focused on the distribution-shift risk in Phase 2: whether neural-aligned embeddings lie close enough to pre-trained text embeddings for the Vec2Text corrector to invert them. That is a real and testable risk, and the absence of explicit embedding-similarity validation is concerning. However, my reading suggests an even more load-bearing issue: the statistical evidence for the central comparative claim is computed at the sentence level while ignoring the clustered data structure (multiple sentences per subject, multiple subjects per story). Because the paper has only three subjects, sentence-level pooling can massively inflate significance, and the paper reports no subject-level or story-level statistics. If clustered inference eliminates the difference, the headline 'outperforms recent state-of-the-art' collapses, independent of how well the embeddings are aligned. I therefore partially agree with the reader: the conditional verdict is appropriate, and the requested subject-level statistics are necessary, but I would prioritize the clustered-inference check over the embedding-similarity check. I keep the verdict as CONDITIONAL/UNCHANGED because the requested analysis could plausibly confirm or refute the claim, and the requested reanalysis is cheap and well-specified.","tokens_in":8064,"tokens_out":4342,"duration_ms":46244,"concrete_test":"Re-analyze Figure 2A and Table 1 using clustered inference: compute Neuro2Semantic and baseline BERTScore/BLEU per subject per held-out story (3 subjects x 6 stories = 18 matched units), then compare methods with a paired test or a linear mixed-effects model with random intercepts for subject and story, or a cluster-bootstrap confidence interval on the mean difference. If the 95% CI for the difference includes zero or p >= 0.05 under clustered inference, the claimed superiority over the baseline is not supported by the data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Neuro2Semantic outperforms the baseline in low-data settings is supported in Section 3.2 and Table 1 by paired t-tests computed over pooled sentence-level BERTScore and BLEU values. With three subjects and six stories, each subject contributes many non-independent sentences drawn from the same conversations, so pooled sentence pairs violate the independence assumption of the t-test. A paired t-test over thousands of correlated sentence pairs can produce p < 0.05 even when no consistent per-subject or per-story improvement exists. The paper reports no subject-level means, no per-story matched comparisons, and no mixed-effects or clustered resampling analysis. Since the claim is explicitly comparative ('outperforming'), the relevant unit of analysis is the subject or story, not the individual sentence. Without clustered inference, the headline result is not statistically established, and the 'strong performance with 30 minutes' claim inherits the same fragility.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Neuro2Semantic, a two-phase transfer-learning framework for decoding perceived-speech semantics from intracranial EEG. Phase 1 trains an LSTM adapter to map high-gamma envelope segments to text-embedding-ada-002 embeddings via a weighted contrastive and triplet loss; Phase 2 freezes the adapter and fine-tunes a Vec2Text corrector to generate continuous text from the aligned neural embeddings. The model is evaluated on three epilepsy patients listening to six podcast-like conversations (about 30 minutes of speech), using BERTScore and BLEU against a version of the Tang et al. (2023) fMRI decoder adapted to iEEG. The authors report ablations, zero-shot out-of-domain results, data- and electrode-scaling curves, and state in the discussion that the small sample size and clinical population limit generalizability, explicitly presenting the study as exploratory. Code and trained models are released.","tokens_in":8241,"tokens_out":7579,"duration_ms":73032,"significance":"If the comparative claims held, Neuro2Semantic would be a useful demonstration that pre-trained text-embedding inversion can be applied to iEEG with very limited training data, enabling open-vocabulary, continuous text generation. The two-phase design, the out-of-domain test, the ablations, and the public code release are genuine strengths. I do not see a circularity problem: the method is a supervised mapping from neural activity to pre-trained text embeddings. However, the evidence as presented is not yet at the level claimed in the abstract: the baseline is not shown to be state of the art for iEEG, the statistical tests ignore sentence-level clustering within subjects and stories, and the absolute BERTScore and BLEU values are low. The paper's practical significance is therefore conditional on a clustered re-analysis and more cautious interpretation of the reported effect sizes.","major_comments":[{"comment":"The central 'outperforms' claim is supported only by paired t-tests computed over pooled sentence-level BERTScore and BLEU values. Because the three subjects each contribute many sentences drawn from the same six stories, the independence assumption of the t-test is violated, and the caption of Table 1 states that the asterisks mark improvements over the random control, not over the baseline. A paired t-test over thousands of correlated sentence pairs can yield p < 0.05 even when no consistent per-subject or per-story improvement exists. The manuscript reports no subject-level or story-level means, no mixed-effects model with random intercepts for subject and story, and no clustered bootstrap. I ask that the authors re-run the comparison with subject or story as the unit of analysis (or a mixed-effects model), report the corresponding effect sizes, and show that the headline result survives that analysis.","section":"Section 3.2, Table 1, Figure 2A"},{"comment":"The baseline is the fMRI Bayesian decoding method of Tang et al. (2023), adapted to iEEG by using high-gamma features and shorter finite impulse response delays, and Section 3.2 refers to it as 'a recent state-of-the-art method.' The paper does not establish that this adapted method is state of the art for iEEG semantic decoding, and it does not compare against any iEEG-specific decoder, such as Makin et al. [23] or other intracranial speech decoders cited in the introduction. Since the abstract's comparative claim depends on this baseline, the authors should either add an appropriate iEEG baseline or explicitly reframe the contribution as showing an advantage over this particular adapted fMRI method, and remove or qualify the 'state-of-the-art' language.","section":"Section 2.3, Section 3.2"},{"comment":"The abstract and text describe BERTScore 0.195 and BLEU 0.079 as 'strong performance,' but these are low in absolute terms, and the only reference point is the random control (BERTScore -0.245). No comparison with previously reported iEEG decoding scores on the same metrics, no human evaluation, and no measure of inter-subject reliability is provided. The practical significance of the 30-minute low-data claim is therefore not established by the reported numbers. Please add external context for these metrics or soften the 'strong performance' characterization to match the evidence.","section":"Abstract, Section 3.2, Table 1"},{"comment":"Section 2.1.2 describes an iterative corrector that refines text hypotheses until cosine similarity converges, but Section 3.1 states that the corrector used only one refinement step and two epochs of fine-tuning; the manuscript should reconcile this discrepancy. More importantly, the zero-shot out-of-domain claim in Section 3.3 assumes that the LSTM adapter maps novel neural inputs into the pre-trained text-embedding distribution well enough for the Vec2Text corrector to invert them, but the paper reports no diagnostic of this distributional match. I ask for quantitative evidence, such as cosine similarity between predicted and true text embeddings on held-out stories, per-story scores with error bars, or a comparison of embedding distributions, before accepting the generalization claim; Figure 2B currently shows a bar plot without significance markers or error bars.","section":"Section 2.1.2, Section 3.1, Figure 2B"}],"minor_comments":[{"comment":"The leave-one-out cross-validation description is ambiguous: 'the last trial of each story was left out for testing' is followed by 'the held-out trial from each story served as the test set for that split,' and it is unclear whether the six folds correspond to trials or whole stories. Please clarify the fold structure and the relation between the training set and test set for each fold.","section":"Section 3.1"},{"comment":"Notation is underspecified: the dimension of the text and neural embeddings, the definitions of R and S in the baseline likelihood p(R|S), and the number of finite impulse response delays used in the adapted baseline are not given.","section":"Section 2.1.1, Section 2.3"},{"comment":"The scaling plots are described as showing 'significant performance improvements,' but no statistical test across the five runs or across training-data percentages is reported; the axes in the bottom-right panel also lack a clear metric label.","section":"Figure 2D, Figure 2E"},{"comment":"The alignment loss is described as a weighted combination of a CLIP contrastive loss and a triplet margin loss, but the connection between the CLIP reference [28] and this specific formulation is not explained, and the sentence 'batch-level similarity optimization' is vague.","section":"Section 3.1, Eq. (1)"},{"comment":"The 'random control' is not defined in the main text; please state how the random baseline was constructed (for example, shuffled neural-to-text pairings) so that the negative BERTScore can be interpreted correctly.","section":"Figure 2A"}],"recommendation":"major_revision","confidential_remarks":"The paper is an exploratory conference-scale study with an interesting two-phase transfer-learning idea and a clean, non-circular training objective. My main reservations are empirical: the statistical analysis ignores subject and story clustering, the baseline is an adapted fMRI method rather than an iEEG-specific decoder, and the absolute metric values are low. These issues are fixable within the manuscript's scope through re-analysis and more cautious claims, so I recommend major revision rather than rejection. I also note that the electrode-selection criterion and the baseline encoding models come from the authors' own prior work; this is not inherently problematic, but the manuscript should disclose the potential dependence when interpreting the comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that the framework is real, but the headline comparison is statistically fragile. The authors build a two-phase iEEG semantic decoder: an LSTM adapter aligns high-gamma envelope features to ada-002 text embeddings with a contrastive plus triplet loss, then a fine-tuned Vec2Text corrector inverts those embeddings into free text. As far as I can tell, no prior iEEG paper does that combination, and the ablations show both phases contribute. Code is released, the method is described in enough detail to reproduce, and the scaling experiments give a useful sense of data and electrode dependence.\n\nThe soft spots are real but not fatal. The main one is the statistics behind 'outperforms the baseline.' The comparison in Table 1 and Figure 2A uses paired t-tests over pooled sentence-level BERTScore/BLEU pairs across three subjects and six stories. Sentences from the same story are not independent, so those p-values are likely anti-conservative. The paper reports no per-subject or per-story means, no mixed-effects model, and no clustered resampling. Since the central claim is explicitly comparative, the unit of analysis should be subject or story, not individual sentences. The stress-test note is correct on this.\n\nSecond, the baseline is an adapted fMRI Bayesian decoder from Tang et al., not an iEEG-specific method. That makes the comparison weak: the bar is low. A proper iEEG decoder baseline, even a simple regression to embeddings, would be more convincing. Third, the hyperparameters (alpha, temperature, learning rate) were chosen by coordinate descent, and the paper doesn't say whether that tuning was done on the held-out test stories or only on validation folds. That matters for the zero-shot claims. Fourth, the neural data aren't released, so independent verification is limited to code and the authors' own recordings.\n\nThe distribution-shift concern about the pre-trained corrector is real but minor: the paper is upfront that fine-tuning adapts the corrector to neural-aligned embeddings, and the out-of-domain results suggest it works reasonably, though without stronger statistics I wouldn't over-read them.\n\nOverall, this is a serious, honest paper. The authors call it exploratory, and that's about right. It deserves a proper peer review: the method is novel, the problem is important, and the weaknesses are fixable. I'd send it out and ask for clustered statistics, a stronger baseline, and clarification of the tuning procedure. I'd probably cite it if I were working on iEEG decoders.","headline":"A plausible iEEG-to-text framework with a novel two-phase design, undercut by a headline 'outperforms' claim that rests on sentence-level t-tests ignoring subject/story clustering.","tokens_in":8781,"tokens_out":3292,"would_cite":true,"duration_ms":28701,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Neuro2Semantic claims that perceived speech semantics can be decoded from intracranial EEG by aligning neural signals with pre-trained text embeddings and then inverting those embeddings into natural language, with strong results from…","keywords":["brain decoding","semantic reconstruction","intracranial EEG","transfer learning","text embedding alignment","Vec2Text","natural language generation"],"falsifier":"Measure the cosine similarity between adapter output embeddings and their target sentence embeddings on held-out stories. If those similarities are no higher than similarity to randomly chosen text embeddings, or if the fine-tuned corrector's single refinement step cannot move a hypothesis text closer to the target on out-of-domain sentences, the transfer claim fails and reconstruction scores should fall to near-random levels.","tokens_in":7863,"feed_emoji":"🧠","tokens_out":10711,"duration_ms":95845,"temperature":0.7,"pith_summary":"Neuro2Semantic sets out to show that continuous, open-vocabulary text can be reconstructed from intracranial EEG (iEEG) recordings of perceived speech by reusing pretrained language models instead of training a decoder from scratch. The proposed pipeline first trains an LSTM adapter to map high-frequency neural activity envelopes into the embedding space of a pretrained text encoder, then fine-tunes a pretrained embedding-to-text corrector so the aligned neural embeddings can be read out as natural sentences. The authors report that 30 minutes of listening data suffices, that the model outperforms a recent continuous-decoding baseline on semantic accuracy, and that it generalizes to entirely held-out stories without additional training. They present the study as exploratory because the dataset contains three epilepsy patients, and they treat the alignment phase as the load-bearing component, supported by ablations in which removing either phase lowers performance.","feed_headline":"30 minutes of brain data reconstruct perceived speech as text","feed_subtitle":"A two-phase transfer-learning model turns intracranial EEG into natural sentences, beating a strong baseline on semantic accuracy.","key_machinery":"The carrying mechanism is the aligned embedding space produced by the adapter. An LSTM reads iEEG high-gamma envelopes and outputs fixed-dimensional embeddings, trained with a weighted combination of contrastive and triplet margin losses against target sentence embeddings. With the adapter frozen, the pretrained Vec2Text corrector — an encoder-decoder transformer that inverts text embeddings by iteratively updating a hypothesis text $x^{(t)}$ to maximize $\\cos(\\hat{e}(x), e)$ — is fine-tuned to read out natural language from the neural-aligned embeddings. Only one refinement step is used during evaluation, so the corrector's generalization depends on how close the adapter's outputs sit to the pretrained embedding manifold.","core_discovery":"The central claim is that semantic decoding of language from iEEG decomposes into two transfer steps: align neural signals to a pretrained text embedding space, then invert that embedding space back into text. After training the frozen LSTM adapter with a weighted contrastive-plus-triplet loss against the pretrained sentence embeddings, the authors fine-tune the Vec2Text corrector for two epochs with a single refinement step and report sentence-level BERTScore of $0.195 \\pm 0.128$ and BLEU of $0.079 \\pm 0.062$ across six cross-validation stories, above the random control and above the baseline on semantic quality. The same configuration retains an out-of-domain advantage when entire stories are held out, and both data and electrode scaling curves rise roughly linearly. The paper's interpretation is that the method captures broad semantic structure rather than memorizing training sentences, and that neither the adapter alone nor the corrector alone reproduces the full-model result.","pith_inferences":["Because the adapter is agnostic to recording modality, the same align-then-invert recipe could be retrained on scalp EEG, MEG, or fMRI features; a direct test would keep the corrector fixed and train only a new adapter on the new input features.","The fact that two fine-tuning epochs and one refinement step suffice suggests alignment quality, not corrector capacity, is the current bottleneck, so future gains may come from better alignment losses rather than larger language models.","The large variance across random electrode subsets implies that electrode placement matters more than raw channel count; an extension would learn to select or weight electrodes, which could improve decoding with fixed coverage."],"forward_implications":["With roughly 30 minutes of per-subject listening data, continuous semantic decoding from iEEG becomes practical in clinical recording sessions rather than requiring many hours of data.","The decoder is not limited to a fixed vocabulary or candidate set, so the generated text is open-vocabulary natural language.","Holding out entire stories still yields above-baseline semantic reconstruction, indicating that the learned mapping transfers to novel semantic content.","Reconstruction quality improves roughly linearly with both training data and electrode coverage, so denser cortical sampling and larger datasets should directly improve decoding."],"supporting_citations":[{"why":"Supplies the pre-trained text embedding space that the LSTM adapter aligns neural signals to.","marker":"[26]"},{"why":"Supplies the Vec2Text corrector, the pre-trained embedding-to-text inversion model fine-tuned in Phase 2.","marker":"[27]"},{"why":"Supplies the contrastive learning objective used in the alignment loss.","marker":"[28]"},{"why":"The continuous semantic decoding baseline that Neuro2Semantic is compared against.","marker":"[6]"},{"why":"Provides the electrode response selection criterion used to choose responsive iEEG electrodes.","marker":"[29]"},{"why":"Supplies the LSTM architecture used as the neural adapter.","marker":"[24]"},{"why":"Supplies BERTScore, the semantic similarity metric used to evaluate reconstruction quality.","marker":"[33]"},{"why":"Supplies BLEU, the surface n-gram metric used to evaluate reconstruction quality.","marker":"[32]"}],"fun_headline_variants":["30-minute iEEG decodes speech to text with transfer learning","Transfer learning turns brain signals into readable speech","Semantic decoding of speech from brief iEEG recordings","Two-step AI reconstructs language from 30 minutes of brain data","iEEG to text: minimal data, maximal semantic reconstruction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that after only two epochs of fine-tuning, the pretrained Vec2Text corrector can invert embeddings produced by the frozen LSTM adapter from held-out and out-of-domain stories, which requires those neural embeddings to fall close enough to the distribution of text embeddings the corrector was originally trained on.","fun_headline_variants_meta":{"raw":{"variants":["30-minute iEEG decodes speech to text with transfer learning","Transfer learning turns brain signals into readable speech","Semantic decoding of speech from brief iEEG recordings","Two-step AI reconstructs language from 30 minutes of brain data","iEEG to text: minimal data, maximal semantic reconstruction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000435,"raw_usage":{"total_tokens":2176,"prompt_tokens":872,"completion_tokens":1304,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":1221}},"tokens_in":488,"tokens_out":1304,"duration_ms":9990,"temperature":1.0,"reasoning_tokens":1221,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:06:13.203217+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the cosine similarity between adapter output embeddings and their target sentence embeddings on held-out stories. If those similarities are no higher than similarity to randomly chosen text embeddings, or if the fine-tuned corrector's single refinement step cannot move a hypothesis text closer to the target on out-of-domain sentences, the transfer claim fails and reconstruction scores should fall to near-random levels.","supporting_citations":[{"cited_title":"Natural speech reveals the semantic maps that tile human cerebral cortex,","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained text embedding space that the LSTM adapter aligns neural signals to."},{"cited_title":"Neural decoding of semantic concepts: A systematic literature review,","cited_arxiv_id":null,"evidence_quote":"Supplies the Vec2Text corrector, the pre-trained embedding-to-text inversion model fine-tuned in Phase 2."},{"cited_title":"Decoding imagined and spo- ken phrases from non-invasive neural (meg) signals,","cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive learning objective used in the alignment loss."},{"cited_title":"Semantic reconstruc- tion of continuous language from non-invasive brain recordings,","cited_arxiv_id":null,"evidence_quote":"The continuous semantic decoding baseline that Neuro2Semantic is compared against."},{"cited_title":"Machine translation of cortical activity to text with an encoder–decoder framework,","cited_arxiv_id":null,"evidence_quote":"Provides the electrode response selection criterion used to choose responsive iEEG electrodes."},{"cited_title":"Second, after aligning the neural embeddings, we fine-tune a pre-trained text reconstruction model [27] to extract coherent text from the neural-aligned embeddings","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM architecture used as the neural adapter."},{"cited_title":"Text Em- beddings Reveal (Almost) As Much As Text,","cited_arxiv_id":null,"evidence_quote":"Supplies BERTScore, the semantic similarity metric used to evaluate reconstruction quality."},{"cited_title":"Openai api: Embeddings,","cited_arxiv_id":null,"evidence_quote":"Supplies BLEU, the surface n-gram metric used to evaluate reconstruction quality."}],"review_version":1}