{"id":"1f11c83d-93de-46d0-b715-46fb0c481345","arxiv_id":"2506.00304","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A frozen LLM with a small EMG adaptor converts unvoiced EMG to text at 0.49 average word error rate on a 67-word closed vocabulary without any voiced audio.","lead":"This paper tests whether frozen large language models can turn unvoiced EMG muscle signals into text using only unvoiced data, with a trainable adaptor feeding EMG features into Llama. It reports a 0.49 word error rate on a 67-word closed vocabulary and says the approach beats specialized models when training data is limited to six minutes. The result is an early proof of concept for silent-speech interfaces that need no voiced audio.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 0.49 WER is not shown to be driven by EMG: no control removes or shuffles the EMG input while keeping the same frozen LLM, prompt, and beam search, so the reported gain could reflect the closed-vocabulary language prior rather than EMG-to-text conversion.","rationale":"The central claim is causal: an EMG adaptor maps unvoiced EMG features into an LLM's input space, and the resulting text output reflects unvoiced speech. The experimental protocol in Section 3.1 compares adaptor-based LLMs with an app-specific model on the same EMG inputs. That comparison can show relative performance, but it cannot separate the contribution of the EMG signal from the contribution of the LLM's priors. In a closed vocabulary of 67 words with short template-like utterances, the language-model prior alone is a nontrivial decoder: many four-word sequences are far more probable than others. Without a condition that destroys EMG information (shuffling utterances, zeroing embeddings, or replacing with noise), the reported 0.49 WER and the 20-26% gains do not establish EMG-to-text conversion.\n\nThe reader's weakest assumption concerned split integrity and selection on test data. That is a legitimate worry, but Appendix B.4's statement that selection used only validation loss and WER mitigates it, and the available text does not show leakage. The missing-input control is more load-bearing because it attacks the attribution of the measured WER directly. The adaptor ablations in Table 2 (e.g., LSTM 0.53 vs transformer 0.79) show the adaptor architecture changes outcomes, but they do not reveal what fraction of the performance comes from EMG content; a constant or shuffled input could still yield different WERs across architectures via different learned biases.\n\nCredit is due where the paper has independent support: it releases code and sample predictions, uses a public dataset, and describes architecture and training in enough detail to reproduce. The internal inconsistency between Table 1's 0.75 and Appendix Table 5's 0.70-0.72 for the same app-specific model should be corrected, but it does not by itself overturn the adaptor result; both baselines leave room for the claimed relative improvement. The limitation section is candid about closed vocabulary but does not mention the absence of an input-dependence control, which is the specific gap this review flags.\n\nThe reader's CONDITIONAL verdict remains appropriate. The condition I would attach is not only split integrity but a demonstrated input-dependence check: WER must degrade substantially when EMG information is removed or shuffled. If the controls pass, the main empirical claim is credible; if they fail, the paper should be reframed as a study of LLM priors in closed-vocabulary settings rather than EMG-to-text conversion. No change to the verdict level is needed; the condition should be specified more precisely.","tokens_in":11644,"tokens_out":6900,"duration_ms":65925,"concrete_test":"Re-run the best configuration (EMG-Ad + Llama3-3B, handcrafted features) with the identical frozen LLM, prompt, optimizer, and beam width under three control inputs: (1) EMG feature tensor shuffled along the utterance axis (breaking EMG-to-text correspondence while preserving input distribution), (2) EMG feature tensor replaced by Gaussian noise with matched mean/variance, (3) EMG feature tensor set to zeros. Also run the same controls on the 6-minute subsampled model used for Figure 2. Compute per-fold WER and the three-fold average. If any control achieves WER within 0.05 of 0.49 (or margins close the reported gains over the app-specific baseline), the headline claim is not supported; if controls degrade to WER >0.8, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 presents the headline result (Table 1: 0.49 WER) and Figure 2 (6-minute subsample) as evidence that a frozen LLM with a small adaptor 'understands unvoiced speech.' For that claim to hold, the output must be causally dependent on the EMG input. The paper never reports a control condition in which the EMG input is removed, zeroed, shuffled across utterances, or replaced by noise while the frozen LLM, adaptor, prompt ('Convert unvoiced EMG embeddings to text'), temperature, and beam width are held fixed. With only 67 words and roughly four words per utterance, the LLM's language-model prior alone can produce plausible closed-vocabulary sequences; without a no-EMG control, the 0.49 WER cannot be attributed to EMG-to-text conversion. The architecture ablations in Table 2 show WER varies with adaptor design (0.53 vs 0.79), which indicates the adaptor matters, but it does not establish that the EMG signal, rather than the adaptor's learned bias or prompt, is the source of the improvement. The central claim therefore rests on an unmeasured quantity: how much of the 0.49 WER survives when EMG information is destroyed. This is distinct from the data-split concern; even a perfectly leakage-free split would not resolve it. A secondary internal inconsistency (Table 1 baseline 0.75 vs Appendix Table 5 baseline 0.70–0.72) further complicates the '20%/26% improvement' claims but is not the primary issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a trainable \"EMG adaptor\" that maps unvoiced surface-EMG features into the input embedding space of a frozen LLM (LLaMA-2-7B or LLaMA-3.2-3B), optionally with LoRA fine-tuning, and evaluates the resulting EMG-to-text conversion on the single-speaker, 67-word closed-vocabulary Gaddy-Klein dataset. The authors report an average WER of 0.49 with handcrafted features, claim that the LLM-based approach outperforms a specialized transducer baseline by roughly 20-30%, and report a 26% gain in a six-minute low-data setting. Additional experiments ablate the adaptor backbone, compare audio versus EMG modality expansion, and include a person-identification pilot. The central claim is that a frozen LLM with a small trainable adaptor can convert unvoiced EMG to text without voiced EMG or audio.","tokens_in":1043,"tokens_out":1728,"duration_ms":75574,"significance":"If the central claim is validated, the paper would make a useful practical contribution to silent-speech interfaces and to the literature on extending LLMs to non-text modalities: the adaptor is lightweight (about 6M trainable parameters), the setting is genuinely unvoiced-only, and the three-fold evaluation on an established dataset is a reasonable starting point. The paper also states that code, data, and sample predictions will be released, which is valuable for reproducibility. However, two issues are load-bearing: the reported gains are not shown to be causally driven by the EMG input, because no input-destruction control is reported; and the baseline WER is internally inconsistent (0.75 in Table 1 versus 0.70-0.72 in Appendix Table 5), which directly affects the percentage-improvement claims. The recommendation is therefore major revision rather than acceptance.","major_comments":[{"comment":"The headline result (WER 0.49) is not shown to be driven by EMG information. The paper compares against a specialized baseline and ablates adaptor architectures, but no condition destroys the EMG input while holding the frozen LLM, adaptor, prompt, temperature, and beam width fixed. Because the task is a 67-word closed vocabulary with roughly four words per utterance, a strong language-model prior acting on the prompt alone could produce plausible transcriptions and account for part of the reported gain. I request control experiments in which the EMG input is zeroed, shuffled across utterances, replaced by noise, or omitted, with the same 8:1:1 split and three-fold averaging, reported alongside Table 1 and Figure 2. Without such controls, the abstract's claim that LLMs 'understand unvoiced speech' is not supported.","section":"Section 3.1, Table 1, Figure 2"},{"comment":"The baseline WER is internally inconsistent. Table 1 reports the App-Specific raw-EMG WER as 0.75 ± 0.06, while Appendix B.1 and Table 5 report the same original 54M model at 0.70 (beam 4) and 0.72 (beam 100). Section 3.1 derives a 30% improvement from the 0.75 number, while the abstract states 'nearly 20%' and the conclusion states 26% for the six-minute setting. Please reconcile these numbers, provide per-fold baseline WERs, specify which baseline and data size underlie each percentage claim, and recompute the abstract and conclusion percentages accordingly. If 0.70 is the correct baseline, the headline gains are materially smaller.","section":"Table 1 vs Appendix B.1/Table 5; Abstract"},{"comment":"The six-minute result is presented only as a figure and a summary percentage (26%), without the underlying WER values. It is therefore not possible to verify the improvement or to check whether the random subsample was drawn only from the training portion. Please provide a table with mean ± std WER for the App-Specific and LLM-based models at the full and subsampled training sizes, state the number of utterances in the six-minute condition, and specify the subsampling procedure relative to the 8:1:1 split (for example, whether validation and test sets were fixed). This is needed to support the abstract's six-minute claim.","section":"Figure 2 and Section 3.1 (data-efficient learning)"}],"minor_comments":[{"comment":"The temperature parameter τ is introduced without motivation; a brief sentence on why a value of 0.8 was chosen would improve reproducibility.","section":"Section 2"},{"comment":"Error bars are not shown in Figure 2; please add mean ± std across the three folds or state that the plotted values are mean WER.","section":"Figure 2"},{"comment":"The NICD reference lacks a full title; consider citing the specific NIH statistics page rather than the abbreviation-only entry.","section":"References"},{"comment":"Some references to the authors' prior work (e.g., Mohapatra et al., 2023c) are used to justify design choices; please clarify in each case whether the prior work used the same EMG setting or a different biosignal, since the relevance may not be obvious to readers.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper reports a useful empirical study but the missing control condition and the baseline inconsistency are serious enough to require a major revision. I would also ask the editor to verify that the released code and data correspond exactly to the experimental protocol described, since the paper relies on the three-fold split for the headline numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's a quick read on arXiv:2506.00304. The paper proposes a trainable EMG adaptor that maps unvoiced EMG features into the embedding space of a frozen LLM, and shows it can achieve a 0.49 word error rate on the Gaddy-Klein closed-vocabulary dataset (67 words, single speaker). The genuinely new element is that it works without any voiced EMG or audio, which is the scenario that matters for people who can't speak. The architecture is straightforward, the ablation over backbone variants is useful, and they release code and sample predictions. I also like the audio-vs-EMG comparison: it quantitatively confirms what people expect, that EMG is much harder than audio for this kind of adaptor.\n\nThe soft spots are real, though. First, the numbers don't fully line up: Table 1 reports the application-specific baseline at 0.75±0.06, while Appendix Table 5 gives the original 54M baseline at 0.70 with beam n=4. If 0.70 is correct, the claimed improvement percentages over that baseline shrink. The abstract says 'nearly 20%', the conclusion says '26%' - these need to be reconciled.\n\nSecond, and more important, there is no control condition where the EMG input is removed, shuffled, or replaced with noise while the frozen LLM, adaptor, prompt, temperature, and beam width are held fixed. With a 67-word closed vocabulary and an average of four words per utterance, the language prior alone can generate plausible command-like sequences. Without a no-EMG control, the 0.49 WER cannot be unambiguously attributed to EMG-to-text conversion. The adaptor ablations show that architecture matters, but that doesn't settle the causal role of the EMG signal. This is a fixable omission, but it's load-bearing for the 'understands unvoiced speech' framing.\n\nThe evaluation is also single-speaker and closed-vocabulary; the limitations section acknowledges this, but the title and abstract overreach.\n\nIn sum: this is a worthwhile first step, and the code and ablations make it useful for people working on silent speech interfaces. It deserves a serious referee, but the referee should ask for a no-EMG control and a reconciliation of the baseline numbers before the quantitative claims are accepted.","headline":"A first step toward silent-speech interfaces with LLMs, with a promising but not fully attributed result and a couple of numeric inconsistencies.","tokens_in":12566,"tokens_out":4194,"would_cite":false,"duration_ms":37800,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that a frozen LLM with a small trainable EMG adaptor can transcribe unvoiced electromyography signals into text without voiced EMG or audio, reaching a 0.49 word error rate on a 67-word closed vocabulary.","keywords":["silent speech interface","unvoiced EMG","EMG-to-text conversion","large language models","adaptor module","closed vocabulary","word error rate","data-efficient learning"],"falsifier":"Re-run the identical adaptor on a second speaker using only six minutes of that speaker's unvoiced EMG; if the WER rises from near 0.49 toward the level of random guessing on the closed vocabulary, the data-efficiency and generalization claims fail. As a second check, replace EMG inputs with shuffled or noise-matched versions while keeping the same text labels; if WER stays low, the model is exploiting dataset artifacts rather than articulatory content.","tokens_in":11411,"feed_emoji":"🧠","tokens_out":7574,"duration_ms":63887,"temperature":0.7,"pith_summary":"This paper asks whether large language models can understand unvoiced speech directly from surface electromyography, with no audio and no voiced EMG recordings. The authors propose a trainable EMG adaptor that converts EMG features into token embeddings for a frozen LLM, and report an average word error rate of 0.49 on a single-speaker, 67-word closed-vocabulary task. With only six minutes of training data, the LLM-based system beats a specialized EMG-to-text model by roughly 20 to 26 percent. The result matters because it suggests that people who cannot produce audible speech could interact with LLM-based assistants using short, personalized silent-speech calibrations.","feed_headline":"LLMs convert silent EMG signals to text at 0.49 WER","feed_subtitle":"Frozen LLM plus small adaptor reads unvoiced muscle signals alone and beats specialized models.","key_machinery":"The load-bearing mechanism is the EMG adaptor, a small trainable network that translates raw or handcrafted EMG time series into token embeddings in the LLM's input space. Concretely, the adaptor uses a stride-6 1D convolution to downsample the input $X_e\\in\\mathbb{R}^{T\\times C}$, two residual 1D-convolution blocks, a BiLSTM to capture temporal dependencies, a stride-2 convolution, and a linear projection to produce embeddings $E\\in\\mathbb{R}^{\\hat{T}\\times F}$ matched to the LLM's embedding dimension. These EMG tokens are wrapped with a text prefix ('Unvoiced EMG:') and a task prompt ('Convert unvoiced EMG embeddings to text') before entering the frozen LLM, which is trained with cross-entropy loss at temperature $\\tau=0.8$ and decoded with beam width 4. The adaptor's job is to feed the LLM features that preserve articulatory order while letting the LLM's pretrained language knowledge do the word prediction.","core_discovery":"The central claim is that a frozen large language model can accept unvoiced electromyography signals as a new input modality through a lightweight trainable adaptor, without needing paired voiced EMG or audio. On a single-speaker, 67-word closed-vocabulary dataset the authors report an average word error rate of 0.49 using handcrafted EMG features, and 0.52 using raw EMG, both with only about 6.4 million trainable parameters in the adaptor. The same setup trained on just six minutes of unvoiced EMG data outperforms a specialized EMG-to-text transducer model by roughly 20 to 26 percent. The authors additionally show that this EMG-to-text expansion is harder than expanding LLMs to audio, that handcrafted features help the LLM adaptor while raw features help the specialized model, and that unvoiced EMG carries speaker-identifying patterns (96 percent accuracy in a four-speaker pilot), which motivates personalized, data-efficient adaptation.","pith_inferences":["An extension the paper leaves implicit: the same frozen-LLM adaptor should be evaluated on a second speaker with a personalized six-minute calibration set; staying near 0.49 WER would make practical, personalized silent-speech LLM assistants plausible.","The 96 percent person-identification result implies EMG embeddings are biometric signatures, so real deployments would need privacy-preserving adaptation and consent safeguards.","Because a small BiLSTM outperformed transformer backbones inside the adaptor, the LLM itself appears to supply long-range language context; pairing the adaptor with a speech-pretrained encoder or a larger frozen LLM may close much of the remaining gap to audio-level performance.","The closed-vocabulary success suggests a stepping-stone: reformulating an open vocabulary as a larger closed set and using target-steering methods, as the paper notes, may be a plausible route to in-the-wild EMG-to-text."],"forward_implications":["With handcrafted temporal and spectral features, the frozen-LLM adaptor reaches a 0.49 average WER on the 67-word closed vocabulary, matching across Llama2-7B and Llama3-3B.","With only six minutes of unvoiced EMG training data, the LLM-based approach beats the specialized transducer baseline by about 26 percent on average, suggesting data-efficient personalized silent-speech interfaces are feasible.","Raw EMG input favors the specialized model (0.75 WER), while handcrafted features favor the LLM adaptor (0.49 WER), so input representation remains a key design choice.","LLMs learn from audio much more easily than from EMG with analogous simple adaptors, indicating that articulatory biosignals are a harder modality and need dedicated treatment.","Adding voiced EMG during training improves the specialized model by about 20 percent but gives little benefit to the LLM adaptor in this closed-vocabulary setting."],"supporting_citations":[{"why":"Supplies the single-speaker, 8-channel, 67-word closed-vocabulary dataset and the application-specific transducer baseline.","marker":"Gaddy and Klein (2021)"},{"why":"Source of the spectral and temporal handcrafted features and the preprocessing style used for EMG inputs.","marker":"Gaddy and Klein (2020)"},{"why":"Provides the embarrassingly simple linear-projector strategy that the paper adapts for connecting a new modality to a frozen LLM.","marker":"Ma et al. (2024)"},{"why":"Prior EMG-to-text work that applies LLMs only as post-processing, the contrast this paper builds on.","marker":"Benster et al. (2024)"},{"why":"Public multi-speaker EMG corpus used for the person-identification pilot analysis.","marker":"Diener et al. (2020b)"},{"why":"Evidence that surface-EMG signals carry person-specific phenotypes, motivating data-efficient personalization.","marker":"Diener et al. (2020a)"},{"why":"Provides the wav2vec 2.0 audio encoder used in the audio-versus-EMG comparison and the GeLU activation used in the adaptor.","marker":"Baevski et al. (2020)"},{"why":"LoRA low-rank fine-tuning baseline used to compare against the frozen-LLM adaptor.","marker":"Hu et al. (2022)"}],"fun_headline_variants":["LLM decodes unvoiced EMG to text at 0.49 WER","Frozen LLM reads silent muscle signals for text","EMG-to-text via LLM adaptor beats specialized models","Six minutes of EMG data improves LLM text conversion","LLMs expand to unvoiced EMG without audio pairing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an 8:1:1 training/validation/test split with three-fold validation on the single-speaker, 67-word closed-vocabulary dataset measures true EMG-to-text ability, with no test-fold leakage and no hyperparameters chosen using test information.","fun_headline_variants_meta":{"raw":{"variants":["LLM decodes unvoiced EMG to text at 0.49 WER","Frozen LLM reads silent muscle signals for text","EMG-to-text via LLM adaptor beats specialized models","Six minutes of EMG data improves LLM text conversion","LLMs expand to unvoiced EMG without audio pairing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1397,"prompt_tokens":962,"completion_tokens":435,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":346}},"tokens_in":578,"tokens_out":435,"duration_ms":4155,"temperature":1.0,"reasoning_tokens":346,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:07:24.541911+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the identical adaptor on a second speaker using only six minutes of that speaker's unvoiced EMG; if the WER rises from near 0.49 toward the level of random guessing on the closed vocabulary, the data-efficiency and generalization claims fail. As a second check, replace EMG inputs with shuffled or noise-matched versions while keeping the same text labels; if WER stays low, the model is exploiting dataset artifacts rather than articulatory content.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the single-speaker, 8-channel, 67-word closed-vocabulary dataset and the application-specific transducer baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the spectral and temporal handcrafted features and the preprocessing style used for EMG inputs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the wav2vec 2.0 audio encoder used in the audio-versus-EMG comparison and the GeLU activation used in the adaptor."}],"review_version":1}