{"id":"d76df086-d036-4ecc-8684-b3d902310c05","arxiv_id":"2505.15378","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A speaker-independent speech classifier using Wav2Vec2 features reaches 88.2% F1 for Parkinson's medication state detection, with prosody-rich continuous speech tasks such as PROS-SENT most informative.","lead":"This paper tests whether short speech recordings can tell whether a Parkinson's patient is in the medication ON state or the OFF state, without needing a personalized model per patient. Using self-supervised speech features, it reports 88.2% F1 on the best task, and finds that prosody-rich continuous speech tasks such as sentence reading are most informative.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ON/OFF labels are confounded with recording session and time of day; the 88.2% F1 may reflect session artifacts rather than medication state, given only 25/74 patients showed MDS-UPDRS-III changes.","rationale":"The reader's weakest assumption correctly identifies label reliability and confounds as the core threat. My stress-test agrees and finds this is the single most load-bearing issue: it directly threatens the validity of the reported 88.2% F1, which is the empirical basis for the paper's central claim of a first speaker-independent medication-state detector. I also note an additional supporting inconsistency in the paper's dysarthria analysis, which fails to find performance trends even among clinically changed patients. The proposed check—a responder/non-responder breakdown—would provide a decisive test because, if the model relies on medication-driven speech changes, it must perform better on responders; if it performs equally on non-responders, session artifacts are the likely explanation. This check does not require new data and can be run with the existing features and folds. The reader's CONDITIONAL verdict is appropriate; the authors should add this analysis before the claim is accepted. Therefore, I recommend no change to the verdict, keeping it CONDITIONAL until the control is provided.","tokens_in":10464,"tokens_out":4894,"duration_ms":48640,"concrete_test":"Re-analyze the FraLusoPark data to report per-subgroup performance: split the 74 patients into responders (25 with MDS-UPDRS-III improvement after medication) and non-responders (49 with no change or worsening). For the PROS-SENT task and the best Wav2Vec2+SVM configuration, compute the F1-score separately for each subgroup using the existing speaker-independent cross-validation folds. Also compute the correlation between per-patient classification accuracy and the change in the MDS-UPDRS-III speech item, if available. If non-responders perform at chance (≈50% F1) and responders significantly above, the medication-state interpretation is supported; if non-responders are also above 80% F1, the result is likely driven by session or order artifacts, undermining the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—a speaker-independent system detecting PD medication state from speech with 88.2% F1—rests on the assumption that the ON/OFF labels represent a true medication-driven change in speech. Section 2 shows this assumption is weak: recordings were made in two separate sessions (OFF after ≥12h drug withdrawal, ON after ≥1h post-medication), and only 25/74 patients had any MDS-UPDRS-III change (2 worsened). For the remaining 49, the clinical labels may not correspond to any measurable physiological change, yet the model reports high accuracy. Moreover, the paper's own 'Dysarthria Level Analysis' reports no correlation between model performance and severity changes, even for subjects whose scores changed—an internal inconsistency with the medication-state interpretation. Because the two recording sessions differ systematically in time of day, fatigue, vocal warm-up, and session order, the classifier could learn these artifacts instead of medication effects. The experimental design has no control condition to rule out this confound, so the 88.2% F1 cannot be attributed to medication state without further evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a speaker-independent system for classifying Parkinson's disease patients' medication state (ON/OFF) from speech. It compares knowledge-based eGeMAPS features against self-supervised Wav2Vec2/XLS-R embeddings, and linear SVMs against attention-based deep networks, across nine clinical speech tasks and three training-data grouping strategies on the FraLusoPark corpus. The reported best result is 88.2% F1 for the PROS-SENT task with Wav2Vec2 features and a linear SVM under the task-specific strategy. The authors conclude that continuous, prosody-rich speech tasks and self-supervised representations are the most effective, and claim this is the first speaker-independent approach to PD medication-state identification from speech.","tokens_in":10771,"tokens_out":4847,"duration_ms":47510,"significance":"If the reported result is attributable to medication state rather than to recording-session artifacts, the paper provides the first speaker-independent PD medication-state classifier from speech and a useful practical step toward remote monitoring. The experimental design is generally sound: speaker-independent nested cross-validation with repeated seeds, gender-balanced folds, multiple tasks and feature sets, and an explicit limitations section. The strong performance of SSL features on continuous speech tasks is plausible and partly supported by prior literature. However, two load-bearing issues—the confounding of ON/OFF labels with recording session and the absence of any multiple-comparison control when selecting the headline configuration—mean the central claim is not yet established at the level the paper states.","major_comments":[{"comment":"The central attribution of the 88.2% F1 score to medication state is not adequately supported because the ON/OFF labels are systematically confounded with the recording protocol. Patients were recorded OFF after at least 12 hours of drug withdrawal and ON at least 1 hour after medication, i.e., in two separate sessions; the text does not describe counterbalancing of session order or a no-medication control condition. The classifier could therefore learn session, time-of-day, fatigue, or vocal-warm-up cues rather than a medication signature. This concern is reinforced by the paper's own observation that only 25 of 74 participants had MDS-UPDRS-III changes after medication (two worsened) and that the dysarthria-level analysis found no correlation with severity changes even for those subjects. The authors should provide a control analysis, for example by evaluating the best model on the subset of patients with documented motor changes, by training on repeated recordings under unchanged medication status, or by demonstrating that the learned decision function is driven by medication-related acoustic changes rather than session artifacts.","section":"Section 2; Section 4, Dysarthria Level Analysis"},{"comment":"The headline result is selected as the maximum over a large grid of configurations: 9 tasks × 3 grouping strategies × 2 feature types × 2 architectures, with the hyperparameters of the SVM also tuned in the inner loop. No multiple-comparison correction, hold-out model-selection procedure, or external validation is applied to this best cell, so the 88.2% F1 may partially reflect selection bias. The reported standard deviations quantify fold and seed variability within one cell but not the variability of the maximum over the whole grid. The authors should either report the distribution of results across all cells with an appropriate correction or permutation test, or validate the single best configuration on a set of speakers never used in any model selection.","section":"Section 4, Table 2"},{"comment":"The claim of 'robustness across genders' is made without any statistical comparison of the male, female, and gender-independent results in Table 3. For the SVM, the differences among 88.2%, 88.9%, and 87.6% are within the reported standard deviations. The authors should state whether these differences are statistically meaningful and, if not, soften the robustness claim accordingly. This is not the central claim of the paper, but it is part of the evidence for the generality of the method.","section":"Section 4, Gender-based Analysis"}],"minor_comments":[{"comment":"The text refers to 'Wav2Vec2.0' but the cited model is XLS-R (reference [19]); please align the terminology used in the text with the actual pretrained model.","section":"Section 3.1"},{"comment":"The dysarthria-level analysis is described only qualitatively; please provide the definition of the severity strata, the number of subjects in each stratum, and the actual performance numbers or statistics supporting the claimed absence of correlation.","section":"Section 4, Dysarthria Level Analysis"},{"comment":"The Limitations paragraph should explicitly acknowledge the session-confounding risk and the small number of patients with measured motor change; the current wording mentions data scarcity but not label validity, which is the more serious threat to interpretation.","section":"Limitations"},{"comment":"The phrase 'the no incorporation of demographic information' is grammatically awkward; it should read 'the failure to incorporate demographic information'.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an important and timely clinical question, and the implementation appears careful on the machine-learning side. My main reservation is not the technical pipeline but the construct validity of the labels: without a control for the session confound, the headline 88.2% F1 may not reflect medication state at all. I would be willing to accept after the authors provide a credible control analysis or substantially soften the causal language. I also suggest the editor ask for the best-cell selection issue to be addressed, as the current reporting invites over-interpretation of a maximum over many configurations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent, clearly-written benchmark that makes a modest but real claim to novelty — first speaker-independent PD medication-state detection from speech — and the systematic comparison across nine clinical tasks is useful. But I would not take the 88.2% F1 at face value as evidence of medication detection; the label reliability and session confounds are serious enough to require re-analysis or strong argument before the headline is accepted.\n\nWhat's good: the evaluation is carefully set up — speaker-independent nested CV, gender-balanced folds, repeated runs, three sensible data-grouping strategies. The finding that continuous, prosody-rich tasks (PROS-SENT, TEXT, FROG) consistently outperform vowels and didadochokinesia is interesting and aligns with prior work. Showing that SSL embeddings beat eGeMAPS without fine-tuning is a useful data point for the community. The comparison with Pompili's speaker-dependent work is honest.\n\nThe soft spots: (1) The ON/OFF labels are clinical protocol labels, not verified speech changes. The paper itself states only 25/74 patients had MDS-UPDRS-III changes after medication, and two worsened. If most patients' motor state doesn't change, what is the model separating? The simplest non-medication explanation is recording-session differences: OFF is recorded after 12h withdrawal, ON after medication, almost certainly in a fixed order. That confounds time of day, fatigue, vocal warm-up, and session acoustics. The paper does not control for this or even discuss it. (2) The 'Dysarthria Level Analysis' section says there was no correlation with severity changes even for subjects whose scores changed. That is an internal tension with the medication-state interpretation, not just a null result. (3) The headline result is selected from hundreds of feature/task/architecture/grouping combinations with no multiple-comparison correction or significance tests. The trend across continuous tasks softens this, but the 88.2% itself is not well supported. (4) No code or data release, which limits reproducibility.\n\nThese are not fatal to the paper's value as a benchmark, but they are fatal to the strong claim that the method detects medication state. The authors need to at least address the session confound (e.g., by analyzing a subset with verified motor response, or by comparing feature distributions across sessions), and temper the language from 'detection' to 'discrimination under the recorded protocol.' As is, I'd send it to peer review with the expectation of major revision.\n\nWho it's for: people building speech-based PD monitoring systems, and reviewers who care about benchmark hygiene. I'd cite the task comparison but not the headline number.","headline":"Useful benchmark with a real novelty claim, but the 88.2% F1 is not well supported as evidence of medication-state detection until the label-validity and session-confound problems are addressed.","tokens_in":11229,"tokens_out":3053,"would_cite":true,"duration_ms":27841,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single, speaker-independent speech model can tell whether a Parkinson's patient is ON or OFF medication, hitting 88.2% F1.","keywords":["Parkinson's disease","medication state detection","speech biomarkers","self-supervised speech representations","Wav2Vec2","prosody","speaker-independent classification","FraLusoPark corpus"],"falsifier":"Record the same patients twice in the same medication state under an identical protocol and retrain the best model to distinguish the two sessions; if the F1 stays near 88%, the model is separating sessions rather than medication states, and the central claim would be refuted. A cheaper check is label permutation within each patient (shuffling ON/OFF while keeping speaker partitions), which should destroy performance if the signal is medication-related.","tokens_in":10257,"feed_emoji":"🗣️","tokens_out":4640,"duration_ms":35600,"temperature":0.7,"pith_summary":"The paper presents the first speaker-independent system that identifies whether a Parkinson's disease patient is in the ON or OFF medication state from voice recordings alone. Using the FraLusoPark corpus of 74 patients recorded both off and on medication, the authors compare hand-crafted acoustic descriptors (eGeMAPS) with self-supervised Wav2Vec2 embeddings and two classifiers, a linear SVM and an attention-based deep network. The best configuration — Wav2Vec2 features with a linear SVM on a sentence-reading task with prosodic contrasts — reaches an F1-score of 88.2%. The results indicate that continuous, prosody-rich speech tasks carry the most medication-state information, and that a single generic model can replace per-patient personalization.","feed_headline":"Speech alone reveals Parkinson's medication ON/OFF state, 88.2% F1","feed_subtitle":"One fixed model with no per-patient training works best on prosodic sentence reading in the FraLusoPark corpus.","key_machinery":"The central objects are two speech representations: eGeMAPS, a compact set of 88 knowledge-based acoustic functionals extracted with openSMILE, and Wav2Vec2.0, a self-supervised model whose 1024-dimensional embeddings are taken from the 7th encoder layer of the multilingual XLS-R variant and aggregated at both frame and utterance levels. These feed either a linear SVM with PCA dimensionality reduction or an attention-based deep network, under three data grouping strategies (task-specific, task-grouping, task-independent) evaluated with nested five-fold cross-validation. The decisive mechanism is the interaction between representation and task: Wav2Vec2 embeddings trained on large multilingual speech corpora capture prosodic variability that eGeMAPS descriptors miss, and that prosodic information is most available in continuous, prosodically demanding speech tasks.","core_discovery":"The central claim is that the ON/OFF medication state of Parkinson's patients can be identified automatically from speech in a speaker-independent manner, without fine-tuning per patient. The discovery that carries this claim is that self-supervised speech representations (Wav2Vec2.0 embeddings) substantially outperform knowledge-based acoustic descriptors (eGeMAPS) in this setting, and that the choice of speech assessment task is decisive: tasks involving continuous, prosody-rich speech — particularly reading sentences with prosodic properties (PROS-SENT) — yield the best accuracy, with the top result of 88.2% F1 achieved by a linear SVM trained on Wav2Vec2 features. Traditional machine learning classifiers are shown to be competitive with attention-based deep networks, which lowers the computational bar for deployment. The authors also report that gender-specific models do not improve over a gender-independent one, suggesting the underlying vocal cues are shared across genders.","pith_inferences":["If the learned signal is genuinely about medication state rather than recording conditions, the approach could enable cheap, passive monitoring of treatment response from brief voice captures in home environments.","The observation that only one-third of patients had changed MDS-UPDRS-III motor scores while the classifier still separated ON from OFF suggests speech may reflect a medication effect not well captured by the clinical motor scale — or that the classifier exploits session-level acoustic confounds; a controlled recording protocol with repeated same-state sessions could distinguish these.","The layer-7 embedding choice hints that intermediate self-supervised layers encode prosodic structure; targeted probing against prosodic annotations could reveal which acoustic dimensions carry the ON/OFF signal and make the black-box model more interpretable.","Combining speaker-independent pre-training with light speaker-dependent adaptation (e.g., a few labeled samples from the target patient) could close the remaining gap toward the 95% accuracy reported in the speaker-dependent literature."],"forward_implications":["A single deployable model can classify a new patient's medication state without any per-patient training data.","Clinical protocols should favor prosodically rich, continuous speech tasks (such as reading prosodic sentences) when speech is used to monitor medication response.","Self-supervised speech features can serve as effective medication-state biomarkers even without fine-tuning on pathological speech.","Linear SVMs offer near-state-of-the-art performance at much lower computational cost, making remote monitoring more feasible.","The reported 88.2% F1 is specific to this corpus; cross-linguistic and cross-dataset validation is the immediate next test."],"supporting_citations":[{"why":"The prior speaker-dependent medication-state system that this work extends and compares against.","marker":"[17]"},{"why":"Introduces the FraLusoPark corpus and its ON/OFF medication-state recording protocol.","marker":"[20]"},{"why":"Supplies the XLS-R Wav2Vec2 self-supervised speech representation used as features.","marker":"[19]"},{"why":"Defines the eGeMAPS knowledge-based acoustic feature set used as the baseline representation.","marker":"[18]"},{"why":"Prior evidence linking (dys)prosody and medication state, motivating the prosody hypothesis.","marker":"[21]"},{"why":"Provides the openSMILE toolkit used to extract eGeMAPS descriptors.","marker":"[26]"},{"why":"The MDS-UPDRS-III scale used to clinically assess medication effects, providing the task's reference labels.","marker":"[22]"},{"why":"Prior comparison of interpretable features versus DNN embeddings in Parkinson's speech, informing the methodology.","marker":"[16]"}],"fun_headline_variants":["Speech AI nails Parkinson's ON/OFF state without per-patient training","Wav2Vec2 beats handcrafted speech features for PD medication state","Prosody-rich speech unlocks Parkinson's drug-state detection","Speaker-independent model hits 88.2% F1 on Parkinson's ON/OFF from speech"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system assumes the clinical ON/OFF labels are reliable ground truth, even though only 25 of 74 patients showed a change in MDS-UPDRS-III after medication and two worsened; if the labels do not correspond to a stable acoustic signature, the classifier may be learning recording-session or time-of-day artifacts instead.","fun_headline_variants_meta":{"raw":{"variants":["Speech AI nails Parkinson's ON/OFF state without per-patient training","Wav2Vec2 beats handcrafted speech features for PD medication state","Prosody-rich speech unlocks Parkinson's drug-state detection","Speaker-independent model hits 88.2% F1 on Parkinson's ON/OFF from speech"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001504,"raw_usage":{"total_tokens":5990,"prompt_tokens":858,"completion_tokens":5132,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":5052}},"tokens_in":474,"tokens_out":5132,"duration_ms":29450,"temperature":1.0,"reasoning_tokens":5052,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:17:47.681530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record the same patients twice in the same medication state under an identical protocol and retrain the best model to distinguish the two sessions; if the F1 stays near 88%, the model is separating sessions rather than medication states, and the central claim would be refuted. A cheaper check is label permutation within each patient (shuffling ON/OFF while keeping speaker partitions), which should destroy performance if the signal is medication-related.","supporting_citations":[{"cited_title":"Automatic Detection of Parkinson’s Disease: An Experimental Analysis of Common Speech Production Tasks Used for Diagnosis,","cited_arxiv_id":null,"evidence_quote":"The prior speaker-dependent medication-state system that this work extends and compares against."},{"cited_title":"On the Use of a Foundation Acoustic Model to Identify Highly Relevant Phonetic Information of Parkinson’s Speech,","cited_arxiv_id":null,"evidence_quote":"Introduces the FraLusoPark corpus and its ON/OFF medication-state recording protocol."},{"cited_title":"Exploiting Foundation Models and Speech Enhancement for Parkinson’s Disease Detection from Speech in Real-World Operative Conditions,","cited_arxiv_id":null,"evidence_quote":"Supplies the XLS-R Wav2Vec2 self-supervised speech representation used as features."},{"cited_title":"Multi-Class Detection of Pathological Speech with Latent Features: How does it Perform on Unseen Data?","cited_arxiv_id":null,"evidence_quote":"Defines the eGeMAPS knowledge-based acoustic feature set used as the baseline representation."},{"cited_title":"Automatic Classification of Parkin- son’s Disease Using Wav2V ec Embeddings at Phoneme, Syllable, and Word Levels,","cited_arxiv_id":null,"evidence_quote":"Prior evidence linking (dys)prosody and medication state, motivating the prosody hypothesis."},{"cited_title":"Dysarthria in Individuals with Parkinson’s Disease: A Protocol for a Binational, Cross-Sectional, Cas e- Controlled Study in French and European Portuguese (FraLu- soPark),","cited_arxiv_id":null,"evidence_quote":"Provides the openSMILE toolkit used to extract eGeMAPS descriptors."},{"cited_title":"Interpretable speech features vs. DNN embed- dings: What to use in the automatic assessment of Parkinson’ s disease in multi-lingual scenarios,","cited_arxiv_id":null,"evidence_quote":"The MDS-UPDRS-III scale used to clinically assess medication effects, providing the task's reference labels."},{"cited_title":"Characterization Methods for the Detection of Multiple V oice Disorders: Neurological, Functional, and Laryngeal Diseases,","cited_arxiv_id":null,"evidence_quote":"Prior comparison of interpretable features versus DNN embeddings in Parkinson's speech, informing the methodology."}],"review_version":1}