{"id":"0f6004e5-cd16-4ba8-93fe-4a75872847b0","arxiv_id":"2504.17739","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A CNN plus Grad-CAM classifies Parkinson’s from Italian speech with 99% reported accuracy, but the evaluation may leak speaker identity and the phonetic interpretations are not statistically tested.","lead":"This paper trains a small convolutional network on word-level speech segments to tell Parkinson’s patients from healthy speakers, and uses Grad-CAM to highlight which words drive the prediction. The authors report near-perfect accuracy on an Italian database of 831 recordings, but the study uses only 65 people and does not state whether the same speaker’s recordings were kept out of the training set.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Participant-level leakage likely explains the 99% accuracy: Section 3 reports a repeated stratified holdout over 831 recordings from 65 speakers but never states whether splits are speaker-disjoint, so the temporal CNN may be recognizing speakers rather than Parkinsonian speech.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing flaw: the paper never states that the repeated stratified holdout splits are participant-disjoint. With 831 recordings from only 65 speakers, recording-level splits would place the same speaker in both training and test sets, allowing the network to exploit speaker identity. This directly threatens the headline result of 99.14% accuracy and 100.0% precision, and it also undermines the claimed superiority over baselines, since all models would be evaluated under the same leakage. The paper provides no code, no training hyperparameters, and no explicit split description, so the concern cannot be dismissed from the manuscript alone. The interpretability analysis is also weakened by the absence of control comparisons, and the 'early detection' claim is not supported by disease-stage data, but these are secondary to the leakage issue. Given that the central empirical claim is not supported as stated, the REJECT verdict is appropriate. The concrete test described above would determine whether the concern actually lands; if participant-disjoint splits were already used and reported in a revision, the verdict could be revisited.","tokens_in":3946,"tokens_out":1995,"duration_ms":22799,"concrete_test":"Re-run the classification experiment using participant-disjoint splits, e.g., leave-one-speaker-out or repeated stratified k-fold at the participant level, and report per-fold accuracy, precision, recall, and F1-score for the CNN and all baselines. Also report the number of speakers shared between train and test in the original nine holdout iterations. If the CNN's extended-speech accuracy drops materially (for example, from ~99% toward the 70–85% range typical of speaker-independent PD speech tasks) or becomes comparable to the baselines, the original claim is an artifact of speaker leakage. This single check would settle whether the central claim survives.","verdict_should_be":"REJECT","load_bearing_attack":"Section 3 states that Table 1 was obtained through a 'rigorous repeated stratified holdout procedure conducted across nine iterations,' but it never specifies whether stratification and splitting were performed at the participant level or at the recording level. This distinction is decisive because the dataset contains 831 audio files from only 65 speakers, i.e., roughly 12.8 recordings per speaker. If recordings from the same speaker appear on both sides of a split, a temporal CNN can learn speaker-specific vocal characteristics and achieve near-perfect accuracy by speaker identification rather than disease detection. The reported 99.14±1.60 accuracy and 100.0±0.0 precision on extended speech signals would then be an artifact of speaker leakage, not evidence of generalizable Parkinson's detection. The classical baselines may also profit from the same leakage, so the comparative claim that 'Ours' is statistically superior (p<0.001) is not trustworthy either. A secondary but independent concern is that the paper's 'early detection' framing is unsupported: no disease-stage information is used and the dataset is not described as containing early-stage patients. The leakage issue is the most load-bearing because it directly invalidates the central empirical claim if it holds, and the manuscript provides no information that would rule it out.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a 1D convolutional neural network (CNN) for classifying Parkinson's disease from speech, using an adaptation of Grad-CAM to highlight word-level speech segments that drive predictions. The method is evaluated on the Italian Parkinson's Voice and Speech Database (831 recordings from 65 speakers) and is reported to achieve 99.14±1.60% accuracy with 100% precision on extended speech signals, outperforming KNN, SVM, RF, and GB baselines. The authors also analyze the phoneme content of 'informative' segments and interpret the results as reflecting articulatory impairment in PD. The central claims are the high classification performance, the statistical superiority over baselines, and the interpretability of the model's decisions.","tokens_in":4234,"tokens_out":4759,"duration_ms":46078,"significance":"If the reported performance is obtained under a participant-disjoint evaluation, the work would be a useful contribution to interpretable PD detection from speech, leveraging a public dataset and a simple, reproducible architecture. The paper's main strengths are its explicit use of a public dataset and its falsifiable headline result (99% accuracy), which would allow direct verification by other researchers. However, the current manuscript does not provide sufficient experimental detail to establish that the result is speaker-independent, and the interpretability analysis lacks the controls needed to support the phonetic conclusions. The significance of the claimed contribution therefore hinges on details that are currently missing or unverified.","major_comments":[{"comment":"The repeated stratified holdout procedure is never described as participant-disjoint. The dataset contains 831 audio files from 65 speakers, so each speaker contributes roughly 12.8 recordings; if recordings from the same speaker appear in both training and test folds, the CNN can exploit speaker identity, making the reported 99.14±1.60 accuracy and 100.0±0.0 precision a measure of speaker recognition rather than Parkinson's detection. The authors must state explicitly whether all recordings per speaker were confined to a single split, and if not, re-run the experiments with speaker-level cross-validation.","section":"Section 3, Table 1"},{"comment":"The features used by the baseline classifiers (KNN, SVM, RF, GB) are never specified. Without knowing whether the baselines use the same word-chunk inputs, hand-crafted acoustic features, or something else, the comparative claim of 'statistically significant improvements' is not interpretable. The authors should describe the exact feature representation for each baseline and, ideally, also compare against a CNN with the same segmentation but without the interpretability components.","section":"Section 3, Table 1"},{"comment":"The claim that specific phonemes such as front vowels /i,e,y/ and plosive alveolar /t,d/ are 'most indicative' of PD is based on the 90th-percentile threshold applied to segment activations, but no comparison is made to the phoneme distribution in healthy speech or to a random-selection baseline. Since the recordings are readings of a phonemically balanced text, the frequency of these words (Cocco, Ciccio, Luna, etc.) may simply reflect the text content. A proper control would compare informative segments against the text's phoneme distribution or against segments from healthy speakers.","section":"Section 3, Table 2"},{"comment":"The paper frames the method as 'early detection' of Parkinson's disease, but the dataset description includes no disease-stage information and no evidence that the included patients are in early stages. As such, the claims should be limited to PD detection from speech, or the authors should provide stage information to support the early-detection framing.","section":"Abstract and Section 3 (Conclusion)"},{"comment":"The statistical significance claim (p<0.001 for accuracy, recall, F1; p=0.061 for precision) lacks a description of the test procedure: which test was used, whether comparisons are paired across the nine holdout iterations, and whether multiple-comparison corrections were applied. The reported 100.0±0.0 precision for the proposed model is also unusually exact; the authors should report the per-iteration test set sizes and the number of test samples underlying these metrics.","section":"Section 3, Results"}],"minor_comments":[{"comment":"The abstract states that the approach is compared against 'state-of-the-art methods,' but the experimental comparisons are against classical ML baselines; I recommend rephrasing to 'classical machine learning baselines' unless a broader comparison is added.","section":"Abstract"},{"comment":"There is a stray comma after the displayed equation for Lc(t); the sentence continues with 'This weighted sum,' so the punctuation should be adjusted for consistency.","section":"Section 2, Equation (2)"},{"comment":"The heatmap panels lack a color scale, axis labels, and a quantitative legend, making it difficult for the reader to interpret the claimed differences between healthy and PD speech patterns.","section":"Figure 1"},{"comment":"Reference [8] contains a typo: 'Hmeypotheses' should be 'Hypotheses'.","section":"References"},{"comment":"The frequency values in Table 2 (e.g., 12, 11, 11, 10, 10, 9, 8, 8, 8, 7) are presented without the total number of informative segments or the baseline word frequency in the reading text, so the 'frequency' measure is not self-contained.","section":"Section 3, Discussion of Table 2"}],"recommendation":"major_revision","confidential_remarks":"The missing participant-level split description is the central concern. If the authors confirm that splits are speaker-disjoint, the empirical claim would be credible; if not, the paper should be rejected. The phonetic interpretation in Table 2 needs a proper control analysis to avoid being purely story telling about the reading text. The authors should also be asked to specify the baseline feature sets and the statistical test used for the p-values."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: the central empirical claim likely does not survive contact with the split protocol. Section 3 reports a \"repeated stratified holdout procedure\" over 831 recordings from 65 speakers but never states whether the splits are at the participant level or the recording level. That distinction is decisive. At roughly 12.8 recordings per speaker, a recording-level split lets a temporal CNN memorize speaker identity and hit near-perfect accuracy by speaker recognition. The paper gives no information that rules this out, and the 99.14% accuracy and 100% precision in Table 1 look exactly like what such leakage would produce. The classical baselines would also benefit, so the p<0.001 superiority claim is not trustworthy either.\n\nWhat is actually new and worth credit: the hybrid word-boundary/amplitude segmentation method for turning long recordings into word chunks is a sensible engineering contribution, and the 1D adaptation of Grad-CAM to attribute predictions to specific phonetic segments is reasonable. The evaluation on the Italian Parkinson's Voice and Speech Database is a specific configuration not reported in the cited references. Table 2's word list is suggestive and could be a useful starting point for clinical hypotheses.\n\nThe soft spots beyond the split: the \"early detection\" framing is overreaching because the dataset contains established patients and no stage information. The phonetic interpretation in Table 2 is purely observational and has no healthy-control comparison, so the \"higher prevalence\" of certain vowels and consonants may simply reflect the common reading text. There is no code, no training details beyond the architecture, and no description of the statistical test that produced the p-values. These are real but secondary. If the authors return with participant-level splits and the accuracy drops to a more modest but honest range, the paper could still be a useful case study in interpretable speech analysis for PD.\n\nWho this is for: a reader working on interpretable audio biomarkers who wants a data point on what can go wrong with speaker leakage, and a referee who can demand a simple but essential methodological fix.\n\nRecommendation: send to peer review rather than desk reject. The flaw is potentially fatal but straightforwardly checkable, and the interpretability angle deserves a careful look. The current version should not be accepted as-is.","headline":"The reported 99% accuracy is not credible until the authors confirm the nine holdout splits are speaker-disjoint, because with 12.8 recordings per speaker a recording-level split would let the CNN recognize speakers, not Parkinson's disease.","tokens_in":4744,"tokens_out":1656,"would_cite":false,"duration_ms":18162,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a temporal convolutional network can detect Parkinson's disease from segmented speech with 99.14% accuracy while using Grad-CAM heatmaps to identify the phonetic segments driving each prediction.","keywords":["Parkinson's disease","speech analysis","temporal convolutional network","Grad-CAM","interpretability","voice biomarkers","deep learning diagnosis","phonetic features"],"falsifier":"Re-evaluate the model using participant-level stratified splits so no speaker appears in both training and test sets; if accuracy falls toward the 85–97% baseline range or below, the 99% result is at least partly speaker recognition.","tokens_in":3746,"feed_emoji":"🗣️","tokens_out":5360,"duration_ms":52047,"temperature":0.7,"pith_summary":"Parkinson's disease affects speech early, so voice recordings are a plausible screening signal. The paper tries to show that a temporal convolutional network, fed word-level chunks of read speech, can classify whether a speaker has Parkinson's, and that adapting Grad-CAM to one-dimensional audio can show which vocal segments drive each prediction. On the Italian Parkinson's Voice and Speech Database (831 recordings, 65 speakers), the model reaches 99.14±1.60% accuracy and 100.0% precision on extended speech, beating KNN, SVM, random forest, and gradient boosting on accuracy, recall, and F1 with p<0.001. The interpretability maps point to front vowels (/i/, /e/, /y/), alveolar plosives (/t/, /d/), and fricatives (/s/, /z/) as the most informative articulations.","feed_headline":"Speech CNN detects Parkinson's at 99% accuracy and shows why","feed_subtitle":"Word-level audio segments drive the decision, and front vowels and alveolar consonants are the telltale signs.","key_machinery":"The load-bearing mechanism is a 1D temporal CNN with two convolutional blocks (48 then 96 channels, kernel size 3) that processes fixed word chunks cut from longer recordings, followed by a fully connected classifier. The interpretability side is Grad-CAM adapted from images to one-dimensional temporal signals: the gradient of the class score with respect to the final convolutional feature maps gives per-segment importance weights, producing an activation map $L_c(t)$ over time. This map is what lets the authors name specific words and phonetic units as the drivers of prediction.","core_discovery":"The paper's central claim is that a simple temporal CNN, rather than hand-crafted feature extraction, can learn disease-related speech dynamics and give interpretable evidence at the same time. The authors report that their model consistently outperforms KNN, SVM, random forest, and gradient boosting baselines on extended speech signals, with statistically significant gains in accuracy, recall, and F1-score (p<0.001) and marginal gains in precision (p=0.061). They also show that the Grad-CAM heatmaps, thresholded at the 90th percentile, isolate words whose articulatory demands are hardest for Parkinson's patients; the resulting phonetic pattern—front vowels, alveolar plosives, and fricatives—is presented as a candidate signature of the underlying neuromuscular impairment.","pith_inferences":["If the evaluation splits were truly participant-level, a practical consequence follows: a few minutes of read speech could be enough for an automated screening test, with no need for engineered voice features.","The phonetic emphasis on alveolar consonants and front vowels suggests a testable hypothesis that tongue-tip rigidity, not just vocal fold stiffness, drives the most informative segments; articulography or surface EMG could verify this.","The same 1D Grad-CAM recipe could be applied to other time-series health signals—heartbeats, gait, breathing—to generate segment-level explanations for their respective diagnoses."],"forward_implications":["On this dataset, the temporal CNN can serve as a speech-only screening classifier with accuracy above 99% on extended read speech.","Word-level segmentation plus one-dimensional Grad-CAM turns a black-box network into a per-word explanation, so clinicians can see which articulations prompted the diagnosis.","The identified phonetic markers (front vowels, /t/ and /d/, /s/ and /z/) align with the articulatory effort involved in tongue-tip and lip coordination, linking the model's choices to known Parkinsonian motor symptoms.","Because the model is trained on temporal dynamics rather than static features, it could in principle track how speech deterioration evolves across repeated recordings.","The same architecture and explanation step should transfer to other languages, pending phonetic mapping."],"supporting_citations":[{"why":"Supplies the dataset: the Italian Parkinson's Voice and Speech Database with 831 audio files from 65 participants.","marker":"[2]"},{"why":"Provides the Grad-CAM method that the paper adapts from images to one-dimensional speech signals.","marker":"[7]"},{"why":"Shows a prior adaptation of Grad-CAM to one-dimensional temporal signals (ECG), which this paper follows for audio.","marker":"[4]"},{"why":"Represents the traditional hand-crafted-feature baseline (KNN, SVM, RF, logistic regression) that the deep model is compared against.","marker":"[3]"},{"why":"Surveys deep and machine learning approaches for early Parkinson's detection, locating the gap the paper fills.","marker":"[9]"},{"why":"Exemplifies feature-driven machine learning approaches whose static features the paper argues miss temporal dependencies.","marker":"[6]"}],"fun_headline_variants":["Speech CNN detects Parkinson's, reveals key phonemes","Interpretable CNN spots Parkinson's speech signs early","Deep learning uncovers Parkinson's vocal markers","Parkinson's detection via CNN with explainable speech clues","CNN model flags Parkinson's, highlights front vowel patterns"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported splits may have been drawn per recording rather than per participant, so the same speaker could appear in both training and test sets; if so, the 99% accuracy could reflect speaker identity rather than Parkinson's detection.","fun_headline_variants_meta":{"raw":{"variants":["Speech CNN detects Parkinson's, reveals key phonemes","Interpretable CNN spots Parkinson's speech signs early","Deep learning uncovers Parkinson's vocal markers","Parkinson's detection via CNN with explainable speech clues","CNN model flags Parkinson's, highlights front vowel patterns"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000338,"raw_usage":{"total_tokens":1807,"prompt_tokens":824,"completion_tokens":983,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":440,"completion_tokens_details":{"reasoning_tokens":909}},"tokens_in":440,"tokens_out":983,"duration_ms":10500,"temperature":1.0,"reasoning_tokens":909,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:32:29.209317+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-evaluate the model using participant-level stratified splits so no speaker appears in both training and test sets; if accuracy falls toward the 85–97% baseline range or below, the 99% result is at least partly speaker recognition.","supporting_citations":[{"cited_title":"doi:10.21227/aw6b-tg17","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset: the Italian Parkinson's Voice and Speech Database with 831 audio files from 65 participants."},{"cited_title":"In: Proceedings of the IEEE International Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Provides the Grad-CAM method that the paper adapts from images to one-dimensional speech signals."},{"cited_title":"In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C","cited_arxiv_id":null,"evidence_quote":"Shows a prior adaptation of Grad-CAM to one-dimensional temporal signals (ECG), which this paper follows for audio."},{"cited_title":"Procedia Computer Science 218, 249--261 (2023)","cited_arxiv_id":null,"evidence_quote":"Represents the traditional hand-crafted-feature baseline (KNN, SVM, RF, logistic regression) that the deep model is compared against."},{"cited_title":"IEEE Access 8, 147635--147646 (2020)","cited_arxiv_id":null,"evidence_quote":"Surveys deep and machine learning approaches for early Parkinson's detection, locating the gap the paper fills."},{"cited_title":"Expert Systems with Applications 110, 182--190 (2018)","cited_arxiv_id":null,"evidence_quote":"Exemplifies feature-driven machine learning approaches whose static features the paper argues miss temporal dependencies."}],"review_version":1}