{"id":"48fd0616-be15-4276-be1e-041b97b80052","arxiv_id":"2505.12192","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"BenSParX provides a 120-speaker Bengali conversational speech dataset and a tuned machine learning framework that reports 95.77% accuracy, 95.57% F1, and 0.982 AUC for Parkinson's disease detection.","lead":"The paper introduces BenSParX, the first Bengali conversational speech dataset for Parkinson's disease detection, and a machine learning pipeline that reports 95.77% accuracy on it. A generalist should read it because voice-based screening is a promising low-cost tool for early PD detection in resource-limited settings, and this is the first such resource for Bengali.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Feature selection and hyperparameter tuning are not nested inside the group-wise CV, so the reported 95.77% accuracy may reflect test-set leakage.","rationale":"The reader's weakest assumption was age confounding, and that is a real threat: Table 3 shows a ~9-year mean age gap (PD 59.4 ± 10.38 vs HC 50.5 ± 6.15), and vocal aging correlates with many of the same acoustic features. I agree this undermines clinical interpretation. But it is not the single most load-bearing issue for the paper's central claim of state-of-the-art performance. A model that correctly detects age would not necessarily achieve 95.77% accuracy with the reported stability unless the age signal is very strong; and age adjustment would not fix a leakage problem. The leakage issue is more fundamental because it can inflate every reported metric, including the external validation results in Table 10, and it is directly testable by re-running the pipeline with nested CV. The manuscript text supports the concern: Section 4.1 only promises group-wise splits for evaluation; Sections 3.4 and 4.3 describe feature selection and hyperparameter tuning using CV but do not say those CVs are nested inside the outer subject-level folds. Thus I flag data leakage as the load-bearing concern, and I agree only partially with the reader's emphasis.","tokens_in":35632,"tokens_out":5296,"duration_ms":54614,"concrete_test":"Re-run the full pipeline with strictly nested group-wise cross-validation: for each of the 10 outer folds, perform all feature-selection steps (RFECV, LASSO, Relief-F+SFS, Mann-Whitney) and RandomizedSearch hyperparameter tuning using only the training-fold segments, then evaluate the selected model on the held-out group. Record accuracy/F1/AUC and compare with Table 8's best row; repeat on at least one external dataset (e.g., Sakar18) for Table 10. If the nested-CV numbers drop by more than ~2 percentage points or the CIs widen substantially, the reported state-of-the-art performance is an artifact of leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline performance numbers rest on the claim that group-wise 10-fold cross-validation (Section 4.1) gives unbiased estimates. But the paper only applies the group-wise split to 'model evaluation.' All four feature-selection strategies (RFECV, LASSO, Relief-F+SFS, Mann-Whitney U; Section 3.4) and RandomizedSearch hyperparameter tuning (Section 4.3) are described as using cross-validation on the full feature/data space, not as being rerun inside each outer training fold. With 900 segments from 120 speakers, selecting features or tuning hyperparameters on all samples lets information from held-out subjects leak into the model: the 'test' fold is no longer unseen. This would inflate accuracy/F1/AUC and artificially narrow the reported CIs, affecting the main 95.77% claim and the external-validation comparisons in Table 10, which use the same pipeline. The paper never states that feature selection and tuning are nested within the outer group-wise CV; the phrase 'group-wise strategy guarantees... model evaluation remains unbiased' in Section 4.1 does not cover feature selection. This is therefore the most load-bearing concern: if leakage is present, the central SOTA claim is not supported even after any age adjustment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces BenSParX, the first Bengali conversational speech dataset for Parkinson's disease (PD) detection, collected from 120 speakers (60 PD, 60 healthy controls) via phone recordings and segmented into 900 ten-second samples. The authors propose a classical machine learning pipeline that extracts 71 acoustic features (MFCCs, jitter, shimmer, formants, pulse, intensity, pitch, harmonicity), applies four feature selection techniques (RFECV, LASSO, Relief-F+SFS, Mann-Whitney U), tunes nine classifiers with RandomizedSearch, and uses SHAP for explainability. The central claims are that the framework achieves state-of-the-art performance on the new Bengali dataset (accuracy 95.77%, F1 95.57%, AUC-ROC 0.982) and consistently outperforms prior published approaches on four external PD voice datasets. The dataset is publicly released.","tokens_in":36016,"tokens_out":3136,"duration_ms":34210,"significance":"If the reported results are valid, the paper makes two useful contributions: a new publicly available Bengali conversational speech dataset for PD research, addressing a genuine linguistic gap, and a reproducible classical-ML pipeline with feature selection, hyperparameter tuning, and SHAP-based interpretability. The external validation across four datasets in different languages is a commendable attempt to demonstrate generalizability, and the authors explicitly recognize the need for group-wise evaluation with multiple segments per speaker. However, the central performance claims currently rest on a methodological ambiguity about whether feature selection and hyperparameter tuning are nested inside the group-wise cross-validation, and on an uncontrolled age difference between the PD and healthy control groups. These issues are load-bearing for the headline accuracy and for the external comparisons.","major_comments":[{"comment":"","section":"§3.4, §4.1, §4.3"},{"comment":"","section":"Table 3, §3.1, §5.1"},{"comment":"","section":"Table 8"},{"comment":"","section":"§4.6, Table 10"}],"minor_comments":[{"comment":"","section":"Figure 7"},{"comment":"","section":"§3.1"},{"comment":"","section":"§3.3"},{"comment":"","section":"§3.4.3"},{"comment":"","section":"Data availability"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the potential leakage from feature selection and hyperparameter tuning outside the group-wise CV. This is fixable by rerunning the pipeline with fully nested group-wise CV, so I recommend major revision rather than rejection. The age confound is also load-bearing and needs a concrete analysis rather than a caveat. The Table 8 inconsistencies make me less confident in the reported numbers, and the external comparisons need a fairer protocol. If the authors address these points with corrected experiments, the paper could make a solid contribution to the PD speech-detection literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading for the dataset; be skeptical of the performance claims. The BenSParX dataset is a real first: 120 speakers, 60 PD and 60 controls, natural conversational Bengali phone recordings, balanced and public. That alone makes this paper useful for the low-resource speech community. The framework itself is solid but standard — RFECV, LASSO, Relief-F+SFS, Mann-Whitney, nine classifiers, randomized search, SHAP. Nothing new there, but the combination on a new dataset is reasonable.\n\nThe soft spots are real and load-bearing. The group-wise 10-fold CV is only described for final model evaluation; feature selection and hyperparameter tuning are done on the full data with standard CV. With 900 segments from 120 speakers, that lets information from held-out subjects leak into the model. The paper never says these steps are nested in the outer group-wise splits, so the 95.77% accuracy and the narrow CIs are likely optimistic. This isn't a small issue — it directly undermines the SOTA claim. Second, the PD group averages 59.4 years vs 50.5 for controls, with no age adjustment. Age-related vocal changes could drive part of the separation. Third, the external validation on four datasets is not apples-to-apples: they apply their own feature extraction and selection pipeline to those datasets, which is fine, but the comparisons to published numbers don't control for different train/test protocols. Fourth, Table 8 has internal inconsistencies (e.g., RF with LASSO rows show recall/specificity values that look copied from LR; KNN with Mann-Whitney shows 100% precision with 6.14% recall and an F1 of 92.4 — that can't be right).\n\nThe limitations section acknowledges short recordings and demographic homogeneity but does not mention the age confound or the leakage risk. These are fixable. A revision with nested group-wise CV, an age-adjusted analysis, and corrected tables would be a solid contribution. As it stands, I'd treat the numbers as optimistic.\n\nThis paper deserves peer review — the dataset is important and the methodological problems are correctable. I'd send it, but the instruction to reviewers should be clear: the main performance claims need re-validation before the work is presented as SOTA.","headline":"The first Bengali PD speech dataset is a genuine contribution, but the headline accuracy of 95.77% is likely inflated by leakage in feature selection and an unadjusted age confound.","tokens_in":707,"tokens_out":695,"would_cite":false,"duration_ms":25626,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Bengali speech dataset and tuned classical pipeline detect Parkinson's disease at 95.77% accuracy, with SHAP-based explanations linking predictions to known vocal impairments.","keywords":["Parkinson's disease","Bengali speech","voice biomarkers","acoustic feature selection","machine learning","SHAP","explainability","conversational speech"],"falsifier":"Retrain the same BenSParX pipeline on age-matched subgroups drawn from the published dataset (for example, restricting both classes to the overlapping 43-57 age range), or add age as a feature and inspect its SHAP value; if accuracy drops substantially below 95.77% or age becomes a top predictor, the central claim of PD-specific detection is weakened. A second test is an independent Bengali cohort recorded with the same phone-call protocol.","tokens_in":35443,"feed_emoji":"🗣️","tokens_out":5252,"duration_ms":44665,"temperature":0.7,"pith_summary":"This paper introduces BenSParX, the first publicly available Bengali conversational speech dataset for Parkinson's disease (PD) detection, together with a machine-learning framework that combines 71 acoustic features, four feature-selection strategies, nine classifiers with hyperparameter tuning, and SHAP explanations. The authors claim state-of-the-art performance on their dataset: 95.77% accuracy, 95.57% F1 score, and an AUC-ROC of 0.982 with a support vector machine using features selected by recursive feature elimination with cross-validation and gradient boosting. They also report that the same framework outperforms previously published results on four external PD voice datasets in other languages. A sympathetic reader would care because the work extends voice-based PD screening to a language spoken by over 230 million people and shows that a rigorously tuned classical pipeline can match or beat deep-learning baselines while remaining interpretable.","feed_headline":"Bengali voice test spots Parkinson's with 95.77% accuracy","feed_subtitle":"First Bengali PD speech dataset, 120 speakers, and a tuned pipeline that beats deep baselines with explainable features.","key_machinery":"The load-bearing mechanism is the multi-stage feature selection pipeline. Seventy-one handcrafted acoustic features, extracted with Praat, are filtered and ranked by four complementary methods: recursive feature elimination with cross-validation (RFECV) run with six different base classifiers, LASSO regularization, Relief-F ranking followed by sequential forward selection, and the Mann-Whitney U test. Each selected subset feeds nine tuned classifiers, and the best overall model is an SVM trained on the 63 features chosen by RFECV with gradient boosting. SHAP (SHapley Additive exPlanations) then assigns per-feature contribution values to the final model, connecting each prediction to interpretable acoustic markers.","core_discovery":"The central claim is that Parkinson's disease can be detected from natural Bengali conversational speech with accuracy comparable to or better than existing voice-based detectors, using a deliberately broad acoustic feature set and systematic feature selection. The paper demonstrates that features often neglected in prior work—formants and glottal-pulse statistics—carry real discriminative signal, with all four formant and all four pulse features retained in the best-performing model. The best configuration (SVM with 63 features selected by RFECV using gradient boosting) reaches 95.77% accuracy and 0.982 AUC on the Bengali dataset, and the framework is claimed to consistently outperform state-of-the-art methods on the Sakar18, Little, Naranjo, and MDVR-KCL datasets. The paper also shows that SHAP attributions for this model line up with clinically established PD speech impairments: energy instability, reduced harmonicity, elevated jitter and shimmer, and articulatory imprecision.","pith_inferences":["The paper compares against published numbers from other studies rather than re-running every competing method under identical train/test splits, so the 'consistently outperforms' claim should be read as accuracy against reported results, not a controlled benchmark.","An immediate testable extension is age-matched re-analysis: because the PD group averages 59.4 years versus 50.5 for healthy controls, retraining on age-balanced subgroups would reveal how much of the 95.77% accuracy is disease-specific versus age-related voice change.","A strong generalization probe would be zero-shot or fine-tuned transfer from Bengali to an unrelated language (e.g., Turkish or Spanish) using the same acoustic pipeline, testing whether the selected features capture PD-specific dysphonia rather than language-specific artifacts.","The framework's reliance on Praat handcrafted features suggests a natural comparison with end-to-end deep embeddings (e.g., Wav2Vec2) on the same dataset to see whether the engineered features retain an advantage."],"forward_implications":["If the reported accuracy holds in independent cohorts, a phone-call-based screening tool could be deployed for Bengali-speaking populations with no specialized recording equipment.","Formant and glottal-pulse features, rarely used in prior PD voice studies, should be included in future acoustic analyses rather than relying on MFCCs, jitter, and shimmer alone.","The framework's external results suggest that careful feature selection plus classical classifiers can be a strong baseline that deep-learning models should be measured against on small tabular acoustic datasets.","SHAP explanations could give clinicians per-patient rationales linking a positive screening result to specific, known dysphonia markers.","Releasing the dataset publicly invites re-analysis and cross-linguistic comparison, which is a precondition for any clinical adoption."],"supporting_citations":[{"why":"Sakar18 dataset serves as the primary external benchmark where BenSParX claims the highest accuracy (95.35%).","marker":"[65]"},{"why":"Little dataset is the external benchmark where BenSParX claims 98.66% accuracy, surpassing prior results.","marker":"[41]"},{"why":"MDVR-KCL dataset provides the external validation set where BenSParX reports 98.59% accuracy.","marker":"[36]"},{"why":"Naranjo dataset is the external benchmark where BenSParX claims 87.91% accuracy.","marker":"[60]"},{"why":"Cai et al. supplies the EF-LSTM and LF-LSTM deep-learning baselines that BenSParX outperforms.","marker":"[10]"},{"why":"Narendra et al. supplies the 1D-CNN baseline and the PC-GITA comparison used in the experiments.","marker":"[47]"},{"why":"Balaha et al. provides prior results on Little and MDVR-KCL datasets used for comparison.","marker":"[4]"},{"why":"Hadjaidji et al. provides the prior 97.44% accuracy on the Little dataset that BenSParX claims to surpass.","marker":"[29]"},{"why":"Polat provides the prior 94.89% accuracy on Sakar18 that BenSParX claims to beat.","marker":"[55]"},{"why":"Naranjo et al. provides the prior 86.2% accuracy on the Naranjo dataset that BenSParX claims to surpass.","marker":"[46]"}],"fun_headline_variants":["First Bengali PD speech dataset yields 95.77% accurate detector","Explainable model spots Parkinson's in Bengali speech, 95.77% accurate","Bengali speech AI for Parkinson's: 95.77% accuracy with SHAP clarity","New Bengali dataset + tuned SVM detects Parkinson's at 95.77%","PD detection from Bengali speech: first dataset, 95.77% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis assumes the PD and healthy control groups differ only in disease status, specifically that the roughly nine-year average age gap (59.4 versus 50.5 years) does not materially drive the voice classification, because the study never adjusts for age or verifies that PD-specific vocal changes—not age-related vocal aging—produce the separation.","fun_headline_variants_meta":{"raw":{"variants":["First Bengali PD speech dataset yields 95.77% accurate detector","Explainable model spots Parkinson's in Bengali speech, 95.77% accurate","Bengali speech AI for Parkinson's: 95.77% accuracy with SHAP clarity","New Bengali dataset + tuned SVM detects Parkinson's at 95.77%","PD detection from Bengali speech: first dataset, 95.77% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000295,"raw_usage":{"total_tokens":1769,"prompt_tokens":1055,"completion_tokens":714,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":671,"completion_tokens_details":{"reasoning_tokens":611}},"tokens_in":671,"tokens_out":714,"duration_ms":6319,"temperature":1.0,"reasoning_tokens":611,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:38:32.626632+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same BenSParX pipeline on age-matched subgroups drawn from the published dataset (for example, restricting both classes to the overlapping 43-57 age range), or add age as a feature and inspect its SHAP value; if accuracy drops substantially below 95.77% or age becomes a top predictor, the central claim of PD-specific detection is weakened. A second test is an independent Bengali cohort recorded with the same phone-call protocol.","supporting_citations":[{"cited_title":"Parkinson’s Disease Classification","cited_arxiv_id":null,"evidence_quote":"Sakar18 dataset serves as the primary external benchmark where BenSParX claims the highest accuracy (95.35%)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Little dataset is the external benchmark where BenSParX claims 98.66% accuracy, surpassing prior results."},{"cited_title":"Mobile device voice recordings at king’s college london (mdvr-kcl) from both early and advanced parkinson’s disease patients and healthy controls","cited_arxiv_id":null,"evidence_quote":"MDVR-KCL dataset provides the external validation set where BenSParX reports 98.59% accuracy."},{"cited_title":"Parkinson Dataset with replicated acoustic features","cited_arxiv_id":null,"evidence_quote":"Naranjo dataset is the external benchmark where BenSParX claims 87.91% accuracy."},{"cited_title":"An essay on the shaking palsy","cited_arxiv_id":null,"evidence_quote":"Polat provides the prior 94.89% accuracy on Sakar18 that BenSParX claims to beat."}],"review_version":1}