{"id":"7acfad28-3874-404e-aeef-759a2dc42952","arxiv_id":"2412.05348","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A compact CNN with Bayesian hyperparameter tuning classifies early Parkinson's disease versus normal on PPMI SPECT images with 99.08% accuracy and flags 76 of 80 SWEDD scans as normal.","lead":"This paper trains convolutional neural networks on SPECT brain scans from the PPMI database to separate early Parkinson's disease from healthy controls, and checks whether scans from SWEDD patients are classified as normal. The best model reaches 99% accuracy in cross-validation, but the study lacks external validation and provides no code.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bayesian hyperparameter selection is not nested inside the 10-fold CV, so the headline 99.08%/99.93% numbers may be optimistically biased; a nested-CV re-run would settle it.","rationale":"The reader's weakest assumption correctly identifies the lack of nested cross-validation as the key soft spot, and my reading agrees. The paper reports strong performance for a compact CNN on PPMI SPECT data, including a useful SWEDD analysis, and it gives credit for comparing several classifiers, performing error analysis, and using a clinically relevant cohort. However, the reported metrics are only trustworthy if the Bayesian hyperparameter optimization did not use the same folds that produced the final confusion matrix. The text does not describe a nested or held-out protocol, so the central numbers carry a real selection-bias risk. This risk is directly testable: a nested 10-fold CV re-run would settle whether the 99.08%/99.93% figures are reproducible or an artifact of model selection on the evaluation folds. Since the reader already reached CONDITIONAL, my assessment does not change that verdict; it sharpens the specific condition that should be verified before the clinical claim is accepted.","tokens_in":10853,"tokens_out":2343,"duration_ms":25214,"concrete_test":"Implement nested 10-fold CV on the same PPMI subset: for each of the 10 outer test folds, run TPE Bayesian optimization on the outer training fold only (e.g., 50 iterations, optimizing inner 9-fold CV accuracy on that training fold), retrain the selected CNN on the full outer training fold, and evaluate on the outer test fold. Aggregate the 10 held-out predictions and recompute accuracy and AUC with fold-level mean and SD. If the nested-CV accuracy falls materially below 99.08% (e.g., by more than 2 percentage points) or AUC drops below roughly 99%, the reported metrics are biased by model selection; if the numbers reproduce within a small tolerance, the concern is resolved. Also report the TPE objective and total number of trials for transparency.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the single-slice CNN accuracy of 99.08% and AUC of 99.93% for early PD vs normal, plus 95% SWEDD detection. The load-bearing condition is that the 10-fold cross-validation estimates generalization of a model whose hyperparameters were chosen independently of the test folds. The paper does not establish this. Section II.E says ten-fold CV is applied 'to evaluate performance', and Section III says for MLP and CNN 'parameters are estimated using Bayesian approximation' (TPE), but no nested CV, no separate validation split, and no description of the TPE objective are given. If TPE evaluated candidate architectures on the same folds later used for the reported confusion matrix, then model selection could fit noise in those folds, inflating accuracy and AUC. This concern is concrete: Table III shows CNN at 99.08% while all other methods cluster at 96-97%, a gap broadly consistent with selection over a large hyperparameter space. The SWEDD result of 76/80 also has no confidence interval or repeated-CV estimate, and the classification threshold is inherited from the PD-vs-normal model, making 95% harder to interpret. Until the evaluation protocol is shown to be unbiased, the clinical-potential claim is not secure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript develops machine learning classifiers, primarily a compact convolutional neural network (CNN), to distinguish early Parkinson's disease (PD) from healthy normal controls using 123I-Ioflupane SPECT images from the PPMI database, and to flag SWEDD subjects as non-PD. Two image representations are used: the 41st transaxial slice and the average of slices 35–48. Ten-fold cross-validation is applied for evaluation. The CNN with Bayesian hyperparameter optimization reportedly achieves 99.08% accuracy and 99.93% AUC for the single-slice early-PD versus normal task, and identifies 76/80 SWEDD scans as normal. The paper concludes that the models could act as diagnostic aids.","tokens_in":11127,"tokens_out":5424,"duration_ms":52096,"significance":"If the reported performance survives an unbiased evaluation, the contribution is meaningful: a small two-convolution-layer CNN operating on a single SPECT slice, without hand-crafted feature extraction, achieving accuracy comparable or superior to prior work on the same PPMI cohort, with an explicit attempt to address the clinically relevant SWEDD group. The use of Bayesian hyperparameter optimization to obtain a compact architecture is a sensible design goal, and the comparison with logistic regression, SVM, and MLP provides context. However, the reliability of the reported numbers hinges entirely on the evaluation protocol, since the claims are empirical and no external cohort, error bars, or code are provided.","major_comments":[{"comment":"The evaluation protocol does not describe how hyperparameters were chosen relative to the cross-validation folds. The text states 'Ten fold cross validation was applied to evaluate the performance' (Sec. II.E) and that for MLP and CNN 'parameters are estimated using Bayesian approximation' (Sec. III), but it does not specify whether the TPE procedure used a separate validation split or was nested within each training fold, nor what objective function was optimized. If the same folds that produced the Table III confusion matrices were used to select architectures, the reported 99.08% accuracy and 99.93% AUC would be optimistically biased. Because the entire clinical-potential claim rests on these numbers, the paper must either describe a nested-CV protocol or re-run the analysis with an independent validation set, and report the TPE search space and number of evaluations.","section":"II.E and III"},{"comment":"The results are single-run point estimates with no uncertainty quantification. For the single-slice CNN, the confusion matrix [439,4;2,208] yields only 6 errors; a single 10-fold CV partition can easily produce a 99% estimate by chance. Without repeated CV with different seeds, bootstrapped confidence intervals, or per-fold metrics, the apparent gap between CNN and the other methods (99.08% vs. 96.0–96.8%) cannot be judged as significant. The SWEDD result '76 out of 80' similarly needs a confidence interval (the exact binomial 95% CI is approximately 87.9–98.7%) to be interpretable.","section":"III, Table III"},{"comment":"The CNN architecture is described inconsistently. The text says 'another convolution layer with 32 filters of size 5 x 5,' but Table II lists Conv2D (3 x 3) for the second convolutional layer, and the output shape (50, 41, 32) is compatible only with a 3x3 kernel on the (52, 43, 64) input. This should be corrected, since the compact architecture is a stated contribution.","section":"III vs. Table II"},{"comment":"The SWEDD evaluation is a transfer of the PD-vs-normal model to an unseen class, but no decision threshold is given. A CNN output is a 2-unit softmax; it is unclear whether the 'accuracy of 95%' uses 0.5 probability or some other cutoff. Reporting a threshold-independent measure (e.g., AUC on the SWEDD set) and the threshold used would make the result reproducible. Additionally, the clinical interpretation would be strengthened by follow-up diagnosis information, which the authors mention in the discussion of Choi et al. but do not analyze here.","section":"III.B"}],"minor_comments":[{"comment":"Table I is misformatted: the HY stage values appear in the Age columns for the early PD and SWEDD groups, making the table hard to read. Please separate the HY stage into its own columns.","section":"Table I"},{"comment":"The phrase 'L1-normalization' to describe L1 regularization in logistic regression and SVM is nonstandard; consider using 'L1 regularization'.","section":"II.E"},{"comment":"The comparison with prior work ('This work notably improves these metrics' in reference to [27]) would be more convincing if the training and test protocols of the cited studies were aligned; a table listing cohort sizes and evaluation schemes of the cited works would help.","section":"III"},{"comment":"The manuscript would benefit from a statement on data and code availability; no repository or implementation details are provided, which limits reproducibility.","section":"General"},{"comment":"The figure captions are not fully self-contained; for example, Fig. 3A and 3B should state in the caption which ground-truth class each misclassified image belongs to, rather than relying on the body text.","section":"Figures 3 and 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript header states the article is already accepted and published with revisions in Artificial Intelligence in Health. This report evaluates the arXiv version; if the published version contains the nested-CV details or threshold descriptions, some major comments may already be addressed. The editor may wish to verify the version under review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, incremental CNN application to PPMI SPECT data. The headline numbers (99.08% accuracy, AUC 99.93% for early PD vs normal, 95% SWEDD detection) are plausible but not fully pinned down because the hyperparameter search isn't nested inside the cross-validation. I'd want that clarified, but the paper deserves a serious referee.\n\nWhat's actually new: a compact two-conv-layer CNN with Bayesian (TPE) hyperparameter selection, trained on a single mid-striatal slice (slice 41) and an average of slices 35–48. The slice choice is inherited from the author's prior work, which is fine. The SWEDD detection result (76/80 correct) is the most clinically interesting piece, and the error analysis with example images is a nice touch. The comparison against logistic regression, SVM, and MLP on the same data is clean, and confusion matrices are provided.\n\nSoft spots: the main one is evaluation. Ten-fold CV is described, but the Bayesian optimization for CNN/MLP hyperparameters appears to have been run on the same dataset with no separate validation split or nested CV. If the TPE trials evaluated on the same folds later used to report the confusion matrix, the 99% numbers could be optimistically biased. The gap between CNN and the other methods (2–3 points) is consistent with selection over a large hyperparameter space. The SWEDD result also lacks a confidence interval and is based on a classifier threshold from the PD-vs-normal model, so the 76/80 count is harder to interpret. There's also a minor internal inconsistency: the text says the second conv layer is 5×5, but Table II lists 3×3. Small, but it suggests a careful proofread is needed. No external validation, no error bars, no code — for a clinical claim, that's a limitation, though PPMI data is public.\n\nThis paper is for someone working on automated SPECT triage, especially the SWEDD problem. It's an incremental but legitimate contribution, and the evaluation issue is fixable. If I were the editor, I'd send it to review and ask for nested CV or an external test set, plus clarification of the hyperparameter protocol. The qualitative conclusion that CNNs work well on PPMI SPECT data is almost certainly right; the precise 99% figure is the part that needs support.","headline":"A competent incremental CNN study on PPMI SPECT with a clinically useful SWEDD finding, but the headline accuracy may be inflated by an unclarified hyperparameter search protocol.","tokens_in":11629,"tokens_out":2835,"would_cite":false,"duration_ms":27025,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single SPECT brain slice reveals early Parkinson's disease with 99.08% accuracy.","keywords":["Parkinson's disease","SPECT imaging","convolutional neural network","early detection","SWEDD","Bayesian optimization","computer-aided diagnosis"],"falsifier":"Run the same two-convolutional-layer CNN with the same slice-41 input on a held-out test set (or an external SPECT cohort) that was never used during Bayesian optimization; if accuracy and AUC fall substantially below 99.08% and 99.93%, the central claim of near-perfect early-PD discrimination does not generalize. For the SWEDD result, check follow-up diagnoses: if a sizable fraction of the 76 scans classified as 'normal' are later diagnosed as clinical PD, then framing that classification as 'detection of non-PD' would be incorrect.","tokens_in":10661,"feed_emoji":"🧠","tokens_out":7389,"duration_ms":68161,"temperature":0.7,"pith_summary":"This paper argues that a compact convolutional neural network, trained on a single normalized transaxial slice from SPECT brain scans, can distinguish early Parkinson's disease from healthy controls with near-perfect performance: 99.08% accuracy and 99.93% AUC in ten-fold cross-validation. It also reports that the network recognizes 95% of SWEDD scans—patients clinically diagnosed with PD whose scans show no dopaminergic deficit—as non-PD. The work matters because early PD diagnosis is error-prone and SWEDD patients are frequently treated with medications that do more harm than good. The design deliberately avoids manual feature extraction and region-of-interest placement, relying instead on Bayesian hyperparameter optimization to find a compact two-convolutional-layer architecture. The authors present the model as a diagnostic aid for clinicians rather than a replacement for clinical judgment.","feed_headline":"Single SPECT slice spots early Parkinson's at 99.08%","feed_subtitle":"A compact CNN also picks out 95% of SWEDD scans that show no dopaminergic deficit.","key_machinery":"The load-bearing mechanism is a convolutional neural network whose architecture was selected by Tree-structured Parzen Estimator (TPE), a form of Bayesian hyperparameter optimization. The chosen network has a 5×5 convolution with 64 filters, a 2×2 max pooling, a 3×3 convolution with 32 filters, another max pooling, a flattened dense layer of 16 neurons with 0.2 dropout, and a final two-neuron output. The input is a SPECT slice normalized by dividing intensities by $2^{15}-1$, either the single slice 41 with maximal striatal uptake or the average of slices 35–48, based on prior localization of striatal activity. What carries the argument is the CNN's ability to learn discriminative textural features directly from these slices, which the paper says produces better results than the manual feature engineering used in prior work.","core_discovery":"The paper's central claim is that a Bayesian-optimized CNN with two convolutional layers (64 and 32 filters) and a 16-unit dense layer, applied to slice 41 of a normalized SPECT volume (or the average of slices 35–48), classifies early PD versus normal with accuracy 99.08% and AUC 99.93% on the single-slice input, with the averaged-slice input close behind at 98.32% accuracy and 99.40% AUC. On the separate SWEDD task, the same CNN correctly labels 76 of 80 (95%) SWEDD scans as normal, outperforming logistic regression, linear SVM, and MLP. The authors report that this improves on their earlier shape-analysis approach, which reached 97.29% accuracy and 99.26% AUC, and that the improvement comes from the CNN's learned filters rather than from hand-crafted features. They interpret the few misclassified SWEDD images as showing patterns that deviate from normal, consistent with prior reports that some such cases are later re-diagnosed as PD.","pith_inferences":["A testable implication the paper leaves implicit is that one transaxial slice carries essentially all the diagnostic information for early PD in this imaging protocol; that claim could be checked by comparing against a 3D CNN taking the full volume, which might matter in sites with different SPECT reconstruction or normalization.","The misclassified SWEDD cases, which the paper describes as looking 'uneven and dull,' may actually be early PD patients whose diagnosis changes on follow-up; linking the model's errors to the longitudinal labels in the public cohort would make the 'clinical aid' claim stronger and is not reported in the paper.","The claimed advantage of Bayesian hyperparameter optimization over manual or grid search in this setting should be verified with nested cross-validation; if verified, the same optimization recipe could transfer to other small-sample medical image classification problems without the optimistic bias risk."],"forward_implications":["A clinician could submit a single SPECT slice to the model as a second reader, potentially reducing early-stage misdiagnosis in the common situation where clinical symptoms are mild or overlap with essential tremor.","If the 95% SWEDD classification rate holds, most SWEDD patients—who currently receive dopaminergic medication despite normal scans—could be spared unnecessary treatment and side effects.","Because the model relies on one slice rather than the full-volume 91-slice scan, it is computationally inexpensive and easy to reproduce in standard clinical imaging pipelines.","The near-equality of single-slice and averaged-slice results suggests that the diagnostic signal is concentrated in the striatal region and is robust to small variations in slice selection."],"supporting_citations":[{"why":"The prior shape-analysis work on the same SPECT cohort that provides the baseline accuracy (97.29%) and AUC (99.26%) this paper claims to beat, plus the slice-range selection (35–48) used for input.","marker":"[27]"},{"why":"Supplies the Tree-structured Parzen Estimator Bayesian hyperparameter optimization method used to select the CNN and MLP architectures.","marker":"[40]"},{"why":"A prior CNN on dopamine-transporter imaging whose SWEDD observations (some atypical SWEDD scans read as abnormal) are used to interpret the CNN's misclassifications.","marker":"[11]"},{"why":"Shows that selecting sub-volumes reduces CNN complexity to two convolutional layers, the precedent for the compact slice-based design here.","marker":"[22]"},{"why":"A five-convolutional-layer CNN using isosurface features that serves as the complexity comparison, supporting the claim that the Bayesian-optimized two-layer network is more compact and more accurate.","marker":"[24]"},{"why":"Describes the longitudinal cohort and the standardized SPECT preprocessing pipeline that produced the images used for training and evaluation.","marker":"[35]"}],"fun_headline_variants":["One SPECT slice plus CNN: 99% accuracy for early PD","CNN spots early Parkinson's from a single SPECT frame","AI reads SPECT slice 41 to catch Parkinson's early","SWEDD scans: CNN correctly flags 95% as non-PD","99% accuracy: CNN detects early PD from one SPECT slice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that ten-fold cross-validation with Bayesian hyperparameter optimization performed on the same dataset gives an unbiased estimate of performance; if the model selection leaked test-fold information, the reported 99.08% accuracy and 99.93% AUC are optimistically inflated.","fun_headline_variants_meta":{"raw":{"variants":["One SPECT slice plus CNN: 99% accuracy for early PD","CNN spots early Parkinson's from a single SPECT frame","AI reads SPECT slice 41 to catch Parkinson's early","SWEDD scans: CNN correctly flags 95% as non-PD","99% accuracy: CNN detects early PD from one SPECT slice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1540,"prompt_tokens":897,"completion_tokens":643,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":552}},"tokens_in":513,"tokens_out":643,"duration_ms":5804,"temperature":1.0,"reasoning_tokens":552,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:49:32.760329+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same two-convolutional-layer CNN with the same slice-41 input on a held-out test set (or an external SPECT cohort) that was never used during Bayesian optimization; if accuracy and AUC fall substantially below 99.08% and 99.93%, the central claim of near-perfect early-PD discrimination does not generalize. For the SWEDD result, check follow-up diagnoses: if a sizable fraction of the 76 scans classified as 'normal' are later diagnosed as clinical PD, then framing that classification as 'detection of non-PD' would be incorrect.","supporting_citations":[{"cited_title":"High -Accuracy Classification of Parkinson's Disease Through Shape Analysis and Surface Fitting in 123I -Ioflupane SPECT Imaging,","cited_arxiv_id":null,"evidence_quote":"The prior shape-analysis work on the same SPECT cohort that provides the baseline accuracy (97.29%) and AUC (99.26%) this paper claims to beat, plus the slice-range selection (35–48) used for input."},{"cited_title":"Algorithms for hyper- parameter optimization,","cited_arxiv_id":null,"evidence_quote":"Supplies the Tree-structured Parzen Estimator Bayesian hyperparameter optimization method used to select the CNN and MLP architectures."},{"cited_title":"Refining diagnosis of Parkinson's disease with deep learning -based interpretation of dopamine transporter imaging,","cited_arxiv_id":null,"evidence_quote":"A prior CNN on dopamine-transporter imaging whose SWEDD observations (some atypical SWEDD scans read as abnormal) are used to interpret the CNN's misclassifications."},{"cited_title":"A 3D Convolutional Neural Network Approach for the Diagnosis of Parkinson’s Disease,","cited_arxiv_id":null,"evidence_quote":"Shows that selecting sub-volumes reduces CNN complexity to two convolutional layers, the precedent for the compact slice-based design here."},{"cited_title":"Parkinson's Disease Detection Using Isosurfaces -Based Features and Convolutional Neural Networks,","cited_arxiv_id":null,"evidence_quote":"A five-convolutional-layer CNN using isosurface features that serves as the complexity comparison, supporting the claim that the Bayesian-optimized two-layer network is more compact and more accurate."},{"cited_title":"The Parkinson's progression markers initiative (PPMI) – establishing a PD biomarker cohort,","cited_arxiv_id":null,"evidence_quote":"Describes the longitudinal cohort and the standardized SPECT preprocessing pipeline that produced the images used for training and evaluation."}],"review_version":1}