REVIEW 3 major objections 5 minor 37 references
Peptidomic-Based Prediction Model for Coronary Heart Disease Using a Multilayer Perceptron Neural Network
T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A multilayer perceptron trained on 50 urinary peptides selected by a genetic algorithm distinguishes coronary heart disease patients from controls with 95.7% accuracy, sensitivity, and specificity, an AUC of 0.9748, and a Matthews correlati
desk verdict The paper's reported test metrics are invalid because SMOTE and GA feature selection leak test-fold information into the model; the central claim of a reliable diagnostic tool does not hold as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a two-stage pipeline: a genetic algorithm performs wrapper-based feature selection, reducing 5,605 measured peptides to 50 using LDA and linear naive Bayes as fitness evaluators; a multilayer perceptron then learns the case-versus-control decision boundary from those 50 features. The MLP has three hidden layers of 60 neurons with tansig activations, is trained with scaled conjugate gradient and a regularization rate of 0.1, and uses early stopping when validation error stalls for six epochs. SMOTE balances the minority class, and stratified 10-fold cross-validation with an inner validation split supplies the performance estimate. The feature-selection stage is what m
What would settle it
Re-run the pipeline with genetic-algorithm feature selection and SMOTE applied separately within each training fold (nested cross-validation), leaving each test fold untouched until prediction; if accuracy and AUC fall materially below 95.7% and 0.9748, the original test metrics are inflated by information leaking from the test set into feature selection and synthetic sample generation.
Extended reading notes
Core claim
On its own terms, the study claims that a multilayer perceptron with three hidden layers of 60 neurons, trained on 50 urinary peptides selected by a genetic algorithm, distinguishes coronary heart disease patients from healthy controls with 95.67% precision, sensitivity, and specificity, a 95.65% F1-score, an AUC of 0.9748, a Matthews correlation coefficient of 0.9134, and a Cohen's kappa of 0.9131. The peptides come from CE-MS mass spectra of urine in four cohort studies; the original 82 cases were synthetically balanced to 345 with SMOTE before training. The author interprets the high, symmetric metrics as evidence that the model generalizes and could serve as a complementary non-invasive
Load-bearing premise
The test fold must be completely uninvolved in the genetic-algorithm peptide selection and the SMOTE oversampling; the Methods order suggests this, but the paper never states that the selection and balancing were done inside each training fold.
Editorial extensions
If this is right
- If the metrics hold up outside the lab, a urine sample plus this MLP could become a non-invasive screening option that avoids imaging- and catheter-based procedures for many patients.
- The 50-peptide signature is small enough to be encoded in a routine CE-MS assay, so the diagnostic could eventually be deployed as a standard panel.
- The near-identical precision and sensitivity across both classes suggest the model is not biased toward patients or controls, which is desirable for a screening tool.
- An MCC and kappa above 0.91 imply that the agreement with true diagnoses is far above chance, which is the paper's central evidence of reliability.
Reading between the lines
- We infer that the genetic-algorithm feature selection and SMOTE balancing were applied before the stratified cross-validation split, because the Methods sections list them in that order without describing a nested procedure. If so, the test metrics do not measure how the model would perform on truly new patients; a nested cross-validation re-run would be needed to know.
- Because 263 of the 345 'case' samples are synthetic interpolations, the model trains on partly fabricated data; real-world sensitivity may be lower once the model sees natural patient variation, a degradation the paper does not estimate.
- The single-pair spectral comparison in Figure 1 is descriptive, not a statistical test, so a reader should not take it as independent evidence of a peptide-level biomarker difference.
- The same pipeline would likely transfer to other proteomic fluids or diseases, but nothing in the paper tests that transfer, so such extensions remain speculative.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multilayer perceptron (MLP) classifier for coronary heart disease (CHD) diagnosis based on 50 urinary peptides selected by a genetic algorithm from 5,605 CE-MS peptide features. SMOTE is used to balance the original 82 cases and 345 controls, and the model is evaluated with a stratified 10-fold cross-validation protocol. Reported final-fold metrics include accuracy 95.65%, precision/sensitivity/specificity 95.67%, F1-score 0.9565, AUC 0.9748, MCC 0.9134, and Cohen's kappa 0.9131, which the Abstract interprets as demonstrating a 'highly accurate and robust non-invasive diagnostic tool' for CHD. The central technical problem is that SMOTE and GA feature selection are applied to the full dataset before the cross-validation split, so the final-fold test set is not independent of feature selection or data augmentation; architecture selection on the same data adds further selection bias.
Significance. If the reported results were obtained under a valid nested evaluation protocol, the work could be clinically relevant: non-invasive urinary peptide biomarkers for CHD are of genuine interest, and a compact 50-peptide classifier with high discrimination would be a useful contribution. The manuscript, however, provides no code, no data repository link, and no external validation cohort, so reproducibility is limited. As submitted, the paper's main value is cautionary: it is a clear example of how pre-split SMOTE and feature selection can inflate apparent predictive performance. The clinical claim in the Abstract is not supported by the experimental design.
major comments (3)
- [§2.2–§2.4, Table 1] The evaluation protocol leaks test information into model construction. SMOTE (§2.2) and genetic-algorithm feature selection (§2.3) are applied to the full 690-sample post-SMOTE dataset before the stratified 10-fold split described in §2.4. The 50-peptide panel is therefore selected using information from all folds, including the final fold later called the 'independent test set' (Table 1, §4.2). Synthetic minority samples in the test fold are generated from minority cases whose nearest neighbours may lie in training folds. The final-fold metrics (accuracy 95.65%, AUC 0.9748, etc.) are consequently not estimates of generalization to new patients. To support the central claim, balancing and feature selection must be nested inside each cross-validation training fold, or an external test cohort must be used.
- [§3] Architecture selection is not nested. Section 3 states that 'multiple neural network architectures were evaluated' and reports the best one (three hidden layers of 60 neurons with tansig). If these evaluations used the same post-SMOTE data that includes the final test fold, the reported test metrics are partly the result of selection on the test set. The manuscript does not describe an inner validation loop or a separate held-out set for choosing the architecture. Without this, the conclusion in §5(c) that the model is 'optimized without apparent overfitting' cannot be assessed.
- [Table 2, §4.2] The final test fold contains only 69 post-SMOTE samples (34 Class 1, 35 Class 2). Because only 82 real cases exist, the majority of Class 1 test instances are synthetic; after 10-fold splitting, roughly 8 real cases appear in the final fold. The headline numbers are point estimates on a small, partly synthetic test set, and no confidence intervals are given. For n=69, a few label changes can move the reported metrics by several percentage points, so the Abstract's 'robust' and 'reliable' claims are substantially overstated.
minor comments (5)
- [Figure 1] The axes are confusing: the x-axis is labelled 'Peptides' with values 0–6000 and the y-axis is labelled 'm/z' with units 10^4, while the text describes intensity versus mass-to-charge ratio. Please clarify the axes and units, and correct the duplicated 'Figure 1a:'/'Figure 1b:' caption text.
- [§2.2] Section 2.2 says 'both the control group and the cases were augmented' to 345 samples, but the original data already contained 345 controls and only 82 cases. Only the minority class should need augmentation; this inconsistency makes the composition of the post-SMOTE dataset unclear.
- [§2.4] The description of balancing is inconsistent: §2.2 describes SMOTE with linear interpolation, while §2.4 says 'an oversampling technique was applied, replicating minority class instances.' Plain replication is not SMOTE. State exactly which resampling procedure was used.
- [Tables 1–3] Several metrics are reported both per class and as macro averages; indicate explicitly that the Abstract's 95.67% values are macro-averaged across the two classes. Also report confidence intervals or a bootstrap estimate for the final-fold metrics, given the small test size.
- [References] Several references are incomplete: for example, Candemir et al. (2019) and Zreik et al. (2019b) lack venue/volume/page information, and many entries have no DOI. A data or code availability statement would also help reproducibility.
Circularity Check
Pre-split SMOTE and GA feature selection invalidate the reported 'independent test set' metrics; the final-fold prediction is partly a fit.
-
fitted input called prediction
[Section 2.2–2.4, Table 1, Section 4.2]
"To ensure representativeness of the datasets and mitigate the imbalance between cases and controls, the Synthetic Minority Over-sampling Technique (SMOTE) was employed, increasing the 82 cases to 345 samples. ... For optimal selection of predictive features, a wrapper-based approach was applied, integrating genetic algorithms (GA) ... The features selected through this strategy were subsequently used to train the primary classifier. ... Next, stratified 10-fold cross-validation was used, following the proposal by Kohavi ... The accuracy on the independent test set corresponding to the final fo"
SMOTE and GA operate on the full 690-sample augmented dataset before the stratified 10-fold split is created. Consequently, the final-fold test cases, as well as synthetic SMOTE cases placed in the test fold, have already influenced which 50 peptides were selected and how the minority class was interpolated. The paper labels this final-fold accuracy 'independent,' but no nested cross-validation is described: feature selection and augmentation were not re-run within each training fold. The reported 95.65% accuracy is therefore not a prediction on untouched data; it is a fit statistic that depends on information from the very samples being scored.
-
fitted input called prediction
[Section 3]
"Multiple neural network architectures were evaluated to identify models with optimal predictive capacity for medical diagnosis. The highest performance was obtained with a multilayer perceptron (MLP) comprising three hidden layers of 60 neurons each, utilizing the tansig activation function."
The architecture (3 hidden layers of 60 neurons with tansig) is selected by evaluating multiple configurations on the same dataset and choosing the one with the best performance. The paper does not state that architecture selection was nested inside the cross-validation loop. Thus the final-fold test metrics are reported for a model whose hyperparameters were chosen using information from the full dataset, including the test fold. This makes the reported sensitivity, specificity, AUC, and MCC partly the product of model selection on the test data, rather than an unbiased estimate of generalization.
full rationale
The paper's central claim is that an MLP trained on 50 GA-selected urinary peptides achieves near-95.7% accuracy, AUC 0.9748, MCC 0.9134, and kappa 0.9131 on an 'independent test set' (Table 2, Table 3, Abstract). However, the Methods section applies SMOTE (Section 2.2) and GA-based feature selection (Section 2.3) to the entire augmented dataset before the stratified 10-fold split (Section 2.4). No step is taken to confine SMOTE and GA to each training fold. As a result, the final-fold test set is not independent: the peptide panel and the synthetic minority samples used in training contain information from the test fold. Additionally, the paper reports that multiple architectures were evaluated and the best one selected (Section 3), again without nesting this selection inside the CV procedure. These are not subtle self-citations or definitional issues; they are fitted inputs being presented as predictions. The abstract and conclusions rest on the final-fold numbers rather than the cross-validation average (90.72%), amplifying the impact of the leakage. There is no evidence of self-citation load-bearing or ansatz smuggling; the circularity is purely the methodological leakage that turns 'predictions' into partial fits. Score 6 reflects that one or more central 'prediction' metrics reduce to fitted values by construction of the experimental pipeline.
Assumptions & free parameters
free parameters (7)
- Number of selected peptides =
50
- MLP architecture =
3 hidden layers x 60 neurons
- Regularization rate =
0.1
- Early stopping patience =
6 epochs
- GA population size =
150
- GA generations =
250
- SMOTE nearest neighbors =
not stated (default 5)
assumptions (4)
- domain assumption Urinary peptide profiles measured by CE-MS contain stable, generalizable signals for coronary artery disease.
- domain assumption SMOTE-generated synthetic peptides are representative of real case samples and preserve biological structure.
- domain assumption The GA-selected 50-peptide subset is predictive and stable across population splits.
- ad hoc to paper The final fold of cross-validation is independent of the feature selection and balancing steps.
Cite this review
Pith. "Pith review of Peptidomic-Based Prediction Model for Coronary Heart Disease Using a Multilayer Perceptron Neural Network." pith.science (2026). https://pith.science/paper/GH7XWWNW
@misc{pith2026250903884,
author = {Pith},
title = {Pith review of: Peptidomic-Based Prediction Model for Coronary Heart Disease Using a Multilayer Perceptron Neural Network},
year = {2026},
howpublished = {\url{https://pith.science/paper/GH7XWWNW}},
note = {Machine review of arXiv:2509.03884}
}
read the original abstract
Coronary heart disease (CHD) is a leading cause of death worldwide and contributes significantly to annual healthcare expenditures. To develop a non-invasive diagnostic approach, we designed a model based on a multilayer perceptron (MLP) neural network, trained on 50 key urinary peptide biomarkers selected via genetic algorithms. Treatment and control groups, each comprising 345 individuals, were balanced using the Synthetic Minority Over-sampling Technique (SMOTE). The neural network was trained using a stratified validation strategy. Using a network with three hidden layers of 60 neurons each and an output layer of two neurons, the model achieved a precision, sensitivity, and specificity of 95.67 percent, with an F1-score of 0.9565. The area under the ROC curve (AUC) reached 0.9748 for both classes, while the Matthews correlation coefficient (MCC) and Cohen's kappa coefficient were 0.9134 and 0.9131, respectively, demonstrating its reliability in detecting CHD. These results indicate that the model provides a highly accurate and robust non-invasive diagnostic tool for coronary heart disease.
Figures
Reference graph
Works this paper leans on
-
[1]
Aebersold, R. and Mann, M. (2003). Mass spectrometry-based proteomics. Nature , 422(6928):198--207
work page 2003
-
[2]
Aebersold, R., Mann, M., Ishihama, Y., Rappsilber, J., Kuster, B., and Vandekerckhove, J. (2018). Perspective on proteomics in biology and medicine. Nature , 537(7620):332--338. [ American Heart Association [AHA] , 2025] AHA2025 American Heart Association [AHA] (2025). Heart disease and stroke statistics---2025 update: At-a-glance. Accessed: 31 Aug 2025
work page 2018
-
[3]
Bom, M. J., Levin, E., Driessen, R. S., Danad, I., van Kuijk, C. C., van Rossum, A. C., Narula, J., Min, J. K., Leipsic, J. A., Belo Pereira, J. P., Taylor, C. A., Nieuwdorp, M., Raijmakers, P. G., Koenig, W., Groen, A. K., Stroes, E. S. G., and Knaapen, P. (2019). Predictive value of targeted proteomics for coronary plaque morphology in patients with sus...
work page 2019
-
[4]
Bytyçi, I., Bytyqi, S., Abazi, G., Bajraktari, G., and Henein, M. Y. (2025). Diagnostic accuracy of noninvasive investigations for coronary artery disease in hypertensive patients: A meta-analysis. Archives of Medical Science , 21(1):167--178
work page 2025
-
[5]
D., Demirer, M., Gupta, V., Bigelow, M
Candemir, S., White, R. D., Demirer, M., Gupta, V., Bigelow, M. T., Prevedello, L. M., and Erdal, B. S. (2019). Automated coronary artery atherosclerosis detection and localization on ccta with a 3d-cnn
work page 2019
-
[6]
Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research , 16:321--357
work page 2002
-
[7]
A 3D deep learning classifier and its explainability when assessing coronary artery disease
Cheung, W. K., Kalindjian, J., Bell, R., Nair, A., Menezes, L. J., Patel, R., Wan, S., Chou, K., Chen, J., Torii, R., Davies, R. H., Moon, J. C., Alexander, D. C., and Jacob, J. (2023). A 3d deep learning classifier and its explainability when assessing coronary artery disease. arXiv preprint arXiv:2308.00009
work page Pith review arXiv 2023
-
[8]
Chin, D., Battistoni, A., Tocci, G., Passerini, J., Parati, G., and Volpe, M. (2012). Noninvasive diagnostic testing for coronary artery disease in the hypertensive patient: Potential advantages of a risk estimation based algorithm. American Journal of Hypertension , 25(12):1226--1235
work page 2012
Show all 37 references
-
[9]
Patient education materials on non-invasive cardiovascular evaluation
Cleerly Health (2025). Patient education materials on non-invasive cardiovascular evaluation
2025
-
[10]
S., Ghosh, N., Patra, A., Mandana, K
Dhaladhuli, J., Dash, A., Rajeshwari, B. S., Ghosh, N., Patra, A., Mandana, K. M., and Khandelwal, S. (2023). Non-invasive coronary artery disease screening based on electrocardiogram characteristics and clinical risk factors. In Proceedings of the Annual International Confere...
2023
-
[11]
M., Davidson, B., Delgado, V., DeMaria, A., Dilsizian, V., Gaemperli, O., Garcia, M
Edvardsen, T., Asch, F. M., Davidson, B., Delgado, V., DeMaria, A., Dilsizian, V., Gaemperli, O., Garcia, M. J., Kamp, O., Lee, D. C., Neglia, D., Nesković, A. N., Pellikka, P. A., Plein, S., Sechtem, U., Shea, E., Sicari, R., Villines, T. C., Lindner, J. R., and Popescu, B. A...
2022
-
[12]
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics , 7(2):179--188
1936
-
[13]
Holland, J. H. (1975). Adaptation in natural and artificial systems . University of Michigan Press. [ Institute for Quality and Efficiency in Health Care [IQWiG] , 2020] IQWiG2020 Institute for Quality and Efficiency in Health Care [IQWiG] (2020). Coronary heart disease: Overview
1975
-
[14]
Kellner, R., Lottspeich, F., and Meyer, H. E. (1999). Microcharacterisation of proteins . Wiley-VCH
1999
-
[15]
D., Mischak, H., Poster, D., Dakna, M., Wüthrich, R
Kistler, A. D., Mischak, H., Poster, D., Dakna, M., Wüthrich, R. P., and Serra, A. L. (2009). Identification of a unique urinary biomarker profile in patients with autosomal dominant polycystic kidney disease. Kidney International , 76(1):89--96
2009
-
[16]
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI Proceedings , volume 2, pages 1137--1143
1995
-
[17]
A., Wojtara, M., and Uwishema, O
Kravarioti, D., Chaito, H., Ouardouz, S., Al Saab, E. A., Wojtara, M., and Uwishema, O. (2025). Noninvasive assessment of coronary artery disease: Recent techniques, diagnostic accuracy, and clinical implications for modern cardiology---a narrative review. Health Science Repor...
2025
-
[18]
Lewis, D. D. (1998). Naive (bayes) at forty: The independence assumption in information retrieval. In Nedellec, C. and Rouveirol, C., editors, Machine Learning: ECML 1998 , pages 4--15. Springer
1998
-
[19]
Luz, P. L. d., Bertini, P. J., and Favarato, D. (2005). Noninvasive detection of coronary artery disease---challenges for prevention of disease and clinical events. Clinics , 60(5):415--428
2005
-
[20]
J., Wu, C
MacCoss, M. J., Wu, C. C., Matthews, D. E., and Yates, J. R. I. (2005). Measurement of isotopic enrichment of stable isotope-labeled proteins using high-resolution mass spectra of peptides. Analytical Chemistry , 77(23):7646--7653
2005
-
[21]
C., Cremer, P., Hachamovitch, R., and Ayoub, C
Matta, M., Harb, S. C., Cremer, P., Hachamovitch, R., and Ayoub, C. (2021). Stress testing and noninvasive coronary imaging: What’s the best test for my patient? Cleveland Clinic Journal of Medicine , 88(9):502--515
2021
-
[22]
Mondal, S., Maity, R., and Nag, A. (2025). An efficient artificial neural network-based optimization technique for the early prediction of coronary heart disease: Comprehensive analysis. Scientific Reports , 15:4827
2025
-
[23]
Perna, G. P. (2020). Preventing myocardial infarction: Use and limitation of non-invasive imaging modalities. European Heart Journal Supplements , 22:E110--E112
2020
-
[24]
I., Neglia, D., and Andreini, D
Pontone, G., Guaricci, A. I., Neglia, D., and Andreini, D. (2017). State of the art noninvasive imaging in ischaemic heart disease. EuroIntervention , 13(6):654--665
2017
-
[25]
Prechelt, L. (1998). Early stopping---but when? In Orr, G. B. and Müller, K.-R., editors, Neural Networks: Tricks of the Trade , pages 55--69. Springer
1998
-
[26]
L., Gallagher, M
Raff, G. L., Gallagher, M. J., O’Neill, W. W., and Goldstein, J. A. (2005). Diagnostic accuracy of noninvasive coronary angiography using 64-slice spiral computed tomography. Journal of the American College of Cardiology , 46(3):552--557
2005
-
[27]
U., Naseem, S., Butt, A
Rehman, M. U., Naseem, S., Butt, A. U. R., Mahmood, T., Khan, A. R., Khan, I., Khan, J., and Jung, Y. (2025). Predicting coronary heart disease with advanced machine learning classifiers for improved cardiovascular risk assessment. Scientific Reports , 15:13361
2025
-
[28]
Shorewala, V. (2021). Early detection of coronary heart disease using ensemble techniques. Informatics in Medicine Unlocked , 26:100655
2021
-
[29]
Smith, A., Elliott, P., Mayr, M., Dehghan, A., and Tzoulaki, I. (2025). Proteomic risk scores for predicting common diseases using linear and neural network models in the uk biobank. Scientific Reports , 15:20520
2025
-
[30]
G., and Wilson, I
Theodoridis, G., Gika, H. G., and Wilson, I. D. (2008). Electrophoresis in metabolic profiling and metabonomics. Electrophoresis , 29(12):2419--2432
2008
-
[31]
L., Thiele, H., and Lurz, P
Unterhuber, M., Kresoja, K.-P., Rommel, K.-P., Besler, C., Baragetti, A., Klöting, N., Ceglarek, U., Blüher, M., Scholz, M., Catapano, A. L., Thiele, H., and Lurz, P. (2021). Proteomics-enabled deep learning machine algorithms can enhance prediction of mortality. Journal of th...
2021
-
[32]
E., and Blacher, J
Vallée, A., Cinaud, A., Blachier, V., Lelong, H., Safar, M. E., and Blacher, J. (2019). Coronary heart disease diagnosis by artificial neural networks including aortic pulse wave velocity index and clinical parameters. Journal of Hypertension , 37(8):1682--1688
2019
-
[33]
D., Van Aelst, L., Vanassche, T., Verhamme, P., Janssens, S., Peter, K., and Zhang, Z.-Y
Wei, D., Melgarejo, J. D., Van Aelst, L., Vanassche, T., Verhamme, P., Janssens, S., Peter, K., and Zhang, Z.-Y. (2023). Prediction of coronary artery disease using urinary proteomics. European Journal of Preventive Cardiology , 30(13):1537--1546
2023
-
[34]
R., Williams, K
Wilkins, M. R., Williams, K. L., Appel, R. D., and Hochstrasser, D. F. (1997). Proteome research: New frontiers in functional genomics . Springer. [ World Health Organization [WHO] , 2022] WHO2022 World Health Organization [WHO] (2022). Cardiovascular diseases (cvds)
1997
-
[35]
Yadav, A., Singh, S., Verma, S., and Gupta, A. (2022). A machine learning model for detection of coronary artery disease using noninvasive clinical parameters. Journal of Healthcare Engineering , 2022:36431068
2022
-
[36]
M., Voskuil, M., Viergever, M
Zreik, M., Lessmann, N., van Hamersvelt, R., Wolterink, J. M., Voskuil, M., Viergever, M. A., Leiner, T., and Isgum, I. (2019a). Automated coronary artery atherosclerosis detection and weakly supervised localization on coronary ct angiography with a deep 3d convolutional neura...
-
[37]
W., Khalili, N., Wolterink, J
Zreik, M., van Hamersvelt, R. W., Khalili, N., Wolterink, J. M., Voskuil, M., Viergever, M. A., and Leiner, T. (2019b). Deep learning analysis of coronary arteries in cardiac ct angiography for detection of patients requiring invasive coronary angiography
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.