Pith. sign in

REVIEW 4 major objections 5 minor 18 references

Mortality Prediction of Pulmonary Embolism Patients with Deep Learning and XGBoost

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that a hybrid CT pipeline predicts 30-day pulmonary embolism mortality with 94% accuracy, outperforming clinical scores and CNN baselines.

desk verdict Imaging-only PE mortality prediction is a real clinical aim and the pipeline is coherent, but the reported metrics contradict each other and the evaluation may leak through SMOTE/PCA, so the headline result is unsupported. read the letter →

arxiv 2411.18063 v1 pith:PTUBFPKH submitted 2024-11-27 eess.IV cs.CVcs.LG

Yalcin Tur , Vedat Cicek , Tufan Cinar , Elif Keles , Bradlay D. Allen , Hatice Savas , Gorkem Durak , Alpay Medetalibeyoglu
show 1 more author
Ulas Bagci
This is my paper · ORCID
classification eess.IVcs.CVcs.LG
keywords pulmonaryembolism30-daymortalitypredictionCTimagingdeeplearningXGBoost3Dresidualnetworkclassimbalanceprognosis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the initial chest CT scan, without any annotation of where the emboli are, carries enough information to predict whether an acute pulmonary embolism patient will die within 30 days. The authors built PEP-Net, a five-stage pipeline that extracts volumetric features with a 3D residual network, reduces them with PCA, balances the classes with Borderline-SMOTE, and classifies with XGBoost. On 193 CT scans, it reports 94.5% accuracy (AUC 0.917) when the input is the lung region and 94.0% accuracy (AUC 0.901) when the input is the cardiac region, clearly above the 71–80% range of the deep-learning baselines. If reliable, this would let clinicians use the scan already taken at admission as a prognostic tool, supplementing clinical scores whose specificity is limited.

What carries the argument

The central object is PEP-Net, a five-step pipeline. Step one uses a conventional 3D U-Net, initialized with publicly available pretrained weights, to segment rough lung and cardiac regions from the CT and crop bounding boxes around them. Step two uses a pretrained 3D ResNet18 adapted for single-channel grayscale input, with the first and last layers replaced, to extract hierarchical volumetric features. Step three applies Borderline-SMOTE, which creates synthetic minority-class samples by interpolating between a minority sample and one of its neighbors, to correct the 20/80 death/survival imbalance. Step four applies PCA and keeps 100 components, chosen by experimenting over 50–150. Step five trains XGBoost with a logistic objective, a learning rate of 0.1, tree depth 3, and 100 boosting rounds, using scale_pos_weight to emphasize the minority class. The machinery's job is to shift the burden from end-to-end deep classification to a feature-extraction-plus-boosting design that the authors argue is better suited to small, imbalanced medical cohorts.

What would settle it

Re-run the five-fold evaluation twice on the same 193 scans: once with Borderline-SMOTE and PCA applied only inside each training fold, and once with the pipeline order exactly as described (oversampling and PCA before cross-validation). If the two accuracies are both near 94%, the claim survives; if the leakage-free version drops toward the 71–80% baseline range, the reported gain is an artifact of synthetic validation samples.

Watch

Extended reading notes

Core claim

On its own terms, the paper's core claim is that 30-day mortality in acute PE can be read from routine CT imaging by a model that never sees the embolus location. PEP-Net takes a rough lung or cardiac region from a pretrained segmentation model, extracts 3D features with a ResNet18 backbone adapted for single-channel CT, compresses those features to 100 principal components, generates synthetic minority-class samples with Borderline-SMOTE, and feeds the result to XGBoost. In five-fold cross-validation on 193 patients (38 deaths, 155 survivors), the Lung-ROI version scored accuracy 0.945±0.003, AUC 0.917±0.007, sensitivity 0.977, specificity 0.874; the Cardiac-ROI version scored accuracy 0.940±0.007, AUC 0.901±0.008. The paper reports that only seven patients were misclassified overall, and that the near-parity of lung and cardiac results suggests the model detects cardiac or vascular changes that are not obvious to the human eye. The authors position the result as a new benchmark for imaging-based PE prognostics and argue that the ResNet-plus-XGBoost combination beats end-to-end deep classifiers on this small dataset.

Load-bearing premise

The load-bearing assumption is that the synthetic samples created by Borderline-SMOTE and the choice of 100 PCA components are kept inside each cross-validation fold, so the validation fold never sees patients synthesized from training data; the paper describes both as preprocessing steps without stating this safeguard, and if they leak, the 94% accuracy could be a memorization artifact rather than a true prognostic signal.

Editorial extensions

If this is right

  • If the reported accuracy holds, an admission CT alone could flag high-risk PE patients for early intensive management without waiting for clinical scoring or biomarker results.
  • The model needs only patient-level outcome labels during training, not radiologist annotations of emboli location or extent, so it could be retrained on existing hospital outcome data.
  • Cardiac-ROI performance nearly matching Lung-ROI performance implies the mortality signal may be partly cardiac (for example, right-heart strain) rather than solely the visible clot burden, which could motivate targeted imaging-biomarker studies.
  • The ResNet-plus-XGBoost architecture, with class balancing and dimensionality reduction, gives a recipe for other small-data medical imaging prognosis tasks where pure deep networks overfit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the reported 94% accuracy is probably optimistic for a single-center 193-scan cohort with synthetic oversampling; a multi-center replication would likely yield a lower figure, and even a drop to the 85–90% range would still be clinically meaningful.
  • Editorial inference: the cardiac-ROI result suggests a testable hypothesis that CT-detectable right ventricular strain, measurable as RV/LV diameter ratio, mediates part of the mortality signal; comparing PEP-Net's cardiac features to this established measurement would separate a genuine biomarker from classifier artifacts.
  • Editorial inference: the same pipeline could be applied to other admission-imaging prognosis tasks, such as predicting sepsis or stroke outcomes, but only if future work strictly confines oversampling and PCA fitting to training folds; otherwise the accuracy gain could be an artifact of validation leakage rather than real signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript presents PEP-Net, a pipeline that combines a 3D ResNet feature extractor with PCA dimensionality reduction, BorderlineSMOTE oversampling, and an XGBoost classifier to predict 30-day in-hospital mortality from CT scans of 193 patients with acute pulmonary embolism. The authors report accuracy of 94.5% (±0.3) for Lung-ROI and 94.0% (±0.7) for Cardiac-ROI inputs, with AUCs of 0.92 and 0.90, and claim that this significantly outperforms baseline CNN models whose accuracies are in the 71–80% range. The paper describes five-fold cross-validation and includes a brief limitations discussion acknowledging the single-center dataset and absence of clinical parameters.

Significance. If the reported performance were valid, the result would be clinically notable: a purely imaging-based model achieving roughly 94% accuracy for 30-day mortality after acute PE would substantially exceed commonly cited prognostic scores such as PESI. The paper also addresses a relatively understudied task, mortality prognosis rather than PE diagnosis, which is a legitimate and important problem. However, the central claim is not supported as stated. The headline metrics in Table 2 are mutually inconsistent with the cohort size and class distribution, and the manuscript does not establish that oversampling and PCA-based dimensionality selection are confined to the training folds. Both issues are load-bearing for the abstract's benchmark. The paper deserves credit for using publicly available segmentation checkpoints, describing the imaging protocol, and framing the task clearly, but the reported quantitative claims cannot be accepted without correction and a precisely specified evaluation protocol.

major comments (4)
  1. [Section 3, Table 2] The reported metrics for PEP-Net with Lung-ROI are internally inconsistent. For N=193 with 38 deaths and 155 survivors, sensitivity 0.977 implies approximately 37 true positives and 1 false negative, while specificity 0.874 implies approximately 135 true negatives and 20 false positives. The resulting accuracy is (37+135)/193 ≈ 0.89, not 0.945. Even after rounding to integer counts, 37 true positives, 1 false negative, 135 true negatives, and 20 false positives give 172 correct out of 193, or 89.1%. No per-fold averaging can reconcile these pooled counts, because the pooled true-positive and true-negative totals are simply the sums of the fold-level counts that define the averaged sensitivity and specificity. The central accuracy claim in the abstract and Table 2 is therefore not supported by the numbers reported.
  2. [Section 2, 'Oversampling and Feature Selection' and 'Further Details on Training'] The manuscript never states that B-SMOTE is applied only inside each training fold. The pipeline description in Figure 2 and the text present B-SMOTE and PCA as sequential steps before the XGBoost classification step, and the PCA component count of 100 is described as selected empirically by testing a range of 50 to 150 components. If B-SMOTE is applied to the full dataset before cross-validation, synthetic minority-class samples will appear in the validation folds, and the classifier can achieve artificially high accuracy by recognizing memorized synthetic examples rather than by generalizing to real patients. Similarly, selecting the PCA dimension using the entire dataset without nested validation makes the reported accuracy partly a fitted quantity. The manuscript must explicitly state that all preprocessing, including oversampling, PCA fitting, and component-count selection, is performed independently within each training fold, or the reported performance cannot be interpreted as a predictive claim.
  3. [Section 3, text after Figure 3] The prose states 'our algorithm misclassified only 7 patients' and then specifies three false positives (alive predicted as dead) and four false negatives (dead predicted as alive). For the Lung-ROI results in Table 2, seven errors would correspond to an accuracy of 186/193 ≈ 96.4%, not 94.5%. For the Cardiac-ROI results in Table 3, seven errors would correspond to 96.4% as well, not 94.0%. Moreover, with three false positives and four false negatives, sensitivity would be 34/38 ≈ 89.5% and specificity would be 152/155 ≈ 98.1%, neither of which matches the sensitivity and specificity in Table 2 or Table 3. Unless the tables and the prose refer to different runs, datasets, or evaluation subsets, which the manuscript does not state, the results are mutually incompatible and cannot be reproduced from the reported confusion counts.
  4. [Section 3, Tables 2 and 3] The standard deviations reported for PEP-Net are implausibly small relative to the baselines: ±0.003 in accuracy and ±0.002 in sensitivity for Lung-ROI, and ±0.007 in accuracy for Cardiac-ROI. With five-fold cross-validation on only 193 patients, such tiny variance would require nearly identical performance across folds. Given the internal inconsistency of the aggregate metrics, the reported standard deviations do not provide evidence of stability. The authors should report per-fold results or a confusion matrix for the pooled predictions so that the reader can verify the reported mean and variance.
minor comments (5)
  1. [Introduction and References] The citation for 3DResNet appears to be [18], but the reference list entry [18] is Zhou et al.'s meta-analysis of the PESI score, not a ResNet architecture paper. The manuscript should cite the original ResNet publications or a correct medical-imaging adaptation of ResNet.
  2. [Section 2, 'Further Details on Training'] There is a typo in 'Sparse Categorical Cross Entropyas the loss function'; it should read 'Sparse Categorical Cross Entropy as the loss function'.
  3. [Title page / Author affiliations] The affiliation for the third author contains a typo: 'Univesity' should be 'University'.
  4. [Tables 2 and 3] Several numbers in the tables contain an errant space after the decimal point, such as '0 .581' and '0 .782'; these should be formatted consistently as '0.581' and '0.782'.
  5. [Section 3, Figure 3] The caption describes left, middle, and right panels, but the text only references the right and middle panels. Please ensure that the panel descriptions are complete and that the ROC curve (left) is explicitly discussed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: PEP-Net's accuracy is an empirical evaluation outcome, not a quantity defined in terms of the input labels.

full rationale

No step in the paper derives a prediction from the prediction target by construction. The pipeline (U-Net ROIs, 3DResNet features, PCA, B-SMOTE, XGBoost) is an empirical classifier trained on image features with mortality labels; the reported accuracy is an evaluated outcome, not a quantity defined to equal the input by definition. The PCA component count and XGBoost hyperparameters are chosen empirically on the same cohort, which is a model-selection/overfitting risk, but the paper never defines those parameters as the mortality prediction, so this is not a circular reduction. Self-citations (refs 4, 6, 12, 13) are used only as generic U-Net segmentation background; the actual ROI segmentation is outsourced to external TotalSegmentator [14], so no load-bearing self-citation is present. The metric-inconsistency and potential data-leakage issues flagged by the reader are correctness/validity concerns rather than circularity under the specified definitions, which require exhibiting a specific reduction of a claimed prediction to its own inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard machine-learning components plus assumptions about label reliability, segmentation accuracy, and data split hygiene. The only numerically fitted choices are the PCA dimension and fixed hyperparameters; no external validation set or independent benchmark is used.

free parameters (6)
  • Number of PCA components = 100
    Selected by experimentation over a range of 50 to 150 components, with the best value reported (Section 2, Oversampling and Feature Selection).
  • XGBoost learning rate = 0.1
    Chosen by hand; no tuning or sensitivity analysis reported (Section 2, Classification with XGBoost).
  • XGBoost max tree depth = 3
    Chosen to limit model complexity; no empirical justification or search reported (Section 2).
  • XGBoost boosting rounds = 100
    Fixed at 100 rounds; no early stopping or ablation reported (Section 2).
  • BorderlineSMOTE k-neighbors and m-neighbors = 5 and 10
    Default parameters used; no sensitivity analysis reported (Section 2, Oversampling and Feature Selection).
  • CNN training epochs and learning rate = 100 epochs, 0.0001
    Fixed for all models; no schedule or early stopping details reported (Section 2, Further Details on Training).
assumptions (5)
  • domain assumption TotalSegmentator's publicly available lung and heart segmentation checkpoints produce sufficiently accurate ROIs for this cohort.
    The ROI extraction relies entirely on pretrained TotalSegmentator models without verification on this dataset (Section 2, Determining Lung-ROI and Cardiac-ROI).
  • domain assumption 3DResNet18 pretrained weights remain useful after replacing the first convolutional layer with a single-channel input trained from scratch.
    The paper replaces the input layer and keeps the rest of the pretrained weights, but does not validate that this transfer is sound for 1 mm slice CT data (Section 2, 3DResNet with a new convolutional layer).
  • domain assumption Patient-level 30-day mortality labels from the hospital record are correct and complete.
    The outcome is taken from a retrospective single-center record with no adjudication or external verification described (Section 2, Data).
  • domain assumption The single-center cohort is representative of the broader acute PE population.
    All 193 scans come from one hospital in Istanbul; the authors acknowledge the need for multi-center validation (Section 4, Discussion).
  • standard math The standard B-SMOTE and XGBoost update formulas are valid as stated.
    Equations (1) and (2) are textbook definitions and are not in dispute.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mortality Prediction of Pulmonary Embolism Patients with Deep Learning and XGBoost." pith.science (2026). https://pith.science/paper/PTUBFPKH

@misc{pith2026241118063,
  author       = {Pith},
  title        = {Pith review of: Mortality Prediction of Pulmonary Embolism Patients with Deep Learning and XGBoost},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PTUBFPKH}},
  note         = {Machine review of arXiv:2411.18063}
}
read the original abstract

Pulmonary Embolism (PE) is a serious cardiovascular condition that remains a leading cause of mortality and critical illness, underscoring the need for enhanced diagnostic strategies. Conventional clinical methods have limited success in predicting 30-day in-hospital mortality of PE patients. In this study, we present a new algorithm, called PEP-Net, for 30-day mortality prediction of PE patients based on the initial imaging data (CT) that opportunistically integrates a 3D Residual Network (3DResNet) with Extreme Gradient Boosting (XGBoost) algorithm with patient level binary labels without annotations of the emboli and its extent. Our proposed system offers a comprehensive prediction strategy by handling class imbalance problems, reducing overfitting via regularization, and reducing the prediction variance for more stable predictions. PEP-Net was tested in a cohort of 193 volumetric CT scans diagnosed with Acute PE, and it demonstrated a superior performance by significantly outperforming baseline models (76-78\%) with an accuracy of 94.5\% (+/-0.3) and 94.0\% (+/-0.7) when the input image is either lung region (Lung-ROI) or heart region (Cardiac-ROI). Our results advance PE prognostics by using only initial imaging data, setting a new benchmark in the field. While purely deep learning models have become the go-to for many medical classification (diagnostic) tasks, combined ResNet and XGBoost models herein outperform sole deep learning models due to a potential reason for having lack of enough data.

Figures

Figures reproduced from arXiv: 2411.18063 by the authors.

Figure 1
Figure 1. We have lobar (left) and distal PE (right) cases are illustrated, respectively. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Proposed PEP-Net architecture for mortality prediction for PE patients include four [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Left: ROC-AUC curve of different DL-based approaches used in this study; TPR– True Positive Rate, FPR– False Positive Rate. Middle: An example CT scan that PEP-Net predicts the 30-day mortality of PE patient with high probability. Right: An example CT scan that PEP-Net fails to predict the patient’s 30-day mortality. Very subtle change is observed [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    Aujesky, D

    D. Aujesky, D. S. Obrosky, R. A. Stone, T. E. Auble, A. Perrier, J. Cornuz, P.-M. Roy, and M. J. Fine. Derivation and validation of a prognostic model for pulmonary embolism. American journal of respiratory and critical care medicine, 172(8):1041–1046, 2005

    work page 2005

  2. [2]

    Aydogdu, N

    M. Aydogdu, N. Topbasi Sinanoglu, N. Dogan, I. Oguzulgen, A. Demircan, F. Bildik, and N. Ekim. Wells score and pulmonary embolism rule out criteria in preventing over investigation of pulmonary embolism in emergency departments. Tuberkuloz ve Toraks-Tuberculosis and Thorax, 2014

    work page 2014

  3. [3]

    Bikdeli et al

    B. Bikdeli et al. Clinical presentation and short-and long-term outcomes in patients with isolated distal deep vein thrombosis vs proximal deep vein thrombosis in the riete registry. JAMA cardiology, 7(8):857–865, 2022

    work page 2022

  4. [4]

    J. J. Caban, J. Yao, U. Bagci, and D. J. Mollura. Monitoring pulmonary fibrosis by fusing clinical, physiological, and computed tomography features. In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 6216–6219. IEEE, 2011

    work page 2011

  5. [5]

    Ceriani, C

    E. Ceriani, C. Combescure, G. Le Gal, M. Nendaz, T. Perneger, H. Bounameaux, A. Perrier, and M. Righini. Clinical prediction rules for pulmonary embolism: a systematic review and meta-analysis. Journal of thrombosis and haemostasis, 8(5):957–970, 2010

    work page 2010

  6. [6]

    Demir, I

    U. Demir, I. Irmakci, E. Keles, A. Topcu, Z. Xu, C. Spampinato, S. Jambawalikar, E. Turkbey, B. Turkbey, and U. Bagci. Information bottleneck attribution for visual explanations of diagnosis and prognosis. In Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27,...

    work page 2021

  7. [7]

    Dentali, W

    F. Dentali, W. Ageno, F. Pomero, L. Fenoglio, A. Squizzato, and M. Bonzini. Time trends and case fatality rate of in-hospital treated pulmonary embolism during 11 years of observation in northwestern italy. Thrombosis and haemostasis, 115(02):399–405, 2016

    work page 2016

  8. [8]

    R. A. Douma, P. W. Kamphuisen, and H. R. Büller. Acute pulmonary embolism. part 1: epidemiology and diagnosis. Nature Reviews Cardiology, 7(10):585–596, 2010

    work page 2010

Show all 18 references
  1. [9]

    Georgilis

    S. Georgilis. Understanding venous thromboembolism and pulmonary embolism. Nursing made Incredibly Easy, 21(2):31–36, 2023

    2023

  2. [10]

    S. Z. Goldhaber, L. Visani, and M. De Rosa. Acute pulmonary embolism: clinical outcomes in the international cooperative pulmonary embolism registry (icoper). The Lancet, 353(9162): 1386–1389, 1999

    1999

  3. [11]

    S. V . Konstantinides, G. Meyer, C. Becattini, H. Bueno, G.-J. Geersing, V .-P. Harjola, M. V . Huisman, M. Humbert, C. S. Jennings, D. Jiménez, et al. 2019 esc guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the european...

    2019

  4. [12]

    Mansoor, U

    A. Mansoor, U. Bagci, and D. J. Mollura. Near-optimal keypoint sampling for fast pathological lung segmentation. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 6032–6035. IEEE, 2014

    2014

  5. [13]

    Mortazi, R

    A. Mortazi, R. Karim, K. Rhode, J. Burt, and U. Bagci. Cardiacnet: Segmentation of left atrium and proximal pulmonary veins from mri using multi-view cnn. In Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC...

    2017

  6. [14]

    Wasserthal, H.-C

    J. Wasserthal, H.-C. Breit, M. T. Meyer, M. Pradella, D. Hinck, A. W. Sauter, T. Heye, D. T. Boll, J. Cyriac, S. Yang, et al. Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence, 5(5), 2023

    2023

  7. [15]

    A. M. Wendelboe and G. E. Raskob. Global burden of thrombosis: epidemiologic aspects. Circulation research, 118(9):1340–1347, 2016

    2016

  8. [16]

    Xue and C

    S. Xue and C. Abhayaratne. Region-of-interest aware 3d resnet for classification of covid-19 chest computerised tomography scans. IEEE Access, 11:28856–28872, 2023

    2023

  9. [17]

    Zheng, A

    B. Zheng, A. Gao, X. Huang, Y . Li, D. Liang, and X. Long. A modified 3d efficientnet for the classification of alzheimer’s disease using structural magnetic resonance images. IET Image Processing, 17(1):77–87, 2023

    2023

  10. [18]

    Zhou, S.-Q

    X.-Y . Zhou, S.-Q. Ben, H.-L. Chen, and S.-S. Ni. The prognostic value of pulmonary embolism severity index in acute pulmonary embolism: a meta-analysis. Respiratory research, 13(1): 1–12, 2012. 8

    2012

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.