Pith. sign in

REVIEW 4 major objections 7 minor 26 references

Interpretable Artificial Intelligence for Detecting Acute Heart Failure on Acute Chest CT Scans

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper develops a Boosted Trees model that predicts radiological signs of acute heart failure from segmented cardiac and pulmonary structures on acute chest CT, reaching AUROC 0.87 on an independent test set.

desk verdict A solid, honest internal validation of an explainable CT-based AHF detector, but the radiologist-comparability claim is not tested and the report-derived labels leave the AUROC uncertain. read the letter →

arxiv 2507.08952 v1 pith:SIZIFJ2Y submitted 2025-07-11 eess.IV cs.CV

Silas Nyboe {O}rting , Kristina Miger , Anne Sophie Overgaard Olesen , Mikael Ploug Boesen , Michael Brun Andersen , Jens Petersen , Olav W. Nielsen , Marleen de Bruijne This is my paper · ORCID
classification eess.IVcs.CV
keywords acuteheartfailurechestCTexplainableAIXGBoostTotalSegmentatorSHAPvaluesradiologyreportsemergencytriage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that an explainable machine-learning model can detect radiological signs of acute heart failure (AHF) on acute chest CT scans with accuracy comparable to thoracic radiologists. Using measurements of segmented cardiac and pulmonary structures from 4,672 patients, a Boosted Trees model achieves an AUROC of 0.87 on an independent test set. The authors' own review of errors found that 38% of the model's apparent mistakes were actually correct predictions whose labels were wrong, suggesting the reported performance may understate the model's accuracy. If correct, the model could flag likely AHF during emergency triage, before the formal radiology report is available.

What carries the argument

The machinery is a Boosted Trees classifier (XGBoost) fed with measurements extracted by TotalSegmentator segmentation of cardiac and pulmonary structures. The final model incorporates twelve features that survive forward feature selection and manual pruning, all directly tied to the pathophysiology of heart failure. Model transparency comes from SHAP values, which assign each feature a contribution to the predicted log odds of AHF. Missing segmentation-derived values are handled as possible splits inside the trees rather than imputed, and predictions from multiple CT acquisitions and reconstructions are averaged at the study level.

What would settle it

Take the independent test set and have a panel of thoracic radiologists re-score each CT for AHF signs using a structured protocol, masking the original reports; if the model's AUROC against these standardized labels falls to near chance (for example around 0.7 or below), the central claim of radiologist-comparable accuracy would be refuted.

Watch

Extended reading notes

Core claim

The central discovery is that a small set of twelve pathophysiologically relevant features—such as pleural effusion volume and ratio, left and right atrial volumes, right ventricle ratio, mean lung boundary density, vena cava inferior diameter and density, and a heart-size $Z$-score—carries enough signal to predict AHF signs on acute chest CT with AUROC 0.87, comparable to clinical radiologists. The model is not a black box: because it is a tree ensemble trained on segmented anatomy, each prediction can be traced through the feature values and visualized with SHAP, and the segmentation overlays can be reviewed directly on the scan.

Load-bearing premise

The model's accuracy is measured against labels mined from radiology reports, where scans without any AHF keyword are assumed negative; if those labels are biased or underreport AHF signs, the AUROC of 0.87 may not reflect the model's true diagnostic performance on the scans.

Editorial extensions

If this is right

  • An emergency physician could see an AHF probability before the formal radiology report, potentially triggering earlier cardiology assessment and treatment.
  • Because each prediction is explained by SHAP feature contributions and visible segmentations, clinicians can audit which anatomical measurement drove the result instead of accepting a black-box output.
  • At the chosen threshold (a 5% false-positive rate on training data), the model detects about half of the test cases labeled AHF; lowering the threshold increases sensitivity at the cost of more false alarms.
  • Since 38% of the model's errors were traced to incorrect report labels, the measured specificity and sensitivity against report-derived labels likely underestimate performance against a standardized radiological review.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to use the model as a second reader that flags scans whose reports may have underreported AHF signs, turning label error into a quality-control signal for radiology reporting.
  • The sex difference in error patterns hinted by the data suggests separate thresholds or sex-specific features may be needed in deployment; the paper only notes the trend, so testing this is a next step.
  • The model's design—off-the-shelf segmentation plus a small tree ensemble—means it could be re-trained on other centers' scans with modest computational cost, provided the report-labeling caveat is addressed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper develops an explainable machine-learning model (XGBoost with TotalSegmentator-derived cardiac and pulmonary measurements) to detect radiological signs of acute heart failure on acute chest CT scans. Using report-derived labels from a single-center retrospective cohort (4,672 subjects), the final twelve-feature model achieves an internal test AUROC of 0.87. The authors also report an expert radiologist error review of 126 misclassifications, finding that roughly 38% of errors were label errors. The central claim is that the model's performance is 'comparable to thoracic radiologists' and that its stepwise, SHAP-based explanations make it useful for triage before the formal radiology report is available.

Significance. If the result holds, the work has practical value: it demonstrates that fully automated extraction of a small set of physiologically motivated CT measurements can flag AHF with reasonably good discrimination in an undifferentiated emergency population, and the use of off-the-shelf segmentation plus tree ensembles lowers the barrier to clinical deployment. The authors are transparent about many limitations, including single-center data, report-derived labels, and the absence of external validation. The strongest assets are the subject-level split, threshold fixed on training data, manual review of positive/negative keyword labels, and the inclusion of an error-review effort. However, the headline claim of comparability to thoracic radiologists is not directly tested, and the reference standard is noisy; these issues are load-bearing for the stated conclusions.

major comments (4)
  1. [3.2, 3.5, 4.3] The ground-truth labels are derived from radiology reports via regular expressions, and reports without any AHF-related keyword are assumed negative. The authors themselves find in the error review that 24/64 false positives and 24/61 false negatives were actually correct model predictions, i.e., 38% of errors are label errors. This makes the reported AUROC of 0.87 an estimate against a noisy reference standard. If the label noise is nondifferential, the true AUROC may be higher, but if it is differential (e.g., certain report styles or patient subgroups underreport AHF signs), the model may be learning associations with report content rather than imaging signs. Because the error review only inspects cases where the model disagrees with the report, it cannot correct the AUROC or quantify overall label accuracy. The authors should re-adjudicate a random sample (ideally all) of the test set with a radiologist blinded to the original report and model prediction, and report the model's AUROC, calibration, and sensitivity/specificity against that adjudicated reference. This would also allow a sensitivity analysis comparing performance on adjudicated versus report-derived labels.
  2. [1, 4.1, Abstract] The claim that the model is 'comparable to thoracic radiologists' is not supported by any direct comparison. No radiologist AUC, sensitivity, specificity, or inter-observer variability on the same cohort is provided. The only radiologist involvement is the error review of misclassified cases, which is not a reader study and cannot establish equivalence. To support the central claim, the authors should either (a) add a reader study in which multiple radiologists independently score a random subset (or all) of the test CTs for AHF likelihood, then compute and statistically compare their ROC curves with the model's, or (b) substantially temper the wording to 'performance in a range consistent with reported radiologist accuracy' and provide explicit literature benchmarks. As written, the abstract and conclusion overstate the evidence.
  3. [3.5] The radiologist error review appears to be unblinded: the text does not state whether the reviewing radiologist was blinded to the model prediction and to the original report label. Without blinding, the estimate that 38% of errors are label errors is vulnerable to confirmation bias, which would directly undermine the authors' interpretation that the model's true accuracy is better than the report-based metrics suggest. The authors should state whether blinding was used and, if not, re-run the review with a blinded protocol or acknowledge this as a serious limitation of the error-analysis result.
  4. [2.5, 3.1] The data-partition description is ambiguous regarding multiple studies per subject. The text says 'For test subjects with multiple studies at different time points, only the latest study was included,' but it does not say whether the same restriction applies to training subjects. Since the total dataset includes 5,111 studies for 4,672 subjects, some subjects contribute more than one study. If training includes all studies per subject while test includes only the latest, the training and test distributions may differ (e.g., in disease severity or scan context), and the threshold calibrated on training data may be mismatched. Please report the number of studies per subject in train and test, and justify the asymmetry or apply the same policy to both partitions.
minor comments (7)
  1. [3.5, Table 2] The text reports 62 false negatives in the test set, but the error-review paragraph says '24 of 61 false negatives' and Table 3 shows 61 false negatives. Clarify that one case was not reviewed due to a PACS issue, and reconcile the counts.
  2. [2.3-2.5] Section numbering jumps from 2.3 to 2.5; a subsection 2.4 appears to be missing or the numbering is misaligned. Please renumber.
  3. [Figure 5] The caption says 'blue errors' where it should say 'blue arrows' when describing features contributing to a negative prediction.
  4. [4.2] The citation sequence '(8–10)(9)' appears to be a formatting error; it should likely be '(8–10)' with appropriate citation placement.
  5. [2.7] The term 'density' is used for measurements that are later described in Hounsfield units (HU). Define the unit at first use and use HU consistently throughout.
  6. [Supplementary Table S1] The 'No decompensation' regular expression is split across two lines in the table, making it difficult to read as a single pattern; format it as one code block.
  7. [6] The trained model is not made publicly available, which limits reproducibility of the exact reported results. Please state explicitly that model weights are available upon reasonable request, or provide a minimal reproducible example with features and code (even if the data cannot be shared).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the model is an empirical supervised learner evaluated on a subject-level held-out test set; report-derived label noise is a data-quality limitation, not a circularity.

full rationale

The paper is an empirical supervised-learning study rather than a derivation. Ground-truth labels are extracted from radiology reports (Section 2.3), and the model is trained to predict those labels from CT-derived measurements (Section 2.8); evaluation is performed on a subject-level held-out test set with the decision threshold fixed on training data (Sections 2.5, 2.9, 3.5). The reported AUROC of 0.87 is therefore an out-of-sample empirical estimate, not a quantity forced by construction. The report-derived labels may be noisy—the error review found 38% of errors were label errors (Sections 3.5 and 4.3)—but noisy labels are a measurement-quality concern rather than circularity, because the model could still succeed or fail at predicting those labels. Feature selection references the authors' prior CT-features study (reference 5), but that prior study was an independent prospective observational study with published empirical content; this citation is not load-bearing in the sense of importing an unverified uniqueness theorem or ansatz that forbids alternatives. No equation or definition reduces a predicted quantity to a fitted parameter, and no self-citation chain forces the model's choices. Hence no circumspect circular steps are present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests primarily on the quality of the report-derived labels and the segmentation-derived measurements. The model itself is standard supervised machine learning; no new physical entities are introduced. The main free parameters are the operating threshold, tuned hyperparameters, and normalization statistics estimated from the training cohort, all of which are standard but should be disclosed.

free parameters (4)
  • Decision threshold t = 0.26840377
    Chosen as the cutoff yielding a 5% false positive rate on training data; all test-set confusion metrics depend on it.
  • XGBoost hyperparameters = eta=0.2, max_depth=1, alpha=4, subsample=1.0 (final)
    Selected by grid search on training data using 3-fold cross-validation; they define the final model.
  • Modified Z-score normalization statistics = Median and MAD of each volume in the training cohort
    Volumes are standardized to modified Z scores; the location and scale are estimated from the data, so the scores are not parameter-free.
  • Lung boundary band width = 10 mm
    The lung boundary feature is defined as a 10 mm band next to the lung boundary; this width is chosen by the authors and affects density features.
assumptions (4)
  • domain assumption Radiology reports, after regex extraction and manual correction, provide valid ground truth for AHF signs.
    The study uses report-derived labels as the outcome for both training and evaluation; reports without any AHF keyword are assumed negative, which can mislabel patients whose findings were underreported. This is acknowledged in Section 4.3.
  • domain assumption TotalSegmentator segmentations are accurate enough for feature extraction in non-contrast and heterogeneous CT scans.
    Features are computed from segmentations; the authors visually inspected segmentations and found acceptable quality for lungs but noted failures for small hearts, small VCI, and pulmonary artery, and boundaries between heart substructures are not visible in non-contrast scans (Section 3.3).
  • domain assumption The 12 final features capture the radiological signs of AHF.
    Features were selected from a prior study and physiological reasoning, then pruned using SHAP values; the model depends on these being informative and measured correctly.
  • standard math Standard ML assumptions for XGBoost and SHAP.
    The paper relies on the correctness of XGBoost's boosting, feature importance, and SHAP explanations as implemented in the cited libraries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpretable Artificial Intelligence for Detecting Acute Heart Failure on Acute Chest CT Scans." pith.science (2026). https://pith.science/paper/SIZIFJ2Y

@misc{pith2026250708952,
  author       = {Pith},
  title        = {Pith review of: Interpretable Artificial Intelligence for Detecting Acute Heart Failure on Acute Chest CT Scans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SIZIFJ2Y}},
  note         = {Machine review of arXiv:2507.08952}
}
read the original abstract

Introduction: Chest CT scans are increasingly used in dyspneic patients where acute heart failure (AHF) is a key differential diagnosis. Interpretation remains challenging and radiology reports are frequently delayed due to a radiologist shortage, although flagging such information for emergency physicians would have therapeutic implication. Artificial intelligence (AI) can be a complementary tool to enhance the diagnostic precision. We aim to develop an explainable AI model to detect radiological signs of AHF in chest CT with an accuracy comparable to thoracic radiologists. Methods: A single-center, retrospective study during 2016-2021 at Copenhagen University Hospital - Bispebjerg and Frederiksberg, Denmark. A Boosted Trees model was trained to predict AHF based on measurements of segmented cardiac and pulmonary structures from acute thoracic CT scans. Diagnostic labels for training and testing were extracted from radiology reports. Structures were segmented with TotalSegmentator. Shapley Additive explanations values were used to explain the impact of each measurement on the final prediction. Results: Of the 4,672 subjects, 49% were female. The final model incorporated twelve key features of AHF and achieved an area under the ROC of 0.87 on the independent test set. Expert radiologist review of model misclassifications found that 24 out of 64 (38%) false positives and 24 out of 61 (39%) false negatives were actually correct model predictions, with the errors originating from inaccuracies in the initial radiology reports. Conclusion: We developed an explainable AI model with strong discriminatory performance, comparable to thoracic radiologists. The AI model's stepwise, transparent predictions may support decision-making.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 25 canonical work pages

  1. [1]

    Department of Computer Science, University of Copenhagen, Denmark

    work page

  2. [2]

    Department of Cardiology, Copenhagen University Hospital – Bispebjerg and Frederiks- berg, Denmark

    work page

  3. [3]

    Department of Clinical Medicine, University of Copenhagen, Denmark

    work page

  4. [4]

    Department of Radiology, Copenhagen University Hospital – Bispebjerg and Frederiks- berg, Denmark

    work page

  5. [5]

    Department of Radiology, Copenhagen University Hospital – Herlev Gentofte, Denmark

    work page

  6. [6]

    ‡: These authors contributed equally as last authors

    Department of Radiology and Nuclear Medicine, Erasmus MC – University Medical Cen- ter Rotterdam, Rotterdam, The Netherlands †: These authors contributed equally as first authors. ‡: These authors contributed equally as last authors. Summary statement: Subjects at risk of acute heart failure can be detected using an explainable tree-based model trained on...

    work page

  7. [7]

    We developed an explainable artificial intelligence model for detecting radiological signs of acute heart failure on acute chest CT scans

    work page

  8. [8]

    The model demonstrated good discriminatory value, comparable to clinical radiologists

    work page

Show all 26 references
  1. [9]

    Abstract Introduction: Chest CT scans are increasingly used in dyspneic patients where acute heart failure (AHF) is a key differential diagnosis

    The combination of key features with a tree-based model and visual segmentations allows for transparent, step-by-step predictions that can easily be interpreted by physicians. Abstract Introduction: Chest CT scans are increasingly used in dyspneic patients where acute heart fa...

    2016

  2. [10]

    It is a major cause of emergency hospital admissions worldwide, and a leading cause of hospitaliza- tions in elderly patients

    Introduction: Acute heart failure (AHF) is a severe health condition with high mortality and rehospitalization rates. It is a major cause of emergency hospital admissions worldwide, and a leading cause of hospitaliza- tions in elderly patients. Early and accurate diagnosis is ...

  3. [11]

    2.1 Study Design A single-center, retrospective study using CT scans to develop an AI model for detecting AHF (Figure 1)

    Materials and Methods We trained a Boosted Trees(4) model to predict AHF based on measurements of cardiac and pul- monary structures. 2.1 Study Design A single-center, retrospective study using CT scans to develop an AI model for detecting AHF (Figure 1). 2.2 Data We included ...

    2016

  4. [12]

    congestion

    Results 8 3.1 Data Inclusion and exclusion are summarized in Figure 2. We extracted radiology reports covering 6101 studies of 5479 subjects. We excluded 157 subjects due to incomplete reports, and 234 subjects due to being included in the FACTUAL study. In total, we included ...

  5. [13]

    The model performed well, achieving an AUROC of 0.87—comparable to assessments by thoracic radiologists

    Discussion 4.1 Summary We developed and internally validated an explainable AI model to detect radiological signs of AHF on emergency chest CT scans. The model performed well, achieving an AUROC of 0.87—comparable to assessments by thoracic radiologists. It relies on a small s...

  6. [14]

    Diagnostik med kunstig intelligens på CT hos patienter med åndenød: udvikling og valid- ering af maskinlæringsalgoritmer (Breath-CT)

    Ethics This study (“Diagnostik med kunstig intelligens på CT hos patienter med åndenød: udvikling og valid- ering af maskinlæringsalgoritmer (Breath-CT)”) is approved by The Danish Research Ethics Commit- tee (Project-id: 1575037), which also waived informed consent. The study...

    2020

  7. [15]

    The Boosted Trees model is available from https://github.com/dmlc/xgboost

    Availability of software and data TotalSegmentator is available from https://github.com/wasserth/TotalSegmentator. The Boosted Trees model is available from https://github.com/dmlc/xgboost. The data is not made publicly available. The trained model is not made publicly availab...

  8. [16]

    2021 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure

    McDonagh TA, Metra M, Adamo M, Gardner RS, Baumbach A, Böhm M, et al. 2021 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur Heart J. 2021;42(36):3599–726

    2021

  9. [17]

    National trends in use of computed tomography in the emergency department

    Kocher KE, Meurer WJ, Fazel R, Scott PA, Krumholz HM, Nallamothu BK. National trends in use of computed tomography in the emergency department. Ann Emerg Med [Internet]. 2011;58(5):452-462.e3. Available from: http://dx.doi.org/10.1016/j.annemergmed.2011.05.020

    2011 doi

  10. [18]

    Impact of Artificial Intelligence Assistance on Chest CT Interpretation Times: A Prospective Randomized Study

    Yacoub B, Varga-Szemes A, Schoepf UJ, Kabakus IM, Baruah D, Burt JR, et al. Impact of Artificial Intelligence Assistance on Chest CT Interpretation Times: A Prospective Randomized Study. AJR Am J Roentgenol. 2022 Nov;219(5):743–51

    2022

  11. [19]

    Xgboost: A scalable tree boosting system

    Chen T, Guestrin C. Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 2016. p. 785– 94

    2016

  12. [20]

    Chest computed tomography features of heart failure: A prospective observational study in patients with acute dyspnea

    Miger K, Fabricius-Bjerre A, Overgaard Olesen AS, Sajadieh A, Høst N, Køber N, et al. Chest computed tomography features of heart failure: A prospective observational study in patients with acute dyspnea. Cardiol J. 2022;29(2):235–44

    2022

  13. [21]

    TotalSegmentator: robust segmentation of 104 anatomical structures in CT images

    Wasserthal J, Meyer M, Breit H-C, Cyriac J, Yang S, Segeroth M. TotalSegmentator: robust segmentation of 104 anatomical structures in CT images. 2022 Aug 11 [cited 2023 Mar 28]; Available from: http://arxiv.org/abs/2208.05868

    2022 arXiv

  14. [22]

    From Local Explanations to Global Understanding with Explainable AI for Trees

    Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From Local Explanations to Global Understanding with Explainable AI for Trees. Nat Mach Intell. 2020 Jan;2(1):56–67. 15

    2020

  15. [23]

    Artificial Intelligence in Cardiovascular CT and MR Imaging

    Lanzafame LRM, Bucolo GM, Muscogiuri G, Sironi S, Gaeta M, Ascenti G, et al. Artificial Intelligence in Cardiovascular CT and MR Imaging. Life (Basel, Switzerland). 2023 Feb;13(2)

    2023

  16. [24]

    Subclinical Pulmonary Congestion and Abnormal Hemodynamics in Heart Failure With Preserved Ejection Fraction

    Jain CC, Tschirren J, Reddy YN V, Melenovsky V, Redfield M, Borlaug BA. Subclinical Pulmonary Congestion and Abnormal Hemodynamics in Heart Failure With Preserved Ejection Fraction. JACC Cardiovasc Imaging. 2022 Apr;15(4):629–37

    2022

  17. [25]

    Development of computer-aided model to differentiate COVID-19 from pulmonary edema in lung CT scan: EDECOVID-net

    Velichko E, Shariaty F, Orooji M, Pavlov V, Pervunina T, Zavjalov S, et al. Development of computer-aided model to differentiate COVID-19 from pulmonary edema in lung CT scan: EDECOVID-net. Comput Biol Med. 2022 Feb;141:105172

    2022

  18. [26]

    Text extraction

    Brady AP, Allen B, Chong J, Kotter E, Kottler N, Mongan J, et al. Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi- Society Statement From the ACR, CAR, ESR, RANZCR & RSNA. Can Assoc Radiol J [Internet]. 2024; Availab...

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.