Pith. sign in

REVIEW 5 major objections 5 minor 2 references

A scoping review of 190 studies argues that machine-learning pipelines for surgical risk prediction are limited less by algorithms than by private data, unreported preprocessing, and neglected explainability.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:47 UTC pith:7IZGETJS

load-bearing objection Useful scoping review with a solid qualitative map, but the quantitative counts are internally inconsistent and need correction and data release before the paper can be relied on. the 5 major comments →

arxiv 2607.29090 v1 pith:7IZGETJS submitted 2026-07-31 cs.LG

What Is Missing in Surgical Risk Stratification and Outcome Prediction: A Scoping Review of End-to-End Machine Learning Approaches

Yizhi Dong , Yuhe Ke , Hairil Rizal Abdullah , Yucheng Xing , Kevan Kai Bing Teo , Ling Huang , Mengling Feng This is my paper
classification cs.LG
keywords surgical risk stratificationmachine learningelectronic health recordsoutcome predictiondata preprocessingmodel explainabilityclass imbalancescoping review
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that machine-learning-based surgical risk prediction, despite a decade of growth, is methodologically immature. Reviewing 190 studies, it finds that nearly all rely on private data, most from a single center, that key preprocessing decisions such as missing-data handling, feature selection, and class-imbalance correction are frequently unreported, and that only about a third of studies attempt any explainability. It positions itself as the first end-to-end methodological review of the area, treating data, preprocessing, modeling, evaluation, and interpretation as one pipeline. A sympathetic reader would care because these gaps — not model choice — are what currently stop ML risk tools from being reproducible, comparable, and trustworthy in perioperative care.

Core claim

The paper's central finding is a frequency map of the field. Of 190 studies published between 2013 and 2023, 94.7% used private datasets, 62.1% were single-center, 97.9% used preoperative features, and 93.7% used only structured tabular data. Feature selection was reported in 48.4% of studies, missing-data handling in 60.0%, and among studies with class-imbalanced outcomes a majority applied no corrective strategy. Conventional models — regularized generalized linear models, random forests, and gradient boosting — dominated; deep learning and multimodal inputs were rare. ROC-AUC was the most common evaluation metric, but calibration was checked in only about 20% of studies and explainability

What carries the argument

The carrying mechanism is a structured data-extraction form that decomposes every study into data-source characteristics, surgical context, input features, preprocessing steps (imputation, feature selection, imbalance handling), algorithm family, evaluation metrics, and explainability methods, then re-aggregates these into frequency counts and workflow visualizations. This pipeline-level taxonomy is what converts 190 heterogeneous papers into a comparable methodological map of the field.

Load-bearing premise

The entire map rests on the assumption that the 190 studies were chosen and coded without bias or error; if the selection rule or the extraction tables misrepresent the literature, every headline percentage inherits the mistake.

What would settle it

Re-run the review's search without the exclusion of pre-2020 papers with fewer than three citations, retrieve the 70 reports that were sought but not obtained, and recompute the headline percentages; if the shares using private data, reporting preprocessing, or providing explainability shift materially, the central characterization of the field is not stable.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the field follows the paper's recommendations, future studies will need to report missing-data handling, feature selection, and imbalance correction explicitly before their results can be compared.
  • Because no perioperative-specific public dataset was used in any reviewed study, reproducibility depends on creating or adopting shared open datasets and standardized evaluation protocols.
  • Since calibration was assessed in only about one-fifth of studies, models with good discrimination may still mislead clinical decisions; calibration reporting should become a standard requirement.
  • The rarity of deep learning and multimodal inputs implies that near-term gains may come from integrating intraoperative time series, clinical text, and imaging with tabular EHR data.
  • With explainability present in under a third of studies, clinical adoption will remain limited until interpretability is treated as a routine component rather than an afterthought.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the review's percentages imply that dataset release, not algorithm innovation, is the binding constraint; a testable prediction is that studies using newly available open perioperative datasets will show more external validation within a few years.
  • Editorial inference: because the review finds calibration reported so rarely, a practical next step is a reporting checklist that treats calibration as mandatory for any model whose probabilities inform clinical decisions.
  • Editorial inference: the near-total reliance on structured tabular data suggests a concrete experiment — add one additional modality, such as intraoperative vitals or clinical notes, to a baseline tabular model and measure whether discrimination and calibration improve enough to justify the added complexity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper is a PRISMA-ScR scoping review of 190 studies applying machine learning to EHR data for surgical risk stratification and postoperative outcome prediction. The authors characterize the full ML pipeline: data sources, input features, preprocessing (missing data, feature selection, class imbalance), algorithm families, evaluation metrics, and explainability. Their headline findings are that most studies use private, single-center datasets; preprocessing reporting is incomplete; conventional ML and shallow networks dominate; benchmarks are absent; and only about one-third of studies employ explainability. The paper positions itself as the first comprehensive end-to-end methodological review of this area.

Significance. If the reported frequency counts are accurate, this would be a valuable reference map of methodological practice and a useful basis for reporting standards in surgical ML. The qualitative themes — private-data dependence, underreporting of preprocessing, narrow modality use, weak explainability adoption — are plausible and consistent with prior reviews. Strengths include the explicit PRISMA-ScR framing, the broad search across five databases, the structured extraction form, and the attempt to cover the entire ML workflow. However, the central contribution is the quantitative distribution of practices, and several of the reported counts are internally inconsistent. Until the underlying data are released and the arithmetic is reconciled, the specific percentages cannot be relied upon.

major comments (5)
  1. [Section III-E1] The missing-data counts do not add up. The text states that 114 (60.0%) studies reported missing data, then says 22 used complete-case analysis and 93 used imputation; 22 + 93 = 115, not 114. Moreover, the four imputation subcategories are given as 45, 29, 12, and 10, which sum to 96, not 93. Since these counts are the empirical backbone of the review, this inconsistency must be resolved and the corrected numbers reported.
  2. [Section III-E3] The class-imbalance percentages are inconsistent with any stated denominator. The text says 6 studies could not be assessed, leaving 184; yet 131 (71.7%) and 77 (41.9%) are not 131/184 or 77/184 (71.2% and 41.8%), nor fractions of 190. In the same section, 75 of 131 did not correct, implying 56 studies addressed imbalance, but 45 (65.2%) and 11 (15.9%) correspond to a denominator of 69, not 56. These cannot be rounding differences; the denominators need to be explicitly defined and recalculated.
  3. [Section III-G] The ROC-AUC frequency is misreported. The text states that ROC-AUC was reported in 117 studies (93.7%), but 117/190 = 61.6%. Even if the denominator is not all 190 studies, no justification is given for 93.7%. The same paragraph gives specificity as 80 (42.63%), which appears to be a typo for 42.1% and should be corrected. Because the evaluation-metric distribution is a key quantitative finding, these numbers must be corrected with a clear denominator.
  4. [Methods Section II] The exclusion criterion 'papers with fewer than three citations published before 2020' is ad hoc and not justified. Using citation count as a quality proxy is problematic: it systematically excludes older but influential work and niche studies with low citation counts, and it is not a standard PRISMA-ScR criterion. In addition, 70 of 291 sought reports were not retrieved (Fig. 1). The authors should either justify and sensitivity-analyze this exclusion or remove it, and should discuss the potential bias from the 70 non-retrieved reports.
  5. [Section IV-C] The taxonomy of algorithms contains a stated factual error: the Discussion calls SVMs 'a type of instance-based learning method.' In Section III-F, SVM is correctly placed under supervised learning, while instance-based learning is reserved for KNN. This contradiction is confusing and should be corrected in revision.
minor comments (5)
  1. [Abstract/III-G] 'Specificity (80, 42.63%)' contains an extra decimal; should be 42.1% if the denominator is 190.
  2. [Funding] 'This work is particially supported' should read 'partially supported.'
  3. [Section III-E2] The feature-selection method percentages (50.0%, 32.6%, 27.2%, 25.0%) sum to 134.8%. This is likely because a study can use multiple methods, but the text does not say so. Please state that categories are not mutually exclusive, or revise the counts.
  4. [Section III-H] Explainability technique counts also appear to overlap: SHAP 26, impurity 16, coefficient 10, perturbation 3 sum to 55, while the text says 56 studies reported explainability. Clarify whether the deep-learning explainability count of 3 is in addition to or included in these categories.
  5. [General] The authors should consider making the full extraction data publicly available as a supplement; this would allow readers to verify all frequency counts and would materially strengthen the reproducibility of the review.

Circularity Check

0 steps flagged

No significant circularity: the review's descriptive corpus synthesis is externally grounded; arithmetic inconsistencies are correctness risks, not circular derivation.

full rationale

This is a scoping review, not a derivation or prediction pipeline. Its central output—characterizing 190 studies with frequency counts of data source, preprocessing, algorithm, evaluation, and explainability practices—is assembled from published primary studies through PRISMA-ScR search, screening, and extraction (Section II, Fig. 1). No parameter is fitted to a subset and then used to predict a related quantity; no equation is derived from an output it is supposed to explain; the 'first comprehensive methodological review' framing is a scope claim, not a reduced-to-input result. The only potentially self-referential item is the recommendation of INSPIRE ([83], Discussion IV-A), which may involve the authors, but it is a forward-looking suggestion and is not load-bearing for the review's empirical findings. The manuscript does contain demonstrable numerator/denominator inconsistencies (e.g., Section III-E1: 22 complete-case + 93 imputation = 115 vs stated 114; Section III-G: ROC-AUC 117 (93.7%) versus 117/190 = 61.6%; Section III-E3: imbalance percentages incompatible with 184 or 190 denominator). Those are serious accuracy/reporting defects and a threat to the review's quantitative backbone, but they are not circularity: the counts are descriptive extractions with no self-referential equivalence between input and output. The reviewer rule to flag limitations is therefore discharged by naming these as correctness risks, not circular steps.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper's main contribution is descriptive, not derivational, so the ledger is short: two hand-chosen thresholds (citation-count and imbalance prevalence), two assumptions about search completeness and reporting accuracy, and one ad-hoc exclusion criterion.

free parameters (2)
  • Citation count exclusion threshold = <3 citations for papers published before 2020
    Arbitrary inclusion criterion that may exclude older relevant studies, affecting the representativeness of the 190-study corpus (Methods, Section II).
  • Class imbalance threshold = minority prevalence <20%
    Hand-chosen definition of class imbalance used to compute prevalence statistics; a different threshold would change reported rates (Results III-E.3).
axioms (3)
  • domain assumption The five databases (PubMed, Embase, IEEE Xplore, ScienceDirect, Google Scholar) and the reported search terms retrieve all relevant ML-for-surgical-risk studies
    A scoping review is only as complete as its search; the exact per-database query strings are not reported (Methods II).
  • domain assumption The 190 included studies' self-reported methods and performance values are accurate and not misleading
    The review does not independently validate the original studies, nor does it conduct risk-of-bias assessment (Results III; Discussion IV).
  • ad hoc to paper The exclusion criterion 'papers with fewer than three citations published before 2020' is a valid quality proxy
    This is an unusual, paper-specific rule with no evidence that citation count tracks methodological quality (Methods II).

pith-pipeline@v1.3.0-daily-deepseek · 20758 in / 13162 out tokens · 121842 ms · 2026-08-03T13:47:47.457226+00:00 · methodology

0 comments
read the original abstract

Postoperative adverse events, including mortality and morbidity, remain a major global burden, many of which are preventable through early identification of high-risk patients and targeted perioperative care. Accurate risk stratification is therefore essential. With the growing availability of large-scale electronic health records (EHRs), machine learning (ML) provides a data-driven approach to model complex clinical patterns. However, existing studies vary widely in design, and methodological practices remain fragmented. This scoping review characterizes ML pipelines for surgical risk stratification and outcome prediction using EHR data. We reviewed 190 studies covering the ML workflow, including data preprocessing, algorithm selection, model evaluation, and explainability. Most studies relied on single-center private datasets with limited data modalities, while the scarcity of open-access surgical datasets constrained reproducibility and generalizability. Reporting of key preprocessing steps, including missing data handling, feature selection, and class imbalance, was often incomplete. Conventional ML models and simple neural networks predominated, whereas deep learning and multimodal approaches remained uncommon. Benchmark datasets and standardized evaluation protocols were largely absent, hindering cross-study comparisons. Only about one-third of studies incorporated explainability methods. This review identifies methodological gaps limiting clinically robust postoperative ML tools and provides a structured reference to support more rigorous, reproducible, and clinically meaningful ML development for perioperative care.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

2 extracted references

  1. [3]

    Surgical adverse events: a systematic review,

    O. Anderson, R. Davis, G. B. Hanna, and C. A. Vincent, “Surgical adverse events: a systematic review,” Am. J. Surg., vol. 206, no. 2, pp. 253–262, Aug. 2013, doi: 10.1016/j.amjsurg.2012.11.009. [4] S. Jhanji, B. Thomas, A. Ely, D. Watson, C. J. Hinds, and R. M. Pearse, “Mortality and utilisation of critical care resources amongst high-risk surgical patien...

    arXiv 2013

  2. [27]

    MIMIC-IV, a freely accessible electronic health record dataset,

    A. E. W. Johnson et al., “MIMIC-IV, a freely accessible electronic health record dataset,” Sci. Data, vol. 10, no. 1, p. 1, Jan. 2023, doi: 10.1038/s41597-022-01899-x. [28] HCUP National Inpatient Sample (NIS). Healthcare Cost and Utilization Project (HCUP). 2012. Agency for Healthcare Research and Quality, Rockville, MD. hcup-us.ahrq.gov/nisoverview.jsp ...

    arXiv 2023