pith. sign in

arxiv: 2605.03560 · v1 · submitted 2026-05-05 · 💻 cs.LG

Enhance the after-discharge mortality rate prediction via learning from the medical notes

Zijiang Yang This is my paper

Pith reviewed 2026-05-07 17:08 UTC · model grok-4.3

classification 💻 cs.LG
keywords medical notesmortality predictiondeep learningpooling mechanismelectronic health recordsAUC-ROCafter-dischargepatient severity
0
0 comments X

The pith

Medical notes improve after-discharge mortality prediction, with a pooling DNN raising AUC-ROC by 2-14% over traditional models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that unstructured medical notes from electronic health records contain useful signals for predicting patient mortality after leaving the hospital. Models that include these notes achieve AUC-ROC values about 0.1 higher than models that ignore them. The authors introduce a deep neural network equipped with a pooling mechanism that focuses on the most relevant notes and outperforms standard machine learning techniques such as tree-based models. This improvement holds for predictions at 15, 30, 60, and 365 days post-discharge. The approach also uncovers links between certain words in the notes and the seriousness of patient conditions.

Core claim

Medical notes are informative for after-discharge mortality rate prediction tasks, as evidenced by a general 0.1 increase in AUC-ROC when included in models. The proposed deep neural network model incorporating a pooling mechanism outperforms traditional machine learning models like tree-based ones, with AUC-ROC improvements ranging from 2% to 14% across 15-days, 30-days, 60-days, and 365-days prediction horizons. The models further reveal relationships between informative keywords in the notes and patient severity.

What carries the argument

The pooling mechanism within the deep neural network, which learns to prioritize the most informative medical notes while handling their messy and redundant nature.

If this is right

  • Incorporating medical notes leads to higher accuracy in mortality predictions compared to using only structured data.
  • The pooling DNN consistently surpasses tree-based models in the specified time-frame tasks.
  • Analysis of the models highlights connections between note keywords and patient severity levels.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar pooling techniques could be tested on other clinical prediction tasks using text data, such as readmission or complication risks.
  • Integration into hospital systems might enable earlier interventions for high-risk patients identified from notes.
  • Further work could examine whether these improvements hold across different healthcare institutions or note-taking styles.

Load-bearing premise

The low-quality medical notes still embed extractable predictive information that the pooling step can isolate reliably.

What would settle it

If the performance gains disappear when the model is tested on notes from a separate hospital or when note content is randomly shuffled while keeping labels, the claim that notes provide genuine signals would be challenged.

Figures

Figures reproduced from arXiv: 2605.03560 by Zijiang Yang.

Figure 1
Figure 1. Figure 1: Flow chart for medical notes representation view at source ↗
Figure 2
Figure 2. Figure 2: AUC-ROC curve on 30-days mortality prediction using multiple machine learn￾ing classifiers without medical notes It is reported that these 3 models have very similar model accuracy. Based on the result, we can see that these models cannot deliver a good result. The AUC is just around 0.58, which barely outperforms the random guessing. 4.2 Baseline model using medical notes information By tokenizing the med… view at source ↗
Figure 3
Figure 3. Figure 3: AUC-ROC curve on 15-days mortality prediction using multiple machine learn￾ing classifier with medical notes view at source ↗
Figure 4
Figure 4. Figure 4: AUC-ROC curve on 30-days mortality prediction using multiple machine learn￾ing classifier with medical notes view at source ↗
Figure 5
Figure 5. Figure 5: AUC-ROC curve on 60-days mortality prediction using multiple machine learn￾ing classifier with medical notes view at source ↗
Figure 7
Figure 7. Figure 7: XGBoost feature importance on 30-days mortality prediction using basic infor￾mation only view at source ↗
Figure 8
Figure 8. Figure 8: XGBoost feature importance on 30-days mortality prediction using basic infor￾mation and medical notes view at source ↗
read the original abstract

With the increase of the Electronic Health Records (EHR) data, more and more researchers are developing machine learning models to learn from the medical notes. These unstructured text data pose significant challenges on the learning process as the quality of data is low. These data are often messy, repetitive and redundant. We have shown these notes data to be informative by conducting the after-discharge mortality rate prediction task. The AUC-ROC for models using the medical note information is generally 0.1 higher than those without the medical notes. Furthermore, we propose the Deep Neural Network(DNN) model with 'pooling' mechanism to enhance the mortality prediction. Based on the experimental results, we demonstrate that the proposed model outperforms the traditional machine learning models like the tree-based models. The proposed method learns from the most informative medical notes and improves the prediction accuracy significantly. The AUC-ROC for the proposed model is 2% to 14% higher than the traditional ones in 15-days, 30-days, 60-days, 365-days after-discharge mortality prediction tasks. Moreover, we can discover some interesting knowledge through the traditional and proposed models. These knowledge are inspiring but also consistent with the previous findings. The models are able to reveal the relationships between the informative keywords and documents from the medical notes and the severity of the patients.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper claims that unstructured medical notes in EHR data contain extractable predictive signals for after-discharge mortality. It reports that models incorporating note information achieve ~0.1 higher AUC-ROC than those without notes, and proposes a DNN with a 'pooling' mechanism that outperforms tree-based baselines by 2-14% AUC-ROC across 15-, 30-, 60-, and 365-day prediction horizons. The work also asserts that the models reveal interpretable relationships between note keywords and patient severity.

Significance. If the empirical claims hold under rigorous validation, the work could advance the use of noisy unstructured EHR text for mortality risk stratification, offering a practical approach to handling low-quality notes via pooling. This would be relevant for clinical ML applications, but the absence of dataset details, controls, and ablations currently limits its contribution to the field.

major comments (3)
  1. [Abstract] Abstract: The central performance claims (0.1 AUC lift with notes; 2-14% gains over tree models) are presented without any dataset size, train/test split details, statistical significance tests, class-imbalance handling, or note-quality preprocessing steps. These omissions render the reported improvements unverifiable and constitute a load-bearing gap for the empirical contribution.
  2. [Proposed Model] Proposed DNN with pooling: The 'pooling' mechanism is described only at a high level with no architecture diagram, layer specifications, regularization, or ablation studies (e.g., shuffled-note controls or attention maps). Without these, it is impossible to determine whether the model extracts genuine mortality signals or fits to noise/artifacts in the messy, repetitive notes.
  3. [Experimental Results] Experimental results: No information is supplied on baseline hyperparameter search, cross-validation strategy, or handling of potential data leakage, which directly undermines the claim that the DNN reliably outperforms traditional models.
minor comments (2)
  1. [Abstract] The abstract and text use vague phrasing such as 'generally 0.1 higher' and '2% to 14% higher' without specifying exact values per task or confidence intervals.
  2. [Introduction] No references are provided to prior work on text pooling in EHR or mortality prediction benchmarks, making it difficult to situate the novelty.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments, which highlight important areas for improving the clarity and rigor of our work. We have revised the manuscript to address each major concern by adding the requested details on datasets, model specifications, and experimental procedures. Point-by-point responses follow.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central performance claims (0.1 AUC lift with notes; 2-14% gains over tree models) are presented without any dataset size, train/test split details, statistical significance tests, class-imbalance handling, or note-quality preprocessing steps. These omissions render the reported improvements unverifiable and constitute a load-bearing gap for the empirical contribution.

    Authors: We agree that these details are essential for verifiability. In the revised manuscript, we have expanded the abstract and added a dedicated 'Data and Preprocessing' subsection that reports the exact dataset size (number of patients, admissions, and notes), the train/test split (temporal 70/15/15 hold-out to respect time order), class-imbalance handling (class-weighted cross-entropy loss), note-quality steps (deduplication, removal of boilerplate sections, lower-casing and stop-word filtering), and statistical significance (DeLong tests with p-values for all AUC comparisons). These additions directly resolve the gap. revision: yes

  2. Referee: [Proposed Model] Proposed DNN with pooling: The 'pooling' mechanism is described only at a high level with no architecture diagram, layer specifications, regularization, or ablation studies (e.g., shuffled-note controls or attention maps). Without these, it is impossible to determine whether the model extracts genuine mortality signals or fits to noise/artifacts in the messy, repetitive notes.

    Authors: We acknowledge the description was insufficient. The revised 'Proposed Model' section now includes a full architecture diagram, precise layer specifications (embedding size 128, multi-head attention pooling followed by two 256-unit ReLU layers), regularization (0.3 dropout and 1e-4 L2), and new ablation results: (i) shuffled-note controls showing a 0.07–0.12 AUC drop, confirming the model relies on genuine content rather than artifacts, and (ii) attention heatmaps highlighting mortality-linked keywords. These experiments demonstrate that the pooling mechanism extracts predictive signals. revision: yes

  3. Referee: [Experimental Results] Experimental results: No information is supplied on baseline hyperparameter search, cross-validation strategy, or handling of potential data leakage, which directly undermines the claim that the DNN reliably outperforms traditional models.

    Authors: We have substantially expanded the 'Experiments' section. It now details the hyperparameter search (grid search over tree depth, learning rate, and batch size with 5-fold inner CV), the outer cross-validation strategy (stratified 5-fold on the training partition with temporal ordering), and explicit leakage controls (notes are truncated to discharge time; no post-discharge information is used). Standard deviations across folds and full hyperparameter tables are reported. These additions substantiate the reliability of the 2–14% gains. revision: yes

Circularity Check

0 steps flagged

No significant circularity in empirical ML evaluation

full rationale

The paper is an empirical supervised learning study that trains models on medical notes to predict after-discharge mortality and reports AUC-ROC lifts on held-out data. No derivation chain, equations, or first-principles result is presented that reduces by construction to its own inputs. Claims rest on experimental comparisons against baselines (tree models) and internal ablations, all evaluated on separate test sets. Hyperparameter tuning introduces the usual dependence on training distribution, but this is standard for ML and does not constitute circularity under the defined patterns. The work is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

Central claim rests on the domain assumption that notes carry mortality signal and on standard ML fitting procedures; no new physical entities or ad-hoc constants are introduced beyond typical neural-network hyperparameters.

free parameters (1)
  • DNN architecture and pooling hyperparameters
    Number of layers, pooling size, learning rate, and regularization chosen or tuned to achieve reported AUC gains.
axioms (1)
  • domain assumption Medical notes contain extractable predictive information for post-discharge mortality despite being messy and redundant
    Invoked when claiming that note-using models improve AUC by 0.1 and that pooling extracts the informative parts.

pith-pipeline@v0.9.0 · 5531 in / 1336 out tokens · 68389 ms · 2026-05-07T17:08:37.029035+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 1 canonical work pages

  1. [1]

    Critical issues in an electronic documentation system,

    Nebeker JR. Weir CR., “Critical issues in an electronic documentation system,” In AMIA Annu Symp Proc,vol. 3, pp. 786–790, 2007

    2007

  2. [2]

    Chronickidney disease and theglobal public health agenda: an international consensus.,

    Francis, A.,Harhay, M.N., Ong,A.C.M. et al. “Chronickidney disease and theglobal public health agenda: an international consensus.,” InNature Reviews Nephrology vol.20, pp.473–485, 2024

    2024

  3. [3]

    Impact of chronic kidney disease on mortality: A nationwide cohort study.,

    Kim KM, Oh HJ, Choi HY, Lee H, Ryu DR. “Impact of chronic kidney disease on mortality: A nationwide cohort study.,” InKidney Res Clin Pract.,vol.38(3), pp.382-390, Sep 30, 2019

    2019

  4. [4]

    Automated Medical Coding on MIMIC-III and MIMIC-IV:A Critical Review and Replicability Study,

    Joakim Edin, Alexander Junge, Jakob D. Havtorn, Lasse Borgholt, Maria Maistro, Tuukka Ruotsalo, and Lars Maaløe, “Automated Medical Coding on MIMIC-III and MIMIC-IV:A Critical Review and Replicability Study,” InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei, Taiwan, pp. 2572–2582, April 2023

    2023

  5. [5]

    Physician burnout: contributors, con se- quences and solutions,

    Shanafelt TD. West CP, Dyrbye LN,. “Physician burnout: contributors, con se- quences and solutions,” InJournal of Internal Medicine, vol.06, pp.516–529, 2018

    2018

  6. [6]

    Physician burnout: A review,

    Sandeep Grover, Himani Adarsh, Chandrima Naskar, and Natarajan Varadharajan., “Physician burnout: A review,” InJournal of Mental Health and Human Behaviour 23,vol. 01, pp. 78–93, 2018

    2018

  7. [7]

    Real-time mortality prediction in the Intensive Care Unit.,

    Johnson AEW, Mark RG. “Real-time mortality prediction in the Intensive Care Unit.,” InAMIA Annu Symp Proc., Apr 16, 2017

    2017

  8. [8]

    Early hospital mortality pre- diction of intensive care unit patients using an ensemble learning approach.,

    Awad A, Bader-El-Den M, McNicholas J, Briggs J. “Early hospital mortality pre- diction of intensive care unit patients using an ensemble learning approach.,” InInt J Med Inform.vol.108, pp.185-195, Dec, 2017

    2017

  9. [9]

    Intensive Care Unit Mortality Prediction: An Improved Patient-Specific Stacking Ensemble Model,

    N. El-Rashidy, S. El-Sappagh, T. Abuhmed, S. Abdelrazek and H. M. El-Bakry, "Intensive Care Unit Mortality Prediction: An Improved Patient-Specific Stacking Ensemble Model," InIEEE Access,vol. 8, pp. 133541-133564, 2020 Enhance the after-discharge mortality rate prediction 13

    2020

  10. [10]

    Sentiment in nursing notes as an indicator of out-of-hospital mortality in intensive care patients.,

    Waudby-Smith IER, Tran N, Dubin JA, Lee J. “Sentiment in nursing notes as an indicator of out-of-hospital mortality in intensive care patients.,” InPLoS One. vol.13(6), Jun 7, 2018

    2018

  11. [11]

    Im- proving mortality prediction for intensive care unit patients using text mining tech- niques,

    Primož Kocbek, Nino Fijačko, Milan Zorman, Simon Kocbek, Gregor Štiglic, “Im- proving mortality prediction for intensive care unit patients using text mining tech- niques,” InProceedings of SiKDD 2017 Conference on Data Mining and Data Ware- houses, 2017

    2017

  12. [12]

    Patient repre- sentation learning and interpretable evaluation using clinical notes,

    Madhumita Sushil, Simon Šuster, Kim Luyckx, Walter Daelemans. “Patient repre- sentation learning and interpretable evaluation using clinical notes,” InJ. Biomed. Inform.,vol.84, pp.103–113, 2018

    2018

  13. [13]

    Estimating patient’s health state using latent structure inferred from clinical time series and text,

    Aaron Zalewski, William Long, Alistair E.W. Johnson, Roger G. Mark, Li-Wei H. Lehman, “Estimating patient’s health state using latent structure inferred from clinical time series and text,” InIEEE EMBS Int Conf Biomed Health Inform, pp. 449–452, February 2017

    2017

  14. [14]

    Unfolding physiological state: mor- tality modelling in intensive care units,

    Marzyeh Ghassemi, Tristan Naumann, Finale Doshi-Velez, Nicole Brimmer, Rohit Joshi, Anna Rumshisky, and Peter Szolovits, “Unfolding physiological state: mor- tality modelling in intensive care units,” InProceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, New York, USA, pp. 75–84, 2014

    2014

  15. [15]

    Using multiple sentiment dimensions of nursing notes to predict mortality in the intensive care unit,

    N. Tran, J. Lee, “Using multiple sentiment dimensions of nursing notes to predict mortality in the intensive care unit,” In:2018 IEEE EMBS International Conference on Biomedical Health Informatics (BHI), pp. 283–286, March 2018

    2018

  16. [16]

    Benchmarking deep learning models on large healthcare datasets,

    Sanjay Purushotham, Chuizheng Meng, Zhengping Che, Yan Liu, “Benchmarking deep learning models on large healthcare datasets,” InJ. Biomed. Inform.,vol.83 pp.112–134, 2018

    2018

  17. [17]

    Johnson, A., Pollard, T., Shen, L. et al. MIMIC-III, a freely accessible critical care database. InSci Data,vol.3, 160035, 2016

    2016

  18. [18]

    Characterizing the Value of Information in Medical Notes,

    Chao-Chun Hsu, Shantanu Karnwal, Sendhil Mullainathan, Ziad Obermeyer, and Chenhao Tan, “Characterizing the Value of Information in Medical Notes,”arxiv, accessed at:https://arxiv.org/abs/2010.03574

    work page arXiv 2010

  19. [19]

    Deep Unordered Composition Rivals Syntactic Methods for Text Classification,

    Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III, “Deep Unordered Composition Rivals Syntactic Methods for Text Classification,” InPro- ceedings of the 53rd Annual Meeting of the Association for Computational Linguis- tics and the 7th International Joint Conference on Natural Language Processing. vol. 01, pp.1681–1691, 2015

    2015