Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hospital prediction models trained without dates lose up to 0.29 AUROC when records switch systems, and grouping measurements into clinical concepts cuts that loss to 0.06.

desk verdict Solid, important empirical study of temporal drift on MIMIC-III; read the headline mitigation numbers as an upper bound from a hand-built ontology, not a deployable strategy. read the letter →

arxiv 1908.00690 v1 pith:2DCJKKME submitted 2019-08-02 cs.LG stat.ML

Bret Nestor , Matthew B. A. McDermott , Willie Boag , Gabriela Berner , Tristan Naumann , Michael C. Hughes , Anna Goldenberg , Marzyeh Ghassemi This is my paper · ORCID
classification cs.LGstat.ML
keywords temporalgeneralizationconceptdriftclinicalmachinelearningelectronichealthrecordsMIMIC-IIIfeaturerepresentationmortalitypredictionlength-of-stay
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Machine learning models trained on electronic health records are usually evaluated on randomly split data, ignoring when care occurred; this paper argues that such benchmarks overstate how well the models will work when deployed on future patients. Using restricted year-of-care information from the MIMIC-III critical care database, the authors show that every model they test loses predictive accuracy when trained on historical data and evaluated on future data, with the largest drop occurring at the 2008 transition from the CareVue to the MetaVision recording system. The paper's main constructive result is that a manually built representation, which groups 181 raw database codes into 68 clinically meaningful measurements, reduces the worst AUROC drop from 0.29 to 0.06 for mortality prediction and from 0.10 to 0.03 for long length-of-stay prediction. The takeaway is that deployable clinical machine learning requires time-aware evaluation and feature representations that are not tied to the changing internals of a hospital's records system.

What carries the argument

The Clinical Aggregation representation carries the argument: it replaces database-specific measurement codes (ItemIDs) with 68 manually curated clinical concepts, each defined by a mapping that explicitly bridges pre-2008 CareVue codes and post-2008 MetaVision codes for the same biophysical quantity, then converts to canonical units and averages. This makes the feature space semantically stable when the software that records measurements changes. The surrounding machinery is the temporal evaluation protocol, which trains under three year-aware regimes (train on 2001-2002 only, prior year only, or full history) and tests on each future year, plus the comparison representations: Raw, PCA, and UMLS-based CUI Code Spanning. The comparison shows that the manual semantic mapping, not automatic dimensionality reduction or ontology grouping, is what preserves performance under drift.

What would settle it

Re-run the Full History experiment but derive the Clinical Aggregation mapping only from pre-2008 ItemIDs and their text descriptions, then evaluate on 2008-2012 data; if the AUROC drop approaches the raw representation's 0.29 rather than the reported 0.06 for mortality, the mitigation is not achievable prospectively. A second check: repeat the same study on another EHR dataset with a documented system transition and see whether expert-defined clinical concepts keep AUROC declines below raw-code declines.

Watch

Extended reading notes

Core claim

On MIMIC-III, prediction models trained on historical data and tested on future data suffer real, measurable decay, and the raw representation commonly used in deep learning is the most fragile. At the 2008 switch from CareVue to MetaVision, a random forest trained on all prior data loses 0.29 AUROC for in-ICU mortality and 0.10 AUROC for long length-of-stay; LSTM and GRU-D models also decay sharply, and many do not recover even with years of additional training data. The paper's proposed Clinical Aggregation representation, which manually maps the 181 raw ItemIDs into 68 clinical concepts with canonical units, cuts these maximum drops to 0.06 and 0.03 and also lowers year-to-year variance. The authors further show that year-agnostic evaluation, the standard benchmark protocol, overstates performance: for random forest mortality models, the raw-representation AUROC falls from a reported 0.82 on random splits to a true time-averaged 0.76, while the clinical representation is much closer, 0.86 versus 0.85.

Load-bearing premise

The load-bearing premise is that a deployer can produce the expert mapping that links every old measurement code to its new equivalent before the new recording system's codes are observed, since the reported 0.06 and 0.03 AUROC drops assume the mapping spans the 2008 transition.

Editorial extensions

If this is right

  • Standard year-agnostic AUROC numbers on MIMIC-III should not be read as expected performance in deployment, especially for raw feature representations.
  • Deploying models with raw ItemID features across a known EHR system transition will likely incur AUROC drops of roughly 0.29 for mortality and 0.10 for length-of-stay unless the feature representation is made system-independent.
  • Expert-defined clinical aggregations substantially reduce temporal decay for all tested model classes, so the choice of representation can matter more than the choice of model in non-stationary settings.
  • Automatic preprocessing such as PCA or UMLS-based concept grouping does not fully close the gap, leaving room for better automatic methods that learn system-spanning semantic mappings.
  • Both mortality and long length-of-stay tasks appear to saturate in predictive quality given little historical data, suggesting that benchmarking progress on these tasks will require harder targets or additional signals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to construct the Clinical Aggregation mapping using only pre-transition code lists and text descriptions, then measure the 2008 drop; if it exceeds 0.06, the reported robustness depends on knowing the future code system.
  • The same logic should transfer to other institutions: if a hospital changes EHR vendors, models built on raw vendor-specific codes should be expected to decay unless a semantic mapping exists, which makes standardized clinical vocabularies a deployment requirement.
  • Because the paper finds smaller subgroups show more erratic drift, fairness evaluations should include subgroup-stratified temporal splits; degradation may not be evenly distributed across protected groups.
  • The saturation result suggests AUROC on these two tasks is becoming a less sensitive benchmark, so temporal robustness, calibration over time, or harder tasks should be added to standard comparisons.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript quantifies the effect of temporal non-stationarity on clinical prediction models trained on MIMIC-III. Using a Limited Data Use Agreement to recover the year of each admission, the authors evaluate four classifiers (logistic regression, random forest, LSTM, GRU-D) on mortality and long length-of-stay tasks, under three chronological training regimes (2001-2002, prior-year, full history) and four feature representations (raw, PCA, CUI-based, and expert-defined clinical aggregations). The main empirical claims are that raw representations show large AUROC drops when models trained on historical data are tested on future data, with the largest drops at the 2008 CareVue-to-MetaVision transition (0.29 for mortality, 0.10 for LOS with random forests), that the clinical aggregation representation reduces these drops to 0.06 and 0.03, and that year-agnostic evaluation overstates deployable performance. The authors release code and the aggregated representations.

Significance. The paper addresses an important gap: because MIMIC-III dates are obscured, researchers routinely report time-agnostic performance, and the deployability of those models is unknown. The temporal evaluation protocol here is clean, the comparison across models and representations is systematic, and the code and representations are released, which makes the core measurement reproducible. The finding that raw ItemID features are fragile across the 2008 system change is likely to be influential. The mitigation result is less secure: the clinical aggregation relies on a mapping from both pre- and post-transition vocabularies, so the reported 0.06/0.03 reductions are best interpreted as an oracle upper bound rather than a demonstrated prospectively achievable improvement. With the framing corrected and the overclaim about "all models" fixed, the paper would be a substantial empirical contribution.

major comments (4)
  1. [Abstract; Table 4 (Appendix E)] The abstract states that "all tested state-of-the-art models decay in prediction quality" when trained on historical data and tested on future data. This is contradicted by the manuscript's own Full History results for the length-of-stay task: in Table 4, GRU-D with the Raw representation has a max AUROC drop of only 0.01, and the table note indicates that 2003 was the worst performing year with consistent improvement in subsequent years; Clinical LR and GRU-D show positive values (+0.02 and +0.06), i.e., improvement. The claim should be restricted to the mortality task, or to the specific models and representations for which decay is actually observed.
  2. [Section 4.1 (Clinical Aggregations) and Table 1] The mitigation evaluation is not prospective. Clinical Aggregations are defined by manually grouping CareVue ItemIDs with MetaVision ItemIDs (e.g., Heart Rate ItemID 211 with ItemID 220045), so the representation decoder contains knowledge of vocabulary that only became observable after the 2008 system transition. Under the Full History regime, models are trained on pre-2008 data and tested on 2008-2012 data, but the feature mapping is constructed using both sides of the transition. A deployer at the moment of transition could not have this mapping, so the reported reductions from 0.29 to 0.06 for mortality and from 0.10 to 0.03 for LOS measure the value of a perfect manually constructed cross-system ontology rather than a deployable algorithm. The conclusion acknowledges this limitation, but the abstract and Section 5.1 present clinical aggregation as a "simple yet effective" mitigation without this caveat. Please either evaluate a variant in which the mapping is built only from pre-2008 data or with a delayed deployment after MetaVision codes are observed, or explicitly label the current mitigation numbers as an oracle upper bound.
  3. [Section 5.2 (Date-Agnostic Training Overstates Performance)] The quantification of the overstatement is confounded with training set size. The year-agnostic RF Raw AUROC of 0.82 ± 0.02 is compared with the Full History year-averaged Raw AUROC of 0.76 ± 0.13, but the year-agnostic model is trained on all 2001-2012 data, whereas a Full History model evaluated on year y is trained only on data from 2001 through y-1. The gap may reflect the larger and more recent training set as well as the unrealistic future information. To support the causal claim that date-agnostic evaluation overstates performance, compare year-agnostic models against chronological models with matched training amounts (for example, train on 2001-2002 and evaluate each future year, or report year-agnostic performance per test year while training only on prior years), or explicitly restrict the interpretation to the combined effect of these differences.
  4. [Tables 1 and 4; Section 5] The headline numbers are maximum single-year AUROC drops relative to 2003, and no measure of uncertainty is reported for the per-year estimates underlying the max. A single noisy 2003 estimate can therefore drive the reported "max loss" values. Please report confidence intervals for year-specific AUROC (e.g., bootstrap over admissions) or a statistical test for the 2008-specific drop, so that the 0.29 and 0.06 numbers can be evaluated against sampling noise.
minor comments (5)
  1. [Appendix B] There is a typo in Appendix B: "ItfemID" should be "ItemID".
  2. [Tables 2 and 3] Several entries in Tables 2 and 3 have formatting artifacts such as "0 .79" and "0 .68"; please fix the typesetting.
  3. [Figure 3 caption] The caption is grammatically incomplete in the sentence "the least deviant and highest performing representation has been attained for most of the models"; please rephrase.
  4. [Section 3.2] The manuscript says it uses "an early version" of the MIMIC-III data extraction code by Wang et al. (2019); please specify which version was used and whether the released code is aligned with the final mimic-extract release.
  5. [Section 4.1 (PCA)] The PCA representation chooses 68 components to match the dimensionality of Clinical Aggregations; a sensitivity analysis over the number of components would help separate the effect of dimensionality from the effect of semantic grouping.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper is an empirical benchmarking study whose central decay result is measured on held-out future years, and the clinical-aggregation caveat is a disclosed prospective-deployment limitation, not a derivation-level reduction.

full rationale

This paper contains no formal derivation chain; it is an empirical evaluation of model generalization under temporal drift on MIMIC-III. The headline result—that raw features decay across the 2008 CareVue-to-MetaVision transition—is measured by training on historical years and evaluating on future years, with no fitted parameter renamed as a prediction. The year-agnostic comparison is also an empirical measurement, not a construction. Self-citations (e.g., Wang et al. 2019 for clinical-aggregation details and prior group papers for cohort criteria) provide implementation details and baseline conventions but are not load-bearing for the central empirical claim. The skeptical concern about Clinical Aggregations is a real deployment caveat: the representation is manually defined using ItemIDs from both sides of the 2008 transition, e.g., grouping Heart Rate ItemID 211 (CareVue) with 220045 (MetaVision), and the paper explicitly acknowledges that 'transfer to other EHR systems would require new taxonomies to be developed with input from clinical experts.' However, this is a limitation about prospective transferability, not circularity: the AUROC numbers are still empirical measurements on held-out data, and the representation does not encode the test labels or the outcome. No equation reduces to an input, and no fitted parameter is relabeled as a prediction, so no circular step can be exhibited under the required standard.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper's central empirical claims rest on access to restricted year labels, on an attribution of performance drops to the 2008 system change, and on a manually curated feature mapping. Each is a domain assumption rather than a fitted equation. No new entities are postulated, and most experimental design choices (PCA dimension, LOS threshold, observation window) are task definitions rather than fitted constants.

free parameters (3)
  • PCA component count = 68
    The PCA representation keeps the first 68 principal components, selected to match the dimensionality of the clinical aggregate features (Section 4.1). This is a hand-chosen design parameter that affects only the PCA comparison, not the central temporal-decay finding.
  • Long-LOS threshold = 3 days
    The binary length-of-stay label is defined as an ICU stay longer than the cohort median of 3 days (Appendix C). This choice determines the task and the reported AUROC drops.
  • Observation window = 24 hours
    All models predict from the first 24 hours of ICU data (Section 3.2), a task-defining choice that bounds the information available to every model.
assumptions (3)
  • domain assumption The year-of-care mapping obtained under the Limited Data Use Agreement reflects the true calendar year of each ICU stay.
    The temporal training regimes in Section 4.3 and all reported AUROC drops depend on these restricted year labels being correct. If the labels are shifted or noisy, the historical/future distinction collapses.
  • domain assumption The 2008 CareVue-to-MetaVision system change is the dominant driver of the sudden AUROC drops, rather than other concurrent changes in population or policy.
    The paper attributes the sharp 2008 decay to the EHR switch using external documentation (Johnson et al. 2016) and descriptive plots (Figure 4), but it does not causally isolate the system change from other temporal confounds.
  • ad hoc to paper The 68 clinically aggregated concepts are semantically consistent across the old and new EHR vocabularies.
    The expert mapping is manually defined (Section 4.1, Appendix B, Wang et al. 2019) and is not validated against any independent gold standard; the mitigation's success depends on this mapping being correct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks." pith.science (2026). https://pith.science/paper/2DCJKKME

@misc{pith2026190800690,
  author       = {Pith},
  title        = {Pith review of: Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2DCJKKME}},
  note         = {Machine review of arXiv:1908.00690}
}
read the original abstract

When training clinical prediction models from electronic health records (EHRs), a key concern should be a model's ability to sustain performance over time when deployed, even as care practices, database systems, and population demographics evolve. Due to de-identification requirements, however, current experimental practices for public EHR benchmarks (such as the MIMIC-III critical care dataset) are time agnostic, assigning care records to train or test sets without regard for the actual dates of care. As a result, current benchmarks cannot assess how well models trained on one year generalise to another. In this work, we obtain a Limited Data Use Agreement to access year of care for each record in MIMIC and show that all tested state-of-the-art models decay in prediction quality when trained on historical data and tested on future data, particularly in response to a system-wide record-keeping change in 2008 (0.29 drop in AUROC for mortality prediction, 0.10 drop in AUROC for length-of-stay prediction with a random forest classifier). We further develop a simple yet effective mitigation strategy: by aggregating raw features into expert-defined clinical concepts, we see only a 0.06 drop in AUROC for mortality prediction and a 0.03 drop in AUROC for length-of-stay prediction. We demonstrate that this aggregation strategy outperforms other automatic feature preprocessing techniques aimed at increasing robustness to data drift. We release our aggregated representations and code to encourage more deployable clinical prediction models.

Figures

Figures reproduced from arXiv: 1908.00690 by the authors.

Figure 1
Figure 1. The full experimental pipeline, spanning four data representations, four model [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Training paradigms. The available training data for each year is shown in grey. In [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Impact of representation (columns) and training style (rows) on the chronological [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The frequency of data collection can change in clinical practice. Shown is an [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: The measured values of data can shift in clinical practice [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Impact of representation (columns) and training style (rows) on the chronological [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Impact of representation on the longevity of performance across two genders. The [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: The number of ICU admissions per year by ethnicity. [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: The performance of the GRU-D on the task of mortality prediction. The classifica [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: The number of ICU admissions per year by insurance type. [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: The performance of the GRU-D on the task of mortality prediction. The [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data

    cs.LG 2025-09 reject novelty 4.0 of 10

    A multiplicative calibration using a six-component diagnostic fidelity score improved simulated dementia predictions by 10 to 33 percent, but only on synthetic data where the score is derived from labels.

Reference graph

Works this paper leans on

21 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    This task has the added benefit of investigating date randomisation effects in a more balanced-class problem without a directly targeted policy change

    This creates a 47.1% positive subject rate in-task. This task has the added benefit of investigating date randomisation effects in a more balanced-class problem without a directly targeted policy change. Appendix D. Year Agnostic Results Tables 2 & 3 contain the model performances when trained without knowledge of the years (5 x 2 fold CV splits (Dietterich...

    work page 1998

  2. [3]

    original-date: 2018-05-13T00:07:42Z

    URLhttps://github.com/zhiyongc/GRU-D. original-date: 2018-05-13T00:07:42Z. 13 Feature Robustness in Non-stationary Health Records Sharon E Davis, Thomas A Lasko, Guanhua Chen, Edward D Siew, and Michael E Matheny. Calibration drift in regression and machine learning models for acute kidney injury.Journal of the American Medical Informatics Association : J...

    work page 2018

  3. [6]

    Opportunities in machine learning for healthcare.arXiv preprint arXiv:1806.00388,

    Marzyeh Ghassemi, Tristan Naumann, Peter Schulam, Andrew L Beam, and Rajesh Ran- ganath. Opportunities in machine learning for healthcare.arXiv preprint arXiv:1806.00388,

    arXiv

  4. [7]

    original-date: 2018-09- 17T19:24:26Z

    URLhttps://github.com/Han-JD/GRU-D. original-date: 2018-09- 17T19:24:26Z. Hrayr Harutyunyan, Hrant Khachatrian, David C Kale, and Aram Galstyan. Multitask learning and benchmarking with clinical time series data.arXiv preprint arXiv:1703.07771,

    arXiv 2018

  5. [9]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift.arXiv preprint arXiv:1502.03167,

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift.arXiv preprint arXiv:1502.03167,

    arXiv

  6. [12]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,

    arXiv

  7. [14]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. InNIPS 2017 Autodiff Workshop,

    work page 2017

  8. [15]

    URL https://www.nature.com/articles/sdata2018178

    doi: 10.1038/sdata.2018.178. URL https://www.nature.com/articles/sdata2018178. Sanjay Purushotham, Chuizheng Meng, Zhengping Che, and Yan Liu. Benchmarking deep learning models on large healthcare datasets.Journal of biomedical informatics, 83:112–134,

    doi:10.1038/sdata.2018.178 2018

Show all 21 references
  1. [16]

    Transfusion: Understand- ing transfer learning with applications to medical imaging.arXiv preprint arXiv:1902.07208,

    Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio. Transfusion: Understand- ing transfer learning with applications to medical imaging.arXiv preprint arXiv:1902.07208,

    1902 arXiv

  2. [17]

    Shirly Wang, Matthew McDermott, Geeticka Chauhan, Michael C Hughes, Tristan Nau- mann, and Marzyeh Ghassemi

    PMLR. Shirly Wang, Matthew McDermott, Geeticka Chauhan, Michael C Hughes, Tristan Nau- mann, and Marzyeh Ghassemi. Mimic-extract: A data extraction, preprocessing, and representation pipeline for mimic-iii.arXiv preprint arXiv:1907.08322,

    1907 arXiv

  3. [18]

    liblinear

    17 Feature Robustness in Non-stationary Health Records Appendix A. Model Training Details Models which do not implicitly handle missingness (LR, RF and LSTM) require data to be imputed. Che et al. (2018) detail a thorough list of imputation schemes in their experiments where t...

    2018

  4. [19]

    based on (Han-JD, 2019; Cui , UW). We use a hidden layer size of 67 units, batch normalisation(Ioffe and Szegedy, 2015), and dropout with a probability of 0.5 on the classification layer like in the original work (Che et al., 2018). The Adam optimizer (Kingma and Ba,

    2019

  5. [20]

    is applied with the early stopping criteria(Che et al., 2018). 18 Feature Robustness in Non-stationary Health Records Tuning Procedure For the RF, LR and LSTM classifiers, 5-fold cross validation was applied to the training data, using a random search to find best parameters for...

    2019

  6. [21]

    long LOS

    as it is a direct signal of acuity strongly associated with EHR signals. The ICU mortality rate of patients in our subset is 7.4%. We also know that policy changes from the Affordable Care Act led to a changes in clinical practice regarding mortality prevention (Kocher et al., ...

    2010

  7. [1997]

    doi: 10.1162/neco.1997.9.8.1735

    ISSN 0899-7667. doi: 10.1162/neco.1997.9.8.1735. URL http://dx.doi.org/10.1162/neco.1997.9.8.1735. Harold Hotelling. Analysis of a complex of statistical variables into principal components. Journal of educational psychology, 24(6):417,

    1997 doi

  8. [1998]

    Sophia: a expedient UMLS concept extraction annotator

    Guy Divita, Qing T Zeng, Adi V Gundlapalli, Scott Duvall, Jonathan Nebeker, and Matthew H Samore. Sophia: a expedient UMLS concept extraction annotator. In Proc. of AMIA Annual Symposium, volume 2014, page

    2014

  9. [2014]

    Disease-atlas: Navigating disease trajectories with deep learning

    Bryan Lim and Mihaela van der Schaar. Disease-atlas: Navigating disease trajectories with deep learning. arXiv preprint arXiv:1803.10254,

    arXiv

  10. [2015]

    doi: 10.1007/s10115-014-0756-9

    ISSN 0219-3116. doi: 10.1007/s10115-014-0756-9. URL https://doi.org/10.1007/s10115-014-0756-9. Olivier Bodenreider. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res, 32(suppl 1):D267–D270,

    doi

  11. [2017]

    doi: 10.1093/jamia/ocx030

    ISSN 1067-5027. doi: 10.1093/jamia/ocx030. URL https://www.ncbi.nlm.nih. gov/pmc/articles/PMC6080675/. Thomas G Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation, 10(7):1895–1923,

    doi

  12. [2018]

    arXiv: 1812.02275

    URL http://arxiv.org/abs/1812.02275. arXiv: 1812.02275. Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic- iii, a freely accessible critical care database.S...

    arXiv

  13. [2019]

    doi: 10.1136/bmjqs-2018-008370

    ISSN 2044-5415, 2044-5423. doi: 10.1136/bmjqs-2018-008370. URL https://qualitysafety.bmj.com/content/28/3/231. ZhengpingChe, SanjayPurushotham, KyunghyunCho, DavidSontag, andYanLiu. Recurrent Neural Networks for Multivariate Time Series with Missing Values.Scientific Reports, 8 (1),

    2018 doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.