Pith. sign in

REVIEW 4 major objections 5 minor 58 references

Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Context-aware 'surprise' scores from a foundation model flag EHR events that predict inpatient mortality and long length-of-stay.

desk verdict A promising but confounded measure of clinical surprise; the redaction experiment needs a token-count control before the headline claim holds. read the letter →

arxiv 2507.22798 v1 pith:IO2PFXJZ submitted 2025-07-30 cs.LG

Michael C. Burkhart , Bashar Ramadan , Luke Solo , William F. Parker , Brett K. Beaulieu-Jones This is my paper
classification cs.LG
keywords foundationmodelselectronichealthrecordsinformationquantificationmodelexplainabilitycontext-awareclinicaleventimportancesurpriseredactionexperiment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the surprise a foundation model shows at each new item in a patient's electronic health record, measured as the negative log probability the model assigns to the observed token given everything seen before, is a usable measure of which events matter. In two hospital datasets, the number of highly surprising tokens and events in the first 24 hours of a stay is positively associated with inpatient death and with stays of seven days or longer. Redacting the most informative events from a timeline degrades the accuracy of representation-based prognostic classifiers, while redacting the least informative events generally does not. If these associations hold, the same self-supervised model that makes predictions could also tell clinicians which events to look at, and could help separate clinically meaningful changes from routine readings that trigger alarm fatigue.

What carries the argument

The machinery is context-aware information: for a realized token $x_t$ with context $x_{<t}$, it is $I_p(x_t|x_{<t}) = -\log_2 p(x_t|x_{<t})$, and for an event (a maximal block of contemporaneous tokens) it is the sum of its tokenwise values, so information is additive. A 67.3-million-parameter transformer is trained from scratch, by next-token prediction, on hospital stays serialized into a 208-token vocabulary in which measured values are binned into deciles (category-value tokenization). The trained model supplies the conditional probabilities and also the 1024-dimensional hidden-state representation used for prognosis; informativeness is then compared with representation jumps and tested by a redaction experiment in which the most, least, or randomly chosen events are deleted from 24-hour timelines.

What would settle it

Re-run the §4.2 logistic regressions with the total number of events in the first 24 hours and a severity-of-illness acuity score as covariates; if the coefficients for the 95th- and 99th-percentile surprise counts fall to zero, the apparent predictive signal is data volume, not surprise. A second check: recompute the redaction experiment after shuffling the order of tokens within each event; if dropping 'most informative' events still hurts performance as much, the context-dependence of the measure is not doing the work.

Watch

Extended reading notes

Core claim

The paper's central claim is that the context-aware information content of an EHR event, $I_p(x_{u:v}|x_{<u}) = -\log_2 p(x_{u:v}|x_{<u})$ computed from a model trained to predict the next token in tokenized hospitalization timelines, identifies events that are clinically significant even when their values fall inside normal ranges. The authors show that counts of tokens and events above the 95th and 99th informativeness percentiles in the first 24 hours are positively and significantly associated with inpatient mortality and long length-of-stay in both of their test sets. They further show that deleting the most informative events from a 24-hour timeline significantly lowers the ROC-AUC of downstream classifiers, whereas deleting the least informative events leaves performance essentially unchanged. The same measure also tracks the model's internal state: more informative tokens produce larger jumps in the latent representation, and event-level informativeness is almost perfectly correlated with the path length the representation travels.

Load-bearing premise

Everything rests on the trained model's next-token probabilities faithfully representing real EHR sequences; if those probabilities are off, a 'surprising' event can be an artifact of hospital ordering patterns or model miscalibration rather than a clinically meaningful change.

Editorial extensions

If this is right

  • Counts of highly informative tokens and events in the first 24 hours are positively associated with inpatient mortality and long length-of-stay in both test datasets.
  • Removing the most informative events significantly reduces the ROC-AUC of representation-based classifiers once 20% or more of them are dropped; removing the least informative events generally does not.
  • More informative tokens coincide with larger jumps in the model's latent representation, and event-level informativeness is almost perfectly proportional to representation path length, so surprise scores double as an explanation of what changed the model's mind.
  • Because informativeness is defined by context, the method flags events that rule-based normal-range checks would miss, such as a within-range blood pressure that is a rapid drop for a stroke patient.
  • The ranking of events by information content provides a clinician-facing list of the most surprising events in a stay, which the paper proposes as a summarization and chart-review tool.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that if the model's probabilities are well calibrated, the same surprise score could be used as the trigger in an alarm system that quietens routine events and escalates contextually rare ones; the paper suggests this but does not run a prospective alerting trial.
  • We infer that the decile tokenization collapses all sub-93% SpO2 readings into one bin, so genuinely critical desaturations inside that bin are invisible to the measure; a finer value encoding would likely surface more of those events, which is a testable extension.
  • We infer that the near-perfect linear relation between event informativeness and representation path length suggests that in models without explicit likelihoods, representation displacement could serve as an approximate surprise score; the paper does not claim this.
  • We infer that because the scores are label-free, surprise patterns could be used to define patient phenotypes without diagnostic codes; the paper mentions phenotyping as a future application but does not validate it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a foundation-model-based method to quantify the informativeness of individual tokens and events in tokenized electronic health records, defining informativeness as the negative log conditional probability assigned by a next-token model trained on MIMIC-IV. The authors report that counts of high-information tokens/events in the first 24 hours are positively associated with inpatient mortality and long length-of-stay in both MIMIC-IV and a held-out UCMC cohort (§4.2), that informative tokens and events produce larger representation changes (§4.3), and that redacting the most informative events significantly reduces the ROC-AUC of downstream prognostic classifiers while redacting the least informative events generally does not (§4.4). The paper includes detailed tokenization tables, example highlighted timelines, and a public code repository.

Significance. The central idea is timely and well-motivated: using a generative model's surprise as a context-aware importance signal for EHR events could address alarm fatigue and improve model interpretability. The external UCMC split and the random-redaction control arm are genuine strengths, as is the transparent decile tokenization and the availability of code and appendices. If the confounds identified below are resolved, the method could become a useful interpretability tool for EHR foundation models. However, the current evidence does not yet cleanly separate 'surprise' from 'data volume' or 'event length', so the headline causal claims are not fully established.

major comments (4)
  1. [§4.4, Table 1] The claim in §4.4 that 'events carrying little information can readily be dropped from a timeline without significantly impacting predictive performance' is contradicted by the paper's own Table 1: for long length-of-stay in MIMIC, bottom redaction at 30% (0.726, p < 0.05) and 40% (0.724, p < 0.01) shows significant ROC-AUC drops, and even random 40% redaction is significant (0.727, p < 0.05). The 'safely dropped' claim should be restricted to the smaller redaction percentages and to mortality, or reinterpreted in light of the fact that at high redaction rates any removal harms performance.
  2. [§3.8, Eq. (3), Table 1] The redaction experiment drops equal percentages of events across arms but not equal numbers of tokens. Because eventwise information is defined as the sum of tokenwise information (Eq. 3), the most informative events are systematically longer, so the 'top' arm removes more tokens per event than the 'bottom' or 'random' arms. The observed AUC reductions (e.g., MIMIC mortality 0.869 to 0.848 at 20% top) may therefore reflect data volume removed rather than the property of surprise. Please report the number of tokens removed in each arm, and ideally include an additional comparison that matches token counts (e.g., drop the N most vs. least informative tokens, or drop events while holding total tokens constant) to make the causal claim clean.
  3. [§4.2] The associations between counts of high-information tokens/events (T≥95, E≥99) and mortality/long length-of-stay are not adjusted for total number of recorded events or tokens, nor for illness severity. If sicker patients generate more data in the first 24 hours, these count variables may track data volume and acuity rather than the specific property of informativeness. Report a sensitivity analysis that adjusts for total event/token counts and a severity score (e.g., SOFA or admission type), or present conditional associations.
  4. [§4.3, Eq. (6), Fig. 6] The near-perfect event-level R²=0.997 between event informativeness and path length Δ_{u:v} is largely mechanical: both quantities are sums over the same tokens (Eqs. 3 and 6), so any correlation between token-level information and token-level representation jump gets amplified by event length. The token-level R²=0.212 is more informative. Please report the event-level regression controlling for event length (number of tokens), or present the per-token averaged relationship, before claiming that informative events 'trace out longer paths in representation space'.
minor comments (5)
  1. [§3.4–3.5] The model is described as a 'foundation model', but it is trained from scratch on MIMIC (67M parameters) rather than being a pretrained foundation model. Consider clarifying the terminology to avoid overclaiming.
  2. [Figure 4 caption] The caption notes that all SpO2 readings at or below 93 are placed in decile Q0, making a clinically significant hypoxemia drop within that bin invisible to the tokenizer. This limitation should be stated in the methods and revisited when interpreting the model's apparent misses.
  3. [General] The paper would benefit from a limitations section, especially regarding the decile binning's loss of granularity and the reliance on the model pθ approximating the empirical distribution (footnote b).
  4. [§4.2] Beta coefficients are reported without confidence intervals; adding them would help assess effect sizes and the practical magnitude of the associations.
  5. [Appendix C3] There is a typo in the caption: 'After a a 30 day 20 hour stay' should read 'After a 30 day 20 hour stay'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: informativeness is externally validated against held-out outcomes and a separate health system, and the representation–information relationships are empirical, not identities.

full rationale

The derivation is self-contained. Informativeness is defined by Eq. (1) as conditional self-information under a model trained by maximum likelihood (Eq. 4); no outcome label enters the training objective or the informativeness calculation. The headline associations (§4.2) relate counts of high-informativeness tokens/events to external outcomes (mortality, long length-of-stay) in a held-out MIMIC test set and in UCMC, an entirely separate health system; the outcome regressions do not set any model constant. The redaction experiment (§3.8/§4.4) trains logistic regressions on representations of redacted timelines and compares against the original; the rankings used for redaction come from pθ, but the AUC differences are measured against external labels, so the result is not forced by construction. The high event-level R²=0.997 in §4.3 is an empirical consequence of both path length and informativeness being summed over tokens (Eqs. 3 and 6), not an identity between the two quantities; token-level R² is only 0.212, so the relationship is not tautological. The paper cites its own CLIF data format [41] and prior tokenization work [9], but these support data preprocessing and are not load-bearing for the surprise–outcome claim. The token-count confound in the redaction experiment (top events are longer because Eq. 3 is additive) is a genuine validity concern but an empirical confound, not circularity: it does not make the AUC drop equal to the informativeness ranking by construction. Overall: no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its central numerical outputs depend on the fitted 67.3M-parameter model, on per-category decile cutoffs computed from the training set, and on hand-chosen percentile and redaction thresholds. The key domain assumption that pθ approximates p̂ is stated only in a footnote.

free parameters (4)
  • per-category decile cutoffs = 9 cutoffs per tokenized category (Appendix B)
    Lab, vital, assessment, and medication values are binned into deciles using cutoffs computed on the MIMIC training set; the token sequence, and therefore every informativeness score, depends on these bin boundaries.
  • 95th and 99th percentile thresholds for informative counts = 95, 99
    T≥95, E≥95,<99, and E≥99 in §4.2 are defined by these hand-chosen thresholds; the reported associations are specific to these cutoffs.
  • redaction percentages = 10, 20, 30, 40 percent
    The redaction experiment in §3.8 and §4.4 uses these chosen percentages, and the significance pattern is reported at these specific values.
  • model parameters θ = 67.3M parameters trained on MIMIC training split
    pθ in Eq. (1) is the fitted Llama-3.2-style model; all self-information values are functions of these learned weights.
assumptions (5)
  • standard math Shannon self-information and its conditional and additive forms (Eqs. 1-3) correctly quantify event informativeness.
    The method relies on standard information theory; the additivity of subsequence information is exact for a fixed distribution.
  • domain assumption Tokenized EHR sequences in CLIF format faithfully represent the clinical care process.
    All conclusions are about events as encoded by the tokenizer in §3.2; information not captured, such as free-text notes or precise timing, is ignored.
  • domain assumption The trained next-token model pθ approximates the true empirical distribution p̂ of EHR sequences.
    Stated in footnote b; if the model is miscalibrated, informativeness scores are model artifacts rather than clinical surprise.
  • domain assumption Decile binning preserves clinically meaningful variation.
    Within-decile changes are invisible to the model; the authors acknowledge in the Figure 4 caption that all SpO2 values below 93 are assigned to Q0.
  • domain assumption The last hidden state R(x1:t) of the language model is a suitable representation for downstream prognosis.
    The redaction experiment trains logistic regressions on these representations without proving that the representation is sufficient for the outcomes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models." pith.science (2026). https://pith.science/paper/IO2PFXJZ

@misc{pith2026250722798,
  author       = {Pith},
  title        = {Pith review of: Quantifying surprise in clinical care: Detecting highly informative events in electronic health records with foundation models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IO2PFXJZ}},
  note         = {Machine review of arXiv:2507.22798}
}
read the original abstract

We present a foundation model-derived method to identify highly informative tokens and events in electronic health records. Our approach considers incoming data in the entire context of a patient's hospitalization and so can flag anomalous events that rule-based approaches would consider within a normal range. We demonstrate that the events our model flags are significant for predicting downstream patient outcomes and that a fraction of events identified as carrying little information can safely be dropped. Additionally, we show how informativeness can help interpret the predictions of prognostic models trained on foundation model-derived representations.

Figures

Figures reproduced from arXiv: 2507.22798 by the authors.

Figure 1
Figure 1. Category-value tokenization. We convert lab results into tokens as follows. For each lab category, we determine decile cutoffs (center) using all results corresponding to that lab category available in the training dataset. Each lab value is then encoded as a decile (with Q0 corresponding to the lowest decile, Q1 to the next, and so on up to Q9) and inserted into the corresponding hospitalization in temporal order. … view at source ↗
Figure 2
Figure 2. Timeline highlighted by tokenwise context-aware information for MIMIC hospitalization [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Timeline highlighted by tokenwise context-aware information for MIMIC hospitalization [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Timeline highlighted by tokenwise context-aware information for MIMIC hospitalization [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Average ∆t versus average token￾wise informativeness by token type for the 24.7 million tokens xt in the MIMIC test set. 0 500 1000 1500 0 500 1000 1500 2000 2500 3000 Information (bits) Path length (in representation space) [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 52 canonical work pages

  1. [1]

    Akiba, S

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A next-generation hyperpa- rameter optimization framework. In KDD, 2019. cSee: https://github.com/bbj-lab/Quantifying-Surprise-EHRs

    work page 2019

  2. [2]

    H. R. Anderson, A. C. Borgen, R. Christnacht, J. Ng, J. G. Weller, H. N. Davison, P. A. Noseworthy, R. Olson, D. O’Laughlin, L. Disrud, and A. H. Kashou. Stats on the desats: alarm fatigue and the implications for patient safety. BMJ Open Qual., 12(3), 2023

    work page 2023

  3. [3]

    Archetti, A

    D. Archetti, A. L. Young, N. P. Oxtoby, D. Ferreira, G. M ˚ artensson, E. Westman, D. C. Alexander, G. B. Frisoni, and A. Redolfi. Inter-cohort validation of sustain model for Alzheimer’s disease. Front. Big Data, 2021

    work page 2021

  4. [4]

    Awais, M

    M. Awais, M. Naseer, S. Khan, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, and F. S. Khan. Foundation models defining a new era in vision: A survey and outlook. IEEE Trans. Pattern Anal. Mach. Intell., 47(4):2245–2264, 2025

    work page 2025

  5. [5]

    B. K. Beaulieu-Jones, P. Orzechowski, and J. H. Moore. Mapping patient trajectories using longitudinal extraction and deep learning in the MIMIC-III critical care database. In PSB, pages 123–132, 2018

    work page 2018

  6. [6]

    B. K. Beaulieu-Jones, W. Yuan, G. A. Brat, A. L. Beam, G. Weber, M. Ruffin, and I. S. Kohane. Machine learning for patient risk stratification: standing on, or looking over, the shoulders of clinicians? npj Digit. Med., 4(1):62, 2021

    work page 2021

  7. [7]

    Bergstrom, B

    N. Bergstrom, B. J. Braden, A. Laguzza, and V. Holman. The Braden scale for predicting pressure sore risk. Nurs. Res., 36(4):205–210, 1987

    work page 1987

  8. [8]

    G. W. Brier. Verification of forecasts expressed in terms of probability. Mon. Weather Rev., 78(1):1–3, 1950

    work page 1950

Show all 58 references
  1. [9]

    M. C. Burkhart, B. Ramadan, Z. Liao, K. Chhikara, J. C. Rojas, W. F. Parker, and B. K. Beaulieu-Jones. Foundation models for electronic health records: representation dynamics and transferability. arXiv:2504.10422, 2025

    2025 arXiv

  2. [10]

    E. Choi, M. T. Bahadori, A. Schuetz, W. F. Stewart, and J. Sun. Doctor AI: Predicting clinical events via recurrent neural networks. arXiv:1511.05942, 2016

    2016 arXiv

  3. [11]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171–4186, 2019

    2019

  4. [12]

    Efron and R

    B. Efron and R. J. Tibshirani. An Introduction to the Bootstrap, volume 57 of Monographs on Statistics and Applied Probability. Chapman and Hall, 1993

    1993

  5. [13]

    Fallahpour, M

    A. Fallahpour, M. Alinoori, W. Ye, X. Cao, A. Afkanpour, and A. Krishnan. EHRMamba: Towards generalizable and scalable foundation models for electronic health records. In ML4H, volume PMLR 259, pages 291–307, 2025

    2025

  6. [14]

    H. M. Fonteijn, M. Modat, M. J. Clarkson, J. Barnes, M. Lehmann, N. Z. Hobbs, R. I. Scahill, S. J. Tabrizi, S. Ourselin, N. C. Fox, and D. C. Alexander. An event-based model for disease pro- gression and its application in familial Alzheimer’s disease and huntington’s disease....

    2012

  7. [15]

    A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. Physiobank, physiotoolkit, and physionet. Circulation, 101(23), 2000

    2000

  8. [16]

    Goodfellow, Y

    I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA, 2016

    2016

  9. [17]

    Grattafiori, A

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, et al. The Llama 3 herd of models. arXiv 2407.21783, 2024

    2024 arXiv

  10. [18]

    Gu and T

    A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces. In COLM, 2024

    2024

  11. [19]

    Hanson and L

    S. Hanson and L. Pratt. Comparing biases for minimal network construction with back-propagation. In Adv. Neural Inf. Process. Syst., 1988

    1988

  12. [20]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 9(8):1735–1780, 1997

    1997

  13. [21]

    M. D. Howell, G. S. Corrado, and K. B. DeSalvo. Three epochs of artificial intelligence in health care. JAMA, 331(3):242–244, 2024

    2024

  14. [22]

    S. K. Inouye, C. H. van Dyck, C. A. Alessi, S. Balkin, A. P. Siegal, and R. I. Horwitz. Clarifying confusion: The confusion assessment method. Ann. Intern. Med., 113(12):941–948, 1990

    1990

  15. [23]

    A. E. W. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, S. Hao, B. Moody, B. Gow, L.-W. H. Lehman, L. A. Celi, and R. G. Mark. MIMIC-IV, a freely accessible electronic health record dataset. Sci. Data, 10, 2023

    2023

  16. [24]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015

    2015

  17. [25]

    Kraljevic, D

    Z. Kraljevic, D. Bean, A. Shek, R. Bendayan, H. Hemingway, J. A. Yeung, A. Deng, A. Baston, J. Ross, E. Idowu, J. T. Teo, and R. J. B. Dobson. Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective mode...

    2024

  18. [26]

    Kullback and R

    S. Kullback and R. A. Leibler. On information and sufficiency. Ann. Math. Statistics, 22:79–86, 1951

    1951

  19. [27]

    Y. Li, S. Rao, J. A. Solares, A. Hassaine, R. Ramakrishnan, D. Canoy, Y. Zhu, K. Rahimi, and G. Salimi-Khorshidi. BEHRT: Transformer for electronic health records. Sci. Rep., 10, 2020

    2020

  20. [28]

    Z. C. Lipton, D. C. Kale, C. Elkan, and R. Wetzel. Learning to diagnose with LSTM recurrent neural networks. arXiv:1511.03677, 2017

    2017 arXiv

  21. [29]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In ICLR, 2019

    2019

  22. [30]

    McDermott, B

    M. McDermott, B. Nestor, P. Argaw, and I. S. Kohane. Event stream GPT: A data pre-processing and modeling library for generative, pre-trained transformers over continuous-time sequences of complex events. In Adv. Neural Inf. Process. Syst., volume 36, pages 24322–24334, 2023

    2023

  23. [31]

    Ng and M

    A. Ng and M. Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes. In Adv. Neur. Inf. Proc. Sys., volume 14, pages 841–848, 2002

    2002

  24. [32]

    H. Niu, O. A. Omitaomu, M. A. Langston, M. Olama, O. Ozmen, H. B. Klasky, A. Laurio, M. Ward, and J. Nebeker. Ehr-bert: A BERT-based model for effective anomaly detection in electronic health records. J. Biomed. Inf., 150:104605, 2024

    2024

  25. [33]

    C. Pang, X. Jiang, K. S. Kalluri, M. Spotnitz, R. Chen, A. Perotte, and K. Natarajan. CEHR- BERT: Incorporating temporal information from structured EHR data to improve prediction tasks. In Proceedings of Machine Learning for Health, volume PMLR 158, pages 239–260, 2021

    2021

  26. [34]

    M. Poli, S. Massaroli, E. Nguyen, D. Y. Fu, T. Dao, S. Baccus, Y. Bengio, S. Ermon, and C. R´ e. Hyena hierarchy: towards larger convolutional language models. In ICML, volume PMLR 202, 2023

    2023

  27. [35]

    Radford, K

    A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever. Improving language understanding by generative pre-training. OpenAI, 2018

    2018

  28. [36]

    Rajkomar, E

    A. Rajkomar, E. Oren, K. Chen, A. M. Dai, N. Hajaj, M. Hardt, P. J. Liu, X. Liu, J. Marcus, M. Sun, et al. Scalable and accurate deep learning with electronic health records. npj Digit. Med., 1(1):18, 2018

    2018

  29. [37]

    Rasmy, Y

    L. Rasmy, Y. Xiang, Z. Xie, C. Tao, and D. Zhi. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digit. Med., 4(1):86, 2021

    2021

  30. [38]

    M. Raza, Z. Jahangir, M. B. Riaz, M. J. Saeed, and M. A. Sattar. Industrial applications of large language models. Sci. Rep., 15, 2025

    2025

  31. [39]

    P. Renc, M. K. Grzeszczyk, N. Oufattole, D. Goode, Y. Jia, S. Bieganski, M. B. A. McDermott, J. Was, A. E. Samir, J. W. Cunningham, D. W. Bates, and A. Sitek. Foundation model of electronic medical records for adaptive risk estimation. arXiv:2502.06124, 2025

    2025 arXiv

  32. [40]

    P. Renc, Y. Jia, A. E. Samir, J. Was, Q. Li, D. W. Bates, and A. Sitek. Zero shot health trajectory prediction using transformer. npj Digit. Med., 7, 2024

    2024

  33. [41]

    J. C. Rojas, P. G. Lyons, K. Chhikara, V. Chaudhari, S. V. Bhavani, M. Nour, K. G. Buell, K. D. Smith, C. A. Gao, S. Amagai, et al. A common longitudinal intensive care unit data format (CLIF) for critical illness research. Intensive Care Med., 2025

    2025

  34. [42]

    C. N. Sessler, M. S. Gosnell, M. J. Grap, G. M. Brophy, P. V. O’Neal, K. A. Keane, E. P. Tesoro, and R. K. Elswick. The Richmond agitation–sedation scale. Am. J. Respir. Crit. Care Med., 166(10):1338–1344, 2002

    2002

  35. [43]

    C. E. Shannon. A mathematical theory of communication. Bell System Tech. J., 27:379–423,623– 656, 1948

    1948

  36. [44]

    C. E. Shannon. Prediction and entropy of printed English. Bell System Tech. J., 30(1):50–64, 1951

    1951

  37. [45]

    Steinberg, J

    E. Steinberg, J. A. Fries, Y. Xu, and N. Shah. MOTOR: A time-to-event foundation model for structured medical records. In ICLR, 2024

    2024

  38. [46]

    Steinberg, K

    E. Steinberg, K. Jung, J. A. Fries, C. K. Corbin, S. R. Pfohl, and N. H. Shah. Language models are an effective representation learning technique for electronic health record data. J. Biomed. Inf., 113, 2021

    2021

  39. [47]

    J. Sun, C. Zheng, E. Xie, Z. Liu, R. Chu, J. Qiu, J. Xu, M. Ding, H. Li, M. Geng, et al. A survey of reasoning with foundation models: Concepts, methodologies, and outlook. ACM Comput. Surv., 57(11), 2025

    2025

  40. [48]

    Medical device alarm safety in hospitals

    The Joint Commission. Medical device alarm safety in hospitals. Sentinel Event Alert, (50), 2013

    2013

  41. [49]

    Hospital: 2025 National Patient Safety Goals

    The Joint Commission. Hospital: 2025 National Patient Safety Goals. 2024

    2025

  42. [50]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Adv. Neural Inf. Process. Syst., volume 30, 2017

    2017

  43. [51]

    J. W. Vogel, A. L. Young, N. P. Oxtoby, R. Smith, R. Ossenkoppele, O. T. Strandberg, et al. Four distinct trajectories of tau deposition identified in Alzheimer’s disease. Nat. Med., 27(5):871–881, 2021

    2021

  44. [52]

    Wornow, S

    M. Wornow, S. Bedi, M. A. F. Hernandez, E. Steinberg, J. Fries, C. R´ e, O. Koyejo, and N. H. Shah. Context clues: Evaluating long context models for clinical prediction tasks on EHRs. In ICLR, 2025

    2025

  45. [53]

    Wornow, R

    M. Wornow, R. Thapa, E. Steinberg, J. A. Fries, and N. Shah. EHRSHOT: An EHR benchmark for few-shot evaluation of foundation models. In Neurips Datasets and Benchmarks Track, volume 36, pages 67125–67137, 2023

    2023

  46. [54]

    Wornow, Y

    M. Wornow, Y. Xu, R. Thapa, B. Patel, E. Steinberg, S. Fleming, M. A. Pfeffer, J. Fries, and N. H. Shah. The shaky foundations of large language models and foundation models for electronic health records. npj Digit. Med., 6(1):135, 2023

    2023

  47. [55]

    D. Xu, F. Liu, X. Ding, J. Ma, Y. Suo, Y.-Y. Peng, J. Li, and X. Fu. Exploring icu nurses’response to alarm management and strategies for alleviating alarm fatigue: a meta-synthesis and systematic review. BMC Nursing, 24, 2025

    2025

  48. [56]

    Z. Yang, A. Mitra, S. Kwon, and H. Yu. ClinicalMamba: A generative clinical language model on longitudinal clinical notes. In Proceedings of the 6th Clinical NLP Workshop, pages 54–63, 2024

    2024

  49. [57]

    A. L. Young, R. V. Marinescu, N. P. Oxtoby, M. Bocchetta, K. Yong, N. C. Firth, et al. Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with subtype and stage inference. Nat. Commun., 9(1), 2018

    2018

  50. [58]

    A. L. Young, N. P. Oxtoby, P. Daga, D. M. Cash, N. C. Fox, S. Ourselin, J. M. Schott, and D. C. Alexander. A data-driven model of biomarker changes in sporadic Alzheimer’s disease. Brain, 137(9):2564–2577, 2014. Appendix A. Vocabulary Our vocabulary consisted of 208 tokens, br...

    2014

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.