Pith. sign in

REVIEW 3 major objections 4 minor 40 references

The paper claims patient-first window sampling beats global-token-stream pretraining on four EHR tasks in MIMIC-IV v2.2 and v3.1.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Patient-aware sampling of pretraining windows, with patients weighted by a tunable exponent, improves downstream AUROC/AUPRC over a global token-stream baseline in autoregressive EHR models.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection A genuinely new and useful design axis for EHR pretraining, but the headline claim is currently riding on single-seed numbers and test-set selection of α; worth a careful look, not yet a firm result. the 3 major comments →

arxiv 2607.22114 v1 pith:4H27LC2B submitted 2026-07-24 cs.LG

Pretraining EHR Foundation Models with Patient-Aware Sampling

Joshua Placidi , Yuxuan Liu , Jinpei Han , Marek Rei , A. Aldo Faisal This is my paper
classification cs.LG
keywords EHR foundation modelspretraining sequence constructionpatient samplingautoregressive transformerszero-shot clinical predictionMIMIC-IVnext-token predictiondata weighting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the default way EHR foundation models are pretrained—concatenating all patients' token sequences into one global stream and cutting training windows from it—is actually a good choice. It argues it is not, because such windows can mix multiple patients and patients with long records dominate optimization updates. The paper proposes Patient Sampling: first sample a patient, then sample a window inside that patient's own sequence, with a single parameter α controlling whether patients are weighted equally or by record length. Across ICU mortality, ICU readmission, ICU admission, and hospital mortality on MIMIC-IV v2.2 and v3.1, Patient Sampling with α=0.6 improves Macro AUROC and AUPRC over Global Stream. The reason to care is that a meaningful accuracy gain comes from a pure data-pipeline change, with no new architecture or data.

Core claim

The central discovery is that sequence construction is a real, controllable design axis for autoregressive EHR foundation models. The paper defines three constructions: Global Stream (windows from the concatenated stream, which can cross patient boundaries and over-weight long records), Patient Chunks (deterministic within-patient windows that preserve boundaries but keep the length bias), and Patient Sampling (stochastic: sample patient i with probability proportional to |W_i|^α, then sample a start index uniformly within that patient). Sweeping α shows that Patient Chunks alone does not beat Global Stream, but Patient Sampling does: with α=0.6 it improves Macro AUROC from 0.834 to 0.842 on

What carries the argument

The key object is the patient-sampling distribution p_α(i) = |W^(i)|^α / Σ_j |W^(j)|^α, where |W^(i)| is the number of valid fixed-length windows inside patient i's trajectory. α=0 samples all patients uniformly; α=1 samples patients in proportion to their number of windows (equivalent to length-weighted). Because window selection is then conditioned on the chosen patient and drawn uniformly from a start-index set that includes left-context offsets, the method decouples 'which patient' from 'which window'. This gives a smooth interpolation between patient-uniform and length-proportional training signals, which is what the paper identifies as the mechanism behind the downstream gains.

Load-bearing premise

The ranking of methods rests on the assumption that a single training run per configuration, evaluated with only 20 stochastic rollouts per prompt, is enough to distinguish Patient Sampling from Global Stream—the AUROC gaps are small enough that run-to-run noise could erase them.

What would settle it

Re-run the α=0.6 Patient Sampling and Global Stream conditions on MIMIC-IV v2.2 with at least five random seeds each, using the same 20-rollout evaluation; if Global Stream matches or beats Patient Sampling on the macro columns in a majority of seeds, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Patient Sampling with α=0.6 improves Macro AUROC by 0.008–0.009 and Macro AUPRC by 0.011 on both MIMIC-IV versions relative to Global Stream.
  • Patient Chunks alone (preserving boundaries) does not improve over Global Stream, so boundary preservation is not the active ingredient.
  • Smaller α values produce the largest positive shifts for shorter ground-truth sequences on ICU mortality, while α=0.2 is anomalously poor, indicating α effects are non-monotonic.
  • Sequence construction is an important and underexplored design choice for autoregressive EHR foundation models, so future pretraining work should report it as a first-class hyperparameter.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The exact optimal α is likely dataset- and task-dependent; a natural extension would tune α on a held-out validation benchmark rather than the same tasks used for final reporting.
  • If the benefit comes from reducing cross-patient mixing, one testable extension is a hybrid that occasionally draws global windows for long-tail patients to keep rare code coverage.
  • The reported macro gains (0.008–0.011 AUROC) sit near the noise floor implied by 20-rollout stochastic evaluation; multi-seed replication would sharpen the ranking before clinical deployment.
  • Patient Sampling could be re-expressed as a reweighted next-token loss with per-patient importance weights, allowing continuous α during training without changing window construction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies sequence-construction strategies for pretraining autoregressive EHR foundation models. The baseline, Global Stream, concatenates all patient trajectories into one token stream and samples fixed-length windows from it. The proposed method, Patient Sampling, first samples a patient according to a tunable weight α (α=0 is patient-uniform, α=1 is proportional to the number of valid windows), then samples a window within that patient. A third, deterministic construction, Patient Chunks, preserves patient boundaries without stochastic sampling. The authors compare these constructions on MIMIC-IV v2.2 and v3.1, each augmented with MIMIC-IV-ED v2.2, using a 6-layer GPT-2 model and the rollout-based evaluation of Renc et al. (2025) on four downstream tasks: ICU Mortality, ICU Readmission, ICU Admission, and Hospital Mortality. They report that Patient Sampling with α=0.6 improves Macro AUROC and AUPRC over Global Stream on both dataset versions, while Patient Chunks alone does not. The paper argues that controlling the distribution of training signal across patients is an important, underexplored design choice for EHR foundation models.

Significance. If the reported improvements are robust, the paper identifies a meaningful and previously underexplored design axis for autoregressive EHR foundation models. The method is simple, interpretable, and plausibly useful in practice, since it directly addresses the length-imbalance and cross-patient-boundary issues inherent in the standard global-stream construction. The study is strengthened by evaluating on two versions of MIMIC-IV, by including a deterministic patient-boundary baseline, and by providing a length-stratified analysis (Figure 4) that goes beyond aggregate metrics. However, the empirical evidence is currently under-powered: the headline comparison rests on a single training run per configuration, a stochastic rollout evaluation with only 20 rollouts per prompt, and an α value selected on the same downstream tasks that later appear in the main result table. These issues limit the strength of the central claim and need to be addressed before the paper can be accepted.

major comments (3)
  1. [Section 4, Tables 1 and 2] The α=0.6 model is selected as the best macro performer in Table 1, which uses exactly the ICU Mortality and ICU Readmission tasks on both MIMIC-IV versions. Table 2 then reports these same tasks as part of its macro metric. The reported macro gains on those tasks are therefore optimistically selected, not independent confirmations. The genuinely out-of-selection tasks, ICU Admission and Hospital Mortality, show weaker and mixed evidence (e.g., ICU Admission v2.2 AUROC +0.000, AUPRC -0.004; Hospital Mortality v3.1 AUROC +0.021 but AUPRC +0.014). Please either choose α on a held-out set or a separate task/dataset, or report the full α sweep for all tasks so the reader can see whether the conclusion depends on the selection step.
  2. [Appendix A, Tables 1 and 2] Appendix A explicitly states: "the rollout-based evaluation is stochastic and can be noisy, and due to computational constraints we were unable to repeat all experiments across multiple random seeds." With one training seed and 20 stochastic rollouts per prompt, the reported differences are small (Table 2 macro AUROC gains: +0.008 on v2.2, +0.009 on v3.1; per-task AUROC gains range from -0.004 to +0.021). The α sweep in Table 1 shows neighboring-α differences of up to 0.017 in macro AUROC (e.g., α=0.4 vs α=0.2 on v2.2: 0.785 vs 0.774; α=0.6 vs α=0.8: 0.792 vs 0.775), which is the same order of magnitude as the headline gains. Without repeated seeds, confidence intervals, or a variance-aware evaluation, the observed differences are within plausible run-to-run noise. The central claim that Patient Sampling "improves Macro AUROC and AUPRC" is therefore not yet supported as stated. Please pr
  3. [Section 4, Figure 4 vs Table 1] Figure 4 shows that on ICU Mortality (MIMIC-IV v2.2), smaller α values produce the largest positive shift relative to Global Stream, with α=0 having the strongest effect overall. Yet Table 1 selects α=0.6 as the best macro configuration. This apparent contradiction is not discussed. It suggests that the optimal α may be task-specific and that the macro-average advantage of α=0.6 could be driven by a subset of tasks. Please reconcile these findings, for example by reporting per-task α sweeps or by explaining why macro performance and the Figure 4 trend diverge. This is relevant to the generalizability of the recommended α=0.6 setting.
minor comments (4)
  1. [Table 1] Several numeric entries are missing spaces or overlap, e.g., "0.8850.432" and "0.8500.446". Please fix the formatting.
  2. [Section 3, Patient-Aware Sampling] The description says the start index is sampled uniformly from W(i)={1-S, ..., L_i-2} and then clipped to max(1,s). After clipping, the actual distribution over start indices is not uniform: all s ≤ 1 map to start index 1, concentrating probability mass on token 1. The footnote acknowledges under-sampling of first/last tokens, but the mass concentration from clipping is stronger than the footnote suggests. Please clarify the intended sampling distribution and its effect on the length coverage.
  3. [Section 4, Experimental setup] Table 2 reports only "Global Stream" and "Patient Sampling (α=0.6)", but the Global Stream baseline was evaluated with two validation constructions (Global Stream and Patient Chunks) in Table 1. Please state explicitly which validation construction is used for the Global Stream row in Table 2, and whether the choice affects the comparison.
  4. [Section 4, Effect of α] The paragraph notes that α=0.2 underperforms relative to neighboring values and calls for further investigation. This non-monotonicity is interesting but is not discussed in the conclusion. A brief comment on possible causes (e.g., noise, interaction with sequence lengths) would strengthen the presentation.

Circularity Check

0 steps flagged

No circularity: the central claim is empirical, and the alpha-selection/test-set overlap is a soundness limitation, not a definitional or self-citation loop.

full rationale

The paper's central claim is an empirical superiority comparison (Patient Sampling vs Global Stream) on MIMIC-IV downstream tasks. The method is defined by p_alpha(i) = |W(i)|^alpha / sum_j |W(j)|^alpha, which is a sequence-construction choice, not a fitted quantity derived from downstream performance. Alpha is tuned by sweeping over {0, 0.2, ..., 1} and selecting the best macro value in Table 1: 'Patient Sampling with α = 0.6 yields the strongest macro performance across both tasks in both datasets, and we therefore select this model for broader evaluation.' This is hyperparameter selection on the same evaluation tasks, so the Table 2 macro numbers are optimistically selected on two of the four tasks; however, the reported improvements are not forced by construction, and two tasks (ICU Admission, Hospital Mortality) were not used in selection and are reported separately. Appendix A's admission that 'the rollout-based evaluation is stochastic and can be noisy, and due to computational constraints we were unable to repeat all experiments across multiple random seeds' is a robustness/statistical limitation, not a circular step. There is no analytic derivation that assumes its conclusion, no fitted parameter relabeled as a prediction, and no load-bearing self-citation: the cited prior work (Renc et al., Kraljevic et al., etc.) is external and supplies the evaluation framework, not the paper's own conclusion. Therefore no circularity is present.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No new physical or architectural entities are introduced; the only new construct is the sampling distribution p_alpha(i), which is a method rather than an entity. The main ledger entries are the tuned alpha and evaluation parameters, plus domain assumptions inherited from prior EHR modeling work.

free parameters (3)
  • alpha = 0.6 (selected from grid 0, 0.2, 0.4, 0.6, 0.8, 1.0)
    Weighting exponent in p_alpha(i)=|W(i)|^alpha / sum; selected using downstream macro performance in Table 1 on the same test tasks that later appear in Table 2.
  • rollouts_per_prompt = 20
    Evaluation hyperparameter; score is the fraction of 20 stochastic rollouts ending in a target token. The choice directly affects AUROC/AUPRC noise and is not justified or sensitivity-analyzed.
  • max_generation_length = 4096
    Manual cap for rollout generation; simulations exceeding it are excluded from benchmark calculations, altering scores. No sensitivity analysis is provided.
axioms (5)
  • domain assumption EHR trajectories can be tokenized into discrete event sequences suitable for next-token prediction.
    Inherited from Renc et al. and prior EHR foundation models; the tokenizer is not detailed or shipped in this paper.
  • domain assumption Next-token prediction pretraining transfers to the rollout-based clinical benchmarks measured by AUROC/AUPRC.
    The evaluation framework is taken from Renc et al. (2025) without independent justification that rollout scores are a fair or reliable downstream measure.
  • domain assumption Patient-level data splitting prevents leakage and validation-loss-based checkpoint selection is appropriate.
    Stated in Section 4; no verification is provided that the lowest-validation-loss checkpoint is the best downstream model.
  • ad hoc to paper The window start-index set W(i)={1-S, 2-S, ..., L_i-2} with clipping is a reasonable design for broadening left-context coverage.
    Introduced as a design choice with intuition; the footnote concedes that first and last tokens remain slightly undersampled, and no independent justification is given.
  • domain assumption Single-seed training and stochastic rollout evaluation are representative enough to rank methods.
    Implicit in the experimental design; Appendix A explicitly disclaims repeated seeds, making this the load-bearing empirical assumption.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Pretraining EHR Foundation Models with Patient-Aware Sampling." pith.science (2026). https://pith.science/paper/4H27LC2B

@misc{pith2026260722114,
  author       = {Pith},
  title        = {Pith review of: Pretraining EHR Foundation Models with Patient-Aware Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4H27LC2B}},
  note         = {Machine review of arXiv:2607.22114}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Autoregressive foundation models for electronic health records (EHRs) typically inherit pretraining methods from language modeling, where patient trajectories are concatenated into a single token stream and windows are sampled from that stream. In EHR data, this choice is consequential: windows may mix multiple patients, and patients with longer records contribute more optimization updates, potentially introducing bias. We propose Patient Sampling, a pretraining sequence-construction method that allows us to control how training signal is distributed across patients. We compare this method to the standard approach, which we refer to as Global Stream. We show that stochastic Patient Sampling with controllable weighting improves performance on real-world EHR data. Across downstream clinical tasks on MIMIC-IV v2.2 and v3.1, Patient Sampling improves Macro AUROC and AUPRC over the Global Stream baseline. These results identify training and validation sequence construction as important and underexplored design choices for autoregressive EHR foundation models.

Figures

Figures reproduced from arXiv: 2607.22114 by A. Aldo Faisal, Jinpei Han, Joshua Placidi, Marek Rei, Yuxuan Liu.

Figure 1
Figure 1. Figure 1: Toy example of an EHR dataset represented as variable￾length token sequences Di. Colored boxes denote EHR tokens, and black-bordered boxes indicate end-of-sequence tokens. Current EHR pretraining methods largely inherit sequence construction strategies from language modeling. In the stan￾dard setup, patient trajectories are concatenated into a global token stream and fixed-length training windows are sampl… view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of the dataset constructions considered in this work. On the left Global Stream forms training windows from a global concatenated token stream. Patient Chunks constructs deterministic within-patient windows. Patient Sampling first samples a patient and then samples a window from that patient alone. Colors denote different patients, and gray boxes indicate padding. 2. Background Autoregressive mo… view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of patient token sequence lengths in the MIMIC-IV v2.2 training set, shown on a log-scaled x-axis. contextual reasoning across document boundaries. These findings motivate studying sequence composition in EHR modelling, where patient trajectories are highly unequal in length and standard sequence construction may further amplify this imbalance. 3. Method We consider a dataset of N patients D =… view at source ↗
Figure 4
Figure 4. Figure 4: ∆ positive score on ICU Mortality (MIMIC-IV v2.2) for Patient Sampling models with different α values, relative to a Global Stream baseline. Positive score is the fraction of simula￾tions that terminate with the correct end token, delta positive score is measured relative to the Global Stream baseline. Datapoints are grouped by the number of tokens of the ground truth sequence. Mortality and ICU Admission … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 7 linked inside Pith

  1. [1]

    arXiv preprint arXiv:1607.06450 , year =

    Layer Normalization , author =. arXiv preprint arXiv:1607.06450 , year =

    Pith/arXiv arXiv

  2. [2]

    Analysing The Impact of Sequence Composition on Language Model Pre-Training

    Zhao, Yu and Qu, Yuanbin and Staniszewski, Konrad and Tworkowski, Szymon and Liu, Wei and Mi o \'s , Piotr and Wu, Yuxiang and Minervini, Pasquale. Analysing The Impact of Sequence Composition on Language Model Pre-Training. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/...

    doi:10.18653/v1/2024.acl-long.427 2024

  3. [3]

    Proceedings of the 38th International Conference on Machine Learning , series =

    Perceiver: General Perception with Iterative Attention , author =. Proceedings of the 38th International Conference on Machine Learning , series =. 2021 , editor =

    2021

  4. [4]

    Advances in Neural Information Processing Systems , editor=

    Flamingo: a Visual Language Model for Few-Shot Learning , author=. Advances in Neural Information Processing Systems , editor=. 2022 , url=

    2022

  5. [5]

    2020 , eprint =

    Yu Gu and Robert Tinn and Hao Cheng and Michael Lucas and Naoto Usuyama and Xiaodong Liu and Tristan Naumann and Jianfeng Gao and Hoifung Poon , title =. 2020 , eprint =

    2020

  6. [6]

    Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) , year =

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , author =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) , year =

    2019

  7. [7]

    OpenAI Technical Report , year =

    Language Models are Unsupervised Multitask Learners , author =. OpenAI Technical Report , year =

  8. [8]

    Workshop on Time Series Learning for Health (TS4H) at ICLR , year =

    Medical Event Data Standard (MEDS): Facilitating Machine Learning for Health , author =. Workshop on Time Series Learning for Health (TS4H) at ICLR , year =

  9. [9]

    npj Digital Medicine , year =

    Zero-shot health trajectory prediction using transformers , author =. npj Digital Medicine , year =. doi:10.1038/s41746-024-01235-0 , url =

    doi:10.1038/s41746-024-01235-0

  10. [10]

    2024 , publisher =

    MIMIC-IV (version 3.1) , author =. 2024 , publisher =. doi:10.13026/kpb9-mt58 , url =

    doi:10.13026/kpb9-mt58 2024

  11. [11]

    2023 , publisher =

    MIMIC-IV-ED (version 2.2) , author =. 2023 , publisher =. doi:10.13026/5ntk-km72 , url =

    doi:10.13026/5ntk-km72 2023

  12. [12]

    Circulation , year =

    PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals , author =. Circulation , year =

  13. [13]

    Advances in neural information processing systems , volume=

    wav2vec 2.0: A framework for self-supervised learning of speech representations , author=. Advances in neural information processing systems , volume=

  14. [14]

    JAMIA open , volume=

    Ecg-fm: An open electrocardiogram foundation model , author=. JAMIA open , volume=. 2025 , publisher=

    2025

  15. [15]

    arXiv preprint arXiv:2507.01045 , year=

    Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals , author=. arXiv preprint arXiv:2507.01045 , year=

    Pith/arXiv arXiv

  16. [16]

    Nature medicine , volume=

    The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care , author=. Nature medicine , volume=. 2018 , publisher=

    2018

  17. [17]

    Journal of biomedical informatics , volume=

    Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies , author=. Journal of biomedical informatics , volume=. 2022 , publisher=

    2022

  18. [18]

    Heliyon , volume=

    Multimodal risk prediction with physiological signals, medical images and clinical notes , author=. Heliyon , volume=. 2024 , publisher=

    2024

  19. [19]

    Journal of the American Medical Informatics Association , volume=

    The first step is the hardest: Pitfalls of representing and tokenizing temporal data for large language models , author=. Journal of the American Medical Informatics Association , volume=. 2024 , publisher=

    2024

  20. [20]

    arXiv preprint arXiv:2508.09182 , year=

    MedPatch: Confidence-Guided Multi-Stage Fusion for Multimodal Clinical Data , author=. arXiv preprint arXiv:2508.09182 , year=

    Pith/arXiv arXiv

  21. [21]

    Authorea Preprints , year=

    Foundation models for biosignals: A survey , author=. Authorea Preprints , year=

  22. [22]

    Machine Learning for Health , pages=

    CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks , author=. Machine Learning for Health , pages=. 2021 , organization=

    2021

  23. [23]

    Artificial Intelligence in Medicine , volume=

    Leveraging VQ-VAE tokenization for autoregressive modeling of medical time series , author=. Artificial Intelligence in Medicine , volume=. 2024 , publisher=

    2024

  24. [24]

    Briefings in bioinformatics , volume=

    Multimodal deep learning for biomedical data fusion: a review , author=. Briefings in bioinformatics , volume=. 2022 , publisher=

    2022

  25. [25]

    Scientific Reports , volume=

    Automated PD-L1 status prediction in lung cancer with multi-modal PET/CT fusion , author=. Scientific Reports , volume=. 2024 , publisher=

    2024

  26. [26]

    arXiv preprint arXiv:2509.09290 , year=

    Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training , author=. arXiv preprint arXiv:2509.09290 , year=

    Pith/arXiv arXiv

  27. [27]

    Machine Learning for Health (ML4H) , pages=

    Med-flamingo: a multimodal medical few-shot learner , author=. Machine Learning for Health (ML4H) , pages=. 2023 , organization=

    2023

  28. [28]

    Nejm Ai , volume=

    Towards generalist biomedical AI , author=. Nejm Ai , volume=. 2024 , publisher=

    2024

  29. [29]

    arXiv preprint arXiv:2510.23639 , year=

    Integrating Genomics into Multimodal EHR Foundation Models , author=. arXiv preprint arXiv:2510.23639 , year=

    arXiv

  30. [30]

    arXiv preprint arXiv:2506.07044 , year=

    Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning , author=. arXiv preprint arXiv:2506.07044 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2306.05109 , year=

    Yet another icu benchmark: A flexible multi-center framework for clinical ml , author=. arXiv preprint arXiv:2306.05109 , year=

    Pith/arXiv arXiv

  32. [32]

    Scientific reports , volume=

    Deep patient: an unsupervised representation to predict the future of patients from the electronic health records , author=. Scientific reports , volume=. 2016 , publisher=

    2016

  33. [33]

    arXiv preprint arXiv:2508.12104 , year=

    Generative medical event models improve with scale , author=. arXiv preprint arXiv:2508.12104 , year=

    arXiv

  34. [34]

    arXiv preprint arXiv:2505.22964 , year=

    Exploring Scaling Laws for EHR Foundation Models , author=. arXiv preprint arXiv:2505.22964 , year=

    Pith/arXiv arXiv

  35. [35]

    Advances in Neural Information Processing Systems , volume=

    Ehrshot: An ehr benchmark for few-shot evaluation of foundation models , author=. Advances in Neural Information Processing Systems , volume=

  36. [36]

    Foresight&

    Kraljevic, Zeljko and Bean, Dan and Shek, Anthony and Bendayan, Rebecca and Hemingway, Harry and Yeung, Joshua Au and Deng, Alexander and Balston, Alfred and Ross, Jack and Idowu, Esther and Teo, James T and Dobson, Richard J B , date =. Foresight&. The Lancet Digital Health , journal1 =. 2024 , year1 =. doi:10.1016/S2589-7500(24)00025-6 , isbn =

    doi:10.1016/s2589-7500(24)00025-6 2024

  37. [37]

    2025 , eprint=

    Generative Medical Event Models Improve with Scale , author=. 2025 , eprint=

    2025

  38. [38]

    GigaScience , volume =

    Renc, Pawel and Grzeszczyk, Michal K and Oufattole, Nassim and Goode, Deirdre and Jia, Yugang and Bieganski, Szymon and McDermott, Matthew B A and Was, Jaroslaw and Samir, Anthony E and Cunningham, Jonathan W and Bates, David W and Sitek, Arkadiusz , title =. GigaScience , volume =. 2025 , month =. doi:10.1093/gigascience/giaf107 , url =

    doi:10.1093/gigascience/giaf107 2025

  39. [39]

    2023 , month = jan, note =

    Johnson, Alistair and Bulgarelli, Lucas and Pollard, Tom and Horng, Steven and Celi, Leo Anthony and Mark, Roger , title =. 2023 , month = jan, note =. doi:10.13026/6mm1-ek67 , url =

    doi:10.13026/6mm1-ek67 2023

  40. [40]

    The Twelfth International Conference on Learning Representations , year=

    In-Context Pretraining: Language Modeling Beyond Document Boundaries , author=. The Twelfth International Conference on Learning Representations , year=

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.