REVIEW 6 cited by
Exploring Scaling Laws for EHR Foundation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Exploring Scaling Laws for EHR Foundation Models
read the original abstract
The emergence of scaling laws has profoundly shaped the development of large language models (LLMs), enabling predictable performance gains through systematic increases in model size, dataset volume, and compute. Yet, these principles remain largely unexplored in the context of electronic health records (EHRs) -- a rich, sequential, and globally abundant data source that differs structurally from natural language. In this work, we present the first empirical investigation of scaling laws for EHR foundation models. By training transformer architectures on patient timeline data from the MIMIC-IV database across varying model sizes and compute budgets, we identify consistent scaling patterns, including parabolic IsoFLOPs curves and power-law relationships between compute, model parameters, data size, and clinical utility. These findings demonstrate that EHR models exhibit scaling behavior analogous to LLMs, offering predictive insights into resource-efficient training strategies. Our results lay the groundwork for developing powerful EHR foundation models capable of transforming clinical prediction tasks and advancing personalized healthcare.
Forward citations
Cited by 6 Pith papers
-
On the Invariance and Generality of Neural Scaling Laws
Neural scaling laws are invariant under bijective data transformations and change predictably with information resolution ρ under non-bijective transformations, enabling cross-domain transport of fitted exponents.
-
Pretraining EHR Foundation Models with Patient-Aware Sampling
Patient-aware sampling of pretraining windows, with patients weighted by a tunable exponent, improves downstream AUROC/AUPRC over a global token-stream baseline in autoregressive EHR models.
-
Foundation Models to Unlock Real-World Evidence from Nationwide Medical Claims
A foundation model trained from scratch on nationwide medical claims data outperforms prior methods on over 1,000 disease prediction tasks and improves real-world evidence analyses.
-
TrajOnco: a multi-agent framework for temporal reasoning over longitudinal EHR for multi-cancer early detection
TrajOnco uses a chain-of-agents LLM architecture with memory to perform temporal reasoning on longitudinal EHR, achieving 0.64-0.80 AUROC for 1-year multi-cancer risk prediction in zero-shot mode on matched cohorts wh...
-
Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction
RAVEN pretrains on over one million EHR sequences via recurrence-aware next-visit event prediction, enabling zero-shot disease incidence forecasting that rivals fine-tuned models and generalizes across cohorts.
-
Autoregressive EHR Foundation Models with Multimodal Inputs
Latent compression improves multimodal fusion, but ICU mortality predictions from EHR plus ECG, CXR, and notes are generally no better than EHR-only.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.