REVIEW 7 cited by
EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Transformers have significantly advanced the modeling of Electronic Health Records (EHR), yet their deployment in real-world healthcare is limited by several key challenges. Firstly, the quadratic computational cost and insufficient context length of these models hinder hospitals' ability in processing the extensive medical histories typical in EHR data. Additionally, existing models employ separate finetuning for each clinical task, complicating maintenance in healthcare environments. Moreover, these models focus exclusively on either clinical prediction or EHR forecasting, lacking proficiency in both tasks. To overcome these limitations, we introduce EHRMamba, a robust foundation model built on the Mamba architecture. EHRMamba can process sequences up to 300% longer than previous models due to its linear computational cost. We also introduce a novel approach to Multitask Prompted Finetuning (MPF) for EHR data, which enables EHRMamba to simultaneously learn multiple clinical tasks in a single finetuning phase, significantly enhancing deployment and cross-task generalization. Furthermore, our model leverages the HL7 FHIR data standard to simplify integration into existing hospital systems. Alongside EHRMamba, we open-source Odyssey, a toolkit designed to support the development and deployment of EHR foundation models, with an emphasis on data standardization and interpretability. Our evaluations on the MIMIC-IV dataset demonstrate that EHRMamba advances state-of-the-art performance across 6 major clinical tasks and excels in EHR forecasting, marking a significant leap forward in the field.
Forward citations
Cited by 7 Pith papers
-
Expanders Meet Reed-Muller: Easy Instances of Noisy k-XOR
Explicit near-optimal expanders exist for which noisy k-XOR is polynomial-time solvable, falsifying conjectures that expansion implies hardness.
-
CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records
CEHR-XGPT unifies feature representation, zero-shot prediction, and synthetic data generation in a single GPT-2 style EHR model using artificial time tokens with time-decomposition and time-to-event losses.
-
Foundation Models for Clinical Records at Health System Scale
A decoder-only Transformer pretrained with next-visit multi-label prediction on EHRs achieves zero-shot dementia and knee OA forecasting comparable to a fully fine-tuned BERT baseline.
-
Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions
A foundation model of wearable behavioral data outperforms simple baselines and complements a PPG sensor model across 57 health detection tasks.
-
EHR2Path: Comprehensive Pathway-Level Modeling of Longitudinal Patient Trajectories from Multimodal Electronic Health Records
A single LLM fine-tuned with a learned summary bottleneck forecasts next-hour EHR states and iteratively simulates multi-hour patient trajectories across ED, ward, and ICU on MIMIC-IV.
-
FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records
FoMoH benchmarks six structured EHR foundation models on 14 tasks and finds they do not consistently outperform supervised baselines, particularly for rare diseases and low-data regimes.
-
HyMaTE: A Hybrid Mamba and Transformer Model for EHR Representation Learning
A hybrid Mamba-Transformer architecture with attention pooling reports higher AUROC and AUPRC than several EHR baselines on five clinical prediction tasks, though gains are small relative to reported variance.
Discussion (0). Continue with ORCID to comment.