Pith. sign in

REVIEW 5 major objections 4 minor 55 references

A hybrid Mamba–Transformer architecture for EHRs claims better predictions than nine baselines across ICU and pediatric tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 14:37 UTC pith:ASJ6J4T7

load-bearing objection A reasonable hybrid architecture, but the pediatric baseline rows are copied from MIMIC-IV and the statistical evidence is thin. the 5 major comments →

arxiv 2509.24118 v2 pith:ASJ6J4T7 submitted 2025-09-28 cs.LG

HyMaTE: A Hybrid Mamba and Transformer Model for EHR Representation Learning

Md Mozaharul Mottalib , Thao-Ly T. Phan , Rahmatollah Beheshti This is my paper
classification cs.LG
keywords EHR representation learningMambaState Space ModelsTransformerhybrid architectureself-supervised pretrainingattention interpretabilityclinical prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes HyMaTE, a hybrid model that combines Mamba's linear-time sequence encoding with Transformer-style attention to learn representations from electronic health records. The central claim is that this combination fixes two weaknesses seen separately in existing models: Transformers scale quadratically and can lose event-level information, while standalone Mamba focuses on sequence mixing but not channel mixing. Evaluated on mortality, length-of-stay, readmission, and pediatric weight-loss prediction, HyMaTE reports the best AUROC and AUPRC on most tasks, ahead of Transformer-based models like DuETT and STraTS and the standalone EHR-Mamba. The paper also argues that the fusion-attention layer yields interpretable temporal and global feature importance aligned with clinical knowledge.

Core claim

HyMaTE layers Mamba blocks, self-attention layers, and a fusion self-attention module over a triplet embedding of (time, feature, value) observations, then concatenates the temporal representation with an embedding of static demographics. A semi-supervised pretraining phase uses masked forecasting on the time-series variables before fine-tuning task heads. The paper reports that this stacked hybrid, trained with a fixed batch size, Adam, and early stopping, outperforms all nine baselines on MIMIC-IV mortality (AUROC 0.907), length of stay (0.802), and readmission (0.798), on pediatric weight-loss prediction (0.752), and is second-best on PhysioNet 2012 mortality (0.868). Ablations show that

What carries the argument

The architecture is a stack: triplet embeddings (time, feature, value) → M blocks of Mamba (each with RMS normalization, two linear projections, a convolution+SiLU, a discretized SSM with ZOH rule, and a multiplicative gate) → self-attention layers for local context → a fusion self-attention layer that computes a softmax-weighted sum of contextual embeddings to give a fixed-length temporal vector. This vector is concatenated with a two-layer FFN embedding of static variables, and the result is trained first with masked MSE forecasting loss over a forecast mask, then with a sigmoid head for the downstream binary task.

Load-bearing premise

The reported advantage over baselines assumes that all nine comparison models were implemented and tuned at least as carefully as HyMaTE, so the performance gaps reflect architecture rather than unequal tuning effort.

What would settle it

Rerun the same five tasks with a full hyperparameter search over learning rate, depth, hidden size, and dropout for every baseline (especially STraTS and DuETT) and compare against reported numbers; if the margin over the second-best model drops below statistical significance, the central claim of superior representation learning is not supported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If HyMaTE's reported results hold, a hybrid of linear-time SSM encoding and attention can match or beat pure-Transformer and pure-Mamba models on EHR prediction without sacrificing long-context processing.
  • The fusion-attention layer produces per-timestamp attention weights that can be inspected per patient, giving a built-in temporal explanation mechanism that may help clinical adoption.
  • The semi-supervised forecasting pretraining appears to be a major contributor to performance; dropping it reduces AUROC noticeably, especially on the pediatric dataset, suggesting a path for learning from unlabeled EHR data.
  • The model processes sequences up to thousands of events (pediatric maximum 8,654 observations) with only minimal AUROC decline in the 4K–8K length range, supporting the scalability claim.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The identical ablation drops for the self-attention and fusion-attention removals hint that the two attention modules may be functionally redundant in this configuration; a single, well-designed attention stage might achieve similar results with fewer parameters, though the paper does not test this.
  • If the baseline scores (e.g., STraTS at 0.692 AUROC on MIMIC-IV mortality) reflect under-tuning rather than true performance, then HyMaTE's advantage over its strongest competitors could shrink under a fair hyperparameter search; a reader should look for external reproductions before treating the margin as architectural.
  • The same triplet-embedding-plus-hybrid recipe could generalize to other sparse, irregular multivariate sequences (e.g., wearable biosignals or financial event streams), where both long-range dependence and inter-channel correlation matter, but the paper only demonstrates this on clinical data.
  • A falsifiable extension is to train HyMaTE without the pretraining phase on a larger labeled cohort; the paper's ablation suggests pretraining matters most on small or imbalanced datasets like the pediatric one, so the benefit may diminish with more labels.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The manuscript proposes HyMaTE, a hybrid architecture for EHR representation learning that combines Mamba blocks for linear-time sequence encoding with Transformer-style self-attention and a fusion attention layer for aggregation. Input events are embedded as time/value/feature triplets; a self-supervised masked-forecasting pretraining phase is followed by supervised fine-tuning. The model is evaluated on PhysioNet Challenge 2012, MIMIC-IV (three prediction tasks), and a private pediatric weight-management dataset, comparing against nine baselines. The authors report mean±SD over 10 runs for AUROC and AUPRC and claim consistent superiority over Transformer and Mamba baselines, plus interpretability via attention weights and final-layer coefficients. Code is released.

Significance. If the empirical claims are established, HyMaTE would be a useful and scalable architecture for longitudinal EHR modeling: it explicitly addresses channel mixing in Mamba, uses a semi-supervised pretraining objective, and provides an interpretability analysis. The paper ships code, reports repeated runs, and uses standard held-out evaluation, which are strengths. However, the central performance claim currently rests on comparison tables with internal inconsistencies, under-specified baseline tuning, and no significance testing. These concerns must be resolved before the claims can be accepted.

major comments (5)
  1. [Table 1 & Appendix Table 4] In the Pediatric weight-loss block, the AUROC/AUPRC entries for GRU, TCN, SAnD, GRU-D, InterpNet, SeFT, and STraTS are exactly the same as the MIMIC-IV mortality block, e.g., STraTS 0.692±0.002/0.295±0.002 and SeFT 0.644±0.007/0.288±0.001 appear identically. The two datasets have different sizes and distributions (14,392 pediatric records vs 53,150 MIMIC-IV admissions), so identical results are not credible. This appears to be a copy/paste error, but as printed it invalidates the pediatric comparison for those baselines and weakens the 'consistent across datasets' claim in Section 5 and the abstract. The authors must replace these rows with actual experimental results (or remove the pediatric dataset from the comparison) and re-evaluate the conclusions.
  2. [Section 4.3] Implementation details only state a fixed batch size of 32, Adam optimizer, and early stopping with patience 10. No hyperparameter ranges, search budgets, or final hyperparameters are reported for either HyMaTE or the baselines. Some baseline scores are far below the values reported in the original publications; for example, STraTS on MIMIC-IV mortality achieves AUROC 0.692 in Table 1, while STraTS was designed for irregularly sampled time series and typically reports substantially higher performance on comparable ICU tasks. This raises the possibility that the reported gains over baselines reflect under-tuning rather than architectural advantage. The authors should either provide full tuning details or demonstrate that baselines were tuned to their optimal performance.
  3. [Section 5, Table 1] Across several tasks, the difference between HyMaTE and the best baseline is within one standard deviation. On PhysioNet mortality, HyMaTE AUROC 0.868±0.045 vs DuETT 0.857±0.018; on MIMIC LOS, 0.802±0.055 vs EHR-Mamba 0.781±0.027; on MIMIC readmission, 0.798±0.035 vs DuETT 0.772±0.024. No significance tests (e.g., paired bootstrap or Wilcoxon) are reported, and the larger standard deviations for HyMaTE on some metrics make it unclear that the observed gaps are reproducible. The abstract's 'outperforms' claim is not supported by the statistics as reported. Add significance tests or temper the claims.
  4. [Section 5.1, Table 2] The text states that removing either the self-attention layer or the attention fusion mechanism 'resulted in identical performance drops across all datasets.' Table 2 shows different values for these two ablations on every task (e.g., PhysioNet 0.855 vs 0.842; MIMIC mortality 0.861 vs 0.868). Also, the sentence 'the hybrid model's performance may not surpass that of the standalone Mamba block model' is unclear and appears to contradict the ablation 'w/o Mamba blocks,' which shows a large drop relative to HyMaTE. Please correct the text and provide a consistent explanation of what each ablation removes and what the results imply.
  5. [Section 5.2, Table 3] The global feature importance scores are computed from the weights of the final predictive dense layer (Eq. 6). Since this layer operates on the concatenated demographic and temporal embeddings, its weights do not directly attribute importance to raw input features; the mapping is nonlinear and confounded by the embedding layers. For example, 'NIMAP' contribution of 4.481 is not an interpretable effect size on the original scale. The interpretability claims should be reformulated as model-internal importance measures, or validated against a proper feature-attribution method (e.g., permutation importance or integrated gradients).
minor comments (4)
  1. [Section 3.2.4] Equation numbering is duplicated: Section 3.2.4 uses equations numbered (1)–(6), but equations (1)–(8) already appeared in Section 3.2.2. Please renumber.
  2. [Section 4.1.1] The sentence 'for predicting mortality within the subsequent 2-hour period' is confusing; the PhysioNet Challenge 2012 task uses the first 48 hours of ICU stay and predicts in-hospital mortality after the observation window. Please clarify the prediction window.
  3. [Section 3.2.1] The embedding dimension is denoted both \(\tau\) and \(d_t\) in the text; use consistent notation.
  4. [Abstract / Conclusion] The phrase 'state-of-the-art performance' is used without a formal comparison to all recently published SOTA methods; consider replacing with 'competitive performance' given the statistical caveats above.

Circularity Check

0 steps flagged

No circular derivation: HyMaTE's predictions are genuine out-of-sample results; the noted baseline-table duplication is a reporting/integrity error, not a circular step.

full rationale

HyMaTE is an empirical architecture paper. The derivation chain is: define triplet embedding, Mamba contextual encoding, self-attention, fusion-attention, pretraining forecast loss (Eq. 14), fine-tuning sigmoid head (Eq. 6). Each component is trained on training/validation splits and evaluated on held-out test sets; no fitted constant is renamed as a prediction and no equation defines the target in terms of itself. The self-citations (e.g., refs 11,12,17,20,31 for the pediatric task) only supply dataset/task definitions and prior context, not the architecture's superiority, so they are not load-bearing. The serious issues in the paper—the pediatric block of Table 1/Table 4 reproducing MIMIC-IV mortality rows for seven baselines (e.g., STraTS 0.692±0.002/0.295±0.002 and SeFT 0.644±0.007/0.288±0.001 appear identically in both), the absence of reported hyperparameter search (§4.3), the lack of significance tests, and the §5.1 admission that 'the hybrid model's performance may not surpass that of the standalone Mamba block model'—bear on result credibility and evaluation fairness, not on circularity. None of these makes the reported predictions equivalent to the inputs by construction. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central claim rests on standard ML training assumptions plus several unstated configuration choices. No new entities are introduced. The free parameters are the model hyperparameters, none of which are reported, making the architecture underspecified.

free parameters (6)
  • Number of Mamba blocks M and layers h = not reported
    Architecture depth chosen by validation performance; values not disclosed anywhere in the paper.
  • Mamba state size psi = not reported
    Internal state dimension of the SSM used in Eq. 1-4; set without reporting.
  • Embedding dimensions (tau, d, d_a) = not reported
    Dimensions of token embeddings, demographic embedding, and fusion attention hidden layer; not stated.
  • Self-attention layers and heads = not reported
    Configuration of the self-attention block in Section 3.2.3 is unspecified.
  • Learning rate and optimizer details = Adam, batch 32; LR not reported
    Section 4.3 gives batch size and optimizer but no learning rate or schedule.
  • Forecast window and masking ratio for pre-training = not reported
    Section 3.3.1 defines the masked forecasting loss but does not specify the forecast horizon or mask fraction.
axioms (4)
  • domain assumption Triplet representation (time, feature, value) captures sufficient information from EHR
    Section 3.2.1 assumes the triplet encoding preserves clinically relevant signal; no analysis justifies this beyond prior work [22, 53].
  • standard math Zero-order hold discretization of the SSM state equation
    Standard discretization from Mamba [15], used in Eqs. 5-6; accepted background.
  • domain assumption Self-supervised forecasting pre-training transfers to downstream classification
    Section 3.3.1 asserts this; the ablation shows a drop when removed, but no external validation or theoretical justification is given.
  • domain assumption Fusion attention weights are interpretable as variable/timestamp importance
    Section 5.2 interprets attention weights without validating them against clinician labels, causal ground truth, or a formal explanation metric.

pith-pipeline@v1.3.0-alltime-deepseek · 15645 in / 12598 out tokens · 359776 ms · 2026-08-04T14:37:22.070909+00:00 · methodology

0 comments
read the original abstract

Electronic health Records (EHRs) have become a cornerstone in modern-day healthcare. They are a crucial part for analyzing the progression of patient health; however, their complexity, characterized by long, multivariate sequences, sparsity, and missing values poses significant challenges in traditional deep learning modeling. While Transformer-based models have demonstrated success in modeling EHR data and predicting clinical outcomes, their quadratic computational complexity and limited context length hinder their efficiency and practical applications. On the other hand, State Space Models (SSMs) like Mamba present a promising alternative offering linear-time sequence modeling and improved efficiency for handling long sequences, but focus mostly on mixing sequence-level information rather than channel-level data. To overcome these challenges, we propose HyMaTE (A Hybrid Mamba and Transformer Model for EHR Representation Learning), a novel hybrid model tailored for representing longitudinal data, combining the strengths of SSMs with advanced attention mechanisms. By testing the model on predictive tasks on multiple clinical datasets, we demonstrate HyMaTE's ability to capture an effective, richer, and more nuanced unified representation of EHR data. Additionally, the interpretability of the outcomes achieved by self-attention illustrates the effectiveness of our model as a scalable and generalizable solution for real-world healthcare applications. Codes are available at: https://github.com/healthylaife/HyMaTE.

Figures

Figures reproduced from arXiv: 2509.24118 by Md Mozaharul Mottalib, Rahmatollah Beheshti, Thao-Ly T. Phan.

Figure 1
Figure 1. Figure 1: The proposed model architecture: HyMaTE 3 Methods We initially introduce the essential notations for problem definition, followed by an outline of the input data structure and the proposed model architecture. We then detail the self-supervised pre-training methodology, which utilizes masked forecasting, and the subsequent fine-tuning process for the downstream task. 3.1 Problem Definition Our model operate… view at source ↗
Figure 2
Figure 2. Figure 2: Mean AUROC vs sequence length [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 7 linked inside Pith

  1. [1]

    Farzana I Adiba, Yifan Zhang, and Rahmatollah Beheshti. 2025. Bias and Fairness in Medical LLMs: An Extensive Scoping Review. doi:10.31219/osf. io/fqejh_v1

    doi:10.31219/osf 2025

  2. [2]

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271(2018)

    Pith/arXiv arXiv 2018

  3. [3]

    João Bento, Pedro Saleiro, André F Cruz, Mário AT Figueiredo, and Pedro Bizarro. 2021. Timeshap: Explaining recurrent models through sequence perturbations. InProceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2565–2573

    2021

  4. [4]

    Aaron Blakeman, Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkar, et al. 2025. Nemotron-h: A family of accurate and efficient hybrid mamba-transformer models.arXiv preprint arXiv:2504.03624(2025)

    Pith/arXiv arXiv 2025

  5. [5]

    Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. 2018. Recurrent neural networks for multivariate time series with missing values.Scientific reports8, 1 (2018), 6085

    2018

  6. [6]

    Xiuwei Chen, Sihao Lin, Xiao Dong, Zisheng Chen, Meng Cao, Jianhua Han, Hang Xu, and Xiaodan Liang. 2025. Transmamba: Fast universal architecture adaption from transformers to mamba.arXiv preprint arXiv:2502.15130(2025)

    arXiv 2025

  7. [7]

    Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. 2016. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism.Advances in neural information processing systems29 (2016)

    2016

  8. [8]

    Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling.arXiv preprint arXiv:1412.3555(2014)

    Pith/arXiv arXiv 2014

  9. [9]

    Ting Deng, Die Wu, Shan-shan Liu, Xing-lin Chen, Zhen-wei Zhao, and Lan-lang Zhang. 2025. Association of blood urea nitrogen with 28-day mortality in critically ill patients: A multi-center retrospective study based on the eICU collaborative research database.Plos one20, 1 (2025), e0317315

    2025

  10. [10]

    Adibvafa Fallahpour, Mahshid Alinoori, Arash Afkanpour, and Amrit Krishnan. 2024. EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records.arXiv preprint arXiv:2405.14567(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Timothy Bunnell, Thao-Ly T

    Hamed Fayyaz, Mehak Gupta, Alejandra Perez Ramirez, Claudine Jurkovitz, H. Timothy Bunnell, Thao-Ly T. Phan, and Rahmatollah Beheshti. 2025. An Interoperable Machine Learning Pipeline for Pediatric Obesity Risk Estimation. InProceedings of the 4th Machine Learning for Health Symposium (Proceedings of Machine Learning Research, Vol. 259), Stefan Hegselmann...

    2025

  12. [12]

    Hamed Fayyaz, Thao-Ly T. Phan, H. Timothy Bunnell, and Rahmatollah Beheshti. 2022. Predicting attrition patterns from pediatric weight management programs. InProceedings of the 2nd Machine Learning for Health symposium (Proceedings of Machine Learning Research, Vol. 193), 14 Mottalib et al. Antonio Parziale, Monica Agrawal, Shalmali Joshi, Irene Y. Chen, ...

    2022

  13. [13]

    Hamed Fayyaz, Raphael Poulain, and Rahmatollah Beheshti. 2025. Enabling Scalable Evaluation of Bias Patterns in Medical LLMs. arXiv:2410.14763 [cs.CL] https://arxiv.org/abs/2410.14763

    Pith/arXiv arXiv 2025

  14. [14]

    Mohan Giri, Lin He, Tianyang Hu, Anju Puri, Xiaozhuo Zheng, Haiyun Dai, and Shuliang Guo. 2022. Blood urea nitrogen is associated with in-hospital mortality in critically ill patients with acute exacerbation of chronic obstructive pulmonary disease: a propensity score matching analysis. Journal of clinical medicine11, 22 (2022), 6709

    2022

  15. [15]

    Albert Gu and Tri Dao. 2024. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. InFirst Conference on Language Modeling

    2024

  16. [16]

    Albert Gu, Karan Goel, and Christopher Re. 2022. Efficiently Modeling Long Sequences with Structured State Spaces. InInternational Conference on Learning Representations

    2022

  17. [17]

    Mehak Gupta, Daniel Eckrich, H Timothy Bunnell, Thao-Ly T Phan, and Rahmatollah Beheshti. 2024. Reliable prediction of childhood obesity using only routinely collected EHRs may be possible.Obesity Pillars12 (2024), 100128

    2024

  18. [18]

    Mehak Gupta, Thao-Ly T. Phan, H. Timothy Bunnell, and Rahmatollah Beheshti. 2021. Concurrent imputation and prediction on EHR data using bi-directional GANs: Bi-GANs for EHR imputation and prediction. InProceedings of the 12th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics(Gainesville, Florida)(BCB ’21). Asso...

    arXiv 2021

  19. [19]

    Mehak Gupta, Thao-Ly T Phan, H Timothy Bunnell, and Rahmatollah Beheshti. 2022. Obesity Prediction with EHR Data: A deep learning approach with interpretable elements.ACM Transactions on Computing for Healthcare (HEALTH)3, 3 (2022), 1–19

    2022

  20. [20]

    Mehak Gupta, Thao-Ly T Phan, Félice Lê-Scherban, Daniel Eckrich, H Timothy Bunnell, and Rahmatollah Beheshti. 2025. Associations of longitudinal BMI-percentile classification patterns in early childhood with neighborhood-level social determinants of health.Childhood obesity21, 1 (2025), 65–75

    2025

  21. [21]

    Mehak Gupta, Raphael Poulain, Thao-Ly T Phan, H Timothy Bunnell, and Rahmatollah Beheshti. 2022. Flexible-window predictions on electronic health records. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 12510–12516

    2022

  22. [22]

    Max Horn, Michael Moor, Christian Bock, Bastian Rieck, and Karsten Borgwardt. 2020. Set functions for time series. InInternational Conference on Machine Learning. PMLR, 4353–4363

    2020

  23. [23]

    Dino Ienco and Roberto Interdonato. 2020. Deep multivariate time series embedding clustering via attentive-gated autoencoder. InAdvances in Knowledge Discovery and Data Mining: 24th Pacific-Asia Conference, PAKDD 2020, Singapore, May 11–14, 2020, Proceedings, Part I 24. Springer, 318–329

    2020

  24. [24]

    Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. 2020. Mimic-iv.PhysioNet. A vailable online at: https://physionet. org/content/mimiciv/1.0/(accessed August 23, 2021)(2020), 49–55

    2020

  25. [25]

    Alex Labach, Aslesha Pokhrel, Xiao Shi Huang, Saba Zuberi, Seung Eun Yi, Maksims Volkovs, Tomi Poutanen, and Rahul G Krishnan. 2023. DuETT: dual event time transformer for electronic health records. InMachine Learning for Healthcare Conference. PMLR, 403–422

    2023

  26. [26]

    Yikuan Li, Mohammad Mamouei, Gholamreza Salimi-Khorshidi, Shishir Rao, Abdelaali Hassaine, Dexter Canoy, Thomas Lukasiewicz, and Kazem Rahimi. 2022. Hi-BEHRT: hierarchical transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records.IEEE journal of biomedical and health informatics27, 2 (2022),...

    2022

  27. [27]

    Yikuan Li, Shishir Rao, José Roberto Ayala Solares, Abdelaali Hassaine, Rema Ramakrishnan, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi-Khorshidi. 2020. BEHRT: transformer for electronic health records.Scientific reports10, 1 (2020), 7155

    2020

  28. [28]

    Yixing Li, Ruobing Xie, Zhen Yang, Xingwu Sun, Shuaipeng Li, Weidong Han, Zhanhui Kang, Yu Cheng, Chengzhong Xu, Di Wang, et al. 2025. Transmamba: Flexibly switching between transformer and mamba.arXiv preprint arXiv:2503.24067(2025)

    arXiv 2025

  29. [29]

    Patrick G Lyons, Jeff Klaus, Colleen A McEvoy, Peter Westervelt, Brian F Gage, and Marin H Kollef. 2019. Factors associated with clinical deterioration among patients hospitalized on the wards at a tertiary cancer hospital.Journal of oncology practice15, 8 (2019), e652–e665

    2019

  30. [30]

    Rupa Makadia and Patrick B Ryan. 2014. Transforming the premier perspective®hospital database into the observational medical outcomes partnership (omop) common data model.Egems2, 1 (2014), 1110

    2014

  31. [31]

    Md Mozaharul Mottalib, Jessica C Jones-Smith, Bethany Sheridan, and Rahmatollah Beheshti. 2023. Subtyping Patients With Chronic Disease Using Longitudinal BMI Patterns.IEEE Journal of Biomedical and Health Informatics27, 4 (2023), 2083–2093. doi:10.1109/JBHI.2023.3237753

    arXiv 2023

  32. [32]

    Michael C Mozer, Denis Kazakov, and Robert V Lindsey. 2017. Discrete event, continuous time rnns.arXiv preprint arXiv:1710.04110(2017)

    Pith/arXiv arXiv 2017

  33. [33]

    Amin Nayebi, Sindhu Tipirneni, Chandan K Reddy, Brandon Foreman, and Vignesh Subbian. 2023. WindowSHAP: An efficient framework for explaining time-series classifiers based on Shapley values.Journal of biomedical informatics144 (2023), 104438

    2023

  34. [34]

    Diana Cristina Buendía Palacios, João André Freitas Silva, Ana Francisca Torres Sarmento, and Michael Gregorio Ortega Sierra. 2023. Mean arterial pressure and outcomes in critically ill patients: is there a difference between high and low target?Revista da Associação Médica Brasileira69, 6 (2023), e20230162

    2023

  35. [35]

    Chao Pang, Xinzhuo Jiang, Krishna S Kalluri, Matthew Spotnitz, RuiJun Chen, Adler Perotte, and Karthik Natarajan. 2021. CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks. InMachine Learning for Health. PMLR, 239–260

    2021

  36. [36]

    Badri Narayana Patro and Vijay Srinivas Agneeswaran. 2025. SiMBA-TS: Simplified Channel Mixing and Mamba for Long-term Time Series Forecasting. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5. HyMaTE 15

    2025

  37. [37]

    Fahmida Liza Piya, Mehak Gupta, and Rahmatollah Beheshti. 2024. Healthgat: Node classifications in electronic health records using graph attention networks. In2024 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE). IEEE, 132–141

    2024

  38. [38]

    Raphael Poulain and Rahmatollah Beheshti. 2024. Graph transformers on EHRs: Better representation improves downstream performance. InThe Twelfth International Conference on Learning Representations

    2024

  39. [39]

    Raphael Poulain, Hamed Fayyaz, and Rahmatollah Beheshti. 2024. Aligning (Medical) LLMs for (Counterfactual) Fairness. arXiv:2408.12055 [cs.CL] https://arxiv.org/abs/2408.12055

    Pith/arXiv arXiv 2024

  40. [40]

    Raphael Poulain, Mehak Gupta, and Rahmatollah Beheshti. 2022. Few-Shot Learning with Semi-Supervised Transformers for Electronic Health Records. InProceedings of the 7th Machine Learning for Healthcare Conference (Proceedings of Machine Learning Research, Vol. 182), Zachary Lipton, Rajesh Ranganath, Mark Sendak, Michael Sjoding, and Serena Yeung (Eds.). P...

    2022

  41. [41]

    Raphael Poulain, Mehak Gupta, Randi Foraker, and Rahmatollah Beheshti. 2021. Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease. In2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). 726–731. doi:10.1109/BIBM52615.2021.9669441

    arXiv 2021

  42. [42]

    Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M Dai, Nissan Hajaj, Michaela Hardt, Peter J Liu, Xiaobing Liu, Jake Marcus, Mimi Sun, et al. 2018. Scalable and accurate deep learning with electronic health records.NPJ digital medicine1, 1 (2018), 1–10

    2018

  43. [43]

    Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. 2021. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction.NPJ digital medicine4, 1 (2021), 86

    2021

  44. [44]

    Houxing Ren, Jingyuan Wang, Wayne Xin Zhao, and Ning Wu. 2021. Rapt: Pre-training of time-aware transformer for learning robust healthcare representation. InProceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 3503–3511

    2021

  45. [45]

    Maurice Rupp, Oriane Peter, and Thirupathi Pattipaka. 2023. Exbehrt: Extended transformer for electronic health records. InInternational Workshop on Trustworthy Machine Learning for Healthcare. Springer, 73–84

    2023

  46. [46]

    Sauradeep Sarkar, Sahib Singh, and Amit Rout. 2022. Mean arterial pressure goal in critically ill patients: a meta-analysis of randomized controlled trials.Journal of Clinical Medicine Research14, 5 (2022), 196

    2022

  47. [47]

    Ying Shao and Jinzhu Hu. 2022. U-Shaped Association between blood pressure and mortality risk in ICU patients with Atrial Fibrillation: the MIMIC-III database.Frontiers in Cardiovascular Medicine9 (2022), 866260

    2022

  48. [48]

    Benjamin Shickel, Tyler J Loftus, Lasith Adhikari, Tezcan Ozrazgat-Baslanti, Azra Bihorac, and Parisa Rashidi. 2019. DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning.Scientific reports9, 1 (2019), 1879

    2019

  49. [49]

    Satya Narayan Shukla and Benjamin Marlin. 2019. Interpolation-Prediction Networks for Irregularly Sampled Time Series. InInternational Conference on Learning Representations

    2019

  50. [50]

    Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. 2012. Predicting in-hospital mortality of icu patients: The phys- ionet/computing in cardiology challenge 2012. In2012 computing in cardiology. IEEE, 245–248

    2012

  51. [51]

    Huan Song, Deepta Rajan, Jayaraman Thiagarajan, and Andreas Spanias. 2018. Attend and diagnose: Clinical time series analysis using attention models. InProceedings of the AAAI conference on artificial intelligence, Vol. 32

    2018

  52. [52]

    Akshay Sood and Mark Craven. 2022. Feature importance explanations for temporal black-box models. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8351–8360

    2022

  53. [53]

    Sindhu Tipirneni and Chandan K Reddy. 2022. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series. ACM Transactions on Knowledge Discovery from Data (TKDD)16, 6 (2022), 1–17

    2022

  54. [54]

    Sana Tonekaboni, Danny Eytan, and Anna Goldenberg. 2021. Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding. InInternational Conference on Learning Representations

    2021

  55. [55]

    Xiaocui Zhu, Qunsheng Ruan, Sai Qian, and Miaohui Zhang. 2025. A hybrid model based on transformer and Mamba for enhanced sequence modeling.Scientific Reports15, 1 (2025), 11428. A Appendix A.1 Extended Baseline Models Here is a complete list of all9baseline models used for evaluation with results in 4. A.1.1 Gated Recurrent Unit (GRU).[ 8] Hourly aggrega...

    2025