Recognition: unknown
Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingness
Pith reviewed 2026-05-09 21:55 UTC · model grok-4.3
The pith
A multimodal encoder and Bayesian filter learn dynamic patient states from clinical time series by treating missingness as informative signal.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We propose a patient representation learning framework that explicitly leverages informative missingness. The framework combines (1) a multimodal encoder that captures signals from structured and textual data together with their observation patterns, (2) a Bayesian filtering module that updates a latent patient state over time from observed multimodal signals, and (3) downstream modules for offline treatment policy learning and patient outcome prediction based on the learned patient state. On ICU sepsis cohorts it improves both policy quality and outcome prediction.
What carries the argument
The multimodal encoder that jointly processes data values and their observation patterns, paired with the Bayesian filtering module that maintains and updates a latent patient state.
If this is right
- Offline treatment policies learned from the representations achieve higher FQE (0.679) than observed clinician behavior (0.528) on MIMIC-III.
- Post-72-hour mortality prediction reaches AUROC 0.886 on MIMIC-III, outperforming methods that do not model informative missingness.
- The same framework applies to sepsis cohorts in MIMIC-IV and eICU, showing consistent gains across data sources.
- Both structured time series and free-text notes are integrated while respecting their distinct recording processes.
Where Pith is reading between the lines
- If missingness patterns prove stable across hospitals, the learned states could support policy transfer to new ICUs without full retraining.
- The Bayesian filter could be extended to longer horizons to handle chronic-disease trajectories beyond acute sepsis episodes.
- Expert review of whether the extracted latent states align with known clinical risk factors would test practical interpretability.
Load-bearing premise
Observation patterns across structured measurements and clinical notes carry recoverable signal about the latent patient state that the encoder and filter can extract without bias or overfitting to dataset-specific recording practices.
What would settle it
An ablation study on a new ICU dataset in which the full model is compared to an otherwise identical version that ignores all missingness indicators; if the performance gap disappears, the claim that missingness patterns supply usable signal would be falsified.
Figures
read the original abstract
Multimodal clinical records contain structured measurements and clinical notes recorded over time, offering rich temporal information about the evolution of patient health. Yet these observations are sparse, and whether they are recorded depends on the patient's latent condition. Observation patterns also differ across modalities, as structured measurements and clinical notes arise under distinct recording processes. While prior work has developed methods that accommodate missingness in clinical time series, how to extract and use the information carried by the observation process itself remains underexplored. We therefore propose a patient representation learning framework for multimodal clinical time series that explicitly leverages informative missingness. The framework combines (1) a multimodal encoder that captures signals from structured and textual data together with their observation patterns, (2) a Bayesian filtering module that updates a latent patient state over time from observed multimodal signals, and (3) downstream modules for offline treatment policy learning and patient outcome prediction based on the learned patient state. We evaluate the framework on ICU sepsis cohorts from MIMIC-III, MIMIC-IV, and eICU. It improves both offline treatment policy learning and adverse outcome prediction, achieving FQE 0.679 versus 0.528 for clinician behavior and AUROC 0.886 for post-72-hour mortality prediction on MIMIC-III.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a patient representation learning framework for multimodal clinical time series (structured measurements and clinical notes) that explicitly incorporates informative missingness. It combines a multimodal encoder capturing signals from both data types and their observation patterns, a Bayesian filtering module to maintain a latent patient state over time, and downstream modules for offline treatment policy learning via FQE and adverse outcome prediction. Evaluation on ICU sepsis cohorts from MIMIC-III, MIMIC-IV, and eICU shows gains over clinician behavior (FQE 0.679 vs. 0.528 on MIMIC-III) and strong predictive performance (AUROC 0.886 for post-72h mortality on MIMIC-III).
Significance. If the empirical gains hold after proper controls, the work would demonstrate that observation patterns in multimodal clinical data carry recoverable, generalizable signal about latent state, advancing both representation learning and offline RL for clinical decision support. The use of multiple public cohorts (MIMIC-III/IV, eICU) and focus on sepsis is a strength for potential reproducibility and impact in critical care.
major comments (2)
- [Evaluation / Experiments] Experimental evaluation: The reported improvements (FQE 0.679 vs. 0.528, AUROC 0.886) are presented without details on baseline implementations, statistical testing (e.g., confidence intervals or p-values across runs), or ablation studies isolating the informative missingness component from the multimodal encoder and Bayesian filter. This leaves the central claim only partially supported, as gains could arise from other architectural choices or dataset-specific artifacts.
- [Evaluation / Experiments] Cross-dataset evaluation: While the abstract states evaluation across MIMIC-III/IV and eICU, no cross-dataset transfer results or ablations are described that would test whether the missingness signal generalizes beyond dataset-specific recording practices (e.g., MIMIC vs. eICU differences in note timing or measurement frequency). This directly bears on the weakest assumption that observation patterns carry recoverable, non-biased signal about latent state.
minor comments (1)
- [Introduction] The abstract and introduction could more clearly distinguish the proposed framework's handling of modality-specific missingness processes from prior work on missingness in clinical time series.
Simulated Author's Rebuttal
We thank the referee for the thoughtful and constructive review. We address each major comment below, indicating where we agree that revisions are needed and outlining the changes we will make to strengthen the manuscript.
read point-by-point responses
-
Referee: Experimental evaluation: The reported improvements (FQE 0.679 vs. 0.528, AUROC 0.886) are presented without details on baseline implementations, statistical testing (e.g., confidence intervals or p-values across runs), or ablation studies isolating the informative missingness component from the multimodal encoder and Bayesian filter. This leaves the central claim only partially supported, as gains could arise from other architectural choices or dataset-specific artifacts.
Authors: We agree that the original manuscript provides insufficient detail on baseline implementations and lacks statistical testing and targeted ablations. In the revised version, we will add: (i) explicit descriptions of all baseline architectures, training procedures, and hyperparameter choices; (ii) performance metrics with 95% confidence intervals and p-values computed across at least five independent runs with different random seeds; and (iii) ablation experiments that systematically disable the informative missingness encoding while keeping the multimodal encoder and Bayesian filter intact. These additions will isolate the contribution of the missingness component and better substantiate the central claims. revision: yes
-
Referee: Cross-dataset evaluation: While the abstract states evaluation across MIMIC-III/IV and eICU, no cross-dataset transfer results or ablations are described that would test whether the missingness signal generalizes beyond dataset-specific recording practices (e.g., MIMIC vs. eICU differences in note timing or measurement frequency). This directly bears on the weakest assumption that observation patterns carry recoverable, non-biased signal about latent state.
Authors: We acknowledge that separate per-dataset results do not directly test whether the informative missingness signal transfers across institutions with differing recording practices. The original submission reports performance on each cohort independently but does not include cross-dataset transfer experiments. In the revision, we will add transfer-learning results (training on one dataset and evaluating on the others) together with ablations that examine the stability of the learned missingness patterns under domain shift. These experiments will provide direct evidence regarding the generalizability of the observation-pattern signal. revision: yes
Circularity Check
No significant circularity detected
full rationale
The paper's core contribution is an empirical framework combining a multimodal encoder, Bayesian filtering for latent state updates, and downstream policy/outcome modules, evaluated via held-out performance metrics (FQE 0.679 vs. 0.528, AUROC 0.886) on public ICU cohorts from MIMIC-III/IV and eICU. No equations, self-citations, or uniqueness theorems are invoked that reduce these results to fitted inputs or definitional equivalences by construction. The derivation chain remains self-contained as standard representation learning plus filtering, with gains demonstrated through external data splits rather than any load-bearing self-referential step.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Johnson, Alistair E. W. and Pollard, Tom J. and Shen, Lu and Lehman, Li-wei H. and Feng, Mengling and Ghassemi, Mohammad and Moody, Benjamin and Szolovits, Peter and Celi, Leo Anthony and Mark, Roger G. , journal =. MIMIC-III, a freely accessible critical care database , volume =
-
[2]
Johnson, Alistair E. W. and Bulgarelli, Lucas and Shen, Lu and Gayles, Alvin and Shammout, Ayad and Horng, Steven and Pollard, Tom J. and Hao, Sicheng and Moody, Benjamin and Gow, Brian and Lehman, Li-wei H. and Celi, Leo A. and Mark, Roger G. , journal =. MIMIC-IV, a freely accessible electronic health record dataset , volume =
-
[3]
Recurrent Neural Networks for Multivariate Time Series with Missing Values , volume =
Che, Zhengping and Purushotham, Sanjay and Cho, Kyunghyun and Sontag, David and Liu, Yan , journal =. Recurrent Neural Networks for Multivariate Time Series with Missing Values , volume =
-
[4]
Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness , year =
Liang, Zihan and Pan, Ziwen and Xiong, Ruoxuan , booktitle =. Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness , year =
-
[5]
Offline Reinforcement Learning with Implicit Q-Learning , year =
Ilya Kostrikov and Ashvin Nair and Sergey Levine , booktitle =. Offline Reinforcement Learning with Implicit Q-Learning , year =
-
[6]
Pollard, Tom J and Johnson, Alistair EW and Raffa, Jesse D and Celi, Leo A and Mark, Roger G and Badawi, Omar , journal =. The
-
[7]
Planning and acting in partially observable stochastic domains , volume =
Kaelbling, Leslie Pack and Littman, Michael L and Cassandra, Anthony R , journal =. Planning and acting in partially observable stochastic domains , volume =
-
[8]
Milos Hauskrecht and Hamish S. F. Fraser , journal =. Planning treatment of ischemic heart disease with partially observable Markov decision processes , volume =
-
[9]
Double Q-Learning , year =
van Hasselt, Hado , booktitle =. Double Q-Learning , year =
-
[10]
Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach , year =
Raghu, Aniruddh and Komorowski, Matthieu and Celi, Leo Anthony and Szolovits, Peter and Ghassemi, Marzyeh , booktitle =. Continuous State-Space Models for Optimal Sepsis Treatment: a Deep Reinforcement Learning Approach , year =
-
[11]
The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care , volume =
Komorowski, Matthieu and Celi, Leo A and Badawi, Omar and Gordon, Anthony C and Faisal, A Aldo , journal =. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care , volume =
-
[12]
Off-Policy Deep Reinforcement Learning without Exploration , year =
Scott Fujimoto and David Meger and Doina Precup , booktitle =. Off-Policy Deep Reinforcement Learning without Exploration , year =
-
[13]
Conservative Q-learning for offline reinforcement learning , year =
Kumar, Aviral and Zhou, Aurick and Tucker, George and Levine, Sergey , booktitle =. Conservative Q-learning for offline reinforcement learning , year =
-
[14]
Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution , year =
Huang, Yong and Cao, Rui and Rahmani, Amir , booktitle =. Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution , year =
-
[15]
Semi-Markov Offline Reinforcement Learning for Healthcare , year =
Fatemi, Mehdi and Wu, Mary and Petch, Jeremy and Nelson, Walter and Connolly, Stuart J and Benz, Alexander and Carnicelli, Anthony and Ghassemi, Marzyeh , booktitle =. Semi-Markov Offline Reinforcement Learning for Healthcare , year =
-
[16]
MedDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support , year =
Xu, Qianyi and Habib, Gousia and Wu, Feng and Perera, Dilruk and Feng, Mengling , journal =. MedDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support , year =
-
[17]
Batch Policy Learning under Constraints , year =
Le, Hoang and Voloshin, Cameron and Yue, Yisong , booktitle =. Batch Policy Learning under Constraints , year =
-
[18]
BRITS: bidirectional recurrent imputation for time series , year =
Cao, Wei and Wang, Dong and Li, Jian and Zhou, Hao and Li, Yitan and Li, Lei , booktitle =. BRITS: bidirectional recurrent imputation for time series , year =
-
[19]
Multi-Time Attention Networks for Irregularly Sampled Time Series , year =
Satya Narayan Shukla and Benjamin Marlin , booktitle =. Multi-Time Attention Networks for Irregularly Sampled Time Series , year =
-
[20]
Deep Reinforcement Learning for Sepsis Treatment , year =
Aniruddh Raghu and Matthieu Komorowski and Imran Ahmed and Leo Celi and Peter Szolovits and Marzyeh Ghassemi , booktitle =. Deep Reinforcement Learning for Sepsis Treatment , year =
-
[21]
Aniruddh Raghu and Matthieu Komorowski and Sumeetpal Singh , title =
-
[22]
Improving sepsis treatment strategies by combining deep and kernel-based reinforcement learning , year =
Peng, Xuefeng and Ding, Yi and Wihl, David and Gottesman, Omer and Komorowski, Matthieu and Lehman, Li-wei H and Ross, Andrew and Faisal, Aldo and Doshi-Velez, Finale , booktitle =. Improving sepsis treatment strategies by combining deep and kernel-based reinforcement learning , year =
-
[23]
Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatment , year =
Sun, Yingchuan and Tang, Shengpu , journal =. Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatment , year =
-
[24]
and Theocharous, Georgios and Ghavamzadeh, Mohammad , booktitle =
Thomas, Philip S. and Theocharous, Georgios and Ghavamzadeh, Mohammad , booktitle =. High confidence off-policy evaluation , year =
-
[25]
Lehman and Matthieu Komorowski and Matthieu Komorowski and Aldo Faisal and Leo Anthony Celi and David Sontag and Finale Doshi-Velez , title =
Omer Gottesman and Fredrik Johansson and Joshua Meier and Jack Dent and Donghun Lee and Srivatsan Srinivasan and Linying Zhang and Yi Ding and David Wihl and Xuefeng Peng and Jiayu Yao and Isaac Lage and Christopher Mosch and Li-wei H. Lehman and Matthieu Komorowski and Matthieu Komorowski and Aldo Faisal and Leo Anthony Celi and David Sontag and Finale D...
-
[26]
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation , year =
Wang, Lu and Zhang, Wei and He, Xiaofeng and Zha, Hongyuan , booktitle =. Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation , year =
-
[27]
Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings , year =
Tang, Shengpu and Wiens, Jenna , booktitle =. Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings , year =
-
[28]
Guidelines for Reinforcement Learning in Healthcare , volume =
Gottesman, Omer and Johansson, Fredrik and Komorowski, Matthieu and Faisal, Aldo and Sontag, David and Doshi-Velez, Finale and Celi, Leo Anthony , journal =. Guidelines for Reinforcement Learning in Healthcare , volume =
-
[29]
Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review , volume =
Liu, Siqi and See, Kay Choong and Ngiam, Kee Yuan and Celi, Leo Anthony and Sun, Xingzhi and Feng, Mengling , journal =. Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review , volume =. 2020 , issn =
2020
-
[30]
Little, Roderick J. A. and Rubin, Donald B. , publisher =. Statistical Analysis with Missing Data , year =
-
[31]
Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series , year =
Lipton, Zachary C and Kale, David and Wetzel, Randall , booktitle =. Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series , year =
-
[32]
Graph-Guided Network for Irregularly Sampled Multivariate Time Series , year =
Xiang Zhang and Marko Zeman and Theodoros Tsiligkaridis and Marinka Zitnik , booktitle =. Graph-Guided Network for Irregularly Sampled Multivariate Time Series , year =
-
[33]
Lin, Ronghao and Hu, Haifeng , journal =
-
[34]
Drfuse: Learning disentangled representation for clinical multi-modal fusion with missing modality and modal inconsistency , year =
Yao, Wenfang and Yin, Kejing and Cheung, William K and Liu, Jia and Qin, Jing , booktitle =. Drfuse: Learning disentangled representation for clinical multi-modal fusion with missing modality and modal inconsistency , year =
-
[35]
Multimodal Patient Representation Learning with Missing Modalities and Labels , year =
Zhenbang Wu and Anant Dadu and Nicholas Tustison and Brian Avants and Mike Nalls and Jimeng Sun and Faraz Faghri , booktitle =. Multimodal Patient Representation Learning with Missing Modalities and Labels , year =
-
[36]
Large dimensional latent factor modeling with missing observations and applications to causal inference , volume =
Ruoxuan Xiong and Markus Pelger , journal =. Large dimensional latent factor modeling with missing observations and applications to causal inference , volume =. 2023 , issn =
2023
-
[37]
Factor Analysis for Causal Inference on Large Non-Stationary Panels with Endogenous Treatment , year =
Duan, Junting and Pelger, Markus and Xiong, Ruoxuan , journal =. Factor Analysis for Causal Inference on Large Non-Stationary Panels with Endogenous Treatment , year =
-
[38]
Target PCA: Transfer learning large dimensional panel data , volume =
Duan, Junting and Pelger, Markus and Xiong, Ruoxuan , journal =. Target PCA: Transfer learning large dimensional panel data , volume =
-
[39]
Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models , year =
Chen, Hongyu and Simchi-Levi, David and Xiong, Ruoxuan , journal =. Partial Identification under Missing Data Using Weak Shadow Variables from Pretrained Models , year =
-
[40]
Representation Learning: A Review and New Perspectives , volume =
Bengio, Yoshua and Courville, Aaron and Vincent, Pascal , journal =. Representation Learning: A Review and New Perspectives , volume =. 2013 , issn =
2013
-
[41]
International Conference on Learning Representations , title =
Wu, Sen and Zhang, Hongyang R and R. International Conference on Learning Representations , title =
-
[42]
Precise high-dimensional asymptotics for quantifying heterogeneous transfers , volume =
Yang, Fan and Zhang, Hongyang R and Wu, Sen and Re, Christopher and Su, Weijie J , journal =. Precise high-dimensional asymptotics for quantifying heterogeneous transfers , volume =
-
[43]
Boosting multitask learning on graphs through higher-order task affinities , year =
Li, Dongyue and Ju, Haotian and Sharma, Aneesh and Zhang, Hongyang R , booktitle =. Boosting multitask learning on graphs through higher-order task affinities , year =
-
[44]
Identification of negative transfers in multitask learning using surrogate models , year =
Li, Dongyue and Nguyen, Huy L and Zhang, Hongyang R , journal =. Identification of negative transfers in multitask learning using surrogate models , year =
-
[45]
Efficient Estimation of Kernel Surrogate Models for Task Attribution , year =
Zhang, Zhenshuo and Duan, Minxuan and Zhang, Hongyang R , journal =. Efficient Estimation of Kernel Surrogate Models for Task Attribution , year =
-
[46]
Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity , year =
Li, Dongyue and Sharma, Aneesh and Zhang, Hongyang R , booktitle =. Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity , year =
-
[47]
Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach , year =
Li, Dongyue and Zhang, Ziniu and Wang, Lu and Zhang, Hongyang R , journal =. Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach , year =
-
[48]
Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets , year =
Li, Dongyue and Zhang, Ziniu and Wang, Lu and Zhang, Hongyang R , journal =. Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets , year =
-
[49]
Scalable multi-objective and meta reinforcement learning via gradient estimation , year =
Zhang, Zhenshuo and Duan, Minxuan and Ye, Youran and Zhang, Hongyang R , booktitle =. Scalable multi-objective and meta reinforcement learning via gradient estimation , year =
-
[50]
The third international consensus definitions for sepsis and septic shock (Sepsis-3) , volume =
Singer, Mervyn and Deutschman, Clifford S and Seymour, Christopher Warren and Shankar-Hari, Manu and Annane, Djillali and Bauer, Michael and Bellomo, Rinaldo and Bernard, Gordon R and Chiche, Jean-Daniel and Coopersmith, Craig M and others , journal =. The third international consensus definitions for sepsis and septic shock (Sepsis-3) , volume =
-
[51]
Publicly Available Clinical
Alsentzer, Emily and Murphy, John and Boag, William and Weng, Wei-Hung and Jindi, Di and Naumann, Tristan and McDermott, Matthew , booktitle =. Publicly Available Clinical
-
[52]
Ashvin Nair and Abhishek Gupta and Murtaza Dalal and Sergey Levine , title =
-
[53]
and Singh, Satinder P
Precup, Doina and Sutton, Richard S. and Singh, Satinder P. , booktitle =. Eligibility Traces for Off-Policy Policy Evaluation , year =
-
[54]
and Brunskill, Emma , booktitle =
Thomas, Philip S. and Brunskill, Emma , booktitle =. Data-efficient off-policy policy evaluation for reinforcement learning , year =
-
[55]
Doubly robust off-policy value evaluation for reinforcement learning , year =
Jiang, Nan and Li, Lihong , booktitle =. Doubly robust off-policy value evaluation for reinforcement learning , year =
-
[56]
and Stone, Peter and Niekum, Scott , booktitle =
Hanna, Josiah P. and Stone, Peter and Niekum, Scott , booktitle =. Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation , year =
-
[57]
Methods and Dimensions of Electronic Health Record Data Quality Assessment: Enabling Reuse for Clinical Research , volume =
Weiskopf, Nicole Gray and Weng, Chunhua , journal =. Methods and Dimensions of Electronic Health Record Data Quality Assessment: Enabling Reuse for Clinical Research , volume =
-
[58]
and Weber, Griffin M
Agniel, Denis and Kohane, Isaac S. and Weber, Griffin M. , journal =. Biases in Electronic Health Record Data Due to Processes Within the Healthcare System: Retrospective Observational Study , volume =
-
[59]
M3Care: Learning with Missing Modalities in Multimodal Healthcare Data , year =
Zhang, Chaohe and Chu, Xu and Ma, Liantao and Zhu, Yinghao and Wang, Yasha and Wang, Jiangtao and Zhao, Junfeng , booktitle =. M3Care: Learning with Missing Modalities in Multimodal Healthcare Data , year =
-
[60]
Learning shadow variable representation for treatment effect estimation under collider bias , year =
Li, Baohong and Li, Haoxuan and Xiong, Ruoxuan and Wu, Anpeng and Wu, Fei and Kuang, Kun , booktitle =. Learning shadow variable representation for treatment effect estimation under collider bias , year =
-
[61]
Two-stage shadow inclusion estimation: An IV approach for causal inference under latent confounding and collider bias , year =
Li, Baohong and Wu, Anpeng and Xiong, Ruoxuan and Kuang, Kun , booktitle =. Two-stage shadow inclusion estimation: An IV approach for causal inference under latent confounding and collider bias , year =
-
[62]
Generalizing Causal Effects from Randomized Controlled Trials to Target Populations across Diverse Environments , year =
Li, Baohong and Wang, Yingrong and Wu, Anpeng and Ma, Ming and Xiong, Ruoxuan and Kuang, Kun , booktitle =. Generalizing Causal Effects from Randomized Controlled Trials to Target Populations across Diverse Environments , year =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.