RECORD
Dynamic Measurement Scheduling for Event Forecasting using Deep RL
Machine Learning · Submitted 24 Jan 2019
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient's health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Imagine a patient in critical condition. | background 1.00 | 0.03 | 0.01 | unclear 0.83 |
| s1 | What and when should be measured to forecast detrimental events, especially under the budget constraints? | background 0.79 | 0.04 | 0.02 | unclear 0.98 |
| s2 | We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. | method 0.56 | 0.93 | 0.35 | asserted 1.00 |
| s3 | We learn our policy to be dynamically dependent on the patient's health history. | method 0.94 | 0.92 | 0.06 | asserted 1.00 |
| s4 | To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. | method 0.96 | 0.91 | 0.04 | asserted 1.00 |
| s5 | In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. | result 1.00 | 0.95 | 0.23 | asserted 1.00 |
| s6 | In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation. | result 1.00 | 0.95 | 0.64 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient's health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 8 sentences without a confident label
Uses the method or tool of 2 papers
-
We use regression-based estimator [ref] to estimate the values of physician and our learned policies using physician collected data.
Doubly Robust Off-policy Value Evaluation for Reinforcement Learning · Off-Policy Policy Evaluation (OPPE) · 0.92
-
Dueling DQN factorizes the computation of Q-value into value stream and advantage stream [ref] , i.e. [equation] where $ξ$, $η$, and $ψ$ are respectively, the parameters of the shared encoder $f_ξ$ of the value stream $V_η$, and of the advantage stream $A_ψ$ (Figure [ref], Policy Learning).
Dueling Network Architectures for Deep Reinforcement Learning · Dueling Deep Q Network (DQN) · 0.85
Cites as background 5 papers
-
[ref] used multi-output Gaussian process (MGP) to impute the irregularly-sampled time series data on the grid points and used those to make predictions via recurrent neural network (RNN).
An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection · Clinical Event Forecasting Models · 0.85
-
Several works [ref] study the problem of selecting a subset of features to achieve the maximum prediction performance for a non-time-series classifier.
Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification · Active feature acquisition · 0.97
-
Several works [ref] study the problem of selecting a subset of features to achieve the maximum prediction performance for a non-time-series classifier.
Active Information Acquisition · Active feature acquisition · 0.97
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.