Transatlantic transferability of a new reinforcement learning model for optimizing haemodynamic treatment for critically ill patients with sepsis.

INTRODUCTION: In recent years, reinforcement learning (RL) has gained traction in the healthcare domain. In particular, RL methods have been explored for haemodynamic optimization of septic patients in the Intensive Care Unit. Most hospitals however, lack the data and expertise for model development, necessitating transfer of models developed using external datasets. This approach assumes model generalizability across different patient populations, the validity of which has not previously been tested. In addition, there is limited knowledge on safety and reliability. These challenges need to be addressed to further facilitate implementation of RL models in clinical practice. METHOD: We developed and validated a new reinforcement learning model for hemodynamic optimization in sepsis on the MIMIC intensive care database from the USA using a dueling double deep Q network. We then transferred this model to the European AmsterdamUMCdb intensive care database. T-Distributed Stochastic Neighbor Embedding and Sequential Organ Failure Assessment scores were used to explore the differences between the patient populations. We apply off-policy policy evaluation methods to quantify model performance. In addition, we introduce and apply a novel deep policy inspection to analyse how the optimal policy relates to the different phases of sepsis and sepsis treatment to provide interpretable insight in order to assess model safety and reliability. RESULTS: The off-policy evaluation revealed that the optimal policy outperformed the physician policy on both datasets despite marked differences between the two patient populations and physician's policies. Our novel deep policy inspection method showed insightful results and unveiled that the model could initiate therapy adequately and adjust therapy intensity to illness severity and disease progression which indicated safe and reliable model behaviour. Compared to current physician behavior, the developed policy prefers a more liberal use of vasopressors with a more restrained use of fluid therapy in line with previous work. CONCLUSION: We created a reinforcement learning model for optimal bedside hemodynamic management and demonstrated model transferability between populations from the USA and Europe for the first time. We proposed new methods for deep policy inspection integrating expert domain knowledge. This is expected to facilitate progression to bedside clinical decision support for the treatment of critically ill patients.

Prediction using patient comparison vs. modeling: a case study for mortality prediction.

Information in Electronic Medical Records (EMRs) can be used to generate accurate predictions for the occurrence of a variety of health states, which can contribute to more pro-active interventions. The very nature of EMRs does make the application of off-the-shelf machine learning techniques difficult. In this paper, we study two approaches to making predictions that have hardly been compared in the past: (1) extracting high-level (temporal) features from EMRs and building a predictive model, and (2) defining a patient similarity metric and predicting based on the outcome observed for similar patients. We analyze and compare both approaches on the MIMIC-II ICU dataset to predict patient mortality and find that the patient similarity approach does not scale well and results in a less accurate model (AUC of 0.68) compared to the modeling approach (0.84). We also show that mortality can be predicted within a median of 72 hours.