Determining and assessing characteristics of data element names impacting the performance of annotation using Usagi.

INTRODUCTION: Hospitals generate large amounts of data and this data is generally modeled and labeled in a proprietary way, hampering its exchange and integration. Manually annotating data element names to internationally standardized data element identifiers is a time-consuming effort. Tools can support performing this task automatically. This study aimed to determine what factors influence the quality of automatic annotations. METHODS: Data element names were used from the Dutch COVID-19 ICU Data Warehouse containing data on intensive care patients with COVID-19 from 25 hospitals in the Netherlands. In this data warehouse, the data had been merged using a proprietary terminology system while also storing the original hospital labels (synonymous names). Usagi, an OHDSI annotation tool, was used to perform the annotation for the data. A gold standard was used to determine if Usagi made correct annotations. Logistic regression was used to determine if the number of characters, number of words, match score (Usagi's certainty) and hospital label origin influenced Usagi's performance to annotate correctly. RESULTS: Usagi automatically annotated 30.5% of the data element names correctly and 5.5% of the synonymous names. The match score is the best predictor for Usagi finding the correct annotation. It was determined that the AUC of data element names was 0.651 and 0.752 for the synonymous names respectively. The AUC for the individual hospital label origins varied between 0.460 to 0.905. DISCUSSION: The results show that Usagi performed better to annotate the data element names than the synonymous names. The hospital origin in the synonymous names dataset was associated with the amount of correctly annotated concepts. Hospitals that performed better had shorter synonymous names and fewer words. Using shorter data element names or synonymous names should be considered to optimize the automatic annotating process. Overall, the performance of Usagi is too poor to completely rely on for automatic annotation.

Augmented intelligence facilitates concept mapping across different electronic health records.

INTRODUCTION: With the advent of artificial intelligence, the secondary use of routinely collected medical data from electronic healthcare records (EHR) has become increasingly popular. However, different EHR systems typically use different names for the same medical concepts. This obviously hampers scalable model development and subsequent clinical implementation for decision support. Therefore, converting original parameter names to a so-called ontology, a standardized set of predefined concepts, is necessary but time-consuming and labor-intensive. We therefore propose an augmented intelligence approach to facilitate ontology alignment by predicting correct concepts based on parameter names from raw electronic health record data exports. METHODS: We used the manually mapped parameter names from the multicenter "Dutch ICU data warehouse against COVID-19" sourced from three types of EHR systems to train machine learning models for concept mapping. Data from 29 intensive care units on 38,824 parameters mapped to 1,679 relevant and unique concepts and 38,069 parameters labeled as irrelevant were used for model development and validation. We used the Natural Language Toolkit (NLTK) to preprocess the parameter names based on WordNet cognitive synonyms transformed by term-frequency inverse document frequency (TF-IDF), yielding numeric features. We then trained linear classifiers using stochastic gradient descent for multi-class prediction. Finally, we fine-tuned these predictions using information on distributions of the data associated with each parameter name through similarity score and skewness comparisons. RESULTS: The initial model, trained using data from one hospital organization for each of three EHR systems, scored an overall top 1 precision of 0.744, recall of 0.771, and F1-score of 0.737 on a total of 58,804 parameters. Leave-one-hospital-out analysis returned an average top 1 recall of 0.680 for relevant parameters, which increased to 0.905 for the top 5 predictions. When reducing the training dataset to only include relevant parameters, top 1 recall was 0.811 and top 5 recall was 0.914 for relevant parameters. Performance improvement based on similarity score or skewness comparisons affected at most 5.23% of numeric parameters. CONCLUSION: Augmented intelligence is a promising method to improve concept mapping of parameter names from raw electronic health record data exports. We propose a robust method for mapping data across various domains, facilitating the integration of diverse data sources. However, recall is not perfect, and therefore manual validation of mapping remains essential.

Evolution of Clinical Phenotypes of COVID-19 Patients During Intensive Care Treatment: An Unsupervised Machine Learning Analysis.

BACKGROUND: Identification of clinical phenotypes in critically ill COVID-19 patients could improve understanding of the disease heterogeneity and enable prognostic and predictive enrichment. However, previous attempts did not take into account temporal dynamics with high granularity. By including the dimension of time, we aim to gain further insights into the heterogeneity of COVID-19. METHODS: We used granular data from 3202 adult COVID patients in the Dutch Data Warehouse that were admitted to one of 25 Dutch ICUs between February 2020 and March 2021. Parameters including demographics, clinical observations, medications, laboratory values, vital signs, and data from life support devices were selected. Twenty-one datasets were created that each covered 24 h of ICU data for each day of ICU treatment. Clinical phenotypes in each dataset were identified by performing cluster analyses. Both evolution of the clinical phenotypes over time and patient allocation to these clusters over time were tracked. RESULTS: The final patient cohort consisted of 2438 COVID-19 patients with a ICU mortality outcome. Forty-one parameters were chosen for cluster analysis. On admission, both a mild and a severe clinical phenotype were found. After day 4, the severe phenotype split into an intermediate and a severe phenotype for 11 consecutive days. Heterogeneity between phenotypes appears to be driven by inflammation and dead space ventilation. During the 21-day period, only 8.2% and 4.6% of patients in the initial mild and severe clusters remained assigned to the same phenotype respectively. The clinical phenotype half-life was between 5 and 6 days for the mild and severe phenotypes, and about 3 days for the medium severe phenotype. CONCLUSIONS: Patients typically do not remain in the same cluster throughout intensive care treatment. This may have important implications for prognostic or predictive enrichment. Prominent dissimilarities between clinical phenotypes are predominantly driven by inflammation and dead space ventilation.

Predicting responders to prone positioning in mechanically ventilated patients with COVID-19 using machine learning.

BACKGROUND: For mechanically ventilated critically ill COVID-19 patients, prone positioning has quickly become an important treatment strategy, however, prone positioning is labor intensive and comes with potential adverse effects. Therefore, identifying which critically ill intubated COVID-19 patients will benefit may help allocate labor resources. METHODS: From the multi-center Dutch Data Warehouse of COVID-19 ICU patients from 25 hospitals, we selected all 3619 episodes of prone positioning in 1142 invasively mechanically ventilated patients. We excluded episodes longer than 24 h. Berlin ARDS criteria were not formally documented. We used supervised machine learning algorithms Logistic Regression, Random Forest, Naive Bayes, K-Nearest Neighbors, Support Vector Machine and Extreme Gradient Boosting on readily available and clinically relevant features to predict success of prone positioning after 4 h (window of 1 to 7 h) based on various possible outcomes. These outcomes were defined as improvements of at least 10% in PaO2/FiO2 ratio, ventilatory ratio, respiratory system compliance, or mechanical power. Separate models were created for each of these outcomes. Re-supination within 4 h after pronation was labeled as failure. We also developed models using a 20 mmHg improvement cut-off for PaO2/FiO2 ratio and using a combined outcome parameter. For all models, we evaluated feature importance expressed as contribution to predictive performance based on their relative ranking. RESULTS: The median duration of prone episodes was 17 h (11-20, median and IQR, N = 2632). Despite extensive modeling using a plethora of machine learning techniques and a large number of potentially clinically relevant features, discrimination between responders and non-responders remained poor with an area under the receiver operator characteristic curve of 0.62 for PaO2/FiO2 ratio using Logistic Regression, Random Forest and XGBoost. Feature importance was inconsistent between models for different outcomes. Notably, not even being a previous responder to prone positioning, or PEEP-levels before prone positioning, provided any meaningful contribution to predicting a successful next proning episode. CONCLUSIONS: In mechanically ventilated COVID-19 patients, predicting the success of prone positioning using clinically relevant and readily available parameters from electronic health records is currently not feasible. Given the current evidence base, a liberal approach to proning in all patients with severe COVID-19 ARDS is therefore justified and in particular regardless of previous results of proning.

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records.

PURPOSE: To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data. METHODS: Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors. RESULTS: A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors. CONCLUSION: In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves.

Right dose, right now: bedside, real-time, data-driven, and personalised antibiotic dosing in critically ill patients with sepsis or septic shock-a two-centre randomised clinical trial.

BACKGROUND: Adequate antibiotic dosing may improve outcomes in critically ill patients but is challenging due to altered and variable pharmacokinetics. To address this challenge, AutoKinetics was developed, a decision support system for bedside, real-time, data-driven and personalised antibiotic dosing. This study evaluates the feasibility, safety and efficacy of its clinical implementation. METHODS: In this two-centre randomised clinical trial, critically ill patients with sepsis or septic shock were randomised to AutoKinetics dosing or standard dosing for four antibiotics: vancomycin, ciprofloxacin, meropenem, and ceftriaxone. Adult patients with a confirmed or suspected infection and either lactate > 2 mmol/L or vasopressor requirement were eligible for inclusion. The primary outcome was pharmacokinetic target attainment in the first 24 h after randomisation. Clinical endpoints included mortality, ICU length of stay and incidence of acute kidney injury. RESULTS: After inclusion of 252 patients, the study was stopped early due to the COVID-19 pandemic. In the ciprofloxacin intervention group, the primary outcome was obtained in 69% compared to 3% in the control group (OR 62.5, CI 11.4-1173.78, p < 0.001). Furthermore, target attainment was faster (26 h, CI 18-42 h, p < 0.001) and better (65% increase, CI 49-84%, p < 0.001). For the other antibiotics, AutoKinetics dosing did not improve target attainment. Clinical endpoints were not significantly different. Importantly, higher dosing did not lead to increased mortality or renal failure. CONCLUSIONS: In critically ill patients, personalised dosing was feasible, safe and significantly improved target attainment for ciprofloxacin. TRIAL REGISTRATION: The trial was prospectively registered at Netherlands Trial Register (NTR), NL6501/NTR6689 on 25 August 2017 and at the European Clinical Trials Database (EudraCT), 2017-002478-37 on 6 November 2017.

Rapid Evaluation of Coronavirus Illness Severity (RECOILS) in intensive care: Development and validation of a prognostic tool for in-hospital mortality.

BACKGROUND: The prediction of in-hospital mortality for ICU patients with COVID-19 is fundamental to treatment and resource allocation. The main purpose was to develop an easily implemented score for such prediction. METHODS: This was an observational, multicenter, development, and validation study on a national critical care dataset of COVID-19 patients. A systematic literature review was performed to determine variables possibly important for COVID-19 mortality prediction. Using a logistic multivariable model with a LASSO penalty, we developed the Rapid Evaluation of Coronavirus Illness Severity (RECOILS) score and compared its performance against published scores. RESULTS: Our development (validation) cohort consisted of 1480 (937) adult patients from 14 (11) Dutch ICUs admitted between March 2020 and April 2021. Median age was 65 (65) years, 31% (26%) died in hospital, 74% (72%) were males, average length of ICU stay was 7.83 (10.25) days and average length of hospital stay was 15.90 (19.92) days. Age, platelets, PaO2/FiO2 ratio, pH, blood urea nitrogen, temperature, PaCO2, Glasgow Coma Scale (GCS) score measured within +/-24 h of ICU admission were used to develop the score. The AUROC of RECOILS score was 0.75 (CI 0.71-0.78) which was higher than that of any previously reported predictive scores (0.68 [CI 0.64-0.71], 0.61 [CI 0.58-0.66], 0.67 [CI 0.63-0.70], 0.70 [CI 0.67-0.74] for ISARIC 4C Mortality Score, SOFA, SAPS-III, and age, respectively). CONCLUSIONS: Using a large dataset from multiple Dutch ICUs, we developed a predictive score for mortality of COVID-19 patients admitted to ICU, which outperformed other predictive scores reported so far.

Predictors for extubation failure in COVID-19 patients using a machine learning approach.

INTRODUCTION: Determining the optimal timing for extubation can be challenging in the intensive care. In this study, we aim to identify predictors for extubation failure in critically ill patients with COVID-19. METHODS: We used highly granular data from 3464 adult critically ill COVID patients in the multicenter Dutch Data Warehouse, including demographics, clinical observations, medications, fluid balance, laboratory values, vital signs, and data from life support devices. All intubated patients with at least one extubation attempt were eligible for analysis. Transferred patients, patients admitted for less than 24 h, and patients still admitted at the time of data extraction were excluded. Potential predictors were selected by a team of intensive care physicians. The primary and secondary outcomes were extubation without reintubation or death within the next 7 days and within 48 h, respectively. We trained and validated multiple machine learning algorithms using fivefold nested cross-validation. Predictor importance was estimated using Shapley additive explanations, while cutoff values for the relative probability of failed extubation were estimated through partial dependence plots. RESULTS: A total of 883 patients were included in the model derivation. The reintubation rate was 13.4% within 48 h and 18.9% at day 7, with a mortality rate of 0.6% and 1.0% respectively. The grandient-boost model performed best (area under the curve of 0.70) and was used to calculate predictor importance. Ventilatory characteristics and settings were the most important predictors. More specifically, a controlled mode duration longer than 4 days, a last fraction of inspired oxygen higher than 35%, a mean tidal volume per kg ideal body weight above 8 ml/kg in the day before extubation, and a shorter duration in assisted mode (< 2 days) compared to their median values. Additionally, a higher C-reactive protein and leukocyte count, a lower thrombocyte count, a lower Glasgow coma scale and a lower body mass index compared to their medians were associated with extubation failure. CONCLUSION: The most important predictors for extubation failure in critically ill COVID-19 patients include ventilatory settings, inflammatory parameters, neurological status, and body mass index. These predictors should therefore be routinely captured in electronic health records.

Some Patients Are More Equal Than Others: Variation in Ventilator Settings for Coronavirus Disease 2019 Acute Respiratory Distress Syndrome.

OBJECTIVES: As coronavirus disease 2019 is a novel disease, treatment strategies continue to be debated. This provides the intensive care community with a unique opportunity as the population of coronavirus disease 2019 patients requiring invasive mechanical ventilation is relatively homogeneous compared with other ICU populations. We hypothesize that the novelty of coronavirus disease 2019 and the uncertainty over its similarity with noncoronavirus disease 2019 acute respiratory distress syndrome resulted in substantial practice variation between hospitals during the first and second waves of coronavirus disease 2019 patients. DESIGN: Multicenter retrospective cohort study. SETTING: Twenty-five hospitals in the Netherlands from February 2020 to July 2020, and 14 hospitals from August 2020 to December 2020. PATIENTS: One thousand two hundred ninety-four critically ill intubated adult ICU patients with coronavirus disease 2019 were selected from the Dutch Data Warehouse. Patients intubated for less than 24 hours, transferred patients, and patients still admitted at the time of data extraction were excluded. MEASUREMENTS AND MAIN RESULTS: We aimed to estimate between-ICU practice variation in selected ventilation parameters (positive end-expiratory pressure, Fio2, set respiratory rate, tidal volume, minute volume, and percentage of time spent in a prone position) on days 1, 2, 3, and 7 of intubation, adjusted for patient characteristics as well as severity of illness based on Pao2/Fio2 ratio, pH, ventilatory ratio, and dynamic respiratory system compliance during controlled ventilation. Using multilevel linear mixed-effects modeling, we found significant (p ≤ 0.001) variation between ICUs in all ventilation parameters on days 1, 2, 3, and 7 of intubation for both waves. CONCLUSIONS: This is the first study to clearly demonstrate significant practice variation between ICUs related to mechanical ventilation parameters that are under direct control by intensivists. Their effect on clinical outcomes for both coronavirus disease 2019 and other critically ill mechanically ventilated patients could have widespread implications for the practice of intensive care medicine and should be investigated further by causal inference models and clinical trials.

The Dutch Data Warehouse, a multicenter and full-admission electronic health records database for critically ill COVID-19 patients.

BACKGROUND: The Coronavirus disease 2019 (COVID-19) pandemic has underlined the urgent need for reliable, multicenter, and full-admission intensive care data to advance our understanding of the course of the disease and investigate potential treatment strategies. In this study, we present the Dutch Data Warehouse (DDW), the first multicenter electronic health record (EHR) database with full-admission data from critically ill COVID-19 patients. METHODS: A nation-wide data sharing collaboration was launched at the beginning of the pandemic in March 2020. All hospitals in the Netherlands were asked to participate and share pseudonymized EHR data from adult critically ill COVID-19 patients. Data included patient demographics, clinical observations, administered medication, laboratory determinations, and data from vital sign monitors and life support devices. Data sharing agreements were signed with participating hospitals before any data transfers took place. Data were extracted from the local EHRs with prespecified queries and combined into a staging dataset through an extract-transform-load (ETL) pipeline. In the consecutive processing pipeline, data were mapped to a common concept vocabulary and enriched with derived concepts. Data validation was a continuous process throughout the project. All participating hospitals have access to the DDW. Within legal and ethical boundaries, data are available to clinicians and researchers. RESULTS: Out of the 81 intensive care units in the Netherlands, 66 participated in the collaboration, 47 have signed the data sharing agreement, and 35 have shared their data. Data from 25 hospitals have passed through the ETL and processing pipeline. Currently, 3464 patients are included in the DDW, both from wave 1 and wave 2 in the Netherlands. More than 200 million clinical data points are available. Overall ICU mortality was 24.4%. Respiratory and hemodynamic parameters were most frequently measured throughout a patient's stay. For each patient, all administered medication and their daily fluid balance were available. Missing data are reported for each descriptive. CONCLUSIONS: In this study, we show that EHR data from critically ill COVID-19 patients may be lawfully collected and can be combined into a data warehouse. These initiatives are indispensable to advance medical data science in the field of intensive care medicine.

Predicting mortality of individual patients with COVID-19: a multicentre Dutch cohort.

OBJECTIVE: Develop and validate models that predict mortality of patients diagnosed with COVID-19 admitted to the hospital. DESIGN: Retrospective cohort study. SETTING: A multicentre cohort across 10 Dutch hospitals including patients from 27 February to 8 June 2020. PARTICIPANTS: SARS-CoV-2 positive patients (age ≥18) admitted to the hospital. MAIN OUTCOME MEASURES: 21-day all-cause mortality evaluated by the area under the receiver operator curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. The predictive value of age was explored by comparison with age-based rules used in practice and by excluding age from the analysis. RESULTS: 2273 patients were included, of whom 516 had died or discharged to palliative care within 21 days after admission. Five feature sets, including premorbid, clinical presentation and laboratory and radiology values, were derived from 80 features. Additionally, an Analysis of Variance (ANOVA)-based data-driven feature selection selected the 10 features with the highest F values: age, number of home medications, urea nitrogen, lactate dehydrogenase, albumin, oxygen saturation (%), oxygen saturation is measured on room air, oxygen saturation is measured on oxygen therapy, blood gas pH and history of chronic cardiac disease. A linear logistic regression and non-linear tree-based gradient boosting algorithm fitted the data with an AUC of 0.81 (95% CI 0.77 to 0.85) and 0.82 (0.79 to 0.85), respectively, using the 10 selected features. Both models outperformed age-based decision rules used in practice (AUC of 0.69, 0.65 to 0.74 for age >70). Furthermore, performance remained stable when excluding age as predictor (AUC of 0.78, 0.75 to 0.81). CONCLUSION: Both models showed good performance and had better test characteristics than age-based decision rules, using 10 admission features readily available in Dutch hospitals. The models hold promise to aid decision-making during a hospital bed shortage.

Risk factors for adverse outcomes during mechanical ventilation of 1152 COVID-19 patients: a multicenter machine learning study with highly granular data from the Dutch Data Warehouse.

BACKGROUND: The identification of risk factors for adverse outcomes and prolonged intensive care unit (ICU) stay in COVID-19 patients is essential for prognostication, determining treatment intensity, and resource allocation. Previous studies have determined risk factors on admission only, and included a limited number of predictors. Therefore, using data from the highly granular and multicenter Dutch Data Warehouse, we developed machine learning models to identify risk factors for ICU mortality, ventilator-free days and ICU-free days during the course of invasive mechanical ventilation (IMV) in COVID-19 patients. METHODS: The DDW is a growing electronic health record database of critically ill COVID-19 patients in the Netherlands. All adult ICU patients on IMV were eligible for inclusion. Transfers, patients admitted for less than 24 h, and patients still admitted at time of data extraction were excluded. Predictors were selected based on the literature, and included medication dosage and fluid balance. Multiple algorithms were trained and validated on up to three sets of observations per patient on day 1, 7, and 14 using fivefold nested cross-validation, keeping observations from an individual patient in the same split. RESULTS: A total of 1152 patients were included in the model. XGBoost models performed best for all outcomes and were used to calculate predictor importance. Using Shapley additive explanations (SHAP), age was the most important demographic risk factor for the outcomes upon start of IMV and throughout its course. The relative probability of death across age values is visualized in Partial Dependence Plots (PDPs), with an increase starting at 54 years. Besides age, acidaemia, low P/F-ratios and high driving pressures demonstrated a higher probability of death. The PDP for driving pressure showed a relative probability increase starting at 12 cmH2O. CONCLUSION: Age is the most important demographic risk factor of ICU mortality, ICU-free days and ventilator-free days throughout the course of invasive mechanical ventilation in critically ill COVID-19 patients. pH, P/F ratio, and driving pressure should be monitored closely over the course of mechanical ventilation as risk factors predictive of these outcomes.

[Clinical course of COVID-19 in the Netherlands: an overview of 2607 patients in hospital during the first wave]. / Klinisch beloop van covid-19 in Nederland.

OBJECTIVE: To systematically collect clinical data from patients with a proven COVID-19 infection in the Netherlands. DESIGN: Data from 2579 patients with COVID-19 admitted to 10 Dutch centers in the period February to July 2020 are described. The clinical data are based on the WHO COVID case record form (CRF) and supplemented with patient characteristics of which recently an association disease severity has been reported. METHODS: Survival analyses were performed as primary statistical analysis. These Kaplan-Meier curves for time to (early) death (3 weeks) have been determined for pre-morbid patient characteristics and clinical, radiological and laboratory data at hospital admission. RESULTS: Total in-hospital mortality after 3 weeks was 22.2% (95% CI: 20.7% - 23.9%), hospital mortality within 21 days was significantly higher for elderly patients (> 70 years; 35, 0% (95% CI: 32.4% - 37.8%) and patients who died during the 21 days and were admitted to the intensive care (36.5% (95% CI: 32.1% - 41.3%)). Apart from that, in this Dutch population we also see a risk of early death in patients with co-morbidities (such as chronic neurological, nephrological and cardiac disorders and hypertension), and in patients with more home medication and / or with increased urea and creatinine levels. CONCLUSION: Early death due to a COVID-19 infection in the Netherlands appears to be associated with demographic variables (e.g. age), comorbidity (e.g. cardiovascular disease) but also disease char-acteristics at admission.

Why we should sample sparsely and aim for a higher target: Lessons from model-based therapeutic drug monitoring of vancomycin in intensive care patients.

AIMS: To explore the optimal data sampling scheme and the pharmacokinetic (PK) target exposure on which dose computation is based in the model-based therapeutic drug monitoring (TDM) practice of vancomycin in intensive care (ICU) patients. METHODS: We simulated concentration data for 1 day following four sampling schemes, Cmin , Cmax + Cmin , Cmax + Cmid-interval + Cmin , and rich sampling where a sample was drawn every hour within a dose interval. The datasets were used for Bayesian estimation to obtain PK parameters, which were used to compute the doses for the next day based on five PK target exposures: AUC24 = 400, 500, and 600 mg·h/L and Cmin = 15 and 20 mg/L. We then simulated data for the next day, adopting the computed doses, and repeated the above procedure for 7 days. Thereafter, we calculated the percentage error and the normalized root mean square error (NRMSE) of estimated against "true" PK parameters, and the percentage of optimal treatment (POT), defined as the percentage of patients who met 400 ≤ AUC24 ≤ 600 mg·h/L and Cmin ≤ 20 mg/L. RESULTS: PK parameters were unbiasedly estimated in all investigated scenarios and the 6-day average NRMSE were 32.5%/38.5% (CL/V, where CL is clearance and V is volume of distribution) in the trough sampling scheme and 27.3%/26.5% (CL/V) in the rich sampling scheme. Regarding POT, the sampling scheme had marginal influence, while target exposure showed clear impacts that the maximum POT of 71.5% was reached when doses were computed based on AUC24 = 500 mg·h/L. CONCLUSIONS: For model-based TDM of vancomycin in ICU patients, sampling more frequently than taking only trough samples adds no value and dosing based on AUC24 = 500 mg·h/L lead to the best POT.

Optimizing Predictive Performance of Bayesian Forecasting for Vancomycin Concentration in Intensive Care Patients.

PURPOSE: Bayesian forecasting is crucial for model-based dose optimization based on therapeutic drug monitoring (TDM) data of vancomycin in intensive care (ICU) patients. We aimed to evaluate the performance of Bayesian forecasting using maximum a posteriori (MAP) estimation for model-based TDM. METHODS: We used a vancomycin TDM data set (n = 408 patients). We compared standard MAP-based Bayesian forecasting with two alternative approaches: (i) adaptive MAP which handles data over multiple iterations, and (ii) weighted MAP which weights the likelihood contribution of data. We evaluated the percentage error (PE) for seven scenarios including historical TDM data from the preceding day up to seven days. RESULTS: The mean of median PEs of all scenarios for the standard MAP, adaptive MAP and weighted MAP method were - 7.7%, -4.5% and - 6.7%. The adaptive MAP also showed the narrowest inter-quartile range of PE. In addition, regardless of MAP method, including historical TDM data further in the past will increase prediction errors. CONCLUSIONS: The proposed adaptive MAP method outperforms standard MAP in predictive performance and may be considered for improvement of model-based dose optimization. The inclusion of historical data beyond either one day (standard MAP and weighted MAP) or two days (adaptive MAP) reduces predictive performance.

Right Dose, Right Now: Development of AutoKinetics for Real Time Model Informed Precision Antibiotic Dosing Decision Support at the Bedside of Critically Ill Patients.

INTRODUCTION: Antibiotic dosing in critically ill patients is challenging because their pharmacokinetics (PK) are altered and may change rapidly with disease progression. Standard dosing frequently leads to inadequate PK exposure. Therapeutic drug monitoring (TDM) offers a potential solution but requires sampling and PK knowledge, which delays decision support. It is our philosophy that antibiotic dosing support should be directly available at the bedside through deep integration into the electronic health record (EHR) system. Therefore we developed AutoKinetics, a clinical decision support system (CDSS) for real time, model informed precision antibiotic dosing. OBJECTIVE: To provide a detailed description of the design, development, validation, testing, and implementation of AutoKinetics. METHODS: We created a development framework and used workflow analysis to facilitate integration into popular EHR systems. We used a development cycle to iteratively adjust and expand AutoKinetics functionalities. Furthermore, we performed a literature review to select and integrate pharmacokinetic models for five frequently prescribed antibiotics for sepsis. Finally, we tackled regulatory challenges, in particular those related to the Medical Device Regulation under the European regulatory framework. RESULTS: We developed a SQL-based relational database as the backend of AutoKinetics. We developed a data loader to retrieve data in real time. We designed a clinical dosing algorithm to find a dose regimen to maintain antibiotic pharmacokinetic exposure within clinically relevant safety constraints. If needed, a loading dose is calculated to minimize the time until steady state is achieved. Finally, adaptive dosing using Bayesian estimation is applied if plasma levels are available. We implemented support for five extensively used antibiotics following model development, calibration, and validation. We integrated AutoKinetics into two popular EHRs (Metavision, Epic) and developed a user interface that provides textual and visual feedback to the physician. CONCLUSION: We successfully developed a CDSS for real time model informed precision antibiotic dosing at the bedside of the critically ill. This holds great promise for improving sepsis outcome. Therefore, we recently started the Right Dose Right Now multi-center randomized control trial to validate this concept in 420 patients with severe sepsis and septic shock.

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy.

PURPOSE: Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis. METHODS: A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance. RESULTS: After screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance. CONCLUSION: This systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.