Effect of an Artificial Intelligence Decision Support Tool on Palliative Care Referral in Hospitalized Patients: A Randomized Clinical Trial.

CONTEXT: Palliative care services are commonly provided to hospitalized patients, but accurately predicting who needs them remains a challenge. OBJECTIVES: To assess the effectiveness on clinical outcomes of an artificial intelligence (AI)/machine learning (ML) decision support tool for predicting patient need for palliative care services in the hospital. METHODS: The study design was a pragmatic, cluster-randomized, stepped-wedge clinical trial in 12 nursing units at two hospitals over a 15-month period between August 19, 2019, and November 17, 2020. Eligible patients were randomly assigned to either a medical service consultation recommendation triggered by an AI/ML tool predicting the need for palliative care services or usual care. The primary outcome was palliative care consultation note. Secondary outcomes included: hospital readmissions, length of stay, transfer to intensive care and palliative care consultation note by unit. RESULTS: A total of 3183 patient hospitalizations were enrolled. Of eligible patients, A total of 2544 patients were randomized to the decision support tool (1212; 48%) and usual care (1332; 52%). Of these, 1717 patients (67%) were retained for analyses. Patients randomized to the intervention had a statistically significant higher incidence rate of palliative care consultation compared to the control group (IRR, 1.44 [95% CI, 1.11-1.92]). Exploratory evidence suggested that the decision support tool group reduced 60-day and 90-day hospital readmissions (OR, 0.75 [95% CI, 0.57, 0.97]) and (OR, 0.72 [95% CI, 0.55-0.93]) respectively. CONCLUSION: A decision support tool integrated into palliative care practice and leveraging AI/ML demonstrated an increased palliative care consultation rate among hospitalized patients and reductions in hospitalizations.

Improving time to palliative care review with predictive modeling in an inpatient adult population: study protocol for a stepped-wedge, pragmatic randomized controlled trial.

BACKGROUND: Palliative care is a medical specialty centered on improving the quality of life (QOL) of patients with complex or life-threatening illnesses. The need for palliative care is increasing and with that the rigorous testing of triage tools that can be used quickly and reliably to identify patients that may benefit from palliative care. METHODS: To that aim, we will conduct a two-armed stepped-wedge cluster randomized trial rolled out to two inpatient hospitals to evaluate whether a machine learning algorithm accurately identifies patients who may benefit from a comprehensive review by a palliative care specialist and decreases time to receiving a palliative care consult in hospital. This is a single-center study which will be conducted from August 2019 to November 2020 at Saint Mary's Hospital & Methodist Hospital both within Mayo Clinic Rochester in Minnesota. Clusters will be nursing units which will be chosen to be a mix of complex patients from Cardiology, Critical Care, and Oncology and had previously established relationships with palliative medicine. The stepped wedge design will have 12 units allocated to a design matrix of 5 treatment wedges. Each wedge will last 75 days resulting in a study period of 12 months of recruitment unless otherwise specified. Data will be analyzed with Bayesian hierarchical models with credible intervals denoting statistical significance. DISCUSSION: This intervention offers a pragmatic approach to delivering specialty palliative care to hospital patients in need using machine learning, thereby leading to high value care and improved outcomes. It is not enough for AI to be utilized by simply publishing research showing predictive performance; clinical trials demonstrating better outcomes are critically needed. Furthermore, the deployment of an AI algorithm is a complex process that requires multiple teams with varying skill sets. To evaluate a deployed AI, a pragmatic clinical trial can accommodate the difficulties of clinical practice while retaining scientific rigor. TRIAL REGISTRATION: ClinicalTrials.gov NCT03976297 . Registered on 6 June 2019, prior to trial start.

Improving the delivery of palliative care through predictive modeling and healthcare informatics.

OBJECTIVE: Access to palliative care (PC) is important for many patients with uncontrolled symptom burden from serious or complex illness. However, many patients who could benefit from PC do not receive it early enough or at all. We sought to address this problem by building a predictive model into a comprehensive clinical framework with the aims to (i) identify in-hospital patients likely to benefit from a PC consult, and (ii) intervene on such patients by contacting their care team. MATERIALS AND METHODS: Electronic health record data for 68 349 inpatient encounters in 2017 at a large hospital were used to train a model to predict the need for PC consult. This model was published as a web service, connected to institutional data pipelines, and consumed by a downstream display application monitored by the PC team. For those patients that the PC team deems appropriate, a team member then contacts the patient's corresponding care team. RESULTS: Training performance AUC based on a 20% holdout validation set was 0.90. The most influential variables were previous palliative care, hospital unit, Albumin, Troponin, and metastatic cancer. The model has been successfully integrated into the clinical workflow making real-time predictions on hundreds of patients per day. The model had an "in-production" AUC of 0.91. A clinical trial is currently underway to assess the effect on clinical outcomes. CONCLUSIONS: A machine learning model can effectively predict the need for an inpatient PC consult and has been successfully integrated into practice to refer new patients to PC.

GM-CSF Neutralization With Lenzilumab in Severe COVID-19 Pneumonia: A Case-Cohort Study.

OBJECTIVE: To assess the efficacy and safety of lenzilumab in patients with severe coronavirus disease 2019 (COVID-19) pneumonia. METHODS: Hospitalized patients with COVID-19 pneumonia and risk factors for poor outcomes were treated with lenzilumab 600 mg intravenously for three doses through an emergency single-use investigational new drug application. Patient characteristics, clinical and laboratory outcomes, and adverse events were recorded. We also identified a cohort of patients matched to the lenzilumab patients for age, sex, and disease severity. Study dates were March 13, 2020, to June 18, 2020. All patients were followed through hospital discharge or death. RESULTS: Twelve patients were treated with lenzilumab; 27 patients comprised the matched control cohort (untreated). Clinical improvement, defined as improvement of at least 2 points on the 8-point ordinal clinical endpoints scale, was observed in 11 of 12 (91.7%) patients treated with lenzilumab and 22 of 27 (81.5%) untreated patients. The time to clinical improvement was significantly shorter for the lenzilumab-treated group compared with the untreated cohort with a median of 5 days versus 11 days (P=.006). Similarly, the proportion of patients with acute respiratory distress syndrome (oxygen saturation/fraction of inspired oxygen<315 mm Hg) was significantly reduced over time when treated with lenzilumab compared with untreated (P<.001). Significant improvement in inflammatory markers (C-reactive protein and interleukin 6) and markers of disease severity (absolute lymphocyte count) were observed in patients who received lenzilumab, but not in untreated patients. Cytokine analysis showed a reduction in inflammatory myeloid cells 2 days after lenzilumab treatment. There were no treatment-emergent adverse events attributable to lenzilumab. CONCLUSION: In high-risk COVID-19 patients with severe pneumonia, granulocyte-macrophage colony-stimulating factor neutralization with lenzilumab was safe and associated with faster improvement in clinical outcomes, including oxygenation, and greater reductions in inflammatory markers compared with a matched control cohort of patients hospitalized with severe COVID-19 pneumonia. A randomized, placebo-controlled clinical trial to validate these findings is ongoing (NCT04351152).

Clinical impact of intraoperative electronic health record downtime on surgical patients.

OBJECTIVE: Despite increased use of electronic health records (EHRs), the clinical impact of system downtime is unknown. MATERIALS AND METHODS: This retrospective matched cohort study evaluated the impact of EHR downtime episodes lasting more than 60 minutes over a 6-year study period. Patients age 18 years or older who underwent surgical procedures at least 60 minutes in duration with an inpatient stay exceeding 24 hours within the study period were eligible for inclusion. Out of 4115 patients exposed to 1 of 176 EHR downtime episodes, 4103 patients were matched to an unexposed cohort in a 1:1 ratio. Multivariable regression analysis, as well as trend analysis for effect of duration of downtime on outcomes, was performed. RESULTS: Downtime-exposed patients had operating room duration 1.1 times longer (p < .001) and postoperative length of stay 1.04 times longer (p = .007) compared to unexposed patients. The 30-day mortality rates were similar between these groups (odds ratio 1.26, p > .05). In trend analysis, there was no association between duration of downtime with respect to evaluated outcomes, postoperative length of stay, and 30-day mortality. CONCLUSION: EHR downtime had no impact on 30-day mortality. Potential associations for increased postoperative length of stay and duration of time spent in the operating room were observed among downtime-exposed patients. No trend effect was observed with respect to duration of downtime and postoperative length of stay and 30-day mortality rates.

Decision Support Tool to Improve Glucose Control Compliance After Cardiac Surgery.

Hyperglycemia control is associated with improved outcomes in patients undergoing cardiac surgery. The Surgical Care Improvement Project metric (SCIP-inf-4) was introduced as a performance measure in surgical patients and included hyperglycemia control. Compliance with the SCIP-inf-4 metric remains suboptimal. A novel real-time decision support tool (DST) with guaranteed feedback that is based on the existing electronic medical record system was developed at a tertiary academic center. Implementation of the DST increased the compliance rate with the SCIP-inf-4 from 87.3% to 96.5%. Changes in tested clinical outcomes were not observed with improved metric compliance. This new framework can serve as a backbone for development of quality control processes for other metrics. Further and, ideally, multicenter studies are required to test if implementation of electronic DSTs will translate into improved resource utilization and outcomes for patients.

The Effect of an Electronic Checklist on Critical Care Provider Workload, Errors, and Performance.

PURPOSE: The strategy used to improve effective checklist use in intensive care unit (ICU) setting is essential for checklist success. This study aimed to test the hypothesis that an electronic checklist could reduce ICU provider workload, errors, and time to checklist completion, as compared to a paper checklist. METHODS: This was a simulation-based study conducted at an academic tertiary hospital. All participants completed checklists for 6 ICU patients: 3 using an electronic checklist and 3 using an identical paper checklist. In both scenarios, participants had full access to the existing electronic medical record system. The outcomes measured were workload (defined using the National Aeronautics and Space Association task load index [NASA-TLX]), the number of checklist errors, and time to checklist completion. Two independent clinician reviewers, blinded to participant results, served as the reference standard for checklist error calculation. RESULTS: Twenty-one ICU providers participated in this study. This resulted in the generation of 63 simulated electronic checklists and 63 simulated paper checklists. The median NASA-TLX score was 39 for the electronic checklist and 50 for the paper checklist (P = .005). The median number of checklist errors for the electronic checklist was 5, while the median number of checklist errors for the paper checklist was 8 (P = .003). The time to checklist completion was not significantly different between the 2 checklist formats (P = .76). CONCLUSION: The electronic checklist significantly reduced provider workload and errors without any measurable difference in the amount of time required for checklist completion. This demonstrates that electronic checklists are feasible and desirable in the ICU setting.

Adverse in-hospital events are associated with increased in-hospital mortality and length of stay in patients with or at risk of acute respiratory distress syndrome.

OBJECTIVE: To explore the effect of various adverse hospital events on short- and long-term outcomes in a cohort of acutely ill hospitalized patients. PATIENTS AND METHODS: In a secondary analysis of a retrospective cohort of acutely ill hospitalized patients with sepsis, shock, or pneumonia or undergoing high-risk surgery who were at risk for or had developed acute respiratory distress syndrome between 2001 and 2010, the effects of potentially preventable hospital exposures and adverse events (AEs) on in-hospital and intensive care unit (ICU) mortality, length of stay, and long-term survival were analyzed. Adverse effects chosen for inclusion were inadequate empiric antimicrobial coverage, hospital-acquired aspiration, medical or surgical misadventure, inappropriate blood product transfusion, and injurious tidal volume while on mechanical ventilation. RESULTS: In 828 patients analyzed, the distribution of 0, 1, 2, and 3 or more cumulative AEs was 521 (63%), 126 (15%), 135 (16%), and 46 (6%) patients, respectively. The adjusted odds ratios (95% CI) for in-hospital mortality in patients who had 1, 2, and 3 or more AEs were 0.9 (0.5-1.7), 0.9 (0.5-1.6), and 1.4 (0.6-3.3), respectively. One AE increased the length of stay, difference between means (95% CI), in the hospital by 8.7 (3.8-13.7) days and in the ICU by 2.4 (0.6-4.2) days. CONCLUSION: Potentially preventable hospital exposure to AEs is associated with prolonged ICU and hospital lengths of stay. Implementation of effective patient safety interventions is of utmost priority in acute care hospitals.

Outcome of adverse events and medical errors in the intensive care unit: a systematic review and meta-analysis.

Adverse events and medical errors (AEs/MEs) are more likely to occur in the intensive care unit (ICU). Information about the incidence and outcomes of such events is conflicting. A systematic review and meta-analysis were conducted to examine the effects of MEs/AEs on mortality and hospital and ICU lengths of stay among ICU patients. Potentially eligible studies were identified from 4 major databases. Of 902 studies screened, 12 met the inclusion criteria, 10 of which are included in the quantitative analysis. Patients with 1 or more MEs/AEs (vs no MEs/AEs) had a nonsignificant increase in mortality (odds ratio = 1.5; 95% confidence interval [CI] = 0.98-2.14) but significantly longer hospital and ICU stays; the mean difference (95% CI) was 8.9 (3.3-14.7) days for hospital stay and 6.8 (0.2-13.4) days for ICU. The ICU environment is associated with a substantial incidence of MEs/AEs, and patients with MEs/AEs have worse outcomes than those with no MEs/AEs.

Connecting the dots: rule-based decision support systems in the modern EMR era.

The intensive care unit (ICU) environment is rich in both medical device and electronic medical record (EMR) data. The ICU patient population is particularly vulnerable to medical error or delayed medical intervention both of which are associated with excess morbidity, mortality and cost. The development and deployment of smart alarms, computerized decision support systems (DSS) and "sniffers" within ICU clinical information systems has the potential to improve the safety and outcomes of critically ill hospitalized patients. However, the current generations of alerts, run largely through bedside monitors, are far from ideal and rarely support the clinician in the early recognition of complex physiologic syndromes or deviations from expected care pathways. False alerts and alert fatigue remain prevalent. In the coming era of widespread EMR implementation novel medical informatics methods may be adaptable to the development of next generation, rule-based DSS.

Derivation and validation of automated electronic search strategies to extract Charlson comorbidities from electronic medical records.

OBJECTIVE: To develop and validate automated electronic note search strategies (automated digital algorithm) to identify Charlson comorbidities. PATIENTS AND METHODS: The automated digital algorithm was built by a series of programmatic queries applied to an institutional electronic medical record database. The automated digital algorithm was derived from secondary analysis of an observational cohort study of 1447 patients admitted to the intensive care unit from January 1 through December 31, 2006, and validated in an independent cohort of 240 patients. The sensitivity, specificity, and positive and negative predictive values of the automated digital algorithm and International Classification of Diseases, Ninth Revision (ICD-9) codes were compared with comprehensive medical record review (reference standard) for the Charlson comorbidities. RESULTS: In the derivation cohort, the automated digital algorithm achieved a median sensitivity of 100% (range, 99%-100%) and a median specificity of 99.7% (range, 99%-100%). In the validation cohort, the sensitivity of the automated digital algorithm ranged from 91% to 100%, and the specificity ranged from 98% to 100%. The sensitivity of the ICD-9 codes ranged from 8% for dementia to 100% for leukemia, whereas specificity ranged from 86% for congestive heart failure to 100% for leukemia, dementia, and AIDS. CONCLUSION: Our results suggest that search strategies that use automated electronic search strategies to extract Charlson comorbidities from the clinical notes contained within the electronic medical record are feasible and reliable. Automated digital algorithm outperformed ICD-9 codes in all the Charlson variables except leukemia, with greater sensitivity, specificity, and positive and negative predictive values.

Clinical review: the hospital of the future - building intelligent environments to facilitate safe and effective acute care delivery.

The translation of knowledge into rational care is as essential and pressing a task as the development of new diagnostic or therapeutic devices, and is arguably more important. The emerging science of health care delivery has identified the central role of human factor ergonomics in the prevention of medical error, omission, and waste. Novel informatics and systems engineering strategies provide an excellent opportunity to improve the design of acute care delivery. In this article, future hospitals are envisioned as organizations built around smart environments that facilitate consistent delivery of effective, equitable, and error-free care focused on patient-centered rather than provider-centered outcomes.

Identification of patient information corruption in the intensive care unit: using a scoring tool to direct quality improvements in handover.

OBJECTIVE: To use a handover assessment tool for identifying patient information corruption and objectively evaluating interventions designed to reduce handover errors and improve medical decision making. The continuous monitoring, intervention, and evaluation of the patient in modern intensive care unit practice generates large quantities of information, the platform on which medical decisions are made. Information corruption, defined as errors of distortion/omission compared with the medical record, may result in medical judgment errors. Identifying these errors may lead to quality improvements in intensive care unit care delivery and safety. DESIGN: Handover assessment instrument development study divided into two phases by the introduction of a handover intervention. SETTING: Closed, 17-bed, university-affiliated mixed surgical/medical intensive care unit. SUBJECTS: Senior and junior medical members of the intensive care unit team. INTERVENTIONS: Electronic handover page. MEASUREMENTS AND MAIN RESULTS: Study subjects were asked to recall clinical information commonly discussed at handover on individual patients. The handover score measured the percentage of information correctly retained for each individual doctor-patient interaction. The clinical intention score, a subjective measure of medical judgment, was graded (1-5) by three blinded intensive care unit experts. A total of 137 interactions were scored. Median (interquartile range) handover scores for phases 1 and 2 were 79.07% (67.44-84.50) and 83.72% (76.16-88.37), respectively. Score variance was reduced by the handover intervention (p < .05). Increasing median handover scores, 68.60 to 83.72, were associated with increases in clinical intention scores from 1 to 5 (chi-square = 23.59, df = 4, p < .0001). CONCLUSIONS: When asked to recall clinical information discussed at handover, medical members of the intensive care unit team provide data that are significantly corrupted compared with the medical record. Low subjective clinical judgment scores are significant associated with low handover scores. The handover/clinical intention scores may, therefore, be useful screening tools for intensive care unit system vulnerability to medical error. Additionally, handover instruments can identify interventions that reduce system vulnerability to error and may be used to guide quality improvements in handover practice.