Use of Community SARS-CoV-2 Case Counts and Instantaneous Reproductive Number to Predict Short-Term COVID-19 Hospital Admission Volumes.

The severe acute respiratory syndrome (SARS-CoV-2) pandemic and high hospitalization rates placed a tremendous strain on hospital resources, necessitating the use of models to predict hospital volumes and the associated resource requirements. Complex epidemiologic models have been developed and published, but many require continued adjustment of input parameters. We developed a simplified model for short-term bed need predictions that self-adjusts to changing patterns of disease in the community and admission rates. The model utilizes public health data on community new case counts for SARS-CoV-2 and projects anticipated hospitalization rates. The model was retrospectively evaluated after the second wave of SARS-CoV-2 in New York, New York (October 2020-April 2021) for its accuracy in predicting numbers of coronavirus disease 2019 (COVID-19) admissions 3, 5, 7, and 10 days into the future, comparing predicted admissions with actual admissions for each day at a large integrated health-care delivery network. The mean absolute percent error of the model was found to be low when evaluated across the entire health system, for a single region of the health system or for a single large hospital (6.1%-7.6% for 3-day predictions, 9.2%-10.4% for 5-day predictions, 12.4%-13.2% for 7-day predictions, and 17.1%-17.8% for 10-day predictions).

Application of a Machine Learning Algorithm to Develop and Validate a Prediction Model for Ambulatory Non-Arrivals.

BACKGROUND: Non-arrivals to scheduled ambulatory visits are common and lead to a discontinuity of care, poor health outcomes, and increased subsequent healthcare utilization. Reducing non-arrivals is important given their association with poorer health outcomes and cost to health systems. OBJECTIVE: To develop and validate a prediction model for ambulatory non-arrivals. DESIGN: Retrospective cohort study. PATIENTS OR SUBJECTS: Patients at an integrated health system who had an outpatient visit scheduled from January 1, 2020, to February 28, 2022. MAIN MEASURES: Non-arrivals to scheduled appointments. KEY RESULTS: There were over 4.3 million ambulatory appointments from 1.2 million adult patients. Patients with appointment non-arrivals were more likely to be single, racial/ethnic minorities, and not having an established primary care provider compared to those who arrived at their appointments. A prediction model using the XGBoost machine learning algorithm had the highest AUC value (0.768 [0.767-0.770]). Using SHAP values, the most impactful features in the model include rescheduled appointments, lead time (number of days from scheduled to appointment date), appointment provider, number of days since last appointment with the same department, and a patient's prior appointment status within the same department. Scheduling visits close to an appointment date is predicted to be less likely to result in a non-arrival. Overall, the prediction model calibrated well for each department, especially over the operationally relevant probability range of 0 to 40%. Departments with fewer observations and lower non-arrival rates generally had a worse calibration. CONCLUSIONS: Using a machine learning algorithm, we developed a prediction model for non-arrivals to scheduled ambulatory appointments usable for all medical specialties. The proposed prediction model can be deployed within an electronic health system or integrated into other dashboards to reduce non-arrivals. Future work will focus on the implementation and application of the model to reduce non-arrivals.

Design, development, and deployment of an indication- and kidney function-based decision support tool to optimize treatment and reduce medication dosing errors.

Delivering clinical decision support (CDS) at the point of care has long been considered a major advantage of computerized physician order entry (CPOE). Despite the widespread implementation of CPOE, medication ordering errors and associated adverse events still occur at an unacceptable level. Previous attempts at indication- and kidney function-based dosing have mostly employed intrusive CDS, including interruptive alerts with poor usability. This descriptive work describes the design, development, and deployment of the Adult Dosing Methodology (ADM) module, a novel CDS tool that provides indication- and kidney-based dosing at the time of order entry. Inclusion of several antimicrobials in the initial set of medications allowed for the additional goal of optimizing therapy duration for appropriate antimicrobial stewardship. The CDS aims to decrease order entry errors and burden on providers by offering automatic dose and frequency recommendations, integration within the native electronic health record, and reasonable knowledge maintenance requirements. Following implementation, early utilization demonstrated high acceptance of automated recommendations, with up to 96% of provided automated recommendations accepted by users.

Examination of Post-discharge Follow-up Appointment Status and 30-Day Readmission.

BACKGROUND: Post-hospital discharge follow-up appointments are intended to evaluate patients' recovery following a hospitalization, but it is unclear how appointment statuses are associated with readmissions. OBJECTIVE: To examine the association between post-discharge ambulatory follow-up status, (1) having a scheduled appointment and (2) arriving to said appointment, and 30-day readmission. DESIGN AND SETTING: A retrospective cohort study of patients hospitalized at 12 hospitals in an Integrated Delivery Network and their ambulatory appointments in that same network. PATIENTS AND MAIN MEASURES: We included 50,772 patients who had an ambulatory appointment within 18 months of an inpatient admission in 2018. Primary outcome was readmission within 30 days post-discharge. KEY RESULTS: There were 32,108 (63.2%) patients with scheduled follow-up appointments and 18,664 (36.8%) patients with no follow-up; 28,313 (88.2%) patients arrived, 3149 (9.8%) missed, and 646 (2.0%) were readmitted prior to their scheduled appointments. Overall 30-day readmission rate was 7.3%; 6.0% [5.75-6.31] for those who arrived, 8.8% [8.44-9.25] for those without follow-up, and 10.3% [9.28-11.40] for those who missed a scheduled appointment (p < 0.001). After adjusting for covariates, patients who arrived at their appointment in the first week following discharge were significantly less likely to be readmitted than those not having any follow-up scheduled (medical adjusted hazard ratio (aHR) 0.57 [0.47-0.69], p < 0.001; surgical aHR 0.58 [0.44-0.75], p < 0.001) There was an increased risk at weeks 3 and 4 for medical patients who arrived at a follow-up compared to those with no follow-up scheduled (week 3 aHR 1.29 [1.10-1.51], p = 0.001; week 4 aHR 1.46 [1.26-1.70], p < 0.001). CONCLUSIONS: The benefit of patients arriving to their post-discharge appointments compared with patients who missed their follow-up visits or had no follow-up scheduled, is only significant during first week post-discharge, suggesting that coordination within 1 week of discharge is critical in reducing 30-day readmissions.

Let Sleeping Patients Lie, avoiding unnecessary overnight vitals monitoring using a clinically based deep-learning model.

Impaired sleep for hospital patients is an all too common reality. Sleep disruptions due to unnecessary overnight vital sign monitoring are associated with delirium, cognitive impairment, weakened immunity, hypertension, increased stress, and mortality. It is also one of the most common complaints of hospital patients while imposing additional burdens on healthcare providers. Previous efforts to forgo overnight vital sign measurements and improve patient sleep used providers' subjective stability assessment or utilized an expanded, thus harder to retrieve, set of vitals and laboratory results to predict overnight clinical risk. Here, we present a model that incorporates past values of a small set of vital signs and predicts overnight stability for any given patient-night. Using data obtained from a multi-hospital health system between 2012 and 2019, a recurrent deep neural network was trained and evaluated using ~2.3 million admissions and 26 million vital sign assessments. The algorithm is agnostic to patient location, condition, and demographics, and relies only on sequences of five vital sign measurements, a calculated Modified Early Warning Score, and patient age. We achieved an area under the receiver operating characteristic curve of 0.966 (95% confidence interval [CI] 0.956-0.967) on the retrospective testing set, and 0.971 (95% CI 0.965-0.974) on the prospective set to predict overnight patient stability. The model enables safe avoidance of overnight monitoring for ~50% of patient-nights, while only misclassifying 2 out of 10,000 patient-nights as stable. Our approach is straightforward to deploy, only requires regularly obtained vital signs, and delivers easily actionable clinical predictions for a peaceful sleep in hospitals.

An approach to predicting patient experience through machine learning and social network analysis.

OBJECTIVE: Improving the patient experience has become an essential component of any healthcare system's performance metrics portfolio. In this study, we developed a machine learning model to predict a patient's response to the Hospital Consumer Assessment of Healthcare Providers and Systems survey's "Doctor Communications" domain questions while simultaneously identifying most impactful providers in a network. MATERIALS AND METHODS: This is an observational study of patients admitted to a single tertiary care hospital between 2016 and 2020. Using machine learning algorithms, electronic health record data were used to predict patient responses to Hospital Consumer Assessment of Healthcare Providers and Systems survey questions in the doctor domain, and patients who are at risk for responding negatively were identified. Model performance was assessed by area under receiver-operating characteristic curve. Social network analysis metrics were also used to identify providers most impactful to patient experience. RESULTS: Using a random forest algorithm, patients' responses to the following 3 questions were predicted: "During this hospital stay how often did doctors. 1) treat you with courtesy and respect? 2) explain things in a way that you could understand? 3) listen carefully to you?" with areas under the receiver-operating characteristic curve of 0.876, 0.819, and 0.819, respectively. Social network analysis found that doctors with higher centrality appear to have an outsized influence on patient experience, as measured by rank in the random forest model in the doctor domain. CONCLUSIONS: A machine learning algorithm identified patients at risk of a negative experience. Furthermore, a doctor social network framework provides metrics for identifying those providers that are most influential on the patient experience.

Higher Imaging Yield When Clinical Decision Support Is Used.

OBJECTIVE: Increased utilization of CT pulmonary angiography (CTPA) for the evaluation of pulmonary embolism has been associated with decreasing diagnostic yields and rising concerns about the harms of unnecessary testing. The objective of this study was to determine whether clinical decision support (CDS) use would be associated with increased imaging yields after controlling for selection bias. METHODS: We performed a retrospective cohort study in the emergency departments of two tertiary care hospitals of all CTPAs performed between August 2015 and September 2018. Providers ordering a CTPA are routed to an optional CDS tool, which allows them to use Wells' Criteria for pulmonary embolism. After propensity score matching, CTPA yield was calculated for the CDS-use and CDS-dismissal groups and stratified by provider type. RESULTS: A total of 7,367 CTPAs were ordered during the study period. Of those, providers used the CDS tool in 2,568 (35%) cases and did not use the tool in 4,799 (65%) of cases. After propensity score matching, CTPA yield was 11.99% in the CDS-use group and 8.70% in the CDS-dismissal group (P < .001). Attending physicians, residents, and physician assistant CDS users demonstrated a 56.5% (P = .006), 38.7% (P = .01), and 16.7% (P = .03) increased yield compared with those who dismissed the tool, respectively. DISCUSSION: Diagnostic yield was 38% higher for CTPAs when the provider used the CDS tool, after controlling for selection bias. Yields were higher for every provider type. Further research is needed to discover successful strategies to increase provider use of these important tools.

ClinRefLink: implementation of infobutton-like functionality in a commercial clinical information system incorporating concepts from textual documents.

Clinical information systems offer an opportunity to provide clinicians with medical reference materials during clinical encounters when the information is most beneficial. Implementation of this "Infobutton" concept has been described by a number of institutions with locally developed clinical information systems and electronic medical records. This article describes the development of an infobutton-like application called ClinRefLink embedded within a commercial clinical information system. ClinRefLink is somewhat unique in that it offers clinicians the option to perform reference searches based on clinical entities identified within narrative documents. In the first 30 days after implementation, 1018 reference searches were performed. The characteristics of the clinicians and the clinical context of the search terms are described. These data support the value of clinical term extraction from narrative documents as a component of an infobutton system.

A framework for analyzing the cognitive complexity of computer-assisted clinical ordering.

Computer-assisted provider order entry is a technology that is designed to expedite medical ordering and to reduce the frequency of preventable errors. This paper presents a multifaceted cognitive methodology for the characterization of cognitive demands of a medical information system. Our investigation was informed by the distributed resources (DR) model, a novel approach designed to describe the dimensions of user interfaces that introduce unnecessary cognitive complexity. This method evaluates the relative distribution of external (system) and internal (user) representations embodied in system interaction. We conducted an expert walkthrough evaluation of a commercial order entry system, followed by a simulated clinical ordering task performed by seven clinicians. The DR model was employed to explain variation in user performance and to characterize the relationship of resource distribution and ordering errors. The analysis revealed that the configuration of resources in this ordering application placed unnecessarily heavy cognitive demands on the user, especially on those who lacked a robust conceptual model of the system. The resources model also provided some insight into clinicians' interactive strategies and patterns of associated errors. Implications for user training and interface design based on the principles of human-computer interaction in the medical domain are discussed.

Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment.

Computerized assistance to clinicians during physician order entry can provide protection against medical errors. However, computer systems that provide too much assistance may adversely affect training of medical students and residents. Trainees may rely on the computer to automatically perform complex calculations and create appropriate orders and are thereby deprived of an important educational exercise. An alternative strategy is to provide a critique at the completion of an order, requiring the trainee to enter the entire order but displaying an alert if an error is made. While this approach preserves the educational components of order-writing, the potential for errors exists if the computerized critique does not induce clinicians to correct the order. The goal of this study was to determine (a) the frequency with which errors are made by trainees in an environment in which renal dosing adjustment calculation for antimicrobials are done by the system after the user has entered an order, and (b) the frequency with which prompts to clinicians regarding these errors leads to correction of those orders.