Clinical outcome prediction using observational supervision with electronic health records and audit logs.

OBJECTIVE: Audit logs in electronic health record (EHR) systems capture interactions of providers with clinical data. We determine if machine learning (ML) models trained using audit logs in conjunction with clinical data ("observational supervision") outperform ML models trained using clinical data alone in clinical outcome prediction tasks, and whether they are more robust to temporal distribution shifts in the data. MATERIALS AND METHODS: Using clinical and audit log data from Stanford Healthcare, we trained and evaluated various ML models including logistic regression, support vector machine (SVM) classifiers, neural networks, random forests, and gradient boosted machines (GBMs) on clinical EHR data, with and without audit logs for two clinical outcome prediction tasks: major adverse kidney events within 120 days of ICU admission (MAKE-120) in acute kidney injury (AKI) patients and 30-day readmission in acute stroke patients. We further tested the best performing models using patient data acquired during different time-intervals to evaluate the impact of temporal distribution shifts on model performance. RESULTS: Performance generally improved for all models when trained with clinical EHR data and audit log data compared with those trained with only clinical EHR data, with GBMs tending to have the overall best performance. GBMs trained with clinical EHR data and audit logs outperformed GBMs trained without audit logs in both clinical outcome prediction tasks: AUROC 0.88 (95% CI: 0.85-0.91) vs. 0.79 (95% CI: 0.77-0.81), respectively, for MAKE-120 prediction in AKI patients, and AUROC 0.74 (95% CI: 0.71-0.77) vs. 0.63 (95% CI: 0.62-0.64), respectively, for 30-day readmission prediction in acute stroke patients. The performance of GBM models trained using audit log and clinical data degraded less in later time-intervals than models trained using only clinical data. CONCLUSION: Observational supervision with audit logs improved the performance of ML models trained to predict important clinical outcomes in patients with AKI and acute stroke, and improved robustness to temporal distribution shifts.

Predictive analytics support for complex chronic medical conditions: An experience-based co-design study of physician managers' needs and preferences.

PURPOSE: The literature suggests predictive technology applications in health care would benefit from physician and manager input during design and development. The aim was to explore the needs and preferences of physician managers regarding the role of predictive analytics in decision support for patients with the highly complex yet common combination of multiple chronic conditions of cardiovascular (Heart) and kidney (Nephrology) diseases and diabetes (HND). METHODS: This qualitative study employed an experience-based co-design model comprised of three data gathering phases: 1. Patient mapping through non-participant observations informed by process mining of electronic health records data, 2. Semi-structured experience-based interviews, and 3. A co-design workshop. Data collection was conducted with physician managers working at or collaborating with the HND center, Danderyd University Hospital (DSAB), in Stockholm, Sweden. HND center is an integrated practice unit offering comprehensive person-centered multidisciplinary care to stabilize disease progression, reduce visits, and develop treatment strategies that enables a transition to primary care. RESULTS: Interview and workshop data described a complex challenge due to the interaction of underlying pathophysiologies and the subsequent need for multiple care givers that hindered care continuity. The HND center partly met this challenge by coordinating care through multiple interprofessional and interdisciplinary shared decision-making interfaces. The large patient datasets were difficult to operationalize in daily practice due to data entry and retrieval issues. Predictive analytics was seen as a potentially effective approach to support decision-making, calculate risks, and improve resource utilization, especially in the context of complex chronic care, and the HND center a good place for pilot testing and development. Simplicity of visual interfaces, a better understanding of the algorithms by the health care professionals, and the need to address professional concerns, were identified as key factors to increase adoption and facilitate implementation. CONCLUSIONS: The HND center serves as a comprehensive integrated practice unit that integrates different medical disciplinary perspectives in a person-centered care process to address the needs of patients with multiple complex comorbidities. Therefore, piloting predictive technologies at the same time with a high potential for improving care represents an extreme, demanding, and complex case. The study findings show that health care professionals' involvement in the design of predictive technologies right from the outset can facilitate the implementation and adoption of such technologies, as well as enhance their predictive effectiveness and performance. Simplicity in the design of predictive technologies and better understanding of the concept and interpretation of the algorithms may result in implementation of predictive technologies in health care. Institutional efforts are needed to enhance collaboration among the health care professionals and IT professionals for effective development, implementation, and adoption of predictive analytics in health care.

Prediction of 30-day unplanned hospital readmission through survival analysis.

Background and Objective: Unplanned hospital readmissions are a severe and recurrent problem that affects all health systems. Estimating the risk of being readmitted the following days after discharge is difficult since many heterogeneous factors can influence this. The extensive work concerning this problem proposes solutions mostly based on classification machine-learning models. Survival analysis methods could make a better match with the assessment of readmission risk and are yet to become well-established in this field. Methods: We compare different statistical and machine learning survival analysis models trained with right-censored all-cause hospital admission data with covariates available at the moment of discharge. The main focus is on tree-ensemble regression methods based on the assumption of proportional hazards. These models are more thoroughly evaluated at a 30-day time period after discharge, although the actual prediction could be set to any time up to 90 days. Results: The mean performance obtained by each of the proposed survival models ranges from 0.707 to 0.716 C-Index and 0.709 to 0.72 ROC-AUC at a 30-day time period after discharge. The model with the lower performance on both metrics was Cox Proportional Hazards, while the model marking the upper end on both ranges is an XGBoost Regression model with a Cox objective function. Conclusions: Our findings indicate that survival models perform well addressing the hospital readmission problem, machine-learning models getting the edge over statistical methods. There seems to be an improvement over classification models when attempting to predict at a 30-day period since discharge, perhaps due to a better handling of cases nearing the 30-day boundary. Some preprocessing steps, such as limiting the observation period to 90 days after discharge, are also highlighted since they resulted in a performance boost.

Association between the LACE+ index and unplanned 30-day hospital readmissions in hospitalized patients with stroke.

Background: The LACE+ index is used to predict unplanned 30-day hospital readmissions, but its utility to predict 30-day readmission in hospitalized patients with stroke is unknown. Methods: We retrospectively analyzed 1,657 consecutive patients presenting with ischemic or hemorrhagic strokes, included in an institutional stroke registry between January 2018 and August 2020. The primary outcome of interest was unplanned 30-day readmission for any reason after index hospitalization for stroke. The 30-day readmission risk was categorized by LACE+ index to high risk (≥78), medium-to-high risk (59-77), medium risk (29-58), and low risk (≤ 28). Kaplan-Meier analysis, Log rank test, and multivariable Cox regression analysis (with backward elimination) were used to determine whether the LACE+ score was an independent predictor for 30-day unplanned readmission. Results: The overall 30-day unplanned readmission rate was 11.7% (194/1,657). The median LACE+ score was higher in the 30-day readmission group compared to subjects that had no unplanned 30-day readmission [74 (IQR 67-79) vs. 70 (IQR 62-75); p < 0.001]. On Kaplan-Meier analysis, the high-risk group had the shortest 30-day readmission free survival time as compared to medium and medium-to-high risk groups (p < 0.01, each; statistically significant). On fully adjusted multivariable Cox-regression, the highest LACE+ risk category was independently associated with the unplanned 30-day readmission risk (per point: HR 1.67 95%CI 1.23-2.26, p = 0.001). Conclusion: Subjects in the high LACE+ index category had a significantly greater unplanned 30-day readmission risk after stroke as compared to lower LACE+ risk groups. This supports the validity of the LACE+ scoring system for predicting unplanned readmission in subjects with stroke. Future studies are warranted to determine whether LACE+ score-based risk stratification can be used to devise early interventions to mitigate the risk for unplanned readmission.

Association of cardiovascular disease with 30-day hospital readmission in Chinese patients receiving maintenance dialysis.

BACKGROUND: Previous studies have shown cardiovascular disease (CVD) to be a risk factor in the prediction of 30-day hospital readmission among patients receiving dialysis. However, studies of Asian populations are limited. In the present study, we examined the association between CVD and 30-day hospital readmission in Chinese patients receiving maintenance dialysis. METHODS: Patients receiving maintenance dialysis were identified by searching a national claims database, the China Health Insurance Research Association (CHIRA) database, using the International Classification of Diseases revision 10 (ICD-10) and items of medical service claims. Patients aged ≥18 years who were discharged after index hospitalization between January 2015 and December 2015 were included in our retrospective analysis. CVD-related diagnoses were divided into three categories: coronary heart disease (CHD), heart failure (HF), and stroke. Thirty-day hospital readmission was defined as any hospital readmission within the 30 days following discharge. Logistic regression models adjusted for logit of propensity scores (PS) were used to assess the association of CVD with 30-day hospital readmission. RESULTS: Of 4,700 patients receiving dialysis, the 30-day hospital readmission rate was 10.4%. Compared with patients without CVD, there was an increased risk of 30-day hospital readmission among maintenance dialysis patients with total CVD [odds ratio (OR): 1.33, 95% confidence interval (CI): 1.06-1.66]. Patients with HF (OR: 1.77, CI: 1.27-2.47) and stroke (OR: 2.14, 95% CI: 1.53-2.98) had a greater risk of 30-day hospital readmission. The fully adjusted OR of CHD for the risk of 30-day hospital readmission was 1.22 (95% CI: 0.97-1.55). CONCLUSIONS: CVDs, especially stroke and HF, are independent predictors of 30-day hospital readmission in Chinese patients receiving dialysis, and could help to guide interventions to improve the quality of care for these patients.

Patient and Provider Perspectives on 30-Day Readmissions, Preventability, and Strategies for Improving Transitions of Care for Patients with HIV at a Safety Net Hospital.

Thirty-day hospital readmissions, a key quality metric, are common among people living with HIV. We assessed perceived causes of 30-day readmissions, factors associated with preventability, and strategies to reduce preventable readmissions and improve continuity of care for HIV-positive individuals. Patient, provider, and staff perspectives toward 30-day readmissions were evaluated in semistructured interviews (n = 86) conducted in triads (HIV-positive patient, medical provider, and case manager) recruited from an inpatient safety net hospital. Iterative analysis included both deductive and inductive themes. Key findings include the following: (1) The 30-day metric should be adjusted for safety net institutions and patients with AIDS; (2) Participants disagreed about preventability, especially regarding patient-level factors; (3) Various stakeholders proposed readmission reduction strategies that spanned the inpatient to outpatient care continuum. Based on these diverse perspectives, we outline multiple interventions, from teach-back patient education to postdischarge home visits, which could substantially decrease hospital readmissions in this underserved population.

The Bundled Hospital Elder Life Program-HELP and HELP in Home Care-and Its Association With Clinical Outcomes Among Older Adults Discharged to Home Healthcare.

OBJECTIVES: To describe the Bundled Hospital Elder Life Program (HELP and HELP in Home Care), an adaptation of HELP, and examine the association of 30-day all-cause unplanned hospital readmission risk among older adults discharged to home care with and without Bundled HELP. DESIGN: Matched case-control study. SETTING: Two medical-surgical units within two midwestern rural hospitals and patient homes (home health). PARTICIPANTS: Hospitalized patients, aged 65 years and older, discharged to home healthcare with and without Bundled HELP exposure between January 1, 2015, and September 30, 2017. Each case (Bundled HELP, n = 148) was matched to a control (non-Bundled HELP, n = 148) on Charlson Comorbidity Index, primary hospital diagnosis of orthopedic condition or injury, and cardiovascular disease using propensity score matching. MEASUREMENTS: The primary study outcome was 30-day all-cause unplanned hospital readmission. Additional outcomes measured were 30-day emergency department (ED) visit, hospital length of stay (LOS), and total number of skilled home care visits. RESULTS: Fewer cases (16.8%) than controls (28.4%) had a 30-day all-cause unplanned hospital readmission. The fully adjusted model showed significantly lower risk of 30-day hospital readmission for case (Bundled HELP) patients (0.41; 95% confidence interval = 0.22-0.77; P < .01). The difference between case (10.8%) and control (15.5%) 30-day ED visit was not significant (P = .23). A lower LOS for the case group was shown (P < .01), while the number of skilled home care visits was not significantly different between groups (P = .28). CONCLUSION: HELP protocol implementation during a patient's hospital stay and as a continued component of home care among older adults at risk for cognitive and/or functional decline appears to be associated with favorable outcomes. Our initial evaluation supports continued study of the Bundled HELP. Further research is needed to confirm the initial findings and to evaluate the impact of the adapted model on functional outcomes and delirium incidence in the home. J Am Geriatr Soc 67:1730-1736, 2019.

Is duration of hospital participation in meaningful use associated with value in Medicare?

OBJECTIVES: "Meaningful Use" (MU) of electronic health records (EHRs) is a measure used by Medicare to determine whether hospitals are comprehensively using electronic tools. Whether hospitals' engagement in value-based initiatives such as MU is associated with value-defined as high quality and low costs-is unknown. Our objectives were to describe hospital participation in MU, and determine whether duration of participation is associated with value. MATERIALS AND METHODS: We linked national Medicare data with MU and other hospital-level and market data. We analyzed bivariate relationships to characterize duration of participation. We estimated inverse probability-weighted multilevel logistic regressions to evaluate whether duration of participation was associated with higher likelihood of value-operationalized as having performance on 30-day readmission and inpatient spending at or below the national average. RESULTS: Of 2860 short-term hospitals, 59% had 4 or 5 years of MU participation by 2015; 7% had 1 or 2 years. There were differences by duration of participation across location, ownership, and size. Seventeen percent of hospitals were classified as high-value. Controlling for hospital characteristics, and holding constant market location, there was no evidence of a statistical association between duration of participation and value (odds ratio = 1.05, 95% confidence interval: 0.91-1.21; P = .51). Examining the 2 outcomes separately, there was a significant relationship between duration of participation and lower Medicare inpatient spending, but not 30-day readmission. DISCUSSION: Sustained participation in MU is associated with lower Medicare spending, but not with lower readmission rates. CONCLUSION: Policy interventions aimed at increasing value may need a broader focus than EHR implementation and use.