Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): A retrospective, single-site study.

BACKGROUND: Pythia is an automated, clinically curated surgical data pipeline and repository housing all surgical patient electronic health record (EHR) data from a large, quaternary, multisite health institute for data science initiatives. In an effort to better identify high-risk surgical patients from complex data, a machine learning project trained on Pythia was built to predict postoperative complication risk. METHODS AND FINDINGS: A curated data repository of surgical outcomes was created using automated SQL and R code that extracted and processed patient clinical and surgical data across 37 million clinical encounters from the EHRs. A total of 194 clinical features including patient demographics (e.g., age, sex, race), smoking status, medications, comorbidities, procedure information, and proxies for surgical complexity were constructed and aggregated. A cohort of 66,370 patients that had undergone 99,755 invasive procedural encounters between January 1, 2014, and January 31, 2017, was studied further for the purpose of predicting postoperative complications. The average complication and 30-day postoperative mortality rates of this cohort were 16.0% and 0.51%, respectively. Least absolute shrinkage and selection operator (lasso) penalized logistic regression, random forest models, and extreme gradient boosted decision trees were trained on this surgical cohort with cross-validation on 14 specific postoperative outcome groupings. Resulting models had area under the receiver operator characteristic curve (AUC) values ranging between 0.747 and 0.924, calculated on an out-of-sample test set from the last 5 months of data. Lasso penalized regression was identified as a high-performing model, providing clinically interpretable actionable insights. Highest and lowest performing lasso models predicted postoperative shock and genitourinary outcomes with AUCs of 0.924 (95% CI: 0.901, 0.946) and 0.780 (95% CI: 0.752, 0.810), respectively. A calculator requiring input of 9 data fields was created to produce a risk assessment for the 14 groupings of postoperative outcomes. A high-risk threshold (15% risk of any complication) was determined to identify high-risk surgical patients. The model sensitivity was 76%, with a specificity of 76%. Compared to heuristics that identify high-risk patients developed by clinical experts and the ACS NSQIP calculator, this tool performed superiorly, providing an improved approach for clinicians to estimate postoperative risk for patients. Limitations of this study include the missingness of data that were removed for analysis. CONCLUSIONS: Extracting and curating a large, local institution's EHR data for machine learning purposes resulted in models with strong predictive performance. These models can be used in clinical settings as decision support tools for identification of high-risk patients as well as patient evaluation and care management. Further work is necessary to evaluate the impact of the Pythia risk calculator within the clinical workflow on postoperative outcomes and to optimize this data flow for future machine learning efforts.

Preliminary results of a clinical research and innovation scholarship to prepare medical students to lead innovations in health care.

There is consensus amongst national organizations to integrate health innovation and augmented intelligence (AI) into medical education. However, there is scant evidence to guide policymakers and medical educators working to revise curricula. This study presents academic, operational, and domain understanding outcomes for the first three cohorts of participants in a clinical research and innovation scholarship program.

Improving Well-Being Among Trainees: A Partnership to Reduce Barriers to Primary Care Services.

BACKGROUND: Improved well-being is a focus for graduate medical education (GME) programs. Residents and fellows often express difficulty with visiting primary care physicians, and this issue has not been thoroughly investigated. OBJECTIVE: We reported implementation and utilization of a primary care concierge scheduling service and a primary care video visit service for GME trainees. METHODS: GME leaders collaborated with Duke Primary Care to offer trainees a concierge scheduling service and opportunity for primary care video visits. This quantitative evaluation included (1) analysis of the institutional GME survey results pre- and post-intervention, and (2) review of use of the concierge scheduling line. RESULTS: Comparison of the 2018 and 2019 internal GME surveys showed a decrease in perceived barriers accessing primary care (58% to 31%, P < .0001), a decrease in perceived delays to access primary care (27% to 21%, P = .023), and an increase in respondents who reported needing health care services in the past year (37% to 62%, P < .0001). Although increased need for health services was reported, there was no difference in the proportion reporting use of health services (63% and 65%, P = .43). Of the 142 concierge line calls reviewed, 127 (87%) callers requested clinic appointments, and 15 (10%) callers requested video appointments. Of callers requesting clinic appointments, 99 (80%) were scheduled. CONCLUSIONS: Providing resources to connect trainees to primary care greatly reduces their perception of barriers to health care and may provide a convenient mechanism to schedule flexible primary care appointments.