Development and Validation of a Machine Learning Model to Predict Near-Term Risk of Iatrogenic Hypoglycemia in Hospitalized Patients.

Importance: Accurate clinical decision support tools are needed to identify patients at risk for iatrogenic hypoglycemia, a potentially serious adverse event, throughout hospitalization. Objective: To predict the risk of iatrogenic hypoglycemia within 24 hours after each blood glucose (BG) measurement during hospitalization using a machine learning model. Design, Setting, and Participants: This retrospective cohort study, conducted at 5 hospitals within the Johns Hopkins Health System, included 54 978 admissions of 35 147 inpatients who had at least 4 BG measurements and received at least 1 U of insulin during hospitalization between December 1, 2014, and July 31, 2018. Data from the largest hospital were split into a 70% training set and 30% test set. A stochastic gradient boosting machine learning model was developed using the training set and validated on internal and external validation. Exposures: A total of 43 clinical predictors of iatrogenic hypoglycemia were extracted from the electronic medical record, including demographic characteristics, diagnoses, procedures, laboratory data, medications, orders, anthropomorphometric data, and vital signs. Main Outcomes and Measures: Iatrogenic hypoglycemia was defined as a BG measurement less than or equal to 70 mg/dL occurring within the pharmacologic duration of action of administered insulin, sulfonylurea, or meglitinide. Results: This cohort study included 54 978 admissions (35 147 inpatients; median [interquartile range] age, 66.0 [56.0-75.0] years; 27 781 [50.5%] male; 30 429 [55.3%] White) from 5 hospitals. Of 1 612 425 index BG measurements, 50 354 (3.1%) were followed by iatrogenic hypoglycemia in the subsequent 24 hours. On internal validation, the model achieved a C statistic of 0.90 (95% CI, 0.89-0.90), a positive predictive value of 0.09 (95% CI, 0.08-0.09), a positive likelihood ratio of 4.67 (95% CI, 4.59-4.74), a negative predictive value of 1.00 (95% CI, 1.00-1.00), and a negative likelihood ratio of 0.22 (95% CI, 0.21-0.23). On external validation, the model achieved C statistics ranging from 0.86 to 0.88, positive predictive values ranging from 0.12 to 0.13, negative predictive values of 0.99, positive likelihood ratios ranging from 3.09 to 3.89, and negative likelihood ratios ranging from 0.23 to 0.25. Basal insulin dose, coefficient of variation of BG, and previous hypoglycemic episodes were the strongest predictors. Conclusions and Relevance: These findings suggest that iatrogenic hypoglycemia can be predicted in a short-term prediction horizon after each BG measurement during hospitalization. Further studies are needed to translate this model into a real-time informatics alert and evaluate its effectiveness in reducing the incidence of inpatient iatrogenic hypoglycemia.

Development, Implementation, and Evaluation of a Physician-Targeted Inpatient Glycemic Management Curriculum.

OBJECTIVE: Diabetes is prevalent among hospitalized patients and there are multiple challenges to attaining glycemic control in the hospital setting. We sought to develop an inpatient glycemic management curriculum with stakeholder input and to evaluate the effectiveness of this educational program on glycemic control in hospitalized patients. METHODS: Using the Six-Step Approach of Kern to Curriculum Development for Medical Education, we developed and implemented an educational curriculum for inpatient glycemic management targeted to internal medicine residents and hospitalists. We surveyed physicians (n = 73) and conducted focus group sessions (n = 18 physicians) to solicit input regarding educational deficits and desired format of the educational intervention. Based on feedback from the surveys and focus groups, we developed educational goals and objectives and a case-based curriculum, which was delivered over a 1-year period via in-person teaching sessions by 2 experienced diabetes physicians at 3 hospitals. Rates of hypoglycemia and hyperglycemia were evaluated among at-risk patient days using an interrupted time-series design. RESULTS: We developed a mnemonic-based (SIGNAL) curriculum consisting of 10 modules, which covers key concepts of inpatient glycemic management and provides an approach to daily glycemic management: S = steroids, I = insulin, G = glucose, N = nutritional status, A = added dextrose, and L = labs. Following implementation of the curriculum, there was no difference in the rates of hyperglycemia in insulin-treated patients following the intervention; however, there was an increase in the rates of hypoglycemia defined as blood glucose (BG) ⩽ 70 mg/dL (5.6% vs 3.0%, P < .001) and clinically significant hypoglycemia defined as BG < 54 mg/dL (1.9% vs 0.8%, P = .01). There was poor penetration of the curriculum, with 60%, 20%, and 90% of the learning modules being delivered at the three participating hospitals, respectively. CONCLUSIONS: In this pilot study, a physician-targeted educational curriculum was not associated with improved glycemic control. Adapting the intervention to increase penetration and integrating the curriculum into existing clinical decision support tools may improve the effectiveness of the educational program on glycemic outcomes.