Performance assessment of different machine learning approaches in predicting diabetic ketoacidosis in adults with type 1 diabetes using electronic health records data.

PURPOSE: To assess the performance of different machine learning (ML) approaches in identifying risk factors for diabetic ketoacidosis (DKA) and predicting DKA. METHODS: This study applied flexible ML (XGBoost, distributed random forest [DRF] and feedforward network) and conventional ML approaches (logistic regression and least absolute shrinkage and selection operator [LASSO]) to 3400 DKA cases and 11 780 controls nested in adults with type 1 diabetes identified from Optum® de-identified Electronic Health Record dataset (2007-2018). Area under the curve (AUC), accuracy, sensitivity and specificity were computed using fivefold cross validation, and their 95% confidence intervals (CI) were established using 1000 bootstrap samples. The importance of predictors was compared across these models. RESULTS: In the training set, XGBoost and feedforward network yielded higher AUC values (0.89 and 0.86, respectively) than logistic regression (0.83), LASSO (0.83) and DRF (0.81). However, the AUC values were similar (0.82) among these approaches in the test set (95% CI range, 0.80-0.84). While the accuracy values >0.8 and the specificity values >0.9 for all models, the sensitivity values were only 0.4. The differences in these metrics across these models were minimal in the test set. All approaches selected some known risk factors for DKA as the top 10 features. XGBoost and DRF included more laboratory measurements or vital signs compared with conventional ML approaches, while feedforward network included more social demographics. CONCLUSIONS: In our empirical study, all ML approaches demonstrated similar performance, and identified overlapping, but different, top 10 predictors. The difference in selected top predictors needs further research.

Development and validation of a predictive model algorithm to identify anaphylaxis in adults with type 2 diabetes in U.S. administrative claims data.

PURPOSE: To use medical record adjudication and predictive modeling methods to develop and validate an algorithm to identify anaphylaxis among adults with type 2 diabetes (T2D) in administrative claims. METHODS: A conventional screening algorithm that prioritized sensitivity to identify potential anaphylaxis cases was developed and consisted of diagnosis codes for anaphylaxis or relevant signs and symptoms. This algorithm was applied to adults with T2D in the HealthCore Integrated Research Database (HIRD) from 2016 to 2018. Clinical experts adjudicated anaphylaxis case status from redacted medical records. We used confirmed case status as an outcome for predictive models developed using lasso regression with 10-fold cross-validation to identify predictors and estimate the probability of confirmed anaphylaxis. RESULTS: Clinical adjudicators reviewed medical records with sufficient information from 272 adults identified by the anaphylaxis screening algorithm, which had an estimated Positive Predictive Value (PPV) of 65% (95% confidence interval [CI]: 60%-71%). The predictive model algorithm had a c-statistic of 0.95. The model's probability threshold of 0.60 excluded 89% (84/94) of false positives identified by the screening algorithm, with a PPV of 94% (95% CI: 91%-98%). The model excluded very few true positives (15 of 178), and identified 92% (95% CI: 87%-96%) of the cases selected by the screening algorithm. CONCLUSIONS: Predictive modeling techniques yielded an accurate algorithm with high PPV and sensitivity for identifying anaphylaxis in administrative claims. This algorithm could be considered in future safety studies using similar claims data to reduce potential outcome misclassification.

Incidence of diabetic ketoacidosis and its trends in patients with type 1 diabetes mellitus identified using a U.S. claims database, 2007-2019.

Diabetic ketoacidosis (DKA) is a common complication of type 1 diabetes mellitus (T1DM). We found that the incidence of DKA was 55.5 per 1000 person-years in US commercially insured patients with T1DM; age-sex-standardized incidence decreased at an average annual rate of 6.1% in 2018-2019 after a steady increase since 2011.