Benchmarking AutoML frameworks for disease prediction using medical claims.

OBJECTIVES: Ascertain and compare the performances of Automated Machine Learning (AutoML) tools on large, highly imbalanced healthcare datasets. MATERIALS AND METHODS: We generated a large dataset using historical de-identified administrative claims including demographic information and flags for disease codes in four different time windows prior to 2019. We then trained three AutoML tools on this dataset to predict six different disease outcomes in 2019 and evaluated model performances on several metrics. RESULTS: The AutoML tools showed improvement from the baseline random forest model but did not differ significantly from each other. All models recorded low area under the precision-recall curve and failed to predict true positives while keeping the true negative rate high. Model performance was not directly related to prevalence. We provide a specific use-case to illustrate how to select a threshold that gives the best balance between true and false positive rates, as this is an important consideration in medical applications. DISCUSSION: Healthcare datasets present several challenges for AutoML tools, including large sample size, high imbalance, and limitations in the available features. Improvements in scalability, combinations of imbalance-learning resampling and ensemble approaches, and curated feature selection are possible next steps to achieve better performance. CONCLUSION: Among the three explored, no AutoML tool consistently outperforms the rest in terms of predictive performance. The performances of the models in this study suggest that there may be room for improvement in handling medical claims data. Finally, selection of the optimal prediction threshold should be guided by the specific practical application.

Real-world retrospective analysis of outcomes in patients undergoing bariatric surgery with class 1 obesity.

BACKGROUND: NIH-established indications for bariatric surgery were set close to 3 decades ago. OBJECTIVES: The purpose of this study was to evaluate outcomes in patients undergoing bariatric surgery with class I obesity, a class that does not fall into current indications. SETTING: University Hospital. METHODS: De-identified records from a clinic system's Electronic Health Record database were accessed to identify adult patients undergoing Roux-en-Y gastric bypass (RYGB) (n = 566) and sleeve gastrectomy (SG) (n = 730). Patients were compared in terms of resolution of co-morbidities and weight loss outcomes at 3 years following surgery. A mixed effects model was used, adjusting for the type of surgery, the number of quarters after the surgery when the averaged measurements were taken, and the interaction between these two variables. RESULTS: Patients lost up to 20% of their initial body mass index (BMI). Being of younger age, female, and having an obesity-related co-morbidity were associated with greater weight loss. At around 2 years after the surgery, the likelihood of being in remission from type 2 diabetes reached 45%. Remission probabilities for hypertension are 60% for RYGB and 50% for SG, 3 years after the surgery. On the other hand, the probabilities of remission from hyperlipidemia are close to 50% and 25% for RYGB and SG at 2 years. There was no difference between the BMI trajectories and remission from type 2 diabetes (T2D) when comparing the 2 groups. CONCLUSIONS: Bariatric surgery is effective in weight loss and resolution of comorbidities in patients with class I obesity. This data further supports the need to revisit the current indication criteria.

Retrospective comparative study of the effectiveness of bariatric surgery on 3-year outcomes in the real-world clinical setting.

BACKGROUND: Bariatric surgery has shown an improvement in obesity and obesity-related disease in many clinical trials and single center studies. However, real-world data, including data from non-centers of excellence, is sparse. OBJECTIVES: To provide clinical outcomes of patients who underwent bariatric surgery in real-world clinical setting. SETTING: Academic Institution. METHODS: Adults with obesity undergoing Roux-en-Y gastric bypass (RYGB), sleeve gastrectomy (SG), and a control group (CG) between 2007 and 2019 were identified. The CG represented patients with a previous visit to a bariatric surgeon without a subsequent surgery. Cohorts were matched on age, gender, ethnicity, baseline body mass index (BMI), and presence of diabetes and hypertension. Groups were compared in terms of co-morbidities, weight loss, and chronic conditions for three years. RESULTS: A total of 61 313 patients were identified. From these, 14 916 RYGB and 20 867 SG patients were matched to the CG (n = 16 562). The median BMI loss three years after surgery was 28.7% (interquartile range [IQR] 20.8%-36.2%) and 20.5% (IQR 13.5%-28.6%) for RYGB and SG groups, respectively. The CG had a median BMI loss of 6.7% with IQR of 20.4% decrease to 1.78% gain. At three years postoperatively, HbA1C decreased by 13% for RYGB and 5.9% for the SG group. The probabilities of remission from diabetes, hypertension, and low high-density lipoprotein cholesterol were significantly higher among patients who had surgery compared to the CG. For both RYGB and SG, the estimated probabilities of remission were similar. CONCLUSION: This study shows that bariatric surgery performed in the real-world clinical setting is an effective therapy for various expressions of the metabolic syndrome with results that are comparable to randomized control trials.

Machine learning does not improve upon traditional regression in predicting outcomes in atrial fibrillation: an analysis of the ORBIT-AF and GARFIELD-AF registries.

AIMS: Prediction models for outcomes in atrial fibrillation (AF) are used to guide treatment. While regression models have been the analytic standard for prediction modelling, machine learning (ML) has been promoted as a potentially superior methodology. We compared the performance of ML and regression models in predicting outcomes in AF patients. METHODS AND RESULTS: The Outcomes Registry for Better Informed Treatment of Atrial Fibrillation (ORBIT-AF) and Global Anticoagulant Registry in the FIELD (GARFIELD-AF) are population-based registries that include 74 792 AF patients. Models were generated from potential predictors using stepwise logistic regression (STEP), random forests (RF), gradient boosting (GB), and two neural networks (NNs). Discriminatory power was highest for death [STEP area under the curve (AUC) = 0.80 in ORBIT-AF, 0.75 in GARFIELD-AF] and lowest for stroke in all models (STEP AUC = 0.67 in ORBIT-AF, 0.66 in GARFIELD-AF). The discriminatory power of the ML models was similar or lower than the STEP models for most outcomes. The GB model had a higher AUC than STEP for death in GARFIELD-AF (0.76 vs. 0.75), but only nominally, and both performed similarly in ORBIT-AF. The multilayer NN had the lowest discriminatory power for all outcomes. The calibration of the STEP modelswere more aligned with the observed events for all outcomes. In the cross-registry models, the discriminatory power of the ML models was similar or lower than the STEP for most cases. CONCLUSION: When developed from two large, community-based AF registries, ML techniques did not improve prediction modelling of death, major bleeding, or stroke.