Prediction of ineffectiveness of biological drugs using machine learning and explainable AI methods: data from the Austrian Biological Registry BioReg.

OBJECTIVES: Machine learning models can support an individualized approach in the choice of bDMARDs. We developed prediction models for 5 different bDMARDs using machine learning methods based on patient data derived from the Austrian Biologics Registry (BioReg). METHODS: Data from 1397 patients and 19 variables with at least 100 treat-to-target (t2t) courses per drug were derived from the BioReg biologics registry. Different machine learning algorithms were trained to predict the risk of ineffectiveness for each bDMARD within the first 26 weeks. Cross-validation and hyperparameter optimization were applied to generate the best models. Model quality was assessed by area under the receiver operating characteristic (AUROC). Using explainable AI (XAI), risk-reducing and risk-increasing factors were extracted. RESULTS: The best models per drug achieved an AUROC score of the following: abatacept, 0.66 (95% CI, 0.54-0.78); adalimumab, 0.70 (95% CI, 0.68-0.74); certolizumab, 0.84 (95% CI, 0.79-0.89); etanercept, 0.68 (95% CI, 0.55-0.87); tocilizumab, 0.72 (95% CI, 0.69-0.77). The most risk-increasing variables were visual analytic scores (VAS) for abatacept and etanercept and co-therapy with glucocorticoids for adalimumab. Dosage was the most important variable for certolizumab and associated with a lower risk of non-response. Some variables, such as gender and rheumatoid factor (RF), showed opposite impacts depending on the bDMARD. CONCLUSION: Ineffectiveness of biological drugs could be predicted with promising accuracy. Interestingly, individual parameters were found to be associated with drug responses in different directions, indicating highly complex interactions. Machine learning can be of help in the decision-process by disentangling these relations.

An AI-Powered Clinical Decision Support System to Predict Flares in Rheumatoid Arthritis: A Pilot Study.

Treat-to-target (T2T) is a main therapeutic strategy in rheumatology; however, patients and rheumatologists currently have little support in making the best treatment decision. Clinical decision support systems (CDSSs) could offer this support. The aim of this study was to investigate the accuracy, effectiveness, usability, and acceptance of such a CDSS-Rheuma Care Manager (RCM)-including an artificial intelligence (AI)-powered flare risk prediction tool to support the management of rheumatoid arthritis (RA). Longitudinal clinical routine data of RA patients were used to develop and test the RCM. Based on ten real-world patient vignettes, five physicians were asked to assess patients' flare risk, provide a treatment decision, and assess their decision confidence without and with access to the RCM for predicting flare risk. RCM usability and acceptance were assessed using the system usability scale (SUS) and net promoter score (NPS). The flare prediction tool reached a sensitivity of 72%, a specificity of 76%, and an AUROC of 0.80. Perceived flare risk and treatment decisions varied largely between physicians. Having access to the flare risk prediction feature numerically increased decision confidence (3.5/5 to 3.7/5), reduced deviations between physicians and the prediction tool (20% to 12% for half dosage flare prediction), and resulted in more treatment reductions (42% to 50% vs. 20%). RCM usability (SUS) was rated as good (82/100) and was well accepted (mean NPS score 7/10). CDSS usage could support physicians by decreasing assessment deviations and increasing treatment decision confidence.

Prediction of Recurrent Ischemic Stroke Using Registry Data and Machine Learning Methods: The Erlangen Stroke Registry.

BACKGROUND: There have been multiple efforts toward individual prediction of recurrent strokes based on structured clinical and imaging data using machine learning algorithms. Some of these efforts resulted in relatively accurate prediction models. However, acquiring clinical and imaging data is typically possible at provider sites only and is associated with additional costs. Therefore, we developed recurrent stroke prediction models based solely on data easily obtained from the patient at home. METHODS: Data from 384 patients with ischemic stroke were obtained from the Erlangen Stroke Registry. Patients were followed at 3 and 12 months after first stroke and then annually, for about 2 years on average. Multiple machine learning algorithms were applied to train predictive models for estimating individual risk of recurrent stroke within 1 year. Double nested cross-validation was utilized for conservative performance estimation and models' learning capabilities were assessed by learning curves. Predicted probabilities were calibrated, and relative variable importance was assessed using explainable artificial intelligence techniques. RESULTS: The best model achieved the area under the curve of 0.70 (95% CI, 0.64-0.76) and relatively good probability calibration. The most predictive factors included patient's family and housing circumstances, rehabilitative measures, age, high calorie diet, systolic and diastolic blood pressures, percutaneous endoscopic gastrotomy, number of family doctor's home visits, and patient's mental state. CONCLUSIONS: Developing fairly accurate models for individual risk prediction of recurrent ischemic stroke within 1 year solely based on registry data is feasible. Such models could be applied in a home setting to provide an initial risk assessment and identify high-risk patients early.