Physician preference for receiving machine learning predictive results: A cross-sectional multicentric study.

Artificial intelligence (AI) algorithms are transforming several areas of the digital world and are increasingly being applied in healthcare. Mobile apps based on predictive machine learning models have the potential to improve health outcomes, but there is still no consensus on how to inform doctors about their results. The aim of this study was to investigate how healthcare professionals prefer to receive predictions generated by machine learning algorithms. A systematic search in MEDLINE, via PubMed, EMBASE and Web of Science was first performed. We developed a mobile app, RandomIA, to predict the occurrence of clinical outcomes, initially for COVID-19 and later expected to be expanded to other diseases. A questionnaire called System Usability Scale (SUS) was selected to assess the usability of the mobile app. A total of 69 doctors from the five regions of Brazil tested RandomIA and evaluated three different ways to visualize the predictions. For prognostic outcomes (mechanical ventilation, admission to an intensive care unit, and death), most doctors (62.9%) preferred a more complex visualization, represented by a bar graph with three categories (low, medium, and high probability) and a probability density graph for each outcome. For the diagnostic prediction of COVID-19, there was also a majority preference (65.4%) for the same option. Our results indicate that doctors could be more inclined to prefer receiving detailed results from predictive machine learning algorithms.

Early identification of older individuals at risk of mobility decline with machine learning.

BACKGROUND: The early identification of individuals at risk of mobility decline can improve targeted strategies of prevention. AIMS: To evaluate the predictive performance of machine learning (ML) algorithms in identifying older individuals at risk of future mobility decline. METHODS: We used data from the SABE Study (Health, Well-being and Aging Study), a representative sample of people aged 60 years and more, living in the Municipality of São Paulo, Brazil. Mobility decline was assessed 5 years after admission in the study by self-reported difficulty to walk a block, climb steps, being able to stoop, crouch and kneel, or lifting or carrying weights greater than 5 kg. Popular machine learning algorithms were trained in 70% of the sample with 10-fold cross-validation, and predictive performance metrics were obtained from applying the trained algorithms to the other 30% (test set). RESULTS: Of the 1,615 individuals, 48% developed difficulty in at least one of the four tasks, 32% in stooping, crouching and kneeling, and 30% in carrying weights. The random forest algorithm had the best predictive performance for most outcomes. The tasks that the algorithm was able to predict with better performance were crouching and kneeling (AUC-ROC: 0.81[0.76-0.85]), and lifting or carrying weights (AUC-ROC: 0.80[0.75-0.84]). Age was the most important variable for the algorithms, followed by education and back pain, according to the SHAP (SHapley Additive exPlanations) values. CONCLUSION: Applications of ML algorithms are a promising tool to identify older patients at risk of mobility decline, with the potential of improving targeted preventive programs.

A multipurpose machine learning approach to predict COVID-19 negative prognosis in São Paulo, Brazil.

The new coronavirus disease (COVID-19) is a challenge for clinical decision-making and the effective allocation of healthcare resources. An accurate prognostic assessment is necessary to improve survival of patients, especially in developing countries. This study proposes to predict the risk of developing critical conditions in COVID-19 patients by training multipurpose algorithms. We followed a total of 1040 patients with a positive RT-PCR diagnosis for COVID-19 from a large hospital from São Paulo, Brazil, from March to June 2020, of which 288 (28%) presented a severe prognosis, i.e. Intensive Care Unit (ICU) admission, use of mechanical ventilation or death. We used routinely-collected laboratory, clinical and demographic data to train five machine learning algorithms (artificial neural networks, extra trees, random forests, catboost, and extreme gradient boosting). We used a random sample of 70% of patients to train the algorithms and 30% were left for performance assessment, simulating new unseen data. In order to assess if the algorithms could capture general severe prognostic patterns, each model was trained by combining two out of three outcomes to predict the other. All algorithms presented very high predictive performance (average AUROC of 0.92, sensitivity of 0.92, and specificity of 0.82). The three most important variables for the multipurpose algorithms were ratio of lymphocyte per C-reactive protein, C-reactive protein and Braden Scale. The results highlight the possibility that machine learning algorithms are able to predict unspecific negative COVID-19 outcomes from routinely-collected data.