Combining Federated Machine Learning and Qualitative Methods to Investigate Novel Pediatric Asthma Subtypes: Protocol for a Mixed Methods Study.

BACKGROUND: Pediatric asthma is a heterogeneous disease; however, current characterizations of its subtypes are limited. Machine learning (ML) methods are well-suited for identifying subtypes. In particular, deep neural networks can learn patient representations by leveraging longitudinal information captured in electronic health records (EHRs) while considering future outcomes. However, the traditional approach for subtype analysis requires large amounts of EHR data, which may contain protected health information causing potential concerns regarding patient privacy. Federated learning is the key technology to address privacy concerns while preserving the accuracy and performance of ML algorithms. Federated learning could enable multisite development and implementation of ML algorithms to facilitate the translation of artificial intelligence into clinical practice. OBJECTIVE: The aim of this study is to develop a research protocol for implementation of federated ML across a large clinical research network to identify and discover pediatric asthma subtypes and their progression over time. METHODS: This mixed methods study uses data and clinicians from the OneFlorida+ clinical research network, which is a large regional network covering linked and longitudinal patient-level real-world data (RWD) of over 20 million patients from Florida, Georgia, and Alabama in the United States. To characterize the subtypes, we will use OneFlorida+ data from 2011 to 2023 and develop a research-grade pediatric asthma computable phenotype and clinical natural language processing pipeline to identify pediatric patients with asthma aged 2-18 years. We will then apply federated learning to characterize pediatric asthma subtypes and their temporal progression. Using the Promoting Action on Research Implementation in Health Services framework, we will conduct focus groups with practicing pediatric asthma clinicians within the OneFlorida+ network to investigate the clinical utility of the subtypes. With a user-centered design, we will create prototypes to visualize the subtypes in the EHR to best assist with the clinical management of children with asthma. RESULTS: OneFlorida+ data from 2011 to 2023 have been collected for 411,628 patients aged 2-18 years along with 11,156,148 clinical notes. We expect to complete the computable phenotyping within the first year of the project, followed by subtyping during the second and third years, and then will perform the focus groups and establish the user-centered design in the fourth and fifth years of the project. CONCLUSIONS: Pediatric asthma subtypes incorporating RWD from diverse populations could improve patient outcomes by moving the field closer to precision pediatric asthma care. Our privacy-preserving federated learning methodology and qualitative implementation work will address several challenges of applying ML to large, multicenter RWD data. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/57981.

Development of a Computable Phenotype for Prehospital Pediatric Asthma Encounters.

INTRODUCTION: Asthma exacerbations are a common cause of pediatric Emergency Medical Services (EMS) encounters. Accordingly, prehospital management of pediatric asthma exacerbations has been designated an EMS research priority. However, accurate identification of pediatric asthma exacerbations from the prehospital record is nuanced and difficult due to the heterogeneity of asthma symptoms, especially in children. Therefore, this study's objective was to develop a prehospital-specific pediatric asthma computable phenotype (CP) that could accurately identify prehospital encounters for pediatric asthma exacerbations. METHODS: This is a retrospective observational study of patient encounters for ages 2-18 years from the ESO Data Collaborative between 2018 and 2021. We modified two existing rule-based pediatric asthma CPs and created three new CPs (one rule-based and two machine learning-based). Two pediatric emergency medicine physicians independently reviewed encounters to assign labels of asthma exacerbation or not. Taking that labeled encounter data, a 50/50 train/test split was used to create training and test sets from the labeled data. A 90/10 split was used to create a small validation set from the training set. We used specificity, sensitivity, positive predictive value (PPV), negative predictive value (NPV) and macro F1 to compare performance across all CP models. RESULTS: After applying the inclusion and exclusion criteria, 24,283 patient encounters remained. The machine-learning models exhibited the best performance for the identification of pediatric asthma exacerbations. A multi-layer perceptron-based model had the best performance in all metrics, with an F1 score of 0.95, specificity of 1.00, sensitivity of 0.91, negative predictive value of 0.98, and positive predictive value of 1.00. CONCLUSION: We modified existing and developed new pediatric asthma CPs to retrospectively identify prehospital pediatric asthma exacerbation encounters. We found that machine learning-based models greatly outperformed rule-based models. Given the high performance of the machine-learning models, the development and application of machine learning-based CPs for other conditions and diseases could help accelerate EMS research and ultimately enhance clinical care by accurately identifying patients with conditions of interest.

MRI-based radiomic features for identifying recurrent prostate cancer after proton radiation therapy.

PURPOSE: Magnetic Resonance Imaging (MRI) evaluation of recurrent prostate cancer (PCa) following proton beam therapy is challenging due to radiation-induced tissue changes. This study aimed to evaluate MRI-based radiomic features so as to identify the recurrent PCa after proton therapy. METHODS: We retrospectively studied 12 patients with biochemical recurrence (BCR) following proton therapy. Two experienced radiologists identified prostate lesions from multi-parametric MRI (mpMRI) images post-proton therapy and marked control regions of interest (ROIs) on the contralateral side of the prostate gland. A total of 210 radiomic features were extracted from lesions and control regions on the T2-weighted (T2WI) and Apparent Diffusion Coefficient (ADC) image series. Recursive Feature Elimination with Cross-Validation method (RFE-CV) was used for feature selection. A Multilayer Perceptron (MLP) neural network was developed to classify three classes: cancerous, benign, and healthy tissue. The 12-core biopsy results were used as the gold standard for the segmentations. The classifier performance was measured using specificity, sensitivity, the area under receiver operating characteristic curve (AUC), and other statistical indicators. RESULTS: Based on biopsy results, 10 lesions were identified as PCa recurrence while eight lesions were confirmed to be benign. Ten radiomic features (10/210) were selected to build the multi-class classifier. The radiomics classifier gave an accuracy of 0.83 in identifying cancerous, benign, and healthy tissue with a sensitivity of 0.80 and specificity of 0.85. The model yielded an AUC of 0.87, 95% CI [0.72-1.00] in differentiating cancer from the benign and healthy tissues. CONCLUSIONS: Our proof-of-concept study demonstrates the potential of using radiomic features as part of the differential diagnosis of PCa on mpMRI following proton therapy. The results need to be validated in a larger cohort.

Data science competition for cross-site individual tree species identification from airborne remote sensing data.

Data on individual tree crowns from remote sensing have the potential to advance forest ecology by providing information about forest composition and structure with a continuous spatial coverage over large spatial extents. Classifying individual trees to their taxonomic species over large regions from remote sensing data is challenging. Methods to classify individual species are often accurate for common species, but perform poorly for less common species and when applied to new sites. We ran a data science competition to help identify effective methods for the task of classification of individual crowns to species identity. The competition included data from three sites to assess each methods' ability to generalize patterns across two sites simultaneously and apply methods to an untrained site. Three different metrics were used to assess and compare model performance. Six teams participated, representing four countries and nine individuals. The highest performing method from a previous competition in 2017 was applied and used as a baseline to understand advancements and changes in successful methods. The best species classification method was based on a two-stage fully connected neural network that significantly outperformed the baseline random forest and gradient boosting ensemble methods. All methods generalized well by showing relatively strong performance on the trained sites (accuracy = 0.46-0.55, macro F1 = 0.09-0.32, cross entropy loss = 2.4-9.2), but generally failed to transfer effectively to the untrained site (accuracy = 0.07-0.32, macro F1 = 0.02-0.18, cross entropy loss = 2.8-16.3). Classification performance was influenced by the number of samples with species labels available for training, with most methods predicting common species at the training sites well (maximum F1 score of 0.86) relative to the uncommon species where none were predicted. Classification errors were most common between species in the same genus and different species that occur in the same habitat. Most methods performed better than the baseline in detecting if a species was not in the training data by predicting an untrained mixed-species class, especially in the untrained site. This work has highlighted that data science competitions can encourage advancement of methods, particularly by bringing in new people from outside the focal discipline, and by providing an open dataset and evaluation criteria from which participants can learn.