Towards a framework for interoperability and reproducibility of predictive models.

The development and deployment of machine learning (ML) models for biomedical research and healthcare currently lacks standard methodologies. Although tools for model replication are numerous, without a unifying blueprint it remains difficult to scientifically reproduce predictive ML models for any number of reasons (e.g., assumptions regarding data distributions and preprocessing, unclear test metrics, etc.) and ultimately, questions around generalizability and transportability are not readily answered. To facilitate scientific reproducibility, we built upon the Predictive Model Markup Language (PMML) to capture essential information. As a key component of the PREdictive Model Index and Exchange REpository (PREMIERE) platform, we present the Automated Metadata Pipeline (AMP) for conversion of a given predictive ML model into an extended PMML file that autocompletes an ML-based checklist, assessing model elements for interoperability and reproducibility. We demonstrate this pipeline on multiple test cases with three different ML algorithms and health-related datasets, providing a foundation for future predictive model reproducibility, sharing, and comparison.

Analgesic Techniques for Rib Fractures-A Comprehensive Review Article.

PURPOSE OF REVIEW: Rib fractures are a common traumatic injury that has been traditionally treated with systemic opioids and non-opioid analgesics. Due to the adverse effects of opioid analgesics, regional anesthesia techniques have become an increasingly promising alternative. This review article aims to explore the efficacy, safety, and constraints of medical management and regional anesthesia techniques in alleviating pain related to rib fractures. RECENT FINDINGS: Recently, opioid analgesia, thoracic epidural analgesia (TEA), and paravertebral block (PVB) have been favored options in the pain management of rib fractures. TEA has positive analgesic effects, and many studies vouch for its efficacy; however, it is contraindicated for many patients. PVB is a viable alternative to those with contraindications to TEA and exhibits promising outcomes compared to other regional anesthesia techniques; however, a failure rate of up to 10% and adverse complications challenge its administration in trauma settings. Serratus anterior plane blocks (SAPB) and erector spinae blocks (ESPB) serve as practical alternatives to TEA or PVB with lower incidences of adverse effects while exhibiting similar levels of analgesia. ESPB can be performed by trained emergency physicians, making it a feasible procedure to perform that is low-risk and efficient in pain management. Compared to the other techniques, intercostal nerve block (ICNB) had less analgesic impact and required concurrent intravenous medication to achieve comparable outcomes to the other blocks. The regional anesthesia techniques showed great success in improving pain scores and expediting recovery in many patients. However, choosing the optimal technique may not be so clear and will depend on the patient's case and the team's preferences. The peripheral nerve blocks have impressive potential in the future and may very well surpass neuraxial techniques; however, further research is needed to prove their efficacy and weaknesses.

AdaDiag: Adversarial Domain Adaptation of Diagnostic Prediction with Clinical Event Sequences.

Early detection of heart failure (HF) can provide patients with the opportunity for more timely intervention and better disease management, as well as efficient use of healthcare resources. Recent machine learning (ML) methods have shown promising performance on diagnostic prediction using temporal sequences from electronic health records (EHRs). In practice, however, these models may not generalize to other populations due to dataset shift. Shifts in datasets can be attributed to a range of factors such as variations in demographics, data management methods, and healthcare delivery patterns. In this paper, we use unsupervised adversarial domain adaptation methods to adaptively reduce the impact of dataset shift on cross-institutional transfer performance. The proposed framework is validated on a next-visit HF onset prediction task using a BERT-style Transformer-based language model pre-trained with a masked language modeling (MLM) task. Our model empirically demonstrates superior prediction performance relative to non-adversarial baselines in both transfer directions on two different clinical event sequence data sources.

Evaluation of an automated phenotyping algorithm for rheumatoid arthritis.

To better understand the challenges of generally implementing and adapting computational phenotyping approaches, the performance of a Phenotype KnowledgeBase (PheKB) algorithm for rheumatoid arthritis (RA) was evaluated on a University of California, Los Angeles (UCLA) patient population, focusing on examining its performance on ambiguous cases. The algorithm was evaluated on a cohort of 4,766 patients, along with a chart review of 300 patients by rheumatologists against accepted diagnostic guidelines. The performance revealed low sensitivity towards specific subtypes of positive RA cases, which suggests revisions in features used for phenotyping. A close examination of select cases also indicated a significant portion of patients with missing data, drawing attention to the need to consider data integrity as an integral part of phenotyping pipelines, as well as issues around the usability of various codes for distinguishing cases. We use patterns in the PheKB algorithm's errors to further demonstrate important considerations when designing a phenotyping algorithm.

Asthma clustering methods: a literature-informed application to the children's health study data.

OBJECTIVE: The heterogeneity of asthma has inspired widespread application of statistical clustering algorithms to a variety of datasets for identification of potentially clinically meaningful phenotypes. There has not been a standardized data analysis approach for asthma clustering, which can affect reproducibility and clinical translation of results. Our objective was to identify common and effective data analysis practices in the asthma clustering literature and apply them to data from a Southern California population-based cohort of schoolchildren with asthma. METHODS: As of January 1, 2020, we reviewed key statistical elements of 77 asthma clustering studies. Guided by the literature, we used 12 input variables and three clustering methods (hierarchical clustering, k-medoids, and latent class analysis) to identify clusters in 598 schoolchildren with asthma from the Southern California Children's Health Study (CHS). RESULTS: Clusters of children identified by latent class analysis were characterized by exhaled nitric oxide, FEV1/FVC, FEV1 percent predicted, asthma control and allergy score; and were predictive of control at two year follow up. Clusters from the other two methods were less clinically remarkable, primarily differentiated by sex and race/ethnicity and less predictive of asthma control over time. CONCLUSION: Upon review of the asthma phenotyping literature, common approaches of data clustering emerged. When applying these elements to the Children's Health Study data, latent class analysis clusters-represented by exhaled nitric oxide and spirometry measures-had clinical relevance over time.

Applying Automated Machine Learning to Predict Mode of Delivery Using Ongoing Intrapartum Data in Laboring Patients.

OBJECTIVE: This study aimed to develop and validate a machine learning (ML) model to predict the probability of a vaginal delivery (Partometer) using data iteratively obtained during labor from the electronic health record. STUDY DESIGN: A retrospective cohort study of deliveries at an academic, tertiary care hospital was conducted from 2013 to 2019 who had at least two cervical examinations. The population was divided into those delivered by physicians with nulliparous term singleton vertex (NTSV) cesarean delivery rates <23.9% (Partometer cohort) and the remainder (control cohort). The cesarean rate among this population of lower risk patients is a standard metric by which to compare provider rates; <23.9% was the Healthy People 2020 goal. A supervised automated ML approach was applied to generate a model for each population. The primary outcome was accuracy of the model developed on the Partometer cohort at 4 hours from admission to labor and delivery. Secondary outcomes included discrimination ability (receiver operating characteristics-area under the curve [ROC-AUC]), precision-recall AUC, and calibration of the Partometer. To assess generalizability, we compared the performance and clinical predictors identified by the Partometer to the control model. RESULTS: There were 37,932 deliveries during the study period; after exclusions, 9,385 deliveries were included in the Partometer cohort and 19,683 in the control cohort. Accuracy of predicting vaginal delivery at 4 hours was 87.1% for the Partometer (ROC-AUC: 0.82). Clinical predictors of greatest importance in the stacked Intrapartum Partometer Model included the Admission Model prediction and ongoing measures of dilatation and station which mirrored those found in the control population. CONCLUSION: Using automated ML and intrapartum factors improved the accuracy of prediction of probability of a vaginal delivery over both previously published models based on logistic regression. Harnessing real-time data and ML could represent the bridge to generating a truly prescriptive tool to augment clinical decision-making, predict labor outcomes, and reduce maternal and neonatal morbidity. KEY POINTS: · Our ML-based model yielded accurate predictions of mode of delivery early in labor.. · Predictors for models created on populations with high and low cesarean rates were the same.. · A ML-based model may provide meaningful guidance to clinicians managing labor..

Long term surgical outcomes for infective endocarditis in people who inject drugs: a systematic review and meta-analysis.

BACKGROUND: In recent years, the number of infective endocarditis (IE) cases associated with injection drug use has increased. Clinical guidelines suggest deferring surgery for IE in people who inject drugs (PWID) due to a concern for worse outcomes in comparison to non-injectors (non-PWID). We performed a systematic review and meta-analysis of long-term outcomes in PWID who underwent cardiac surgery and compared these outcomes to non-PWID. METHODS: We systematically searched for studies reported between 1965 and 2018. We used an algorithm to estimate individual patient data (eIPD) from Kaplan-Meier (KM) curves and combined it with published individual patient data (IPD) to analyze long-term outcomes after cardiac surgery for IE in PWID. Our primary outcome was survival. Secondary outcomes were reoperation and mortality at 30-days, one-, five-, and 10-years. Random effects Cox regression was used for estimating survival. RESULTS: We included 27 studies in the systematic review and 19 provided data (KM or IPD) for the meta-analysis. PWID were younger and more likely to have S. aureus than non-PWID. Survival at 30-days, one-, five-, and 10-years was 94.3, 81.0, 62.1, and 56.6% in PWID, respectively; and 96.4, 85.0, 70.3, and 63.4% in non-PWID. PWID had 47% greater hazard of death (HR 1.47, 95% CI, 1.05-2.05) and more than twice the hazard of reoperation (HR 2.37, 95% CI, 1.25-4.50) than non-PWID. CONCLUSION: PWID had shorter survival that non-PWID. Implementing evidence-based interventions and testing new modalities are urgently needed to improve outcomes in PWID after cardiac surgery.

Rationale and design of a multicenter Chronic Kidney Disease (CKD) and at-risk for CKD electronic health records-based registry: CURE-CKD.

BACKGROUND: Chronic kidney disease (CKD) is a global public health problem, exhibiting sharp increases in incidence, prevalence, and attributable morbidity and mortality. There is a critical need to better understand the demographics, clinical characteristics, and key risk factors for CKD; and to develop platforms for testing novel interventions to improve modifiable risk factors, particularly for the CKD patients with a rapid decline in kidney function. METHODS: We describe a novel collaboration between two large healthcare systems (Providence St. Joseph Health and University of California, Los Angeles Health) supported by leadership from both institutions, which was created to develop harmonized cohorts of patients with CKD or those at increased risk for CKD (hypertension/HTN, diabetes/DM, pre-diabetes) from electronic health record data. RESULTS: The combined repository of candidate records included more than 3.3 million patients with at least a single qualifying measure for CKD and/or at-risk for CKD. The CURE-CKD registry includes over 2.6 million patients with and/or at-risk for CKD identified by stricter guide-line based criteria using a combination of administrative encounter codes, physical examinations, laboratory values and medication use. Notably, data based on race/ethnicity and geography in part, will enable robust analyses to study traditionally disadvantaged or marginalized patients not typically included in clinical trials. DISCUSSION: CURE-CKD project is a unique multidisciplinary collaboration between nephrologists, endocrinologists, primary care physicians with health services research skills, health economists, and those with expertise in statistics, bio-informatics and machine learning. The CURE-CKD registry uses curated observations from real-world settings across two large healthcare systems and has great potential to provide important contributions for healthcare and for improving clinical outcomes in patients with and at-risk for CKD.

An Interpretable Deep Hierarchical Semantic Convolutional Neural Network for Lung Nodule Malignancy Classification.

While deep learning methods have demonstrated performance comparable to human readers in tasks such as computer-aided diagnosis, these models are difficult to interpret, do not incorporate prior domain knowledge, and are often considered as a "black-box." The lack of model interpretability hinders them from being fully understood by end users such as radiologists. In this paper, we present a novel interpretable deep hierarchical semantic convolutional neural network (HSCNN) to predict whether a given pulmonary nodule observed on a computed tomography (CT) scan is malignant. Our network provides two levels of output: 1) low-level semantic features; and 2) a high-level prediction of nodule malignancy. The low-level outputs reflect diagnostic features often reported by radiologists and serve to explain how the model interprets the images in an expert-interpretable manner. The information from these low-level outputs, along with the representations learned by the convolutional layers, are then combined and used to infer the high-level output. This unified architecture is trained by optimizing a global loss function including both low- and high-level tasks, thereby learning all the parameters within a joint framework. Our experimental results using the Lung Image Database Consortium (LIDC) show that the proposed method not only produces interpretable lung cancer predictions but also achieves significantly better results compared to using a 3D CNN alone.

Envisioning the future of 'big data' biomedicine.

Through the increasing availability of more efficient data collection procedures, biomedical scientists are now confronting ever larger sets of data, often finding themselves struggling to process and interpret what they have gathered. This, while still more data continues to accumulate. This torrent of biomedical information necessitates creative thinking about how the data are being generated, how they might be best managed, analyzed, and eventually how they can be transformed into further scientific understanding for improving patient care. Recognizing this as a major challenge, the National Institutes of Health (NIH) has spearheaded the "Big Data to Knowledge" (BD2K) program - the agency's most ambitious biomedical informatics effort ever undertaken to date. In this commentary, we describe how the NIH has taken on "big data" science head-on, how a consortium of leading research centers are developing the means for handling large-scale data, and how such activities are being marshalled for the training of a new generation of biomedical data scientists. All in all, the NIH BD2K program seeks to position data science at the heart of 21st Century biomedical research.

Developing a framework for digital objects in the Big Data to Knowledge (BD2K) commons: Report from the Commons Framework Pilots workshop.

The volume and diversity of data in biomedical research have been rapidly increasing in recent years. While such data hold significant promise for accelerating discovery, their use entails many challenges including: the need for adequate computational infrastructure, secure processes for data sharing and access, tools that allow researchers to find and integrate diverse datasets, and standardized methods of analysis. These are just some elements of a complex ecosystem that needs to be built to support the rapid accumulation of these data. The NIH Big Data to Knowledge (BD2K) initiative aims to facilitate digitally enabled biomedical research. Within the BD2K framework, the Commons initiative is intended to establish a virtual environment that will facilitate the use, interoperability, and discoverability of shared digital objects used for research. The BD2K Commons Framework Pilots Working Group (CFPWG) was established to clarify goals and work on pilot projects that address existing gaps toward realizing the vision of the BD2K Commons. This report reviews highlights from a two-day meeting involving the BD2K CFPWG to provide insights on trends and considerations in advancing Big Data science for biomedical research in the United States.

Feasibility of a Secure Wireless Sensing Smartwatch Application for the Self-Management of Pediatric Asthma.

To address the need for asthma self-management in pediatrics, the authors present the feasibility of a mobile health (mHealth) platform built on their prior work in an asthmatic adult and child. Real-time asthma attack risk was assessed through physiological and environmental sensors. Data were sent to a cloud via a smartwatch application (app) using Health Insurance Portability and Accountability Act (HIPAA)-compliant cryptography and combined with online source data. A risk level (high, medium or low) was determined using a random forest classifier and then sent to the app to be visualized as animated dragon graphics for easy interpretation by children. The feasibility of the system was first tested on an adult with moderate asthma, then usability was examined on a child with mild asthma over several weeks. It was found during feasibility testing that the system is able to assess asthma risk with 80.10 ± 14.13% accuracy. During usability testing, it was able to continuously collect sensor data, and the child was able to wear, easily understand and enjoy the use of the system. If tested in more individuals, this system may lead to an effective self-management program that can reduce hospitalization in those who suffer from asthma.

Improving Biomedical Signal Search Results in Big Data Case-Based Reasoning Environments.

Time series subsequence matching has importance in a variety of areas in healthcare informatics. These include case-based diagnosis and treatment as well as discovery of trends among patients. However, few medical systems employ subsequence matching due to high computational and memory complexities. This manuscript proposes a randomized Monte Carlo sampling method to broaden search criteria with minimal increases in computational and memory complexities over R-NN indexing. Information gain improves while producing result sets that approximate the theoretical result space, query results increase by several orders of magnitude, and recall is improved with no signi cant degradation to precision over R-NN matching.

Remote Ischemic Conditioning Alters Methylation and Expression of Cell Cycle Genes in Aneurysmal Subarachnoid Hemorrhage.

BACKGROUND AND PURPOSE: Remote ischemic conditioning (RIC) is a phenomenon in which short periods of nonfatal ischemia in 1 tissue confers protection to distant tissues. Here we performed a longitudinal human pilot study in patients with aneurysmal subarachnoid hemorrhage undergoing RIC by limb ischemia to compare changes in DNA methylation and transcriptome profiles before and after RIC. METHODS: Thirteen patients underwent 4 RIC sessions over 2 to 12 days after rupture of an intracranial aneurysm. We analyzed whole blood transcriptomes using RNA sequencing and genome-wide DNA methylomes using reduced representation bisulfite sequencing, both before and after RIC. We tested differential expression and differential methylation using an intraindividual paired study design and then overlapped the differential expression and differential methylation results for analyses of functional categories and protein-protein interactions. RESULTS: We observed 164 differential expression genes and 3493 differential methylation CpG sites after RIC, of which 204 CpG sites overlapped with 103 genes, enriched for pathways of cell cycle (P<3.8×10(-4)) and inflammatory responses (P<1.4×10(-4)). The cell cycle pathway genes form a significant protein-protein interaction network of tightly coexpressed genes (P<0.00001). CONCLUSIONS: Gene expression and DNA methylation changes in aneurysmal subarachnoid hemorrhage patients undergoing RIC are involved in coordinated cell cycle and inflammatory responses.

An integrated, ontology-driven approach to constructing observational databases for research.

The electronic health record (EHR) contains a diverse set of clinical observations that are captured as part of routine care, but the incomplete, inconsistent, and sometimes incorrect nature of clinical data poses significant impediments for its secondary use in retrospective studies or comparative effectiveness research. In this work, we describe an ontology-driven approach for extracting and analyzing data from the patient record in a longitudinal and continuous manner. We demonstrate how the ontology helps enforce consistent data representation, integrates phenotypes generated through analyses of available clinical data sources, and facilitates subsequent studies to identify clinical predictors for an outcome of interest. Development and evaluation of our approach are described in the context of studying factors that influence intracranial aneurysm (ICA) growth and rupture. We report our experiences in capturing information on 78 individuals with a total of 120 aneurysms. Two example applications related to assessing the relationship between aneurysm size, growth, gene expression modules, and rupture are described. Our work highlights the challenges with respect to data quality, workflow, and analysis of data and its implications toward a learning health system paradigm.

Early prediction of end-stage kidney disease using electronic health record data: a machine learning approach with a 2-year horizon.

Objectives: In the United States, end-stage kidney disease (ESKD) is responsible for high mortality and significant healthcare costs, with the number of cases sharply increasing in the past 2 decades. In this study, we aimed to reduce these impacts by developing an ESKD model for predicting its occurrence in a 2-year period. Materials and Methods: We developed a machine learning (ML) pipeline to test different models for the prediction of ESKD. The electronic health record was used to capture several kidney disease-related variables. Various imputation methods, feature selection, and sampling approaches were tested. We compared the performance of multiple ML models using area under the ROC curve (AUCROC), area under the Precision-Recall curve (PR-AUC), and Brier scores for discrimination, precision, and calibration, respectively. Explainability methods were applied to the final model. Results: Our best model was a gradient-boosting machine with feature selection and imputation methods as additional components. The model exhibited an AUCROC of 0.97, a PR-AUC of 0.33, and a Brier score of 0.002 on a holdout test set. A chart review analysis by expert physicians indicated clinical utility. Discussion and Conclusion: An ESKD prediction model can identify individuals at risk for ESKD and has been successfully deployed within our health system.

Data-driven prediction of continuous renal replacement therapy survival.

Continuous renal replacement therapy (CRRT) is a form of dialysis prescribed to severely ill patients who cannot tolerate regular hemodialysis. However, as the patients are typically very ill to begin with, there is always uncertainty whether they will survive during or after CRRT treatment. Because of outcome uncertainty, a large percentage of patients treated with CRRT do not survive, utilizing scarce resources and raising false hope in patients and their families. To address these issues, we present a machine learning-based algorithm to predict short-term survival in patients being initiated on CRRT. We use information extracted from electronic health records from patients who were placed on CRRT at multiple institutions to train a model that predicts CRRT survival outcome; on a held-out test set, the model achieves an area under the receiver operating curve of 0.848 (CI = 0.822-0.870). Feature importance, error, and subgroup analyses provide insight into bias and relevant features for model prediction. Overall, we demonstrate the potential for predictive machine learning models to assist clinicians in alleviating the uncertainty of CRRT patient survival outcomes, with opportunities for future improvement through further data collection and advanced modeling.

Impact of Potential Case Misclassification by Administrative Diagnostic Codes on Outcome Assessment of Observational Study for People Who Inject Drugs.

Introduction: Initiation of medications for opioid use disorder (MOUD) within the hospital setting may improve outcomes for people who inject drugs (PWID) hospitalized because of an infection. Many studies used International Classification of Diseases (ICD) codes to identify PWID, although these may be misclassified and thus, inaccurate. We hypothesized that bias from misclassification of PWID using ICD codes may impact analyses of MOUD outcomes. Methods: We analyzed a cohort of 36 868 cases of patients diagnosed with Staphylococcus aureus bacteremia at 124 US Veterans Health Administration hospitals between 2003 and 2014. To identify PWID, we implemented an ICD code-based algorithm and a natural language processing (NLP) algorithm for classification of admission notes. We analyzed outcomes of prescribing MOUD as an inpatient using both approaches. Our primary outcome was 365-day all-cause mortality. We fit mixed-effects Cox regression models with receipt or not of MOUD during the index hospitalization as the primary predictor and 365-day mortality as the outcome. Results: NLP identified 2389 cases as PWID, whereas ICD codes identified 6804 cases as PWID. In the cohort identified by NLP, receipt of inpatient MOUD was associated with a protective effect on 365-day survival (adjusted hazard ratio, 0.48; 95% confidence interval, .29-.81; P < .01) compared with those not receiving MOUD. There was no significant effect of MOUD receipt in the cohort identified by ICD codes (adjusted hazard ratio, 1.00; 95% confidence interval, .77-1.30; P = .99). Conclusions: MOUD was protective of all-cause mortality when NLP was used to identify PWID, but not significant when ICD codes were used to identify the analytic subjects.