Predictability of buprenorphine-naloxone treatment retention: A multi-site analysis combining electronic health records and machine learning.

BACKGROUND AND AIMS: Opioid use disorder (OUD) and opioid dependence lead to significant morbidity and mortality, yet treatment retention, crucial for the effectiveness of medications like buprenorphine-naloxone, remains unpredictable. Our objective was to determine the predictability of 6-month retention in buprenorphine-naloxone treatment using electronic health record (EHR) data from diverse clinical settings and to identify key predictors. DESIGN: This retrospective observational study developed and validated machine learning-based clinical risk prediction models using EHR data. SETTING AND CASES: Data were sourced from Stanford University's healthcare system and Holmusk's NeuroBlu database, reflecting a wide range of healthcare settings. The study analyzed 1800 Stanford and 7957 NeuroBlu treatment encounters from 2008 to 2023 and from 2003 to 2023, respectively. MEASUREMENTS: Predict continuous prescription of buprenorphine-naloxone for at least 6 months, without a gap of more than 30 days. The performance of machine learning prediction models was assessed by area under receiver operating characteristic (ROC-AUC) analysis as well as precision, recall and calibration. To further validate our approach's clinical applicability, we conducted two secondary analyses: a time-to-event analysis on a single site to estimate the duration of buprenorphine-naloxone treatment continuity evaluated by the C-index and a comparative evaluation against predictions made by three human clinical experts. FINDINGS: Attrition rates at 6 months were 58% (NeuroBlu) and 61% (Stanford). Prediction models trained and internally validated on NeuroBlu data achieved ROC-AUCs up to 75.8 (95% confidence interval [CI] = 73.6-78.0). Addiction medicine specialists' predictions show a ROC-AUC of 67.8 (95% CI = 50.4-85.2). Time-to-event analysis on Stanford data indicated a median treatment retention time of 65 days, with random survival forest model achieving an average C-index of 65.9. The top predictor of treatment retention identified included the diagnosis of opioid dependence. CONCLUSIONS: US patients with opioid use disorder or opioid dependence treated with buprenorphine-naloxone prescriptions appear to have a high (∼60%) treatment attrition by 6 months. Machine learning models trained on diverse electronic health record datasets appear to be able to predict treatment continuity with accuracy comparable to that of clinical experts.

Leveraging Artificial Intelligence to Predict Health Belief Model and COVID-19 Vaccine Uptake Using Survey Text from US Nurses.

We investigated how artificial intelligence (AI) reveals factors shaping COVID-19 vaccine hesitancy among healthcare providers by examining their open-text comments. We conducted a longitudinal survey starting in Spring of 2020 with 38,788 current and former female nurses in three national cohorts to assess how the pandemic has affected their livelihood. In January and March-April 2021 surveys, participants were invited to contribute open-text comments and answer specific questions about COVID-19 vaccine uptake. A closed-ended question in the survey identified vaccine-hesitant (VH) participants who either had no intention or were unsure of receiving a COVID-19 vaccine. We collected 1970 comments from VH participants and trained two machine learning (ML) algorithms to identify behavioral factors related to VH. The first predictive model classified each comment into one of three health belief model (HBM) constructs (barriers, severity, and susceptibility) related to adopting disease prevention activities. The second predictive model used the words in January comments to predict the vaccine status of VH in March-April 2021; vaccine status was correctly predicted 89% of the time. Our results showed that 35% of VH participants cited barriers, 17% severity, and 7% susceptibility to receiving a COVID-19 vaccine. Out of the HBM constructs, the VH participants citing a barrier, such as allergic reactions and side effects, had the most associated change in vaccine status from VH to later receiving a vaccine.

Session Introduction: Artificial Intelligence in Clinical Medicine: Generative and Interactive Systems at the Human-Machine Interface.

Artificial Intelligence (AI) models are substantially enhancing the capability to analyze complex and multi-dimensional datasets. Generative AI and deep learning models have demonstrated significant advancements in extracting knowledge from unstructured text, imaging as well as structured and tabular data. This recent breakthrough in AI has inspired research in medicine, leading to the development of numerous tools for creating clinical decision support systems, monitoring tools, image interpretation, and triaging capabilities. Nevertheless, comprehensive research is imperative to evaluate the potential impact and implications of AI systems in healthcare. At the 2024 Pacific Symposium on Biocomputing (PSB) session entitled "Artificial Intelligence in Clinical Medicine: Generative and Interactive Systems at the Human-Machine Interface", we spotlight research that develops and applies AI algorithms to solve real-world problems in healthcare.

High-throughput target trial emulation for Alzheimer's disease drug repurposing with real-world data.

Target trial emulation is the process of mimicking target randomized trials using real-world data, where effective confounding control for unbiased treatment effect estimation remains a main challenge. Although various approaches have been proposed for this challenge, a systematic evaluation is still lacking. Here we emulated trials for thousands of medications from two large-scale real-world data warehouses, covering over 10 years of clinical records for over 170 million patients, aiming to identify new indications of approved drugs for Alzheimer's disease. We assessed different propensity score models under the inverse probability of treatment weighting framework and suggested a model selection strategy for improved baseline covariate balancing. We also found that the deep learning-based propensity score model did not necessarily outperform logistic regression-based methods in covariate balancing. Finally, we highlighted five top-ranked drugs (pantoprazole, gabapentin, atorvastatin, fluticasone, and omeprazole) originally intended for other indications with potential benefits for Alzheimer's patients.

Mild Cognitive Impairment: Data-Driven Prediction, Risk Factors, and Workup.

Over 78 million people will suffer from dementia by 2030, emphasizing the need for early identification of patients with mild cognitive impairment (MCI) at risk, and personalized clinical evaluation steps to diagnose potentially reversible causes. Here, we leverage real-world electronic health records in the observational medical outcomes partnership (OMOP) data model to develop machine learning models to predict MCI up to a year in advance of recorded diagnosis. Our experimental results with logistic regression, random forest, and xgboost models trained and evaluated on more than 531K patient visits show random forest model can predict MCI onset with ROC-AUC of 68.2±0.7. We identify the clinical factors mentioned in clinician notes that are most predictive of MCI. Using similar association mining techniques, we develop a data-driven list of clinical procedures commonly ordered in the workup of MCI cases, that could be used as a basis for guidelines and clinical order set templates.

Graph-based clinical recommender: Predicting specialists procedure orders using graph representation learning.

OBJECTIVE: To determine whether graph neural network based models of electronic health records can predict specialty consultation care needs for endocrinology and hematology more accurately than the standard of care checklists and other conventional medical recommendation algorithms in the literature. METHODS: Demand for medical expertise far outstrips supply, with tens of millions in the US alone with deficient access to specialty care. Rather than potentially months long delays to initiate diagnostic workup and medical treatment with a specialist, referring primary care supported by an automated recommender algorithm could anticipate and directly initiate patient evaluation that would otherwise be needed at subsequent a specialist appointment. We propose a novel graph representation learning approach with a heterogeneous graph neural network to model structured electronic health records and formulate recommendation/prediction of subsequent specialist orders as a link prediction problem. RESULTS: Models are trained and assessed in two specialty care sites: endocrinology and hematology. Our experimental results show that our model achieves an 8% improvement in ROC-AUC for endocrinology (ROC-AUC = 0.88) and 5% improvement for hematology (ROC-AUC = 0.84) personalized procedure recommendations over prior medical recommender systems. These recommender algorithm approaches provide medical procedure recommendations for endocrinology referrals more effectively than manual clinical checklists (recommender: precision = 0.60, recall = 0.27, F1-score = 0.37) vs. (checklist: precision = 0.16, recall = 0.28, F1-score = 0.20), and similarly for hematology referrals (recommender: precision = 0.44, recall = 0.38, F1-score = 0.41) vs. (checklist: precision = 0.27, recall = 0.71, F1-score = 0.39). CONCLUSION: Embedding graph neural network models into clinical care can improve digital specialty consultation systems and expand the access to medical experience of prior similar cases.

A Comparative Effectiveness Study on Opioid Use Disorder Prediction Using Artificial Intelligence and Existing Risk Models.

Opioid use disorder (OUD) is a leading cause of death in the United States placing a tremendous burden on patients, their families, and health care systems. Artificial intelligence (AI) can be harnessed with available healthcare data to produce automated OUD prediction tools. In this retrospective study, we developed AI based models for OUD prediction and showed that AI can predict OUD more effectively than existing clinical tools including the unweighted opioid risk tool (ORT). Data include 474,208 patients' data over 10 years; 269,748 were females with an average age of 56.78 years. Cases are prescription opioid users with at least one diagnosis of OUD or at least one prescription for buprenorphine or methadone. Controls are prescription opioid users with no OUD diagnoses or buprenorphine or methadone prescriptions. On 100 randomly selected test sets including 47,396 patients, our proposed transformer-based AI model can predict OUD more efficiently (AUC = 0.742 ± 0.021) compared to logistic regression (AUC = 0.651 ± 0.025), random forest (AUC = 0.679 ± 0.026), xgboost (AUC = 0.690 ± 0.027), long short-term memory model (AUC = 0.706 ± 0.026), transformer (AUC = 0.725 ± 0.024), and unweighted ORT model (AUC = 0.559 ± 0.025). Our results show that embedding AI algorithms into clinical care may assist clinicians in risk stratification and management of patients receiving opioid therapy.

Predicting premature discontinuation of medication for opioid use disorder from electronic medical records.

Medications such as buprenorphine-naloxone are among the most effective treatments for opioid use disorder, but limited retention in treatment limits long-term outcomes. In this study, we assess the feasibility of a machine learning model to predict retention vs. attrition in medication for opioid use disorder (MOUD) treatment using electronic medical record data including concepts extracted from clinical notes. A logistic regression classifier was trained on 374 MOUD treatments with 68% resulting in potential attrition. On a held-out test set of 157 events, the full model achieved an area under the receiver operating characteristic curve (AUROC) of 0.77 (95% CI: 0.64-0.90) and AUROC of 0.74 (95% CI: 0.62-0.87) with a limited model using only structured EMR data. Risk prediction for opioid MOUD retention vs. attrition is feasible given electronic medical record data, even without necessarily incorporating concepts extracted from clinical notes.

T cells dominate peripheral inflammation in a cross-sectional analysis of obesity-associated diabetes.

OBJECTIVE: Myeloid cells dominate metabolic disease-associated inflammation (metaflammation) in mouse obesity, but the contributions of myeloid cells to the peripheral inflammation that fuels sequelae of human obesity are untested. This study used unbiased approaches to rank contributions of myeloid and T cells to peripheral inflammation in people with obesity across the spectrum of metabolic health. METHODS: Peripheral blood mononuclear cells (PBMCs) from people with obesity with or without prediabetes or type 2 diabetes were stimulated with T cell-targeting CD3/CD28 or myeloid-targeting lipopolysaccharide for 20 to 72 hours to assess cytokine production using Bio-Plex. Bioinformatic modeling ranked cytokines with respect to their predictive power for metabolic health. Intracellular tumor necrosis factor α was quantitated as a classical indicator of metaflammation. RESULTS: Cytokines increased over 72 hours following T cell-, but not myeloid-, targeted stimulation to indicate that acute myeloid inflammation may shift to T cell inflammation over time. T cells contributed more tumor necrosis factor α to peripheral inflammation regardless of metabolic status. Bioinformatic combination of cytokines from all cohorts, stimuli, and time points indicated that T cell-targeted stimulation was most important for differentiating inflammation in diabetes, consistent with previous identification of a mixed T helper type 1/T helper type 17 cytokine profile in diabetes. CONCLUSIONS: T cells dominate peripheral inflammation in obesity; therefore, targeting T cells may be an effective approach for prevention/management of metaflammation.

Identifying Opioid Use Disorder from Longitudinal Healthcare Data using a Multi-stream Transformer.

Opioid Use Disorder (OUD) is a public health crisis costing the US billions of dollars annually in healthcare, lost workplace productivity, and crime. Analyzing longitudinal healthcare data is critical in addressing many real-world problems in healthcare. Leveraging the real-world longitudinal healthcare data, we propose a novel multi-stream transformer model called MUPOD for OUD identification. MUPOD is designed to simultaneously analyze multiple types of healthcare data streams, such as medications and diagnoses, by attending to segments within and across these data streams. Our model tested on the data from 392,492 patients with long-term back pain problems showed significantly better performance than the traditional models and recently developed deep learning models.

Integrating data science into the translational science research spectrum: A substance use disorder case study.

The availability of large healthcare datasets offers the opportunity for researchers to navigate the traditional clinical and translational science research stages in a nonlinear manner. In particular, data scientists can harness the power of large healthcare datasets to bridge from preclinical discoveries (T0) directly to assessing population-level health impact (T4). A successful bridge from T0 to T4 does not bypass the other stages entirely; rather, effective team science makes a direct progression from T0 to T4 impactful by incorporating the perspectives of researchers from every stage of the clinical and translational science research spectrum. In this exemplar, we demonstrate how effective team science overcame challenges and, ultimately, ensured success when a diverse team of researchers worked together, using healthcare big data to test population-level substance use disorder (SUD) hypotheses generated from preclinical rodent studies. This project, called Advancing Substance use disorder Knowledge using Big Data (ASK Big Data), highlights the critical roles that data science expertise and effective team science play in quickly translating preclinical research into public health impact.

Predicting substance use disorder using long-term attention deficit hyperactivity disorder medication records in Truven.

About 20% of individuals with attention deficit hyperactivity disorder are first diagnosed during adolescence. While preclinical experiments suggest that adolescent-onset exposure to attention deficit hyperactivity disorder medication is an important factor in the development of substance use disorder phenotypes in adulthood, the long-term impact of attention deficit hyperactivity disorder medication initiated during adolescence has been largely unexplored in humans. Our analysis of 11,624 adolescent enrollees with attention deficit hyperactivity disorder in the Truven database indicates that temporal medication features, rather than stationary features, are the most important factors on the health consequences related to substance use disorder and attention deficit hyperactivity disorder medication initiation during adolescence.

Deep Learning Prediction of Mild Cognitive Impairment using Electronic Health Records.

About 44.4 million people have been diagnosed with dementia worldwide, and it is estimated that this number will be almost tripled by 2050. Predicting mild cognitive impairment (MCI), an intermediate state between normal cognition and dementia and an important risk factor for the development of dementia is crucial in aging populations. MCI is formally determined by health professionals through a comprehensive cognitive evaluation, together with a clinical examination, medical history and often the input of an informant (an individual that know the patient very well). However, this is not routinely performed in primary care visits, and could result in a significant delay in diagnosis. In this study, we used deep learning and machine learning techniques to predict the progression from cognitively unimpaired to MCI and also to analyze the potential for patient clustering using routinely-collected electronic health records (EHRs). Our analysis of EHRs indicates that temporal characteristics of patient data incorporated in a deep learning model provides increased power in predicting MCI.

Predicting Substance Use Disorder in ADHD Patients using Long-Short Term Memory Model.

About 20% of individuals with attention deficit hyperactivity disorder (ADHD) are first diagnosed during adolescence. While preclinical experiments suggest that adolescent-onset exposure to ADHD medication is an important factor in the development of substance use disorder (SUD) phenotypes in adulthood, the long-term impact of ADHD medication initiated during adolescence has been largely unexplored in humans. Our analysis of 11,624 adolescent ADHD patients in the Truven database indicates that temporal medication features are the important factors on the health consequences related to SUD and ADHD medication initiation during adolescence.