Implementing a pragmatic clinical trial to tailor opioids for chronic pain on behalf of the IGNITE ADOPT PGx investigators.

Chronic pain is a prevalent condition with enormous economic burden. Opioids such as tramadol, codeine, and hydrocodone are commonly used to treat chronic pain; these drugs are activated to more potent opioid receptor agonists by the hepatic CYP2D6 enzyme. Results from clinical studies and mechanistic understandings suggest that CYP2D6-guided therapy will improve pain control and reduce adverse drug events. However, CYP2D6 is rarely used in clinical practice due in part to the demand for additional clinical trial evidence. Thus, we designed the ADOPT-PGx (A Depression and Opioid Pragmatic Trial in Pharmacogenetics) chronic pain study, a multicenter, pragmatic, randomized controlled clinical trial, to assess the effect of CYP2D6 testing on pain management. The study enrolled 1048 participants who are taking or being considered for treatment with CYP2D6-impacted opioids for their chronic pain. Participants were randomized to receive immediate or delayed (by 6 months) genotyping of CYP2D6 with clinical decision support (CDS). CDS encouraged the providers to follow the CYP2D6-guided trial recommendations. The primary study outcome is the 3-month absolute change in the composite pain intensity score assessed using Patient-Reported Outcomes Measurement Information System (PROMIS) measures. Follow-up will be completed in July 2024. Herein, we describe the design of this trial along with challenges encountered during enrollment.

Using large language model to guide patients to create efficient and comprehensive clinical care message.

OBJECTIVE: This study aims to investigate the feasibility of using Large Language Models (LLMs) to engage with patients at the time they are drafting a question to their healthcare providers, and generate pertinent follow-up questions that the patient can answer before sending their message, with the goal of ensuring that their healthcare provider receives all the information they need to safely and accurately answer the patient's question, eliminating back-and-forth messaging, and the associated delays and frustrations. METHODS: We collected a dataset of patient messages sent between January 1, 2022 to March 7, 2023 at Vanderbilt University Medical Center. Two internal medicine physicians identified 7 common scenarios. We used 3 LLMs to generate follow-up questions: (1) Comprehensive LLM Artificial Intelligence Responder (CLAIR): a locally fine-tuned LLM, (2) GPT4 with a simple prompt, and (3) GPT4 with a complex prompt. Five physicians rated them with the actual follow-ups written by healthcare providers on clarity, completeness, conciseness, and utility. RESULTS: For five scenarios, our CLAIR model had the best performance. The GPT4 model received higher scores for utility and completeness but lower scores for clarity and conciseness. CLAIR generated follow-up questions with similar clarity and conciseness as the actual follow-ups written by healthcare providers, with higher utility than healthcare providers and GPT4, and lower completeness than GPT4, but better than healthcare providers. CONCLUSION: LLMs can generate follow-up patient messages designed to clarify a medical question that compares favorably to those generated by healthcare providers.

Rationale and design for a pragmatic randomized trial to assess gene-based prescribing for SSRIs in the treatment of depression.

Specific selective serotonin reuptake inhibitors (SSRIs) metabolism is strongly influenced by two pharmacogenes, CYP2D6 and CYP2C19. However, the effectiveness of prospectively using pharmacogenetic variants to select or dose SSRIs for depression is uncertain in routine clinical practice. The objective of this prospective, multicenter, pragmatic randomized controlled trial is to determine the effectiveness of genotype-guided selection and dosing of antidepressants on control of depression in participants who are 8 years or older with ≥3 months of depressive symptoms who require new or revised therapy. Those randomized to the intervention arm undergo pharmacogenetic testing at baseline and receive a pharmacy consult and/or automated clinical decision support intervention based on an actionable phenotype, while those randomized to the control arm have pharmacogenetic testing at the end of 6-months. In both groups, depression and drug tolerability outcomes are assessed at baseline, 1 month, 3 months (primary), and 6 months. The primary end point is defined by change in Patient-Reported Outcomes Measurement Information System (PROMIS) Depression score assessed at 3 months versus baseline. Secondary end points include change inpatient health questionnaire (PHQ-8) measure of depression severity, remission rates defined by PROMIS score < 16, medication adherence, and medication side effects. The primary analysis will compare the PROMIS score difference between trial arms among those with an actionable CYP2D6 or CYP2C19 genetic result or a CYP2D6 drug-drug interaction. The trial has completed accrual of 1461 participants, of which 562 were found to have an actionable phenotype to date, and follow-up will be complete in April of 2024.

Large language models facilitate the generation of electronic health record phenotyping algorithms.

OBJECTIVES: Phenotyping is a core task in observational health research utilizing electronic health records (EHRs). Developing an accurate algorithm demands substantial input from domain experts, involving extensive literature review and evidence synthesis. This burdensome process limits scalability and delays knowledge discovery. We investigate the potential for leveraging large language models (LLMs) to enhance the efficiency of EHR phenotyping by generating high-quality algorithm drafts. MATERIALS AND METHODS: We prompted four LLMs-GPT-4 and GPT-3.5 of ChatGPT, Claude 2, and Bard-in October 2023, asking them to generate executable phenotyping algorithms in the form of SQL queries adhering to a common data model (CDM) for three phenotypes (ie, type 2 diabetes mellitus, dementia, and hypothyroidism). Three phenotyping experts evaluated the returned algorithms across several critical metrics. We further implemented the top-rated algorithms and compared them against clinician-validated phenotyping algorithms from the Electronic Medical Records and Genomics (eMERGE) network. RESULTS: GPT-4 and GPT-3.5 exhibited significantly higher overall expert evaluation scores in instruction following, algorithmic logic, and SQL executability, when compared to Claude 2 and Bard. Although GPT-4 and GPT-3.5 effectively identified relevant clinical concepts, they exhibited immature capability in organizing phenotyping criteria with the proper logic, leading to phenotyping algorithms that were either excessively restrictive (with low recall) or overly broad (with low positive predictive values). CONCLUSION: GPT versions 3.5 and 4 are capable of drafting phenotyping algorithms by identifying relevant clinical criteria aligned with a CDM. However, expertise in informatics and clinical experience is still required to assess and further refine generated algorithms.

Risk factors affecting polygenic score performance across diverse cohorts.

Apart from ancestry, personal or environmental covariates may contribute to differences in polygenic score (PGS) performance. We analyzed effects of covariate stratification and interaction on body mass index (BMI) PGS (PGSBMI) across four cohorts of European (N=491,111) and African (N=21,612) ancestry. Stratifying on binary covariates and quintiles for continuous covariates, 18/62 covariates had significant and replicable R2 differences among strata. Covariates with the largest differences included age, sex, blood lipids, physical activity, and alcohol consumption, with R2 being nearly double between best and worst performing quintiles for certain covariates. 28 covariates had significant PGSBMI-covariate interaction effects, modifying PGSBMI effects by nearly 20% per standard deviation change. We observed overlap between covariates that had significant R2 differences among strata and interaction effects - across all covariates, their main effects on BMI were correlated with their maximum R2 differences and interaction effects (0.56 and 0.58, respectively), suggesting high-PGSBMI individuals have highest R2 and increase in PGS effect. Using quantile regression, we show the effect of PGSBMI increases as BMI itself increases, and that these differences in effects are directly related to differences in R2 when stratifying by different covariates. Given significant and replicable evidence for context-specific PGSBMI performance and effects, we investigated ways to increase model performance taking into account non-linear effects. Machine learning models (neural networks) increased relative model R2 (mean 23%) across datasets. Finally, creating PGSBMI directly from GxAge GWAS effects increased relative R2 by 7.8%. These results demonstrate that certain covariates, especially those most associated with BMI, significantly affect both PGSBMI performance and effects across diverse cohorts and ancestries, and we provide avenues to improve model performance that consider these effects.

Leveraging large language models for generating responses to patient messages-a subjective analysis.

OBJECTIVE: This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal. MATERIALS AND METHODS: Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate fine-tuned models, we used 10 representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness. RESULTS: The dataset consisted of 499 794 pairs of patient messages and corresponding responses from the patient portal, with 5000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness. CONCLUSION: This subjective analysis suggests that leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and healthcare providers.

Leveraging generative AI to prioritize drug repurposing candidates for Alzheimer's disease with real-world clinical validation.

Drug repurposing represents an attractive alternative to the costly and time-consuming process of new drug development, particularly for serious, widespread conditions with limited effective treatments, such as Alzheimer's disease (AD). Emerging generative artificial intelligence (GAI) technologies like ChatGPT offer the promise of expediting the review and summary of scientific knowledge. To examine the feasibility of using GAI for identifying drug repurposing candidates, we iteratively tasked ChatGPT with proposing the twenty most promising drugs for repurposing in AD, and tested the top ten for risk of incident AD in exposed and unexposed individuals over age 65 in two large clinical datasets: (1) Vanderbilt University Medical Center and (2) the All of Us Research Program. Among the candidates suggested by ChatGPT, metformin, simvastatin, and losartan were associated with lower AD risk in meta-analysis. These findings suggest GAI technologies can assimilate scientific insights from an extensive Internet-based search space, helping to prioritize drug repurposing candidates and facilitate the treatment of diseases.

Leveraging explainable artificial intelligence to optimize clinical decision support.

OBJECTIVE: To develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches. METHODS: We extracted data on alerts generated from January 1, 2019 to December 31, 2020, at Vanderbilt University Medical Center. We developed machine learning models to predict user responses to alerts. We applied XAI techniques to generate global explanations and local explanations. We evaluated the generated suggestions by comparing with alert's historical change logs and stakeholder interviews. Suggestions that either matched (or partially matched) changes already made to the alert or were considered clinically correct were classified as helpful. RESULTS: The final dataset included 2 991 823 firings with 2689 features. Among the 5 machine learning models, the LightGBM model achieved the highest Area under the ROC Curve: 0.919 [0.918, 0.920]. We identified 96 helpful suggestions. A total of 278 807 firings (9.3%) could have been eliminated. Some of the suggestions also revealed workflow and education issues. CONCLUSION: We developed a data-driven process to generate suggestions for improving alert criteria using XAI techniques. Our approach could identify improvements regarding clinical decision support (CDS) that might be overlooked or delayed in manual reviews. It also unveils a secondary purpose for the XAI: to improve quality by discovering scenarios where CDS alerts are not accepted due to workflow, education, or staffing issues.

Large Language Models Facilitate the Generation of Electronic Health Record Phenotyping Algorithms.

Objectives: Phenotyping is a core task in observational health research utilizing electronic health records (EHRs). Developing an accurate algorithm demands substantial input from domain experts, involving extensive literature review and evidence synthesis. This burdensome process limits scalability and delays knowledge discovery. We investigate the potential for leveraging large language models (LLMs) to enhance the efficiency of EHR phenotyping by generating high-quality algorithm drafts. Materials and Methods: We prompted four LLMs-GPT-4 and GPT-3.5 of ChatGPT, Claude 2, and Bard-in October 2023, asking them to generate executable phenotyping algorithms in the form of SQL queries adhering to a common data model (CDM) for three phenotypes (i.e., type 2 diabetes mellitus, dementia, and hypothyroidism). Three phenotyping experts evaluated the returned algorithms across several critical metrics. We further implemented the top-rated algorithms and compared them against clinician-validated phenotyping algorithms from the Electronic Medical Records and Genomics (eMERGE) network. Results: GPT-4 and GPT-3.5 exhibited significantly higher overall expert evaluation scores in instruction following, algorithmic logic, and SQL executability, when compared to Claude 2 and Bard. Although GPT-4 and GPT-3.5 effectively identified relevant clinical concepts, they exhibited immature capability in organizing phenotyping criteria with the proper logic, leading to phenotyping algorithms that were either excessively restrictive (with low recall) or overly broad (with low positive predictive values). Conclusion: GPT versions 3.5 and 4 are capable of drafting phenotyping algorithms by identifying relevant clinical criteria aligned with a CDM. However, expertise in informatics and clinical experience is still required to assess and further refine generated algorithms.

Diagnostic delay in monogenic disease: A scoping review.

PURPOSE: Diagnostic delay in monogenic disease is reportedly common. We conducted a scoping review investigating variability in study design, results, and conclusions. METHODS: We searched the academic literature on January 17, 2023, for original peer reviewed journals and conference articles that quantified diagnostic delay in monogenic disease. We abstracted the reported diagnostic delay, relevant study design features, and definitions. RESULTS: Our search identified 259 articles quantifying diagnostic delay in 111 distinct monogenetic diseases. Median reported diagnostic delay for all studies collectively in monogenetic diseases was 5.0 years (IQR 2-10). There was major variation in the reported delay within individual monogenetic diseases. Shorter delay was associated with disorders of childhood metabolism, immunity, and development. The majority (67.6%) of articles that studied delay reported an improvement with calendar time. Study design and definitions of delay were highly heterogenous. Three gaps were identified: (1) no studies were conducted in the least developed countries, (2) delay has not been studied for the majority of known, or (3) most prevalent genetic diseases. CONCLUSION: Heterogenous study design and definitions of diagnostic delay inhibit comparison across studies. Future efforts should focus on standardizing delay measurements, while expanding the research to low-income countries.