Implementing a pragmatic clinical trial to tailor opioids for chronic pain on behalf of the IGNITE ADOPT PGx investigators.

Chronic pain is a prevalent condition with enormous economic burden. Opioids such as tramadol, codeine, and hydrocodone are commonly used to treat chronic pain; these drugs are activated to more potent opioid receptor agonists by the hepatic CYP2D6 enzyme. Results from clinical studies and mechanistic understandings suggest that CYP2D6-guided therapy will improve pain control and reduce adverse drug events. However, CYP2D6 is rarely used in clinical practice due in part to the demand for additional clinical trial evidence. Thus, we designed the ADOPT-PGx (A Depression and Opioid Pragmatic Trial in Pharmacogenetics) chronic pain study, a multicenter, pragmatic, randomized controlled clinical trial, to assess the effect of CYP2D6 testing on pain management. The study enrolled 1048 participants who are taking or being considered for treatment with CYP2D6-impacted opioids for their chronic pain. Participants were randomized to receive immediate or delayed (by 6 months) genotyping of CYP2D6 with clinical decision support (CDS). CDS encouraged the providers to follow the CYP2D6-guided trial recommendations. The primary study outcome is the 3-month absolute change in the composite pain intensity score assessed using Patient-Reported Outcomes Measurement Information System (PROMIS) measures. Follow-up will be completed in July 2024. Herein, we describe the design of this trial along with challenges encountered during enrollment.

Using large language model to guide patients to create efficient and comprehensive clinical care message.

OBJECTIVE: This study aims to investigate the feasibility of using Large Language Models (LLMs) to engage with patients at the time they are drafting a question to their healthcare providers, and generate pertinent follow-up questions that the patient can answer before sending their message, with the goal of ensuring that their healthcare provider receives all the information they need to safely and accurately answer the patient's question, eliminating back-and-forth messaging, and the associated delays and frustrations. METHODS: We collected a dataset of patient messages sent between January 1, 2022 to March 7, 2023 at Vanderbilt University Medical Center. Two internal medicine physicians identified 7 common scenarios. We used 3 LLMs to generate follow-up questions: (1) Comprehensive LLM Artificial Intelligence Responder (CLAIR): a locally fine-tuned LLM, (2) GPT4 with a simple prompt, and (3) GPT4 with a complex prompt. Five physicians rated them with the actual follow-ups written by healthcare providers on clarity, completeness, conciseness, and utility. RESULTS: For five scenarios, our CLAIR model had the best performance. The GPT4 model received higher scores for utility and completeness but lower scores for clarity and conciseness. CLAIR generated follow-up questions with similar clarity and conciseness as the actual follow-ups written by healthcare providers, with higher utility than healthcare providers and GPT4, and lower completeness than GPT4, but better than healthcare providers. CONCLUSION: LLMs can generate follow-up patient messages designed to clarify a medical question that compares favorably to those generated by healthcare providers.

Rationale and design for a pragmatic randomized trial to assess gene-based prescribing for SSRIs in the treatment of depression.

Specific selective serotonin reuptake inhibitors (SSRIs) metabolism is strongly influenced by two pharmacogenes, CYP2D6 and CYP2C19. However, the effectiveness of prospectively using pharmacogenetic variants to select or dose SSRIs for depression is uncertain in routine clinical practice. The objective of this prospective, multicenter, pragmatic randomized controlled trial is to determine the effectiveness of genotype-guided selection and dosing of antidepressants on control of depression in participants who are 8 years or older with ≥3 months of depressive symptoms who require new or revised therapy. Those randomized to the intervention arm undergo pharmacogenetic testing at baseline and receive a pharmacy consult and/or automated clinical decision support intervention based on an actionable phenotype, while those randomized to the control arm have pharmacogenetic testing at the end of 6-months. In both groups, depression and drug tolerability outcomes are assessed at baseline, 1 month, 3 months (primary), and 6 months. The primary end point is defined by change in Patient-Reported Outcomes Measurement Information System (PROMIS) Depression score assessed at 3 months versus baseline. Secondary end points include change inpatient health questionnaire (PHQ-8) measure of depression severity, remission rates defined by PROMIS score < 16, medication adherence, and medication side effects. The primary analysis will compare the PROMIS score difference between trial arms among those with an actionable CYP2D6 or CYP2C19 genetic result or a CYP2D6 drug-drug interaction. The trial has completed accrual of 1461 participants, of which 562 were found to have an actionable phenotype to date, and follow-up will be complete in April of 2024.

Large language models facilitate the generation of electronic health record phenotyping algorithms.

OBJECTIVES: Phenotyping is a core task in observational health research utilizing electronic health records (EHRs). Developing an accurate algorithm demands substantial input from domain experts, involving extensive literature review and evidence synthesis. This burdensome process limits scalability and delays knowledge discovery. We investigate the potential for leveraging large language models (LLMs) to enhance the efficiency of EHR phenotyping by generating high-quality algorithm drafts. MATERIALS AND METHODS: We prompted four LLMs-GPT-4 and GPT-3.5 of ChatGPT, Claude 2, and Bard-in October 2023, asking them to generate executable phenotyping algorithms in the form of SQL queries adhering to a common data model (CDM) for three phenotypes (ie, type 2 diabetes mellitus, dementia, and hypothyroidism). Three phenotyping experts evaluated the returned algorithms across several critical metrics. We further implemented the top-rated algorithms and compared them against clinician-validated phenotyping algorithms from the Electronic Medical Records and Genomics (eMERGE) network. RESULTS: GPT-4 and GPT-3.5 exhibited significantly higher overall expert evaluation scores in instruction following, algorithmic logic, and SQL executability, when compared to Claude 2 and Bard. Although GPT-4 and GPT-3.5 effectively identified relevant clinical concepts, they exhibited immature capability in organizing phenotyping criteria with the proper logic, leading to phenotyping algorithms that were either excessively restrictive (with low recall) or overly broad (with low positive predictive values). CONCLUSION: GPT versions 3.5 and 4 are capable of drafting phenotyping algorithms by identifying relevant clinical criteria aligned with a CDM. However, expertise in informatics and clinical experience is still required to assess and further refine generated algorithms.

Risk factors affecting polygenic score performance across diverse cohorts.

Apart from ancestry, personal or environmental covariates may contribute to differences in polygenic score (PGS) performance. We analyzed effects of covariate stratification and interaction on body mass index (BMI) PGS (PGSBMI) across four cohorts of European (N=491,111) and African (N=21,612) ancestry. Stratifying on binary covariates and quintiles for continuous covariates, 18/62 covariates had significant and replicable R2 differences among strata. Covariates with the largest differences included age, sex, blood lipids, physical activity, and alcohol consumption, with R2 being nearly double between best and worst performing quintiles for certain covariates. 28 covariates had significant PGSBMI-covariate interaction effects, modifying PGSBMI effects by nearly 20% per standard deviation change. We observed overlap between covariates that had significant R2 differences among strata and interaction effects - across all covariates, their main effects on BMI were correlated with their maximum R2 differences and interaction effects (0.56 and 0.58, respectively), suggesting high-PGSBMI individuals have highest R2 and increase in PGS effect. Using quantile regression, we show the effect of PGSBMI increases as BMI itself increases, and that these differences in effects are directly related to differences in R2 when stratifying by different covariates. Given significant and replicable evidence for context-specific PGSBMI performance and effects, we investigated ways to increase model performance taking into account non-linear effects. Machine learning models (neural networks) increased relative model R2 (mean 23%) across datasets. Finally, creating PGSBMI directly from GxAge GWAS effects increased relative R2 by 7.8%. These results demonstrate that certain covariates, especially those most associated with BMI, significantly affect both PGSBMI performance and effects across diverse cohorts and ancestries, and we provide avenues to improve model performance that consider these effects.

Leveraging large language models for generating responses to patient messages-a subjective analysis.

OBJECTIVE: This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal. MATERIALS AND METHODS: Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate fine-tuned models, we used 10 representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness. RESULTS: The dataset consisted of 499 794 pairs of patient messages and corresponding responses from the patient portal, with 5000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness. CONCLUSION: This subjective analysis suggests that leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and healthcare providers.

Leveraging explainable artificial intelligence to optimize clinical decision support.

OBJECTIVE: To develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches. METHODS: We extracted data on alerts generated from January 1, 2019 to December 31, 2020, at Vanderbilt University Medical Center. We developed machine learning models to predict user responses to alerts. We applied XAI techniques to generate global explanations and local explanations. We evaluated the generated suggestions by comparing with alert's historical change logs and stakeholder interviews. Suggestions that either matched (or partially matched) changes already made to the alert or were considered clinically correct were classified as helpful. RESULTS: The final dataset included 2 991 823 firings with 2689 features. Among the 5 machine learning models, the LightGBM model achieved the highest Area under the ROC Curve: 0.919 [0.918, 0.920]. We identified 96 helpful suggestions. A total of 278 807 firings (9.3%) could have been eliminated. Some of the suggestions also revealed workflow and education issues. CONCLUSION: We developed a data-driven process to generate suggestions for improving alert criteria using XAI techniques. Our approach could identify improvements regarding clinical decision support (CDS) that might be overlooked or delayed in manual reviews. It also unveils a secondary purpose for the XAI: to improve quality by discovering scenarios where CDS alerts are not accepted due to workflow, education, or staffing issues.

Leveraging generative AI to prioritize drug repurposing candidates for Alzheimer's disease with real-world clinical validation.

Drug repurposing represents an attractive alternative to the costly and time-consuming process of new drug development, particularly for serious, widespread conditions with limited effective treatments, such as Alzheimer's disease (AD). Emerging generative artificial intelligence (GAI) technologies like ChatGPT offer the promise of expediting the review and summary of scientific knowledge. To examine the feasibility of using GAI for identifying drug repurposing candidates, we iteratively tasked ChatGPT with proposing the twenty most promising drugs for repurposing in AD, and tested the top ten for risk of incident AD in exposed and unexposed individuals over age 65 in two large clinical datasets: (1) Vanderbilt University Medical Center and (2) the All of Us Research Program. Among the candidates suggested by ChatGPT, metformin, simvastatin, and losartan were associated with lower AD risk in meta-analysis. These findings suggest GAI technologies can assimilate scientific insights from an extensive Internet-based search space, helping to prioritize drug repurposing candidates and facilitate the treatment of diseases.

Large Language Models Facilitate the Generation of Electronic Health Record Phenotyping Algorithms.

Objectives: Phenotyping is a core task in observational health research utilizing electronic health records (EHRs). Developing an accurate algorithm demands substantial input from domain experts, involving extensive literature review and evidence synthesis. This burdensome process limits scalability and delays knowledge discovery. We investigate the potential for leveraging large language models (LLMs) to enhance the efficiency of EHR phenotyping by generating high-quality algorithm drafts. Materials and Methods: We prompted four LLMs-GPT-4 and GPT-3.5 of ChatGPT, Claude 2, and Bard-in October 2023, asking them to generate executable phenotyping algorithms in the form of SQL queries adhering to a common data model (CDM) for three phenotypes (i.e., type 2 diabetes mellitus, dementia, and hypothyroidism). Three phenotyping experts evaluated the returned algorithms across several critical metrics. We further implemented the top-rated algorithms and compared them against clinician-validated phenotyping algorithms from the Electronic Medical Records and Genomics (eMERGE) network. Results: GPT-4 and GPT-3.5 exhibited significantly higher overall expert evaluation scores in instruction following, algorithmic logic, and SQL executability, when compared to Claude 2 and Bard. Although GPT-4 and GPT-3.5 effectively identified relevant clinical concepts, they exhibited immature capability in organizing phenotyping criteria with the proper logic, leading to phenotyping algorithms that were either excessively restrictive (with low recall) or overly broad (with low positive predictive values). Conclusion: GPT versions 3.5 and 4 are capable of drafting phenotyping algorithms by identifying relevant clinical criteria aligned with a CDM. However, expertise in informatics and clinical experience is still required to assess and further refine generated algorithms.

Diagnostic delay in monogenic disease: A scoping review.

PURPOSE: Diagnostic delay in monogenic disease is reportedly common. We conducted a scoping review investigating variability in study design, results, and conclusions. METHODS: We searched the academic literature on January 17, 2023, for original peer reviewed journals and conference articles that quantified diagnostic delay in monogenic disease. We abstracted the reported diagnostic delay, relevant study design features, and definitions. RESULTS: Our search identified 259 articles quantifying diagnostic delay in 111 distinct monogenetic diseases. Median reported diagnostic delay for all studies collectively in monogenetic diseases was 5.0 years (IQR 2-10). There was major variation in the reported delay within individual monogenetic diseases. Shorter delay was associated with disorders of childhood metabolism, immunity, and development. The majority (67.6%) of articles that studied delay reported an improvement with calendar time. Study design and definitions of delay were highly heterogenous. Three gaps were identified: (1) no studies were conducted in the least developed countries, (2) delay has not been studied for the majority of known, or (3) most prevalent genetic diseases. CONCLUSION: Heterogenous study design and definitions of diagnostic delay inhibit comparison across studies. Future efforts should focus on standardizing delay measurements, while expanding the research to low-income countries.

Prospective, multi-site study of healthcare utilization after actionable monogenic findings from clinical sequencing.

As large-scale genomic screening becomes increasingly prevalent, understanding the influence of actionable results on healthcare utilization is key to estimating the potential long-term clinical impact. The eMERGE network sequenced individuals for actionable genes in multiple genetic conditions and returned results to individuals, providers, and the electronic health record. Differences in recommended health services (laboratory, imaging, and procedural testing) delivered within 12 months of return were compared among individuals with pathogenic or likely pathogenic (P/LP) findings to matched individuals with negative findings before and after return of results. Of 16,218 adults, 477 unselected individuals were found to have a monogenic risk for arrhythmia (n = 95), breast cancer (n = 96), cardiomyopathy (n = 95), colorectal cancer (n = 105), or familial hypercholesterolemia (n = 86). Individuals with P/LP results more frequently received services after return (43.8%) compared to before return (25.6%) of results and compared to individuals with negative findings (24.9%; p < 0.0001). The annual cost of qualifying healthcare services increased from an average of $162 before return to $343 after return of results among the P/LP group (p < 0.0001); differences in the negative group were non-significant. The mean difference-in-differences was $149 (p < 0.0001), which describes the increased cost within the P/LP group corrected for cost changes in the negative group. When stratified by individual conditions, significant cost differences were observed for arrhythmia, breast cancer, and cardiomyopathy. In conclusion, less than half of individuals received billed health services after monogenic return, which modestly increased healthcare costs for payors in the year following return.

The phenotype-genotype reference map: Improving biobank data science through replication.

Population-scale biobanks linked to electronic health record data provide vast opportunities to extend our knowledge of human genetics and discover new phenotype-genotype associations. Given their dense phenotype data, biobanks can also facilitate replication studies on a phenome-wide scale. Here, we introduce the phenotype-genotype reference map (PGRM), a set of 5,879 genetic associations from 523 GWAS publications that can be used for high-throughput replication experiments. PGRM phenotypes are standardized as phecodes, ensuring interoperability between biobanks. We applied the PGRM to five ancestry-specific cohorts from four independent biobanks and found evidence of robust replications across a wide array of phenotypes. We show how the PGRM can be used to detect data corruption and to empirically assess parameters for phenome-wide studies. Finally, we use the PGRM to explore factors associated with replicability of GWAS results.

Leveraging Generative AI to Prioritize Drug Repurposing Candidates: Validating Identified Candidates for Alzheimer's Disease in Real-World Clinical Datasets.

Drug repurposing represents an attractive alternative to the costly and time-consuming process of new drug development, particularly for serious, widespread conditions with limited effective treatments, such as Alzheimer's disease (AD). Emerging generative artificial intelligence (GAI) technologies like ChatGPT offer the promise of expediting the review and summary of scientific knowledge. To examine the feasibility of using GAI for identifying drug repurposing candidates, we iteratively tasked ChatGPT with proposing the twenty most promising drugs for repurposing in AD, and tested the top ten for risk of incident AD in exposed and unexposed individuals over age 65 in two large clinical datasets: 1) Vanderbilt University Medical Center and 2) the All of Us Research Program. Among the candidates suggested by ChatGPT, metformin, simvastatin, and losartan were associated with lower AD risk in meta-analysis. These findings suggest GAI technologies can assimilate scientific insights from an extensive Internet-based search space, helping to prioritize drug repurposing candidates and facilitate the treatment of diseases.

Leveraging Large Language Models for Generating Responses to Patient Messages.

Objective: This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal. Methods: Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate the fine-tuned models, we used ten representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness. Results: The dataset consisted of a total of 499,794 pairs of patient messages and corresponding responses from the patient portal, with 5,000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness. Conclusion: Leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and primary care providers.

Leveraging Generative AI to Prioritize Drug Repurposing Candidates: Validating Identified Candidates for Alzheimer's Disease in Real-World Clinical Datasets.

Drug repurposing represents an attractive alternative to the costly and time-consuming process of new drug development, particularly for serious, widespread conditions with limited effective treatments, such as Alzheimer's disease (AD). Emerging generative artificial intelligence (GAI) technologies like ChatGPT offer the promise of expediting the review and summary of scientific knowledge. To examine the feasibility of using GAI for identifying drug repurposing candidates, we iteratively tasked ChatGPT with proposing the twenty most promising drugs for repurposing in AD, and tested the top ten for risk of incident AD in exposed and unexposed individuals over age 65 in two large clinical datasets: 1) Vanderbilt University Medical Center and 2) the All of Us Research Program. Among the candidates suggested by ChatGPT, metformin, simvastatin, and losartan were associated with lower AD risk in meta-analysis. These findings suggest GAI technologies can assimilate scientific insights from an extensive Internet-based search space, helping to prioritize drug repurposing candidates and facilitate the treatment of diseases.

Studying the impact of translational genomic research: Lessons from eMERGE.

Two major goals of the Electronic Medical Record and Genomics (eMERGE) Network are to learn how best to return research results to patient/participants and the clinicians who care for them and also to assess the impact of placing these results in clinical care. Yet since its inception, the Network has confronted a host of challenges in achieving these goals, many of which had ethical, legal, or social implications (ELSIs) that required consideration. Here, we share impediments we encountered in recruiting participants, returning results, and assessing their impact, all of which affected our ability to achieve the goals of eMERGE, as well as the steps we took to attempt to address these obstacles. We divide the domains in which we experienced challenges into four broad categories: (1) study design, including recruitment of more diverse groups; (2) consent; (3) returning results to participants and their health care providers (HCPs); and (4) assessment of follow-up care of participants and measuring the impact of research on participants and their families. Since most phases of eMERGE have included children as well as adults, we also address the particular ELSI posed by including pediatric populations in this research. We make specific suggestions for improving translational genomic research to ensure that future projects can effectively return results and assess their impact on patient/participants and providers if the goals of genomic-informed medicine are to be achieved.

Phenotypic presentation of Mendelian disease across the diagnostic trajectory in electronic health records.

PURPOSE: To investigate the phenotypic presentation of Mendelian disease across the diagnostic trajectory in the electronic health record (EHR). METHODS: We applied a conceptual model to delineate the diagnostic trajectory of Mendelian disease to the EHRs of patients affected by 1 of 9 Mendelian diseases. We assessed data availability and phenotype ascertainment across the diagnostic trajectory using phenotype risk scores and validated our findings via chart review of patients with hereditary connective tissue disorders. RESULTS: We identified 896 individuals with genetically confirmed diagnoses, 216 (24%) of whom had fully ascertained diagnostic trajectories. Phenotype risk scores increased following clinical suspicion and diagnosis (P < 1 × 10-4, Wilcoxon rank sum test). We found that of all International Classification of Disease-based phenotypes in the EHR, 66% were recorded after clinical suspicion, and manual chart review yielded consistent results. CONCLUSION: Using a novel conceptual model to study the diagnostic trajectory of genetic disease in the EHR, we demonstrated that phenotype ascertainment is, in large part, driven by the clinical examinations and studies prompted by clinical suspicion of a genetic disease, a process we term diagnostic convergence. Algorithms designed to detect undiagnosed genetic disease should consider censoring EHR data at the first date of clinical suspicion to avoid data leakage.