Retrieval Augmented Generation Enabled Generative Pre-Trained Transformer 4 (GPT-4) Performance for Clinical Trial Screening.

Background: Subject screening is a key aspect of all clinical trials; however, traditionally, it is a labor-intensive and error-prone task, demanding significant time and resources. With the advent of large language models (LLMs) and related technologies, a paradigm shift in natural language processing capabilities offers a promising avenue for increasing both quality and efficiency of screening efforts. This study aimed to test the Retrieval-Augmented Generation (RAG) process enabled Generative Pretrained Transformer Version 4 (GPT-4) to accurately identify and report on inclusion and exclusion criteria for a clinical trial. Methods: The Co-Operative Program for Implementation of Optimal Therapy in Heart Failure (COPILOT-HF) trial aims to recruit patients with symptomatic heart failure. As part of the screening process, a list of potentially eligible patients is created through an electronic health record (EHR) query. Currently, structured data in the EHR can only be used to determine 5 out of 6 inclusion and 5 out of 17 exclusion criteria. Trained, but non-licensed, study staff complete manual chart review to determine patient eligibility and record their assessment of the inclusion and exclusion criteria. We obtained the structured assessments completed by the study staff and clinical notes for the past two years and developed a workflow of clinical note-based question answering system powered by RAG architecture and GPT-4 that we named RECTIFIER (RAG-Enabled Clinical Trial Infrastructure for Inclusion Exclusion Review). We used notes from 100 patients as a development dataset, 282 patients as a validation dataset, and 1894 patients as a test set. An expert clinician completed a blinded review of patients' charts to answer the eligibility questions and determine the "gold standard" answers. We calculated the sensitivity, specificity, accuracy, and Matthews correlation coefficient (MCC) for each question and screening method. We also performed bootstrapping to calculate the confidence intervals for each statistic. Results: Both RECTIFIER and study staff answers closely aligned with the expert clinician answers across criteria with accuracy ranging between 97.9% and 100% (MCC 0.837 and 1) for RECTIFIER and 91.7% and 100% (MCC 0.644 and 1) for study staff. RECTIFIER performed better than study staff to determine the inclusion criteria of "symptomatic heart failure" with an accuracy of 97.9% vs 91.7% and an MCC of 0.924 vs 0.721, respectively. Overall, the sensitivity and specificity of determining eligibility for the RECTIFIER was 92.3% (CI) and 93.9% (CI), and study staff was 90.1% (CI) and 83.6% (CI), respectively. Conclusion: GPT-4 based solutions have the potential to improve efficiency and reduce costs in clinical trial screening. When incorporating new tools such as RECTIFIER, it is important to consider the potential hazards of automating the screening process and set up appropriate mitigation strategies such as final clinician review before patient engagement.

Implementation of a Scalable Online Weight Management Programme in Clinical Settings: Protocol for the PROPS 2.0 Programme (Partnerships for Reducing Overweight and Obesity with Patient-Centered Strategies 2.0).

INTRODUCTION: There is an urgent need for scalable strategies for treating overweight and obesity in clinical settings. PROPS 2.0 (Partnerships for Reducing Overweight and Obesity with Patient-Centered Strategies 2.0) aims to adapt and implement the combined intervention from the PROPS Study at scale, in a diverse cross-section of patients and providers. METHODS AND ANALYSIS: We are implementing PROPS 2.0 across a variety of clinics at Brigham and Women's Hospital, targeting enrolment of 5000 patients. Providers can refer patients or patients can self-refer. Eligible patients must be ≥20 years old and have a body mass index (BMI) of ≥30 kg/m2 or a BMI of 25-29.9 kg/m2 plus another cardiovascular risk factor or obesity-related condition. After enrolment, patients register for the RestoreHealth online programme/app (HealthFleet Inc.) and participate for 12 months. Patients can engage with the programme and receive personalized feedback from a coach. Patient navigators help to enrol patients, enter updates in the electronic health record, and refer patients to additional resources. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework is guiding the evaluation. ETHICS AND DISSEMINATION: The Mass General Brigham Human Research Committee approved this protocol. An implementation guide will be created and disseminated, to help other sites adopt the intervention in the future. TRIAL REGISTRATION NUMBER: NCT0555925.

Results of a Remotely Delivered Hypertension and Lipid Program in More Than 10 000 Patients Across a Diverse Health Care Network.

Importance: Blood pressure (BP) and cholesterol control remain challenging. Remote care can deliver more effective care outside of traditional clinician-patient settings but scaling and ensuring access to care among diverse populations remains elusive. Objective: To implement and evaluate a remote hypertension and cholesterol management program across a diverse health care network. Design, Setting, and Participants: Between January 2018 and July 2021, 20 454 patients in a large integrated health network were screened; 18 444 were approached, and 10 803 were enrolled in a comprehensive remote hypertension and cholesterol program (3658 patients with hypertension, 8103 patients with cholesterol, and 958 patients with both). A total of 1266 patients requested education only without medication titration. Enrolled patients received education, home BP device integration, and medication titration. Nonlicensed navigators and pharmacists, supported by cardiovascular clinicians, coordinated care using standardized algorithms, task management and automation software, and omnichannel communication. BP and laboratory test results were actively monitored. Main Outcomes and Measures: Changes in BP and low-density lipoprotein cholesterol (LDL-C). Results: The mean (SD) age among 10 803 patients was 65 (11.4) years; 6009 participants (56%) were female; 1321 (12%) identified as Black, 1190 (11%) as Hispanic, 7758 (72%) as White, and 1727 (16%) as another or multiple races (including American Indian or Alaska Native, Asian, Native Hawaiian or Other Pacific Islander, unknown, other, and declined to respond; consolidated owing to small numbers); and 142 (11%) reported a preferred language other than English. A total of 424 482 BP readings and 139 263 laboratory reports were collected. In the hypertension program, the mean (SD) office BP prior to enrollment was 150/83 (18/10) mm Hg, and the mean (SD) home BP was 145/83 (20/12) mm Hg. For those engaged in remote medication management, the mean (SD) clinic BP 6 and 12 months after enrollment decreased by 8.7/3.8 (21.4/12.4) and 9.7/5.2 (22.2/12.6) mm Hg, respectively. In the education-only cohort, BP changed by a mean (SD) -1.5/-0.7 (23.0/11.1) and by +0.2/-1.9 (30.3/11.2) mm Hg, respectively (P < .001 for between cohort difference). In the lipids program, patients in remote medication management experienced a reduction in LDL-C by a mean (SD) 35.4 (43.1) and 37.5 (43.9) mg/dL at 6 and 12 months, respectively, while the education-only cohort experienced a mean (SD) reduction in LDL-C of 9.3 (34.3) and 10.2 (35.5) mg/dL at 6 and 12 months, respectively (P < .001). Similar rates of enrollment and reductions in BP and lipids were observed across different racial, ethnic, and primary language groups. Conclusions and Relevance: The results of this study indicate that a standardized remote BP and cholesterol management program may help optimize guideline-directed therapy at scale, reduce cardiovascular risk, and minimize the need for in-person visits among diverse populations.