Factors Affecting Post-trial Sustainment or De-implementation of Study Interventions: A Narrative Review.

In contrast to traditional randomized controlled trials, embedded pragmatic clinical trials (ePCTs) are conducted within healthcare settings with real-world patient populations. ePCTs are intentionally designed to align with health system priorities leveraging existing healthcare system infrastructure and resources to ease intervention implementation and increase the likelihood that effective interventions translate into routine practice following the trial. The NIH Pragmatic Trials Collaboratory, funded by the National Institutes of Health (NIH), supports the conduct of large-scale ePCT Demonstration Projects that address major public health issues within healthcare systems. The Collaboratory has a unique opportunity to draw on the Demonstration Project experiences to generate lessons learned related to ePCTs and the dissemination and implementation of interventions tested in ePCTs. In this article, we use case studies from six completed Demonstration Projects to summarize the Collaboratory's experience with post-trial interpretation of results, and implications for sustainment (or de-implementation) of tested interventions. We highlight three key lessons learned. First, ineffective interventions (i.e., ePCT is null for the primary outcome) may be sustained if they have other measured benefits (e.g., secondary outcome or subgroup) or even perceived benefits (e.g., staff like the intervention). Second, effective interventions-even those solicited by the health system and/or designed with significant health system partner buy-in-may not be sustained if they require significant resources. Third, alignment with policy incentives is essential for achieving sustainment and scale-up of effective interventions. Our experiences point to several recommendations to aid in considering post-trial sustainment or de-implementation of interventions tested in ePCTs: (1) include secondary outcome measures that are salient to health system partners; (2) collect all appropriate data to allow for post hoc analysis of subgroups; (3) collect experience data from clinicians and staff; (4) engage policy-makers before starting the trial.

Modifications of the readiness assessment for pragmatic trials tool for appropriate use with Indigenous populations.

BACKGROUND: Inequities in health access and outcomes exist between Indigenous and non-Indigenous populations. Embedded pragmatic randomized, controlled trials (ePCTs) can test the real-world effectiveness of health care interventions. Assessing readiness for ePCT, with tools such as the Readiness Assessment for Pragmatic Trials (RAPT) model, is an important component. Although equity must be explicitly incorporated in the design, testing, and widespread implementation of any health care intervention to achieve equity, RAPT does not explicitly consider equity. This study aimed to identify adaptions necessary for the application of the 'Readiness Assessment for Pragmatic Trials' (RAPT) tool in embedded pragmatic randomized, controlled trials (ePCTs) with Indigenous communities. METHODS: We surveyed and interviewed participants (researchers with experience in research involving Indigenous communities) over three phases (July-December 2022) in this mixed-methods study to explore the appropriateness and recommended adaptions of current RAPT domains and to identify new domains that would be appropriate to include. We thematically analyzed responses and used an iterative process to modify RAPT. RESULTS: The 21 participants identified that RAPT needed to be modified to strengthen readiness assessment in Indigenous research. In addition, five new domains were proposed to support Indigenous communities' power within the research processes: Indigenous Data Sovereignty; Acceptability - Indigenous Communities; Risk of Research; Research Team Experience; Established Partnership). We propose a modified tool, RAPT-Indigenous (RAPT-I) for use in research with Indigenous communities to increase the robustness and cultural appropriateness of readiness assessment for ePCT. In addition to producing a tool for use, it outlines a methodological approach to adopting research tools for use in and with Indigenous communities by drawing on the experience of researchers who are part of, and/or working with, Indigenous communities to undertake interventional research, as well as those with expertise in health equity, implementation science, and public health. CONCLUSION: RAPT-I has the potential to provide a useful framework for readiness assessment prior to ePCT in Indigenous communities. RAPT-I also has potential use by bodies charged with critically reviewing proposed pragmatic research including funding and ethics review boards.

Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results From the RCT DUPLICATE Initiative.

BACKGROUND: Regulators are evaluating the use of noninterventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT DUPLICATE initiative (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) uses a structured process to design RWE studies emulating randomized, controlled trials (RCTs) and compare results. We report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications. METHODS: We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion and exclusion criteria, selected primary end points, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 preexposure confounders. All study measures were prospectively defined and protocols registered before hazard ratios and 95% CIs were computed. Success criteria for the primary analysis were prespecified for each replication. RESULTS: Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a hazard ratio estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation. CONCLUSIONS: Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. Registration: URL: https://www.clinicaltrials.gov; Unique identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922.

Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis.

BACKGROUND: Use of patient-reported outcomes (PROs) and patient and public engagement are critical ingredients of pragmatic trials, which are intended to be patient centered. Engagement of patients and members of the public in selecting the primary trial outcome and determining the target difference can better ensure that the trial is designed to inform the decisions of those who ultimately stand to benefit. However, to the best of our knowledge, the use and reporting of PROs and patient and public engagement in pragmatic trials have not been described. The objectives of this study were to review a sample of pragmatic trials to describe (1) the prevalence of reporting patient and public engagement; (2) the prevalence and types of PROs used; (3) how its use varies across trial characteristics; and (4) how sample sizes and target differences are determined for trials with primary PROs. METHODS AND FINDINGS: This was a methodological review of primary reports of pragmatic trials. We used a published electronic search filter in MEDLINE to identify pragmatic trials, published in English between January 1, 2014 and April 3, 2019; we identified the subset that were registered in ClinicalTrials.gov and explicitly labeled as pragmatic. Trial descriptors were downloaded from ClinicalTrials.gov; information about PROs and sample size calculations were extracted from the manuscript. Chi-squared, Cochran-Armitage, and Wilcoxon rank sum tests were used to examine associations between trial characteristics and use of PROs. Among 4,337 identified primary trial reports, 1,988 were registered in CT.gov, of which 415 were explicitly labeled as pragmatic. Use of patient and public engagement was identified in 39 (9.4%). PROs were measured in 235 (56.6%): 144 (34.7%) used PROs as primary outcomes and 91 (21.9%) as only secondary outcomes. Primary PROs were symptoms (64; 44%), health behaviors (36; 25.0%), quality of life (17; 11.8%), functional status (16; 11.1%), and patient experience (10; 6.9%). Trial characteristics with lower prevalence of use of PROs included being conducted exclusively in children or adults over age 65 years, cluster randomization, recruitment in low- and middle-income countries, and primary purpose of prevention; trials conducted in Europe had the highest prevalence of PROs. For the 144 trials with a primary PRO, 117 (81.3%) reported a sample size calculation for that outcome; of these, 71 (60.7%) justified the choice of target difference, most commonly, using estimates from pilot studies (31; 26.5%), standardized effect sizes (20; 17.1%), or evidence reviews (16; 13.7%); patient or stakeholder opinions were used to justify the target difference in 8 (6.8%). Limitations of this study are the need for trials to be registered in ClinicalTrials.gov, which may have reduced generalizability, and extracting information only from the primary trial report. CONCLUSIONS: In this study, we observed that pragmatic trials rarely report patient and public engagement and do not commonly use PROs as primary outcomes. When provided, target differences are often not justified and rarely informed by patients and stakeholders. Research funders, scientific journals, and institutions should support trialists to incorporate patient engagement to fulfill the mandate of pragmatic trials to be patient centered.

Methodological challenges in pragmatic trials in Alzheimer's disease and related dementias: Opportunities for improvement.

BACKGROUND AND AIMS: We need more pragmatic trials of interventions to improve care and outcomes for people living with Alzheimer's disease and related dementias. However, these trials present unique methodological challenges in their design, analysis, and reporting-often, due to the presence of one or more sources of clustering. Failure to account for clustering in the design and analysis can lead to increased risks of Type I and Type II errors. We conducted a review to describe key methodological characteristics and obtain a "baseline assessment" of methodological quality of pragmatic trials in dementia research, with a view to developing new methods and practical guidance to support investigators and methodologists conducting pragmatic trials in this field. METHODS: We used a published search filter in MEDLINE to identify trials more likely to be pragmatic and identified a subset that focused on people living with Alzheimer's disease or other dementias or included them as a defined subgroup. Pairs of reviewers extracted descriptive information and key methodological quality indicators from each trial. RESULTS: We identified N = 62 eligible primary trial reports published across 36 different journals. There were 15 (24%) individually randomized, 38 (61%) cluster randomized, and 9 (15%) individually randomized group treatment designs; 54 (87%) trials used repeated measures on the same individual and/or cluster over time and 17 (27%) had a multivariate primary outcome (e.g. due to measuring an outcome on both the patient and their caregiver). Of the 38 cluster randomized trials, 16 (42%) did not report sample size calculations accounting for the intracluster correlation and 13 (34%) did not account for intracluster correlation in the analysis. Of the 9 individually randomized group treatment trials, 6 (67%) did not report sample size calculations accounting for intracluster correlation and 8 (89%) did not account for it in the analysis. Of the 54 trials with repeated measurements, 45 (83%) did not report sample size calculations accounting for repeated measurements and 19 (35%) did not utilize at least some of the repeated measures in the analysis. No trials accounted for the multivariate nature of their primary outcomes in sample size calculation; only one did so in the analysis. CONCLUSION: There is a need and opportunity to improve the design, analysis, and reporting of pragmatic trials in dementia research. Investigators should pay attention to the potential presence of one or more sources of clustering. While methods for longitudinal and cluster randomized trials are well developed, accessible resources and new methods for dealing with multiple sources of clustering are required. Involvement of a statistician with expertise in longitudinal and clustered designs is recommended.

Plasma trial: Pilot randomized clinical trial to determine safety and efficacy of plasma transfusions.

BACKGROUND: Plasma is frequently administered to patients with prolonged INR prior to invasive procedures. However, there is limited evidence evaluating efficacy and safety. STUDY DESIGN AND METHODS: We performed a pilot trial in hospitalized patients with INR between 1.5 and 2.5 undergoing procedures conducted outside the operating room. We excluded patients undergoing procedures proximal to the central nervous system, platelet counts <40,000/µl, or congenital or acquired coagulation disorders unresponsive to plasma. We randomly allocated patients stratified by hospital and history of cirrhosis to receive plasma transfusion (10-15 cc/kg) or no transfusion. The primary outcome was change in hemoglobin concentration within 2 days of procedure. RESULTS: We enrolled 57 patients, mean age 56.0, 34 (59.6%) with cirrhosis, and mean INR 1.92 (SD = 0.27). In the intention to treat analysis, there were 10 of 27 (38.5%) participants in the plasma arm with a post procedure INR <1.5 and one of 30 (3.6%) in the no treatment arm (p < .01). The mean INR after receiving plasma transfusion was -0.24 (SD 0.26) lower than baseline. The change from pre-procedure hemoglobin level to lowest level within 2 days was -0.6 (SD = 1.0) in the plasma transfusion arm and -0.4 (SD = 0.6) in the no transfusion arm (p = .29). Adverse outcomes were uncommon. DISCUSSION: We found no differences in change in hemoglobin concentration in those treated with plasma compared to no treatment. The change in INR was small and corrected to less than 1.5 in minority of patients. Large trials are required to establish if plasma is safe and efficacious.

Pragmatic patient engagement in designing pragmatic oncology clinical trials.

Oncology trials are the cornerstone of effective and safe therapeutic discoveries. However, there is increasing demand for pragmatism and patient engagement in the design, implementation and dissemination of oncology trials. Many researchers are uncertain about making trials more practical and even less knowledgeable about how to meaningfully engage patients without compromising scientific rigor to meet regulatory requirements. The present work provides practical guidance for addressing both pragmaticism and meaningful patient engagement. Applying evidence-based approaches like PRECIS-2-tool and the 10-Step Engagement Framework offer practical guidance to make future trials in oncology truly pragmatic and patient-centered. Consequently, such patient-centered trials have improved participation, faster recruitment and greater retention, and uptake of innovative technologies in community-based care.

The use of community advisory boards in pragmatic clinical trials: The case of the adult day services plus project.

Community advisory boards (CABs) have become increasingly common and important in translational research in health care including studies focusing on home and community-based services. CABs are composed of stakeholders who share interest in research projects and typically include patients/clients, practitioners, community members, policymakers, and researchers. CABs advise researchers on issues ranging from research design and recruitment to implementation and dissemination. In this article, the researchers detail their experiences with the CAB for a pragmatic clinical trail of Adult Day Services (ADS) Plus, an education and support intervention for family caregivers of older adults with dementia using adult day services. Lessons learned, guidelines, and best practices are then presented for developing and working with a CAB in healthcare research.

Real-world effectiveness of BNT162b2 mRNA vaccine: a meta-analysis of large observational studies.

This paper aims to summarize through meta-analyses the overall vaccine effectiveness of the BNT162b2 mRNA vaccine from observational studies. A systematic literature search with no language restriction was performed in electronic databases to identify eligible observational studies which reported the adjusted effectiveness of the BNT162b2 mRNA vaccine to prevent RT-PCR confirmed COVID-19. Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR) and pooled incidence rate ratio (IRR) at 95% confidence intervals, and the vaccine effectiveness was indicated as (pooled HR - 1)/HR or (pooled IRR - 1)/IRR. Nineteen studies were included for this meta-analysis. The meta-analysis revealed significant protective effect against RT-PCR confirmed COVID-19 ≥ 14 days after the first dose, with vaccine effectiveness of 53% (95% confidence interval 32-68%), and ≥ 7 days after the second dose, with vaccine effectiveness of 95% (95% confidence interval: 96-97%). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval. There have been limited data about vaccine effectiveness among immunocompromised patients; thus, the vaccine should be used cautiously in this patient population.

The RICH LIFE Project: A cluster randomized pragmatic trial comparing the effectiveness of health system only vs. health system Plus a collaborative/stepped care intervention to reduce hypertension disparities.

Disparities in the control of hypertension and other cardiovascular disease risk factors are well-documented in the United States, even among patients seen regularly in the healthcare system. Few existing approaches explicitly address disparities in hypertension care and control. This paper describes the RICH LIFE Project (Reducing Inequities in Care of Hypertension: Lifestyle Improvement for Everyone) design. METHODS: RICH LIFE is a two-arm, cluster-randomized trial, comparing the effectiveness of enhanced standard of care, "Standard of Care Plus" (SCP), to a multi-level intervention, "Collaborative Care/Stepped Care" (CC/SC), for improving blood pressure (BP) control and patient activation and reducing disparities in BP control among 1890 adults with uncontrolled hypertension and at least one other cardiovascular disease risk factor treated at 30 primary care practices in Maryland and Pennsylvania. Fifteen practices randomized to the SCP arm receive standardized BP measurement training; race/ethnicity-specific audit and feedback of BP control rates; and quarterly webinars in management practices, quality improvement and disparities reduction. Fifteen practices in the CC/SC arm receive the SCP interventions plus implementation of the collaborative care model with stepped-care components (community health worker referrals and virtual specialist-panel consults). The primary clinical outcome is BP control (<140/90 mm Hg) at 12 months. The primary patient-reported outcome is change from baseline in self-reported patient activation at 12 months. DISCUSSION: This study will provide knowledge about the feasibility of leveraging existing resources in routine primary care and potential benefits of adding supportive community-facing roles to improve hypertension care and reduce disparities. TRIAL REGISTRATION: Clinicaltrials.govNCT02674464.

Methodological considerations when analysing and interpreting real-world data.

In the absence of relevant data from randomized trials, nonexperimental studies are needed to estimate treatment effects on clinically meaningful outcomes. State-of-the-art study design is imperative for minimizing the potential for bias when using large healthcare databases (e.g. claims data, electronic health records, and product/disease registries). Critical design elements include new-users (begin follow-up at treatment initiation) reflecting hypothetical interventions and clear timelines, active-comparators (comparing treatment alternatives for the same indication), and consideration of induction and latent periods. Propensity scores can be used to balance measured covariates between treatment regimens and thus control for measured confounding. Immortal-time bias can be avoided by defining initiation of therapy and follow-up consistently between treatment groups. The aim of this manuscript is to provide a non-technical overview of study design issues and solutions and to highlight the importance of study design to minimize bias in nonexperimental studies using real-world data.

What does digitalization hold for the creation of real-world evidence?

Health-related information is increasingly being collected and stored digitally. These data, either structured or unstructured, are becoming the ubiquitous assets that might enable us to comprehensively map out a patient's health journey from an asymptomatic state of wellness to disease onset and its trajectory. These new data could provide rich real-world evidence for better clinical care and research, if they can be accessed, linked and analyzed-all of which are possible. In this review, these opportunities will be explored through a case vignette of a patient with OA, followed by discussion on how this digitalized real-world evidence could best be utilized, as well as the challenges of data access, quality and maintaining public trust.

Design and rationale of a multi-center, pragmatic, open-label randomized trial of antimicrobial therapy - the study of clinical efficacy of antimicrobial therapy strategy using pragmatic design in Idiopathic Pulmonary Fibrosis (CleanUP-IPF) clinical trial.

Compelling data have linked disease progression in patients with idiopathic pulmonary fibrosis (IPF) with lung dysbiosis and the resulting dysregulated local and systemic immune response. Moreover, prior therapeutic trials have suggested improved outcomes in these patients treated with either sulfamethoxazole/ trimethoprim or doxycycline. These trials have been limited by methodological concerns. This trial addresses the primary hypothesis that long-term treatment with antimicrobial therapy increases the time-to-event endpoint of respiratory hospitalization or all-cause mortality compared to usual care treatment in patients with IPF. We invoke numerous innovative features to achieve this goal, including: 1) utilizing a pragmatic randomized trial design; 2) collecting targeted biological samples to allow future exploration of 'personalized' therapy; and 3) developing a strong partnership between the NHLBI, a broad range of investigators, industry, and philanthropic organizations. The trial will randomize approximately 500 individuals in a 1:1 ratio to either antimicrobial therapy or usual care. The site principal investigator will declare their preferred initial antimicrobial treatment strategy (trimethoprim 160 mg/ sulfamethoxazole 800 mg twice a day plus folic acid 5 mg daily or doxycycline 100 mg once daily if body weight is < 50 kg or 100 mg twice daily if ≥50 kg) for the participant prior to randomization. Participants randomized to antimicrobial therapy will receive a voucher to help cover the additional prescription drug costs. Additionally, those participants will have 4-5 scheduled blood draws over the initial 24 months of therapy for safety monitoring. Blood sampling for DNA sequencing and genome wide transcriptomics will be collected before therapy. Blood sampling for transcriptomics and oral and fecal swabs for determination of the microbiome communities will be collected before and after study completion. As a pragmatic study, participants in both treatment arms will have limited in-person visits with the enrolling clinical center. Visits are limited to assessments of lung function and other clinical parameters at time points prior to randomization and at months 12, 24, and 36. All participants will be followed until the study completion for the assessment of clinical endpoints related to hospitalization and mortality events. TRIAL REGISTRATION: ClinicalTrials.gov identifier NCT02759120.

Ixazomib-based regimens for relapsed/refractory multiple myeloma: are real-world data compatible with clinical trial outcomes? A multi-site Israeli registry study.

Ixazomib, the first oral proteasome inhibitor (PI), has been approved for the treatment of relapsed refractory multiple myeloma (RRMM) in combination with lenalidomide and dexamethasone, based on the TOURMALINE-MM1 phase 3 trial, which demonstrated the efficacy and safety of this all-oral triplet, compared with lenalidomide-dexamethasone. However, clinical trial outcomes do not always translate into real-world outcomes. The aim of this study was to assess the outcomes of ixazomib-based combination for treatment of patients with RRMM in a real-world setting. All consecutive RRMM patients who received at least one cycle of ixazomib-based treatment combination between June 2013 and June 2018 were identified. Data was extracted from medical charts focusing on demographics, disease characteristics, prior treatment, and responses. Primary endpoint was progression-free survival (PFS); secondary endpoints included overall response rate (ORR), overall survival (OS), safety, and tolerability. A total of 78 patients across 7 sites were retrospectively included. Median follow-up was 22 months. Median age was 68 (range 38-90). Sixty-four percent received ixazomib in 2nd line, 19% in 3rd line. Overall, 89% of patients had been exposed to PIs (bortezomib 87%) prior to IRd, 41% to IMiDs. Twenty-nine (48%, of 60 available) had high (t(4:14), t(14:16), del17p) or intermediate (+1q21) risk aberrations. Most patients (82%) received ixazomib in combination with lenalidomide and dexamethasone. An exploratory assessment for disease aggressiveness at diagnosis was classified by a treating physician as indolent (rapid control to protect from target organ damage not required) vs aggressive (imminent target organ damage) in 63% vs 37%, respectively. Treatment was well tolerated, with a low discontinuation rate (11%). Median PFS on ixazomib therapy was 24 months (95% CI 17-30). PFS was 77% and 47% at 12 and 24 months, respectively. Median OS was not reached; OS was 91% and 80% at 12 and 24 months, respectively. Higher LDH, older age, and worse clinical aggressiveness were associated with worse PFS, whereas a deeper response to ixazomib (≥ VGPR) and a longer response to first-line bortezomib (≥ 24 m) were associated with an improved PFS on ixazomib. No effect on PFS was found for cytogenetic risk by FISH, ISS/rISS, and prior anti-myeloma treatment. Ixazomib-based combinations are efficacious and safe regimens in RRMM patients in the real-world setting, regardless to cytogenetic risk, with a PFS of 24 months comparable with clinical trial data. This regimen had most favorable outcomes among patients who remained progression-free more than 24 months after a bortezomib induction and for those who have a more indolent disease phenotype.

Use of Real-World Evidence to Support FDA Approval of Oncology Drugs.

OBJECTIVES: Real-world evidence (RWE) has gained increased attention in recent years as a complement to traditional clinical trials. The use of RWE to establish the efficacy of oncology drugs for Food and Drug Administration (FDA) approval has not been described. In this paper, we review 5 recent examples where RWE was submitted in support of the FDA approvals of original or supplementary indications for oncology drugs. METHODS: To identify cases where RWE was used, we reviewed drug approval packages available at Drugs@FDA for oncology drugs approved between 2017 and 2019. Five cases were selected to present a broad overview of different types of RWE, different circumstances under which RWE has been used for regulatory approvals, and how FDA evaluated the data in each case. The type of RWE submitted, the indication, limitations identified by FDA reviewers, and the outcome of the submission are discussed. RESULTS: RWE, particularly historical controls for rare or orphan indications, has been used to support both original and supplementary oncology drug approvals. Types of RWE included data from electronic health records, claims, post-marketing safety reports, retrospective medical record reviews, and expanded access studies. Small sample sizes, data quality, and methodological issues were among concerns cited by FDA reviewers. CONCLUSION: By bridging the gap between the constraints of the trial setting and the realities of clinical practice, RWE can add value to a regulatory submission. These early examples provide insight into how regulators evaluated RWE submitted as evidence of efficacy for oncology drugs.

Real-world evidence to support regulatory decision-making for medicines: Considerations for external control arms.

Randomized clinical trials (RCTs) are the gold standard in producing clinical evidence of efficacy and safety of medical interventions. More recently, a new paradigm is emerging-specifically within the context of preauthorization regulatory decision-making-for some novel uses of real-world evidence (RWE) from a variety of real-world data (RWD) sources to answer certain clinical questions. Traditionally reserved for rare diseases and other special circumstances, external controls (eg, historical controls) are recognized as a possible type of control arm for single-arm trials. However, creating and analyzing an external control arm using RWD can be challenging since design and analytics may not fully control for all systematic differences (biases). Nonetheless, certain biases can be attenuated using appropriate design and analytical approaches. The main objective of this paper is to improve the scientific rigor in the generation of external control arms using RWD. Here we (a) discuss the rationale and regulatory circumstances appropriate for external control arms, (b) define different types of external control arms, and (c) describe study design elements and approaches to mitigate certain biases in external control arms. This manuscript received endorsement from the International Society for Pharmacoepidemiology (ISPE).

Trial designs using real-world data: The changing landscape of the regulatory approval process.

PURPOSE: There is a need to develop hybrid trial methodology combining the best parts of traditional randomized controlled trials (RCTs) and observational study designs to produce real-world evidence (RWE) that provides adequate scientific evidence for regulatory decision-making. METHODS: This review explores how hybrid study designs that include features of RCTs and studies with real-world data (RWD) can combine the advantages of both to generate RWE that is fit for regulatory purposes. RESULTS: Some hybrid designs include randomization and use pragmatic outcomes; other designs use single-arm trial data supplemented with external comparators derived from RWD or leverage novel data collection approaches to capture long-term outcomes in a real-world setting. Some of these approaches have already been successfully used in regulatory decisions, raising the possibility that studies using RWD could increasingly be used to augment or replace traditional RCTs for the demonstration of drug effectiveness in certain contexts. These changes come against a background of long reliance on RCTs for regulatory decision-making, which are labor-intensive, costly, and produce data that can have limited applicability in real-world clinical practice. CONCLUSIONS: While RWE from observational studies is well accepted for satisfying postapproval safety monitoring requirements, it has not commonly been used to demonstrate drug effectiveness for regulatory purposes. However, this position is changing as regulatory opinions, guidance frameworks, and RWD methodologies are evolving, with growing recognition of the value of using RWE that is acceptable for regulatory decision-making.

From screening to ascertainment of the primary outcome using electronic health records: Challenges in the STRIDE trial.

Electronic health record data are a rich resource and can be utilized to answer a wealth of research questions. It is important when using electronic health record data in clinical trials that systems be put in place and vetted prior to enrollment to ensure data elements can be collected consistently across all health care systems. It is often overlooked how something conceptualized on paper (e.g. use of the electronic health record in a study) can be difficult to implement in practice. This article discusses some of the challenges in using electronic health records in the conduct of the STRIDE (Strategies to Reduce Injuries and Develop Confidence in Elders) trial, how we handled those challenges, and the lessons we learned for the conduct of future trials looking to employ the electronic health record.

Five analytic challenges in working with electronic health records data to support clinical trials with some solutions.

Electronic health records data are becoming a key data resource in clinical research. Owing to issues of data efficiency, electronic health records data are being used for clinical trials. This includes both large-scale pragmatic trails and smaller-more focused-point-of-care trials. While electronic health records data open up a number of scientific opportunities, they also present a number of analytic challenges. This article discusses five particular challenges related to organizing electronic health records data for analytic purposes. These are as follows: (1) data are not organized for research purposes, (2) data are both densely and irregularly observed, (3) we don't have all data elements we may want or need, (4) data are both cross-sectional and longitudinal, and (5) data may be informatively observed. While laying out these challenges, the article notes how many of these challenges can be addressed by careful and thoughtful study design as well as by integration of clinicians and informaticians into the analytic team.

Role of health plan administrative claims data in participant recruitment for pragmatic clinical trials: An Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) example.

AIM: The purpose of this study is to evaluate HealthCore/Anthem Research Network recruitment strategies, compare response and enrollment rates for different recruitment strategies, and describe demographic and clinical characteristics of responders and enrollees. METHODS: HealthCore/Anthem Research Network, a part of the Health Plan Research Network of the Patient-Centered Clinical Data Research Network, used administrative claims data to identify eligible health plan members for potential participation in the Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness study. We approached health plan members, identified with a validated Patient-Centered Clinical Data Research Network common data model computable phenotype, and their clinical providers during November 2017 to August 2018. Providers were offered the option to exclude their patients' participation in Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness prior to our direct patient (member) outreach. Member identification was in two phases: Phase 1: 1 January 2006 to 1 April 2017, and Phase 2: 1 January 2006 to 2 February 2018. Phase 1 consisted of two batches of mail and one phone call per patient. In Phase 2, which included two similar batches of patients, outreach was via either mail or brochure and one phone call. RESULTS: Phase 1 and Phase 2 included 133,373 and 51,777 members, respectively. We engaged 28,593 providers in Phase 1, and 5077 in Phase 2. In Phase 1, 264,158 mixed email/mail messages were delivered to 133,373 members, followed by 90,481 phone calls from November 2017 to February 2018. In Phase 2, after simple randomization to letter or brochure, 51,777 members were sent email/mail or mailed brochure in three waves from May 2018 to July 2018. In this 9-week period, 51,623 communications were sent to 25,914 members in the email/mail group, and 50,160 brochures to 25,863 in the brochure group. Following email/mail or mailed brochure outreach, 16,624 and 16,580 calls were made to the groups, respectively. Overall, 1549 health plan members visited the study portal by 1 September 2018; 355 electronically signed the Informed Consent Form and enrolled. Mailed brochures drove more portal visits in Phase 2, but a lower percentage of responders enrolled. Recruitment was better in Phase 2-2.3 enrollees per 1000 outreach members versus 1.8 in Phase 1. CONCLUSION: This study showed the ability of a health plan within Patient-Centered Clinical Data Research Network to identify potential study participants with administrative claims, and use different outreach methods to facilitate recruitment and enrollment for pragmatic clinical trials.