Pragmaticism in Cancer Clinical Trials.

Clinical trials are essential for advancing oncology treatment strategies and have contributed significantly to the decline in cancer mortality rates over the past decades. Traditional explanatory trials, focused on establishing intervention efficacy in ideal settings, often lack generalizability and may not reflect real-world patient care scenarios. Furthermore, increasing complexity in cancer clinical trial design has led to challenges such as protocol deviations, slow enrollment leading to lengthened durations of trial, and escalating costs. By contrast, pragmatic trials aim to assess intervention effectiveness in more representative patient populations under routine clinical conditions. Here, we review the principles, methodologies, challenges, and advantages of incorporating pragmatic features (PFs) into cancer clinical trials. We illustrate the application of pragmatic trial designs in oncology and discuss the QUASAR collaborative, TAPUR study, and the ongoing PRAGMATICA-LUNG trial. Although not all oncology trials may be amenable to adopting fully pragmatic designs, integration of PFs when feasible will enhance trial generalizability and real-world applicability. Project Pragmatica and similar initiatives advocate for the integration of real-world practice with clinical trials, fostering a nuanced approach to oncology research that balances efficacy and effectiveness assessments, ultimately with a goal of improving patient outcomes.

Factors Affecting Post-trial Sustainment or De-implementation of Study Interventions: A Narrative Review.

In contrast to traditional randomized controlled trials, embedded pragmatic clinical trials (ePCTs) are conducted within healthcare settings with real-world patient populations. ePCTs are intentionally designed to align with health system priorities leveraging existing healthcare system infrastructure and resources to ease intervention implementation and increase the likelihood that effective interventions translate into routine practice following the trial. The NIH Pragmatic Trials Collaboratory, funded by the National Institutes of Health (NIH), supports the conduct of large-scale ePCT Demonstration Projects that address major public health issues within healthcare systems. The Collaboratory has a unique opportunity to draw on the Demonstration Project experiences to generate lessons learned related to ePCTs and the dissemination and implementation of interventions tested in ePCTs. In this article, we use case studies from six completed Demonstration Projects to summarize the Collaboratory's experience with post-trial interpretation of results, and implications for sustainment (or de-implementation) of tested interventions. We highlight three key lessons learned. First, ineffective interventions (i.e., ePCT is null for the primary outcome) may be sustained if they have other measured benefits (e.g., secondary outcome or subgroup) or even perceived benefits (e.g., staff like the intervention). Second, effective interventions-even those solicited by the health system and/or designed with significant health system partner buy-in-may not be sustained if they require significant resources. Third, alignment with policy incentives is essential for achieving sustainment and scale-up of effective interventions. Our experiences point to several recommendations to aid in considering post-trial sustainment or de-implementation of interventions tested in ePCTs: (1) include secondary outcome measures that are salient to health system partners; (2) collect all appropriate data to allow for post hoc analysis of subgroups; (3) collect experience data from clinicians and staff; (4) engage policy-makers before starting the trial.

Rationale and design of the THIRST Alert feasibility study: a pragmatic, single-centre, parallel-group randomised controlled trial of an interruptive alert for oral fluid restriction in patients treated with intravenous furosemide.

INTRODUCTION: Acute heart failure (HF) is a major cause of unplanned hospitalisation characterised by excess body water. A restriction in oral fluid intake is commonly imposed on patients as an adjunct to pharmacological therapy with loop diuretics, but there is a lack of evidence from traditional randomised controlled trials (RCTs) to support the safety and effectiveness of this intervention in the acute setting.This study aims to explore the feasibility of using computer alerts within the electronic health record (EHR) system to invite clinical care teams to enrol patients into a pragmatic RCT at the time of clinical decision-making. It will additionally assess the effectiveness of using an alert to help address the clinical research question of whether oral fluid restriction is a safe and effective adjunct to pharmacological therapy for patients admitted with fluid overload. METHODS AND ANALYSIS: THIRST (Randomised Controlled Trial within the electronic Health record of an Interruptive alert displaying a fluid Restriction Suggestion in patients with the treatable Trait of congestion) Alert is a single-centre, parallel-group, open-label pragmatic RCT embedded in the EHR system that will be conducted as a feasibility study at an National Health Service (NHS) hospital in London. The clinical care team will be invited to enrol suitable patients in the study using a point-of-care alert with a target sample size of 50 patients. Enrolled patients will then be randomised to either restricted or unrestricted oral fluid intake. Two primary outcomes will be explored (1) the proportion of eligible patients enrolled in the study and (2) the mean difference in oral fluid intake between randomised groups. A series of secondary outcomes are specified to evaluate the effectiveness of the alert, adherence to the randomised treatment allocation and the quality of data generated from routine care, relevant to the outcomes of interest. ETHICS AND DISSEMINATION: This study was approved by Riverside Research Ethics Committee (Ref: 22/LO/0889) and will be published on completion. TRIAL REGISTRATION NUMBER: NCT05869656.

Iterative approaches to the use of electronic health records data for large pragmatic studies.

Randomized controlled trials (RCT) are the gold standard for evaluating the effectiveness and safety of interventions and treatments, yet traditional clinical trials have relied on cumbersome and redundant processes such as electronic data entry which involves manual abstraction of already available electronic health record (EHR) data. This review focuses on the opportunities to expand the use of EHR data for pragmatic clinical trials using methods and lessons learned from the Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-Term Effectiveness (ADAPTABLE) study, the demonstration project from PCORnet® (the National Patient-Centered Clinical Research Network).

Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis.

BACKGROUND: Use of patient-reported outcomes (PROs) and patient and public engagement are critical ingredients of pragmatic trials, which are intended to be patient centered. Engagement of patients and members of the public in selecting the primary trial outcome and determining the target difference can better ensure that the trial is designed to inform the decisions of those who ultimately stand to benefit. However, to the best of our knowledge, the use and reporting of PROs and patient and public engagement in pragmatic trials have not been described. The objectives of this study were to review a sample of pragmatic trials to describe (1) the prevalence of reporting patient and public engagement; (2) the prevalence and types of PROs used; (3) how its use varies across trial characteristics; and (4) how sample sizes and target differences are determined for trials with primary PROs. METHODS AND FINDINGS: This was a methodological review of primary reports of pragmatic trials. We used a published electronic search filter in MEDLINE to identify pragmatic trials, published in English between January 1, 2014 and April 3, 2019; we identified the subset that were registered in ClinicalTrials.gov and explicitly labeled as pragmatic. Trial descriptors were downloaded from ClinicalTrials.gov; information about PROs and sample size calculations were extracted from the manuscript. Chi-squared, Cochran-Armitage, and Wilcoxon rank sum tests were used to examine associations between trial characteristics and use of PROs. Among 4,337 identified primary trial reports, 1,988 were registered in CT.gov, of which 415 were explicitly labeled as pragmatic. Use of patient and public engagement was identified in 39 (9.4%). PROs were measured in 235 (56.6%): 144 (34.7%) used PROs as primary outcomes and 91 (21.9%) as only secondary outcomes. Primary PROs were symptoms (64; 44%), health behaviors (36; 25.0%), quality of life (17; 11.8%), functional status (16; 11.1%), and patient experience (10; 6.9%). Trial characteristics with lower prevalence of use of PROs included being conducted exclusively in children or adults over age 65 years, cluster randomization, recruitment in low- and middle-income countries, and primary purpose of prevention; trials conducted in Europe had the highest prevalence of PROs. For the 144 trials with a primary PRO, 117 (81.3%) reported a sample size calculation for that outcome; of these, 71 (60.7%) justified the choice of target difference, most commonly, using estimates from pilot studies (31; 26.5%), standardized effect sizes (20; 17.1%), or evidence reviews (16; 13.7%); patient or stakeholder opinions were used to justify the target difference in 8 (6.8%). Limitations of this study are the need for trials to be registered in ClinicalTrials.gov, which may have reduced generalizability, and extracting information only from the primary trial report. CONCLUSIONS: In this study, we observed that pragmatic trials rarely report patient and public engagement and do not commonly use PROs as primary outcomes. When provided, target differences are often not justified and rarely informed by patients and stakeholders. Research funders, scientific journals, and institutions should support trialists to incorporate patient engagement to fulfill the mandate of pragmatic trials to be patient centered.

When Are Treatment Blinding and Treatment Standardization Necessary in Real-World Clinical Trials?

Concerns regarding both the limited generalizability and the slow pace of traditional randomized trials have led to calls for greater use of real-world evidence in the evaluation of new treatments or products. Real-world clinical trials or pragmatic trials often differ from traditional clinical trials in the use of open-label or nonblinded treatments delivered by real-world clinicians in community practice settings. Blinding and standardization of treatment may sometimes be necessary for internal validity, but they may also obscure or distort meaningful differences between treatments. When investigators consider whether blinding of clinicians, patients, or assessors is necessary, we suggest they consider several specific questions: Will clinicians, patients, and assessors have expectations or preferences regarding benefits or adverse effects? How might those expectations affect treatment uptake, treatment adherence, or assessment of outcomes? Will expectations differ in the settings where trial results will be applied? How would blinding of treatment reduce biases? How would blinding obscure true differences between treatments? How would procedures necessary for blinding reduce acceptability or increase risk of trial participation? When investigators consider how strictly treatments should be standardized, we suggest they consider several specific questions: How would treatment effectiveness or safety vary according to clinician experience or expertise? What level of experience or expertise is available in potential trial settings and settings where trial results would be applied? Is some level of standardization necessary for valid inference? Considering any special vulnerabilities of the study population, is some level of standardization necessary to assure participant safety?

Methodological challenges in pragmatic trials in Alzheimer's disease and related dementias: Opportunities for improvement.

BACKGROUND AND AIMS: We need more pragmatic trials of interventions to improve care and outcomes for people living with Alzheimer's disease and related dementias. However, these trials present unique methodological challenges in their design, analysis, and reporting-often, due to the presence of one or more sources of clustering. Failure to account for clustering in the design and analysis can lead to increased risks of Type I and Type II errors. We conducted a review to describe key methodological characteristics and obtain a "baseline assessment" of methodological quality of pragmatic trials in dementia research, with a view to developing new methods and practical guidance to support investigators and methodologists conducting pragmatic trials in this field. METHODS: We used a published search filter in MEDLINE to identify trials more likely to be pragmatic and identified a subset that focused on people living with Alzheimer's disease or other dementias or included them as a defined subgroup. Pairs of reviewers extracted descriptive information and key methodological quality indicators from each trial. RESULTS: We identified N = 62 eligible primary trial reports published across 36 different journals. There were 15 (24%) individually randomized, 38 (61%) cluster randomized, and 9 (15%) individually randomized group treatment designs; 54 (87%) trials used repeated measures on the same individual and/or cluster over time and 17 (27%) had a multivariate primary outcome (e.g. due to measuring an outcome on both the patient and their caregiver). Of the 38 cluster randomized trials, 16 (42%) did not report sample size calculations accounting for the intracluster correlation and 13 (34%) did not account for intracluster correlation in the analysis. Of the 9 individually randomized group treatment trials, 6 (67%) did not report sample size calculations accounting for intracluster correlation and 8 (89%) did not account for it in the analysis. Of the 54 trials with repeated measurements, 45 (83%) did not report sample size calculations accounting for repeated measurements and 19 (35%) did not utilize at least some of the repeated measures in the analysis. No trials accounted for the multivariate nature of their primary outcomes in sample size calculation; only one did so in the analysis. CONCLUSION: There is a need and opportunity to improve the design, analysis, and reporting of pragmatic trials in dementia research. Investigators should pay attention to the potential presence of one or more sources of clustering. While methods for longitudinal and cluster randomized trials are well developed, accessible resources and new methods for dealing with multiple sources of clustering are required. Involvement of a statistician with expertise in longitudinal and clustered designs is recommended.

Pragmatic patient engagement in designing pragmatic oncology clinical trials.

Oncology trials are the cornerstone of effective and safe therapeutic discoveries. However, there is increasing demand for pragmatism and patient engagement in the design, implementation and dissemination of oncology trials. Many researchers are uncertain about making trials more practical and even less knowledgeable about how to meaningfully engage patients without compromising scientific rigor to meet regulatory requirements. The present work provides practical guidance for addressing both pragmaticism and meaningful patient engagement. Applying evidence-based approaches like PRECIS-2-tool and the 10-Step Engagement Framework offer practical guidance to make future trials in oncology truly pragmatic and patient-centered. Consequently, such patient-centered trials have improved participation, faster recruitment and greater retention, and uptake of innovative technologies in community-based care.

Lessons Learned Using Real-World Data to Emulate Randomized Trials: A Case Study of Treatment Effectiveness for Newly Diagnosed Immune Thrombocytopenia.

Regulatory agencies are increasingly considering real-world evidence (RWE) to support label expansions of approved medicines. We conducted a comparative effectiveness study to emulate a proposed randomized trial of romiplostim vs. standard-of-care (SOC) therapy among patients with recently diagnosed (≤12 months) immune thrombocytopenia (ITP), that could support expansion of the romiplostim label. We discuss challenges that we encountered and solutions that were developed to address those challenges. Study size was a primary concern, particularly for romiplostim initiators, given the rarity of ITP and the stringent trial eligibility criteria. For this reason, we leveraged multiple data sources (Nordic Country Patient Registry for Romiplostim; chart review study of romiplostim initiators in Europe; Flatiron Health EMR linked with MarketScan claims). Additionally, unlike the strictly controlled clinical trial setting, platelet counts were not measured at regular intervals in the observational data sources, and therefore the end point of durable platelet response often used in trials could not be reliably measured. Instead, the median platelet count was chosen as the primary end point. Ultimately, while we observed a slightly higher median platelet count in the romiplostim group vs. SOC, precision was limited because of small study size (median difference was 11 × 109 /L (95% CI: -59, 81)). We underscore the importance of conducting comprehensive feasibility assessments to identify fit-for-purpose data sources with sufficient sample size, data elements, and follow-up. Beyond technical challenges, we also discuss approaches to increase the credibility of RWE, including systematic incorporation of clinical expertise into study design decisions, and separation between decision makers and the data.

Real-world effectiveness of BNT162b2 mRNA vaccine: a meta-analysis of large observational studies.

This paper aims to summarize through meta-analyses the overall vaccine effectiveness of the BNT162b2 mRNA vaccine from observational studies. A systematic literature search with no language restriction was performed in electronic databases to identify eligible observational studies which reported the adjusted effectiveness of the BNT162b2 mRNA vaccine to prevent RT-PCR confirmed COVID-19. Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR) and pooled incidence rate ratio (IRR) at 95% confidence intervals, and the vaccine effectiveness was indicated as (pooled HR - 1)/HR or (pooled IRR - 1)/IRR. Nineteen studies were included for this meta-analysis. The meta-analysis revealed significant protective effect against RT-PCR confirmed COVID-19 ≥ 14 days after the first dose, with vaccine effectiveness of 53% (95% confidence interval 32-68%), and ≥ 7 days after the second dose, with vaccine effectiveness of 95% (95% confidence interval: 96-97%). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval. There have been limited data about vaccine effectiveness among immunocompromised patients; thus, the vaccine should be used cautiously in this patient population.

Plasma trial: Pilot randomized clinical trial to determine safety and efficacy of plasma transfusions.

BACKGROUND: Plasma is frequently administered to patients with prolonged INR prior to invasive procedures. However, there is limited evidence evaluating efficacy and safety. STUDY DESIGN AND METHODS: We performed a pilot trial in hospitalized patients with INR between 1.5 and 2.5 undergoing procedures conducted outside the operating room. We excluded patients undergoing procedures proximal to the central nervous system, platelet counts <40,000/µl, or congenital or acquired coagulation disorders unresponsive to plasma. We randomly allocated patients stratified by hospital and history of cirrhosis to receive plasma transfusion (10-15 cc/kg) or no transfusion. The primary outcome was change in hemoglobin concentration within 2 days of procedure. RESULTS: We enrolled 57 patients, mean age 56.0, 34 (59.6%) with cirrhosis, and mean INR 1.92 (SD = 0.27). In the intention to treat analysis, there were 10 of 27 (38.5%) participants in the plasma arm with a post procedure INR <1.5 and one of 30 (3.6%) in the no treatment arm (p < .01). The mean INR after receiving plasma transfusion was -0.24 (SD 0.26) lower than baseline. The change from pre-procedure hemoglobin level to lowest level within 2 days was -0.6 (SD = 1.0) in the plasma transfusion arm and -0.4 (SD = 0.6) in the no transfusion arm (p = .29). Adverse outcomes were uncommon. DISCUSSION: We found no differences in change in hemoglobin concentration in those treated with plasma compared to no treatment. The change in INR was small and corrected to less than 1.5 in minority of patients. Large trials are required to establish if plasma is safe and efficacious.

A review of pragmatic trials found a high degree of diversity in design and scope, deficiencies in reporting and trial registry data, and poor indexing.

OBJECTIVE: We established a large database of trials to serve as a resource for future methodological and ethical analyses. Here, we use meta-data to describe the broad landscape of pragmatic trials including research areas, identification as pragmatic, quality of trial registry data and enrolment. STUDY DESIGN AND SETTING: Trials were identified by a validated search filter and included if a primary report of a health-related randomized trial published January 2014-April 2019. Data were collated from MEDLINE, Web of Science, ClinicalTrials.gov, and full text. RESULTS: 4337 eligible trials were identified from 13,065 records, of which 1988 were registered in ClinicalTrials.gov. Research areas were diverse, with the most common being general and internal medicine; public, environmental and occupational health; and health care sciences and services. The term "pragmatic" was seldom used in titles or abstracts. Several domains in ClinicalTrials.gov had questionable data quality. We estimated that one-fifth of trials under-accrued by at least 15%. CONCLUSION: There is a need to improve reporting of pragmatic trials and quality of trial registry data. Under accrual remains a challenge in pragmatic RCTs despite calls for more streamlined recruitment approaches. The diversity of pragmatic trials should be reflected in future ethical analyses.

PCORnet® 2020: current state, accomplishments, and future directions.

OBJECTIVE: To describe PCORnet, a clinical research network developed for patient-centered outcomes research on a national scale. STUDY DESIGN AND SETTING: Descriptive study of the current state and future directions for PCORnet. We conducted cross-sectional analyses of the health systems and patient populations of the 9 Clinical Research Networks and 2 Health Plan Research Networks that are part of PCORnet. RESULTS: Within the Clinical Research Networks, electronic health data are currently collected from 337 hospitals, 169,695 physicians, 3,564 primary care practices, 338 emergency departments, and 1,024 community clinics. Patients can be recruited for prospective studies from any of these clinical sites. The Clinical Research Networks have accumulated data from 80 million patients with at least one visit from 2009 to 2018. The PCORnet Health Plan Research Network population of individuals with a valid enrollment segment from 2009 to 2019 exceeds 60 million individuals, who on average have 2.63 years of follow-up. CONCLUSION: PCORnet's infrastructure comprises clinical data from a diverse cohort of patients and has the capacity to rapidly access these patient populations for pragmatic clinical trials, epidemiological research, and patient-centered research on rare diseases.

The use of community advisory boards in pragmatic clinical trials: The case of the adult day services plus project.

Community advisory boards (CABs) have become increasingly common and important in translational research in health care including studies focusing on home and community-based services. CABs are composed of stakeholders who share interest in research projects and typically include patients/clients, practitioners, community members, policymakers, and researchers. CABs advise researchers on issues ranging from research design and recruitment to implementation and dissemination. In this article, the researchers detail their experiences with the CAB for a pragmatic clinical trail of Adult Day Services (ADS) Plus, an education and support intervention for family caregivers of older adults with dementia using adult day services. Lessons learned, guidelines, and best practices are then presented for developing and working with a CAB in healthcare research.

Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results From the RCT DUPLICATE Initiative.

BACKGROUND: Regulators are evaluating the use of noninterventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT DUPLICATE initiative (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) uses a structured process to design RWE studies emulating randomized, controlled trials (RCTs) and compare results. We report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications. METHODS: We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion and exclusion criteria, selected primary end points, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 preexposure confounders. All study measures were prospectively defined and protocols registered before hazard ratios and 95% CIs were computed. Success criteria for the primary analysis were prespecified for each replication. RESULTS: Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a hazard ratio estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation. CONCLUSIONS: Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. Registration: URL: https://www.clinicaltrials.gov; Unique identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922.

Use of Real-World Evidence to Support FDA Approval of Oncology Drugs.

OBJECTIVES: Real-world evidence (RWE) has gained increased attention in recent years as a complement to traditional clinical trials. The use of RWE to establish the efficacy of oncology drugs for Food and Drug Administration (FDA) approval has not been described. In this paper, we review 5 recent examples where RWE was submitted in support of the FDA approvals of original or supplementary indications for oncology drugs. METHODS: To identify cases where RWE was used, we reviewed drug approval packages available at Drugs@FDA for oncology drugs approved between 2017 and 2019. Five cases were selected to present a broad overview of different types of RWE, different circumstances under which RWE has been used for regulatory approvals, and how FDA evaluated the data in each case. The type of RWE submitted, the indication, limitations identified by FDA reviewers, and the outcome of the submission are discussed. RESULTS: RWE, particularly historical controls for rare or orphan indications, has been used to support both original and supplementary oncology drug approvals. Types of RWE included data from electronic health records, claims, post-marketing safety reports, retrospective medical record reviews, and expanded access studies. Small sample sizes, data quality, and methodological issues were among concerns cited by FDA reviewers. CONCLUSION: By bridging the gap between the constraints of the trial setting and the realities of clinical practice, RWE can add value to a regulatory submission. These early examples provide insight into how regulators evaluated RWE submitted as evidence of efficacy for oncology drugs.