Modifications of the readiness assessment for pragmatic trials tool for appropriate use with Indigenous populations.

BACKGROUND: Inequities in health access and outcomes exist between Indigenous and non-Indigenous populations. Embedded pragmatic randomized, controlled trials (ePCTs) can test the real-world effectiveness of health care interventions. Assessing readiness for ePCT, with tools such as the Readiness Assessment for Pragmatic Trials (RAPT) model, is an important component. Although equity must be explicitly incorporated in the design, testing, and widespread implementation of any health care intervention to achieve equity, RAPT does not explicitly consider equity. This study aimed to identify adaptions necessary for the application of the 'Readiness Assessment for Pragmatic Trials' (RAPT) tool in embedded pragmatic randomized, controlled trials (ePCTs) with Indigenous communities. METHODS: We surveyed and interviewed participants (researchers with experience in research involving Indigenous communities) over three phases (July-December 2022) in this mixed-methods study to explore the appropriateness and recommended adaptions of current RAPT domains and to identify new domains that would be appropriate to include. We thematically analyzed responses and used an iterative process to modify RAPT. RESULTS: The 21 participants identified that RAPT needed to be modified to strengthen readiness assessment in Indigenous research. In addition, five new domains were proposed to support Indigenous communities' power within the research processes: Indigenous Data Sovereignty; Acceptability - Indigenous Communities; Risk of Research; Research Team Experience; Established Partnership). We propose a modified tool, RAPT-Indigenous (RAPT-I) for use in research with Indigenous communities to increase the robustness and cultural appropriateness of readiness assessment for ePCT. In addition to producing a tool for use, it outlines a methodological approach to adopting research tools for use in and with Indigenous communities by drawing on the experience of researchers who are part of, and/or working with, Indigenous communities to undertake interventional research, as well as those with expertise in health equity, implementation science, and public health. CONCLUSION: RAPT-I has the potential to provide a useful framework for readiness assessment prior to ePCT in Indigenous communities. RAPT-I also has potential use by bodies charged with critically reviewing proposed pragmatic research including funding and ethics review boards.

Cost-effectiveness of direct oral anticoagulants versus vitamin K antagonist in atrial fibrillation: A study protocol using Real-World Data from Catalonia (FantasTIC Study).

BACKGROUND: Anticoagulant therapy is used for stroke prevention and proved to be effective and safe in the long term. The study aims to analyse the cost-effectiveness relationship of using of direct-acting oral anticoagulants vs vitamin K antagonists to prevent ischaemic stroke in patients with nonvalvular atrial fibrillation, including all the active ingredients marketed in Spain, prescribed for 2 years in the Primary Care service of the Institut Català de la Salut. METHODS: Population-based cohort study, in which the cost of the 2 treatment groups will be evaluated. Direct costs (pharmacy, primary care, emergency and hospitalization) and indirect costs (lost productivity) will be included from a social perspective. Effectiveness (assessed as the occurrence of a health event, the 1 of primary interest being stroke) will be determined, with a 2-year time horizon and a 3% discount rate. The average cost of the 2 groups of drugs will be compared using a regression model to determine the factors with the greatest influence on determining costs. We will carry out a univariate ('one-way') deterministic sensitivity analysis. DISCUSSION: We hope to provide relevant information about direct and indirect costs of oral anticoagulants, which, together with aspects of effectiveness and safety, could help shape the consensual decision-making of evaluating bodies.

Burden of diabetes mellitus and its impact on COVID-19 patients: A meta-analysis of real-world evidence.

BACKGROUND & AIMS: Coronavirus disease 2019 (COVID-19) spreads rapidly and within no time, it has been declared a pandemic by the World Health Organization. Evidence suggests diabetes to be a risk factor for the progression and poor prognosis of COVID-19. Therefore, we aimed to understand the pooled prevalence of diabetes in patients infected with COVID-19. We also aimed to compute the risk of mortality and ICU admissions in COVID-19 patients with and without diabetes. METHODS: A comprehensive literature search was performed in PubMed to identify the articles reporting the diabetes prevalence and risk of mortality or ICU admission in COVID-19 patients. The primary outcome was to compute the pooled prevalence of diabetes in COVID-19 patients. Secondary outcomes included risk of mortality and ICU admissions in COVID-19 patients with diabetes compared to patients without diabetes. RESULTS: This meta-analysis was based on a total of 23007 patients from 43 studies. The pooled prevalence of diabetes in patients infected with COVID-19 was found to be 15% (95% CI: 12%-18%), p = <0.0001. Mortality risk was found to be significantly higher in COVID-19 patients with diabetes as compared to COVID-19 patients without diabetes with a pooled risk ratio of 1.61 (95% CI: 1.16-2.25%), p = 0.005. Likewise, risk of ICU admission rate was significantly higher in COVID-19 patients with diabetes as compared to COVID-19 patients without diabetes with a pooled risk ratio of 1.88 (1.20%-2.93%), p = 0.006. CONCLUSION: This meta-analysis found a high prevalence of diabetes and higher mortality and ICU admission risk in COVID-19 patients with diabetes.

A Call to Action to Track Generic Drug Quality Using Real-World Data and the FDA's Sentinel Initiative.

DISCLOSURES: The author reports funding from the U.S. Food and Drug Administration to study real-world data approaches to detect generic drug quality issues. There are no further conflicts or disclosures.

Methodologies for pragmatic and efficient assessment of benefits and harms: Application to the SOCRATES trial.

BACKGROUND/AIMS: Standard approaches to trial design and analyses can be inefficient and non-pragmatic. Failure to consider a range of outcomes impedes evidence-based interpretation and reduces power. Traditional approaches synthesizing information obtained from separate analysis of each outcome fail to incorporate associations between outcomes and recognize the cumulative nature of outcomes in individual patients, suffer from competing risk complexities during interpretation, and since efficacy and safety analyses are often conducted on different populations, generalizability is unclear. Pragmatic and efficient approaches to trial design and analyses are needed. METHODS: Approaches providing a pragmatic assessment of benefits and harms of interventions, summarizing outcomes experienced by patients, and providing sample size efficiencies are described. Ordinal outcomes recognize finer gradations of patient responses. Desirability of outcome ranking is an ordinal outcome combining benefits and harms within patients. Analysis of desirability of outcome ranking can be based on rank-based methodologies including the desirability of outcome ranking probability, the win ratio, and the proportion in favor of treatment. Partial credit analyses, involving grading the levels of the desirability of outcome ranking outcome similar to an academic test, provides an alternative approach. The methodologies are demonstrated using the acute stroke or transient ischemic attack treated with aspirin or ticagrelor and patient outcomes study (SOCRATES; NCT01994720), a randomized clinical trial. RESULTS: Two 5-level ordinal outcomes were developed for SOCRATES. The first was based on a modified Rankin scale. The odds ratio is 0.86 (95% confidence interval = 0.75, 0.99; p = 0.04) indicating that the odds of worse stroke categorization for a trial participant assigned to ticagrelor is 0.86 times that of a trial participant assigned to aspirin. The 5-level desirability of outcome ranking outcome incorporated and prioritized survival; the number of strokes, myocardial infarction, and major bleeding events; and whether a stroke event was disabling. The desirability of outcome ranking probability and win ratio are 0.504 (95% confidence interval = 0.499, 0.508; p = 0.10) and 1.11 (95% confidence interval = 0.98, 1.26; p = 0.10), respectively, implying that the probability of a more desirable result with ticagrelor is 50.4% and that a more desirable result occurs 1.11 times more frequently on ticagrelor versus aspirin. CONCLUSION: Ordinal outcomes can improve efficiency through required pre-specification, careful construction, and analyses. Greater pragmatism can be obtained by composing outcomes within patients. Desirability of outcome ranking provides a global assessment of the benefits and harms that more closely reflect the experience of patients. The desirability of outcome ranking probability, the proportion in favor of treatment, the win ratio, and partial credit can more optimally inform patient treatment, enhance the understanding of the totality of intervention effects on patients, and potentially provide efficiencies over standard analyses. The methods provide the infrastructure for incorporating patient values and estimating personalized effects.

Practical challenges in the conduct of pragmatic trials embedded in health plans: Lessons of IMPACT-AFib, an FDA-Catalyst trial.

IMPACT-AFib was an 80,000-patient randomized clinical trial implemented by five US insurance companies (health plans) aimed at increasing the use of oral anticoagulants by individuals with atrial fibrillation who were at high risk of stroke and not on treatment. The underlying thesis was that patients could be change agents to initiate prescribing discussions with their providers. We tested the effect of mailing information to both patients and their providers. We used administrative medical claims and pharmacy dispensing data to identify eligible patients, to randomize them to an early or delayed intervention, and to assess clinical outcomes. The core data were analysis-ready datasets each site had created and curated for the FDA's Sentinel System, supplemented by updated "fresh" pharmacy and enrollment data to ensure eligibility at the time of intervention. Following mutually agreed upon procedures, sites linked to additional internal source data to implement the intervention-educational information mailed to patients and their providers in the early intervention arm, and to providers of patients in the delayed intervention arm approximately 12 months later. The primary analysis compares the early intervention arm to the delayed intervention arm, prior to the delayed intervention being conducted (i.e. compares intervention to non-intervention). The endpoints of interest were evidence of initiation of anticoagulation (primary) as well as clinical endpoints, including stroke and hospitalization for bleeding. Major challenges, some unanticipated, identified during the planning phase include convening multi-stakeholder investigator teams and advisors, addressing ethical concerns about not intervening in a usual care comparison group, and identifying and avoiding interference with sites' routine programs that were similar to the intervention. Needs and challenges during the implementation phase included the fact that even limited site-specific programming greatly increased time and effort, the need to refresh research data extracts immediately before outreach to patients and providers, potential difficulty identifying low-cost medications such as warfarin that may not be reimbursed by health plans and so not discoverable in dispensing data, the need to develop workarounds when "providers" in claims data were facilities, difficulty addressing clustering of patients by provider because providers can have multiple identifiers within and between health plans, and the need to anticipate loss to follow up because of health plan disenrollment or change in benefits. As pragmatic trials begin to shape evidence generation within clinical practice, investigators should anticipate issues inherent to claims data and working with multiple large sites. In IMPACT-AFib, we found that investing in collaboration and communication among all parties throughout all phases of the study helped ensure common understanding, early identification of challenges, and streamlined actual implementation.

Real-World Clinical and Economic Outcomes Associated with Palbociclib for HR-Positive/HER2 Negative Metastatic Breast Cancer: A Commentary.

Despite the achieved advancement in pharmacological cancer treatments, the majority of postmenopausal women with hormone receptor-positive metastatic breast cancer (mBC) will experience disease progression. Research into alternative therapies with improved efficacy and reduced side effects has led to the development of a new class of oral anticancer medications, the cyclin-dependent kinase (CDK) 4/6 inhibitors, which include palbociclib, ribociclib, and abemaciclib. Nonetheless, there is growing evidence that the effectiveness of oral anticancer medications is sub-optimal, being influenced by low adherence, sociodemographic factors, and adverse effect profiles. In addition, there is a disconnect between the high price tags of CDK 4/6 inhibitors and their observed effectiveness, raising questions about their value. Currently, the existing knowledge base on the effectiveness and cost-effectiveness of newer oral anticancer medications in understudied populations with possible health disparities is scant. This commentary discusses what is known about palbociclib's clinical effectiveness, safety, and adherence and suggests the need for further studies of real-world effectiveness and cost-effectiveness to help establish the value of newer oncologic drugs, such as palbociclib. DISCLOSURES: No funding supported the writing of this article. The authors have nothing to disclose.

Design and rationale of a multi-center, pragmatic, open-label randomized trial of antimicrobial therapy - the study of clinical efficacy of antimicrobial therapy strategy using pragmatic design in Idiopathic Pulmonary Fibrosis (CleanUP-IPF) clinical trial.

Compelling data have linked disease progression in patients with idiopathic pulmonary fibrosis (IPF) with lung dysbiosis and the resulting dysregulated local and systemic immune response. Moreover, prior therapeutic trials have suggested improved outcomes in these patients treated with either sulfamethoxazole/ trimethoprim or doxycycline. These trials have been limited by methodological concerns. This trial addresses the primary hypothesis that long-term treatment with antimicrobial therapy increases the time-to-event endpoint of respiratory hospitalization or all-cause mortality compared to usual care treatment in patients with IPF. We invoke numerous innovative features to achieve this goal, including: 1) utilizing a pragmatic randomized trial design; 2) collecting targeted biological samples to allow future exploration of 'personalized' therapy; and 3) developing a strong partnership between the NHLBI, a broad range of investigators, industry, and philanthropic organizations. The trial will randomize approximately 500 individuals in a 1:1 ratio to either antimicrobial therapy or usual care. The site principal investigator will declare their preferred initial antimicrobial treatment strategy (trimethoprim 160 mg/ sulfamethoxazole 800 mg twice a day plus folic acid 5 mg daily or doxycycline 100 mg once daily if body weight is < 50 kg or 100 mg twice daily if ≥50 kg) for the participant prior to randomization. Participants randomized to antimicrobial therapy will receive a voucher to help cover the additional prescription drug costs. Additionally, those participants will have 4-5 scheduled blood draws over the initial 24 months of therapy for safety monitoring. Blood sampling for DNA sequencing and genome wide transcriptomics will be collected before therapy. Blood sampling for transcriptomics and oral and fecal swabs for determination of the microbiome communities will be collected before and after study completion. As a pragmatic study, participants in both treatment arms will have limited in-person visits with the enrolling clinical center. Visits are limited to assessments of lung function and other clinical parameters at time points prior to randomization and at months 12, 24, and 36. All participants will be followed until the study completion for the assessment of clinical endpoints related to hospitalization and mortality events. TRIAL REGISTRATION: ClinicalTrials.gov identifier NCT02759120.

Effect of community active case-finding strategies for detection of tuberculosis in Cambodia: study protocol for a pragmatic cluster randomized controlled trial.

BACKGROUND: Cambodia has made notable progress in the fight against tuberculosis (TB). However, these gains are impeded by a significant proportion of undiagnosed cases. To effectively reach people with TB, active case-finding (ACF) strategies have been adopted by countries affected by the epidemic, including Cambodia, alongside passive case finding (PCF). Despite increased efforts to improve case detection, approximately 40% of TB cases in Cambodia remained undiagnosed in 2018. In Cambodia, several community-based TB ACF modalities have been implemented, but their effectiveness has yet to be systematically assessed. METHODS: This pragmatic cluster randomized controlled trial will be conducted between December 2019 and June 2021. We will randomize eight operational districts (clusters) in seven provinces (Kampong Cham, Kampong Thom, Prey Veng, Thbong Khmum, Kampong Chhnang, Kandal, and Kampong Speu) to either the control group (PCF) or the intervention groups (ACF using a seed-and-recruit model, ACF targeting household and neighborhood contacts, and ACF targeting persons aged ≥ 55 years using mobile screening units). The primary endpoints will be TB case notification rates, additionality, and cumulative yield of TB cases. The secondary endpoints include treatment outcomes, the number needed to screen to find one TB case, and cost-effectiveness outcome measures. We will analyze the primary and secondary endpoints by intention to treat. We will compare cluster and individual-level characteristics using Student's t test and hierarchical or mixed-effect models to estimate the ratio of these means. The incremental cost-effectiveness ratio per disability-adjusted life year averted will also be considered as a benchmark to determine whether the interventions are cost-effective. DISCUSSION: This study will build an evidence base to inform future scale-up, implementation, and sustainability of ACF strategies in Cambodia and other similar settings. Implementation of this study will also complement TB control strategies in Cambodia by conducting ACF in operational districts without active interventions to find TB cases currently. Those who are ill and might have TB will be promptly screened, diagnosed, and linked to care. Early diagnosis and treatment initiation will also benefit their community by interrupting transmission and prevent further infections. The experience gained from this project will inform future attempts in conducting pragmatic trials in low-resource settings. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04094350. Registered on 18 September 2019.

Role of health plan administrative claims data in participant recruitment for pragmatic clinical trials: An Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) example.

AIM: The purpose of this study is to evaluate HealthCore/Anthem Research Network recruitment strategies, compare response and enrollment rates for different recruitment strategies, and describe demographic and clinical characteristics of responders and enrollees. METHODS: HealthCore/Anthem Research Network, a part of the Health Plan Research Network of the Patient-Centered Clinical Data Research Network, used administrative claims data to identify eligible health plan members for potential participation in the Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness study. We approached health plan members, identified with a validated Patient-Centered Clinical Data Research Network common data model computable phenotype, and their clinical providers during November 2017 to August 2018. Providers were offered the option to exclude their patients' participation in Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness prior to our direct patient (member) outreach. Member identification was in two phases: Phase 1: 1 January 2006 to 1 April 2017, and Phase 2: 1 January 2006 to 2 February 2018. Phase 1 consisted of two batches of mail and one phone call per patient. In Phase 2, which included two similar batches of patients, outreach was via either mail or brochure and one phone call. RESULTS: Phase 1 and Phase 2 included 133,373 and 51,777 members, respectively. We engaged 28,593 providers in Phase 1, and 5077 in Phase 2. In Phase 1, 264,158 mixed email/mail messages were delivered to 133,373 members, followed by 90,481 phone calls from November 2017 to February 2018. In Phase 2, after simple randomization to letter or brochure, 51,777 members were sent email/mail or mailed brochure in three waves from May 2018 to July 2018. In this 9-week period, 51,623 communications were sent to 25,914 members in the email/mail group, and 50,160 brochures to 25,863 in the brochure group. Following email/mail or mailed brochure outreach, 16,624 and 16,580 calls were made to the groups, respectively. Overall, 1549 health plan members visited the study portal by 1 September 2018; 355 electronically signed the Informed Consent Form and enrolled. Mailed brochures drove more portal visits in Phase 2, but a lower percentage of responders enrolled. Recruitment was better in Phase 2-2.3 enrollees per 1000 outreach members versus 1.8 in Phase 1. CONCLUSION: This study showed the ability of a health plan within Patient-Centered Clinical Data Research Network to identify potential study participants with administrative claims, and use different outreach methods to facilitate recruitment and enrollment for pragmatic clinical trials.

Building internal capacity in pragmatic trials: a workshop for program scientists at the US National Cancer Institute.

BACKGROUND: Building capacity in research funding organizations to support the conduct of pragmatic clinical trials is an essential component of advancing biomedical and public health research. To date, efforts to increase the ability to design and carry out pragmatic trials have largely focused on training researchers. To complement these efforts, we developed an interactive workshop tailored to meet the roles and responsibilities of program scientists at the National Cancer Institute-the leading cancer research funding agency in the USA. The objectives of the workshop were to improve the understanding of pragmatic trials and enhance the capacity to distinguish between elements that make a trial more pragmatic or more explanatory among key programmatic staff. To our knowledge, this is the first reported description of such a workshop. MAIN BODY: The workshop was developed to meet the needs of program scientists as researchers and stewards of research funds, which often includes promoting scientific initiatives, advising prospective applicants, collaborating with grantees, and creating training programs. The workshop consisted of presentations from researchers with expertise in the design and interpretation of trials across the explanatory-pragmatic continuum. Presentations were followed by interactive, small-group exercises to solidify participants' understanding of the purpose and conduct of these trials, which were tailored to attendees' areas of expertise across the cancer control continuum and designed to reflect their scope of work as program scientists at NCI. A total of 29 program scientists from the Division of Cancer Control and Population Sciences and the Division of Cancer Prevention participated; 19 completed a post-workshop evaluation. Attendees were very enthusiastic about the workshop: they reported improved knowledge, significant relevance of the material to their work, and increased interest in pragmatic trials across the cancer control continuum. CONCLUSION: Training program scientists at major biomedical research agencies who are responsible for developing funding opportunities and advising grantees is essential for increasing the quality and quantity of pragmatic trials. Together with workshops for other target audiences (e.g., academic researchers), this approach has the potential to shape the future of pragmatic trials and continue to generate more and better actionable evidence to guide decisions that are of critical importance to health care practitioners, policymakers, and patients.

Addressing guideline and policy changes during pragmatic clinical trials.

While conducting a set of large-scale multi-site pragmatic clinical trials involving high-impact public health issues such as end-stage renal disease, opioid use, and colorectal cancer, there were substantial changes to both policies and guidelines relevant to the trials. These external changes gave rise to unexpected challenges for the trials, including decisions regarding how to respond to new clinical practice guidelines, increased difficulty in implementing trial interventions, achieving separation between treatment groups, and differential responses across sites. In this article, we describe these challenges and the approaches used to address them. When deliberating appropriate action in the face of external changes during a pragmatic clinical trial, we recommend considering the well-being of the participants, clinical equipoise, and the strength and quality of the evidence associated with the change; involving those charged with data and safety monitoring; and where possible, planning for potential external changes as the trial is being designed. Any solution must balance the primary obligation to protect the well-being of participants with the secondary obligation to protect the integrity of the trial in order to gain meaningful answers to important public health questions.

Determining the Comparative Value of Pharmaceutical Risk-Sharing Policies in Non-Small Cell Lung Cancer Using Real-World Data.

BACKGROUND: Risk-sharing arrangements (RSAs) can be used to mitigate uncertainty about the value of a drug by sharing the financial risk between payer and pharmaceutical company. We evaluated the projected impact of alternative RSAs for non-small cell lung cancer (NSCLC) therapies based on real-world data. METHODS: Data on treatment patterns of Dutch NSCLC patients from four different hospitals were used to perform "what-if" analyses, evaluating the costs and benefits likely associated with various RSAs. In the scenarios, drug costs or refunds were based on response evaluation criteria in solid tumors (RECIST) response, survival compared to the pivotal trial, treatment duration, or a fixed cost per patient. Analyses were done for erlotinib, gemcitabine/cisplatin, and pemetrexed/platinum for metastatic NSCLC, and gemcitabine/cisplatin, pemetrexed/cisplatin, and vinorelbine/cisplatin for nonmetastatic NSCLC. RESULTS: Money-back guarantees led to moderate cost reductions to the payer. For conditional treatment continuation schemes, costs and outcomes associated with the different treatments were dispersed. When price was linked to the outcome, the payer's drug costs reduced by 2.5% to 26.7%. Discounted treatment initiation schemes yielded large cost reductions. Utilization caps mainly reduced the costs of erlotinib treatment (by 16%). Given a fixed cost per patient based on projected average use of the drug, risk sharing was unfavorable to the payer because of the lower than projected use. The impact of RSAs on a national scale was dispersed. CONCLUSIONS: For erlotinib and pemetrexed/platinum, large cost reductions were observed with risk sharing. RSAs can mitigate uncertainty around the incremental cost-effectiveness or budget impact of drugs, but only when the type of arrangement matches the setting and type of uncertainty.

FDA-Catalyst-Using FDA's Sentinel Initiative for large-scale pragmatic randomized trials: Approach and lessons learned during the planning phase of the first trial.

BACKGROUND: The US Food and Drug Administration's Sentinel Initiative is well positioned to support pragmatic clinical trials. FDA-Catalyst combines direct contact with health plan members and/or providers with data in the Sentinel infrastructure. Here, we describe the rationale, feasibility analyses, and lessons learned from the planning phase of the first large pragmatic trial conducted using the Sentinel Initiative's delivery system capabilities-IMplementation of a randomized controlled trial to imProve treatment with oral AntiCoagulanTs in patients with Atrial Fibrillation (the IMPACT-AFib trial). METHODS: During the planning phase, we convened representatives from five commercial health plans, FDA, study coordinating centers, and a patient representative for protocol development, institutional review board preparation, and other activities. Administrative claims data from the plans were included in a retrospective cohort analysis to assess sample size for the trial. Members ≥30 years old with ≥365 days of medical/pharmacy coverage, ≥2 diagnosis codes for atrial fibrillation, a guideline-based indication for oral anticoagulant use for stroke prevention, and no evidence of oral anticoagulant use in the 365 days prior to the index atrial fibrillation diagnosis in 2013 were included. Exclusions for the analysis included other conditions requiring anticoagulation, history of intracranial hemorrhage, and gastrointestinal bleed. We calculated rates of oral anticoagulant use, transient ischemic attack or stroke, and bleeding in the 365 days following the index atrial fibrillation diagnosis. RESULTS: A total of 44,786 members with atrial fibrillation with no evidence of recent oral anticoagulant use were identified. In total, 87% (n = 38,759) were classified as having a guideline-based indication for oral anticoagulants. Of those, 33% (n = 12,867) had a new oral anticoagulant dispensed during the following year, 15% (n = 5917) were hospitalized for stroke or transient ischemic attack, and 9% (n = 3469) for bleeding events. This information was used to develop the trial protocol including sample size, power calculations, and level of randomization. CONCLUSION: Sentinel infrastructure generated preliminary data that supported planning and implementation of a large pragmatic trial embedded in health plans. This planning identified unanticipated challenges that must be addressed in similar trials.

Commentary: considerations for using the 'Trials within Cohorts' design in a clinical trial of an investigational medicinal product.

BACKGROUND: The 'trials within cohorts' (TwiC) design is a pragmatic approach to randomised trials in which trial participants are randomly selected from an existing cohort. The design has multiple potential benefits, including the option of conducting multiple trials within the same cohort. MAIN TEXT: To date, the TwiC design methodology been used in numerous clinical settings but has never been applied to a clinical trial of an investigational medicinal product (CTIMP). We have recently secured the necessary approvals to undertake the first CTIMP using the TwiC design. In this paper, we describe some of the considerations and modifications required to ensure such a trial is compliant with Good Clinical Practice and international clinical trials regulations. We advocate using a two-stage consent process and using the consent stages to explicitly differentiate between trial participants and cohort participants who are providing control data. This distinction ensured compliance but had consequences with respect to costings, recruitment and the trial assessment schedule. CONCLUSION: We have demonstrated that it is possible to secure ethical and regulatory approval for a CTIMP TwiC. By including certain considerations at the trial design stage, we believe this pragmatic and efficient methodology could be utilised in other CTIMPs in future.

Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making.

PURPOSE: Real-world evidence (RWE) includes data from retrospective or prospective observational studies and observational registries and provides insights beyond those addressed by randomized controlled trials. RWE studies aim to improve health care decision making. METHODS: The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) created a task force to make recommendations regarding good procedural practices that would enhance decision makers' confidence in evidence derived from RWD studies. Peer review by ISPOR/ISPE members and task force participants provided a consensus-building iterative process for the topics and framing of recommendations. RESULTS: The ISPOR/ISPE Task Force recommendations cover seven topics such as study registration, replicability, and stakeholder involvement in RWE studies. These recommendations, in concert with earlier recommendations about study methodology, provide a trustworthy foundation for the expanded use of RWE in health care decision making. CONCLUSION: The focus of these recommendations is good procedural practices for studies that test a specific hypothesis in a specific population. We recognize that some of the recommendations in this report may not be widely adopted without appropriate incentives from decision makers, journal editors, and other key stakeholders.

Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory.

BACKGROUND: The clinical research enterprise is not producing the evidence decision makers arguably need in a timely and cost effective manner; research currently involves the use of labor-intensive parallel systems that are separate from clinical care. The emergence of pragmatic clinical trials (PCTs) poses a possible solution: these large-scale trials are embedded within routine clinical care and often involve cluster randomization of hospitals, clinics, primary care providers, etc. Interventions can be implemented by health system personnel through usual communication channels and quality improvement infrastructure, and data collected as part of routine clinical care. However, experience with these trials is nascent and best practices regarding design operational, analytic, and reporting methodologies are undeveloped. METHODS: To strengthen the national capacity to implement cost-effective, large-scale PCTs, the Common Fund of the National Institutes of Health created the Health Care Systems Research Collaboratory (Collaboratory) to support the design, execution, and dissemination of a series of demonstration projects using a pragmatic research design. RESULTS: In this article, we will describe the Collaboratory, highlight some of the challenges encountered and solutions developed thus far, and discuss remaining barriers and opportunities for large-scale evidence generation using PCTs. CONCLUSION: A planning phase is critical, and even with careful planning, new challenges arise during execution; comparisons between arms can be complicated by unanticipated changes. Early and ongoing engagement with both health care system leaders and front-line clinicians is critical for success. There is also marked uncertainty when applying existing ethical and regulatory frameworks to PCTS, and using existing electronic health records for data capture adds complexity.