Analysis of the characteristics and the degree of pragmatism exhibited by pragmatic-labelled trials of antineoplastic treatments.

BACKGROUND: Pragmatic clinical trials (PCTs) are designed to reflect how an investigational treatment would be applied in clinical practice. As such, unlike their explanatory counterparts, they measure therapeutic effectiveness and are capable of generating high-quality real-world evidence. However, the conduct of PCTs remains extremely rare. The scarcity of such studies has contributed to the emergence of the efficacy-effectiveness gap and has led to calls for launching more of them, including in the field of oncology. This analysis aimed to identify self-labelled pragmatic trials of antineoplastic interventions and to evaluate whether their use of this label was justified. METHODS: We searched PubMed® and Embase® for publications corresponding with studies that investigated antitumor therapies and that were tagged as pragmatic in their titles, abstracts and/or index terms. Subsequently, we consulted all available source documents for the included trials and extracted relevant information from them. The data collected were then used to appraise the degree of pragmatism displayed by the PCTs with the help of the validated PRECIS-2 tool. RESULTS: The literature search returned 803 unique records, of which 46 were retained upon conclusion of the screening process. This ultimately resulted in the identification of 42 distinct trials that carried the 'pragmatic' label. These studies examined eight different categories of neoplasms and were mostly randomized, open-label, multicentric, single-country trials sponsored by non-commercial parties. On a scale of one (very explanatory) to five (very pragmatic), the median PCT had a PRECIS-2 score per domain of 3.13 (interquartile range: 2.57-3.53). The most and least pragmatic studies in the sample had a score of 4.44 and 1.57, respectively. Only a minority of trials were described in sufficient detail to allow them to be graded across all domains of the PRECIS-2 instrument. Many of the studies examined also had features that arguably precluded them from being pragmatic altogether, such as being monocentric or placebo-controlled in nature. CONCLUSION: PCTs of antineoplastic treatments are generally no more pragmatic than they are explanatory.

The Targeted Agent and Profiling Utilization Registry Study: A pragmatic clinical trial.

The conceptual framework of pragmatism in clinical trials is explored using the American Society of Clinical Oncology's pragmatic, non-randomized, phase II, multi-center basket clinical trial, the Targeted Agent and Profiling Utilization Registry Study (NCT02693535) as a model. The Targeted Agent and Profiling Utilization Registry Study aims to identify signals of drug activity when Food and Drug Administration approved drugs are matched to pre-specified genomic targets in patients with advanced cancer outside of their approved indication(s). The objectives of the study are to generate evidence of potential signals of activity in targeted therapies prescribed in an off-label setting as well as to expose and educate community cancer centers to genomic testing and precision medicine through the study protocol. The principles of pragmatic trial design can be applied across a broad spectrum of evidence-generation strategies, from explanatory trials to real-world evidence studies, and are briefly discussed. American Society of Clinical Oncology's Targeted Agent and Profiling Utilization Registry Study falls closer to the pragmatic end of this spectrum as it seeks to assess the efficacy of Food and Drug Administration approved drugs used outside their approved indications under usual care conditions, yielding results generalizable to the population that would likely receive the intervention in practice, while still adhering to rigorous data quality standards. The Targeted Agent and Profiling Utilization Registry Study's pragmatic objectives, characteristics, strengths, and limitations in its implementation are discussed and demonstrate that a large, multi-center, precision medicine basket trial can be mounted in the context of community practice and can generate clinically useful information with minimal burden to patients and clinical trial sites.

Applying dissemination and implementation research methods to translate a school-based asthma program.

Our school-based asthma program has reduced asthma exacerbations for youth with health disparities in the Denver metropolitan area, due partly to addressing social determinants of health, such as access to health care and medications. Dissemination and implementation (D&I) science approaches accelerate the translation of evidence-based programs into routine practice. D&I methods are being applied more commonly to improve health equity. The purpose of this publication was to give an overview of D&I research methods, using our school-based asthma program as an example. To successfully scale out our program across the state of Colorado, we are applying a D&I framework that guides the adaptation of our existing implementation approach to better meet our stakeholders' local context-the Exploration, Preparation, Implementation, Sustainment framework. In a pragmatic trial design, we will evaluate the outcomes of implementing the program across 5 Colorado regions, with attention to health equity, using a second commonly used D&I framework-Reach, Effectiveness, Adoption, Implementation, and Maintenance. Our central hypothesis is that our program will have broad and equitable reach to eligible students (primary outcome) and will reduce asthma attacks and symptoms. This D&I approach accelerates dissemination of our program and is an applicable process for translating other effective allergy/asthma programs to address asthma and allergy-related disparities.

The PRECIS-2 tool seems not to be useful to discriminate the degree of pragmatism of medicine masked trials from that of open-label trials.

PURPOSE: To assess, with all available trial information, whether the assessment of the PRECIS-2 nine domains could provide a clear distinction between medicine masked pragmatic randomized controlled trials (pRCTs) and open-label pRCTs. METHODS: A search was conducted of participant-level pRCTs on medicines published on 25 influential medical journals in July 2018-December 2019. All pre-licensing (phases 1-3) and cluster pRCTs were excluded. All trials' available reports were searched through the published article information, Google Scholar, and trial websites. Instead of providing a score to each PRECIS-2 domain, these were classified as E (explanatory), N (neutral), or P (pragmatic). RESULTS: Of 128 pRCTs, 18 (14%) were participant-level pRCTs on medicines. The full trial protocol was available for 14 trials; 12 had published the protocol and nine had additional reports published. All trials were prospectively registered, and none was funded by industry. Ten and eight were masked and open-label trials, respectively. Masked pRCTS had 34% of pragmatic and 60% of explanatory domains; open-label pRCTS had 45% pragmatic and 45% explanatory domains. Among the 10 masked trials, only one had a majority of five pragmatic domains; among the eight open-label trials, four had a majority of six or five pragmatic domains. "Follow-up" was considered explanatory in the 18 pRCTs; "primary analysis" was pragmatic in 17 pRCTs. CONCLUSION: The PRECIS-2 tool seems not to be sensitive enough to clearly discriminate between medicine masked pRCTs and open-label pRCTs. When conducting systematic reviews, it is suggested that the PRECIS-2 tool should not be used to support placing masked trials in the pragmatic side of the explanatory/pragmatic continuum.

Protocol refinement for a diabetes pragmatic trial using the PRECIS-2 framework.

BACKGROUND: This report describes how we refined a protocol for a pragmatic comparative effectiveness study of two models of an evidence-based diabetes shared medical appointment intervention and used the PRECIS-2 rating system to evaluate these adaptations. METHODS: We report primary data collected between June and August 2019, and protocol refinements completed between 2018 and 2020. Twenty-two members of the study team collaborated in protocol refinement and completed the PRECIS-2 ratings of study pragmatism. We discuss study design refinements made to achieve the desired level of pragmatism vs. experimental control for each of the nine PRECIS-2 dimensions. Study team members received training on PRECIS-2 scoring and were asked to rate the study protocol on the nine PRECIS-2 dimensions. Ratings were compared using descriptive statistics. RESULTS: In general, the PRECIS-2 ratings revealed high levels of pragmatism, but somewhat less pragmatic ratings on the categories of Delivery and Organization (costs and resources). This variation was purposeful, and we provide the rationale for and steps taken to obtain the targeted level of pragmatism on each PRECIS-2 dimension, as well as detail design changes made to a) make the design more pragmatic and b) address COVID-19 issues. There was general agreement among team members and across different types of stakeholders on PRECIS-2 ratings. CONCLUSIONS: We discuss lessons learned from use of PRECIS-2 and experiences in refining the study to be maximally pragmatic on some dimensions and less so on other dimensions. This paper expands on prior research by describing actions to achieve higher levels of pragmatism and revise our protocol fit to the changed context. We make recommendations for future use of PRECIS-2 to help address changing context and other strategies for the planning of and transparent reporting on pragmatic research and comparative effectiveness research. TRIAL REGISTRATION: Clinicaltrials.gov Registration ID: NCT03590041 .

Articles provided insufficient information to conduct an appropriate retrospective assessment of the pragmatic/explanatory features of medicine trials with the PRECIS-2 tool.

PURPOSE: To assess whether, in the retrospective assessment of the pragmatic/explanatory features of pragmatic randomized controlled trials (pRCTs), the nine PRECIS-2 domain scores using the information provided in articles were modified after using the information reported in other publicly available sources. METHODS: This is a cross-sectional study of participant-level pRCTs published in July 2018 to December 2019 in the four highest-impact general medicine journals. The articles described the main results of pRCTs assessing medicines in one or more arms that were not in the pre-licensing phases. The information reported in trial full protocols, published protocols, and other publications, registries, and trial websites were assessed and scored, and compared with that previously obtained after reviewing the information reported in the articles. RESULTS: Out of 76 articles on pRCTs, 13 (17%) were included in the analysis. All were two-arm trials, assessing medicines only (n = 7), medicine vs device (n = 2), medicine vs surgery (n = 1), or medicine vs placebo (n = 3). Seven were open-label trials, and six had any type of masking. All except one had the full protocol available and/or published protocol; seven had other types of publication available. The assessment of the nine PRECIS-2 domains with the information reported in the 13 articles was changed in all trials after using the information included in other additional available sources. Between one (n = 1 article) and six (n = 2) domains were modified in each pRCT. The domains that most commonly changed were "organization" (n = 12), "recruitment" (n = 11), and "follow-up" (n = 8). "Primary outcome" and "primary analysis" were not modified in any trial. Eight percent of all domains could not be assessed due to inadequate or lack of information in seven articles; those were "recruitment" (n = 3), "organization" (n = 3), "setting" (n = 2), and "flexibility:adherence" (n = 1). CONCLUSION: Articles describing the trial main results are usually insufficient for the appropriate retrospective assessment of the pragmatic/explanatory features of a pRCT by authors not involved in the conduct of the trial. To address this issue, editors should require the submission of the original full protocol and final full protocol with the history of amendments to be published as supplementary material to the article.

Informing real-world practice with real-world evidence: the value of PRECIS-2.

Real-world evidence is needed to inform real-world practice. Pragmatic controlled trials are intended to provide such evidence by assessing the effectiveness of medicines and other interventions in real-world settings, as opposed to explanatory trials that assess efficacy in highly controlled settings. Dal-Ré and colleagues (BMC Med 16:49, 2018) recently performed a literature review of studies published between 2014 and 2017 to assess the degree to which studies that self-identified as pragmatic were truly so. The authors found that over one-third of randomized controlled trials of drugs and biologics that were self-labeled as pragmatic used placebo controls (as opposed to usual care), tested medicines before licensing, or were conducted in a single site. Further, they proposed that, in order to improve the reliability of the 'pragmatic' label, investigators should assess their trials using the PRECIS-2 tool upon submission to funders, ethics boards, or journals. We appreciate the value of PRECIS-2 as an indicator to assess the pragmatic versus explanatory features in a trial, and we herein highlight the potential challenges and opportunities that may arise with its systematic and widespread use.

Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic?

INTRODUCTION: Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and pragmatic are the extremes of a continuum. This debate article seeks to evaluate and provide recommendation on how to characterize pragmatic RCTs in light of the current landscape of RCTs. It is supported by findings from a PubMed search conducted in August 2017, which retrieved 615 RCTs self-labeled in their titles as "pragmatic" or "naturalistic". We focused on 89 of these trials that assessed medicines (drugs or biologics). DISCUSSION: 36% of these 89 trials were placebo-controlled, performed before licensing of the medicine, or done in a single-center. In our opinion, such RCTs overtly deviate from usual care and pragmatism. It follows, that the use of the term 'pragmatic' to describe them, conveys a misleading message to patients and clinicians. Furthermore, many other trials among the 615 coined as 'pragmatic' and assessing other types of intervention are plausibly not very pragmatic; however, this is impossible for a reader to tell without access to the full protocol and insider knowledge of the trial conduct. The degree of pragmatism should be evaluated by the trial investigators themselves using the PRECIS-2 tool, a tool that comprises 9 domains, each scored from 1 (very explanatory) to 5 (very pragmatic). CONCLUSIONS: To allow for a more appropriate characterization of the degree of pragmatism in clinical research, submissions of RCTs to funders, research ethics committees and to peer-reviewed journals should include a PRECIS-2 tool assessment done by the trial investigators. Clarity and accuracy on the extent to which a RCT is pragmatic will help understand how much it is relevant to real-world practice.

Applying the pragmatic-explanatory continuum indicator summary to the implementation of a physical activity coaching trial in chronic obstructive pulmonary disease.

BACKGROUND: Observational studies show that physical inactivity is associated with worse outcomes in chronic obstructive pulmonary disease (COPD). Despite practice guidelines recommending regular physical activity (PA), there are no large-scale experimental studies to confirm that patients at high risk for COPD exacerbations can increase their PA and consequently, have improved outcomes. PURPOSE: The purpose of this case study is to describe the use of a widely accepted pragmatic trials framework for the design and implementation of a pragmatic clinical trial (PCT) of PA coaching for COPD in a real-world setting. METHOD: The aim of the trial was to determine the effectiveness of a 12-month PA coaching intervention (Walk On!) compared to standard care for 2,707 patients at high risk for COPD exacerbations from a large integrated health care system. The descriptions of our implementation experiences are anchored within the pragmatic-explanatory continuum indicator summary (PRECIS-2) framework. DISCUSSION: Facilitators of PCT implementation include early and ongoing engagement and support of multiple stakeholders including patients, health system leaders, administrators, physician champions, and frontline clinicians, an organizational/setting that prioritizes positive lifestyle behaviors, and a flexible intervention that allows for individualization. Pragmatic challenges include reliance on electronic data that are not complete or available in real-time for patient identification, timing of outreach may not synchronize with patients' readiness for change, and high turnover of clinical staff drawn from the existing workforce. DISCUSSION: PRECIS-2 is a useful guide for organizing decisions about study designs and implementation approaches to help diverse stakeholders recognize the compromises between internal and external validity with those decisions.

PRECIS-2 analysis of pragmatic acupuncture trials: a systematic review.

BACKGROUND: Pragmatic acupuncture trials (PATs) are a research tool for assessing the effectiveness of acupuncture treatments in a real-world setting. This study aimed to provide a comprehensive methodological analysis of PATs using the PRECIS-2(PRagmatic Explanatory Continuum Indicator Summary-2) tool to determine their pragmatism. METHODS: The MEDLINE, EMBASE, Cochrane Central Register for Controlled Trials, CINAHL, Allied and Complementary Medicine Database, China National Knowledge Infrastructure, VIP, WANFANG, Taiwan Periodical Literature Database, KoreaMed, KMbase, Research Information Service System, Oriental Medicine Advanced Searching Integrated System, CiNii and ClinicalTrials.gov were searched. The search included randomised controlled trials (RCTs) and protocols of RCTs that investigated all types of acupuncture and used self-declared pragmatic design. Two authors independently collected the basic information and characteristics of the studies and assessed their pragmatism using the nine PRECIS-2 domains and the additional domain of control. RESULTS: A total of 93 studies were included. The means of eligibility, recruitment, organisation, primary outcome, primary analysis, and control domains were statistically larger than three and were shown to be pragmatic. The means of setting, flexibility:delivery, and follow-up domains were not greater than three and were shown to be non-pragmatic. For flexibility:adherence domain was inappropriate for assessment owing to insufficient information in the studies. CONCLUSIONS: PATs were pragmatic in the domain of eligibility, recruitment, organisation, primary outcome, primary analysis, and control and were not pragmatic in the domain of setting, flexibility:delivery, and follow-up. Future PATs need to strengthen the pragmatism in the setting, flexibility:delivery, and follow-up domains and to describe the flexibility:adherence domain in more detail. TRIAL REGISTRATION: CRD42021236975.

Randomized controlled trials comparing gastric bypass, gastric band, and sleeve gastrectomy: A systematic review examining validity and applicability to wider clinical practice.

Consideration of how applicable the results of surgical trials are to clinical practice is important to inform decision-making. Randomized controlled trials comparing at least two surgical interventions (of gastric bypass, gastric band, and sleeve gastrectomy) for severe and complex obesity were examined using the PRagmatic Explanatory Continuum Indicator Summary-2 tool, to consider how applicable the trial results are to clinical practice, and the Risk of Bias 2 tool, to examine validity. MEDLINE, Embase, and CENTRAL databases were searched for studies published between November 2013 and June 2021, and 15 were identified. Using the PRagmatic Explanatory Continuum Indicator Summary-2 tool, three were classified as pragmatic, with good applicability to clinical practice. Ten had more explanatory domains but did include some pragmatic characteristics, and two were predominantly explanatory. This was due to some trial design features that would not be considered applicable to the wider clinical setting, including being single-centered, having prescribed intervention delivery methods, and intensive follow-up regimens. Only two trials had low risk of bias, of which one was considered pragmatic. Three had high risk of bias. Overall, few trials in bariatric surgery are pragmatic with low risk of bias. Well-designed pragmatic trials are needed to inform practice and reduce research waste.

Intervention Fidelity in Pain Pragmatic Trials for Nonpharmacologic Pain Management: Nuanced Considerations for Determining PRECIS-2 Flexibility in Delivery and Adherence.

Nonpharmacological treatments are considered first-line pain management strategies, but they remain clinically underused. For years, pain-focused pragmatic clinical trials (PCTs) have generated evidence for the enhanced use of nonpharmacological interventions in routine clinical settings to help overcome implementation barriers. The Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) framework describes the degree of pragmatism across 9 key domains. Among these, "flexibility in delivery" and "flexibility in adherence," address a key goal of pragmatic research by tailoring approaches to settings in which people receive routine care. However, to maintain scientific and ethical rigor, PCTs must ensure that flexibility features do not compromise delivery of interventions as designed, such that the results are ethically and scientifically sound. Key principles of achieving this balance include clear definitions of intervention core components, intervention monitoring and documentation that is sufficient but not overly burdensome, provider training that meets the demands of delivering an intervention in real-world settings, and use of an ethical lens to recognize and avoid potential trial futility when necessary and appropriate. PERSPECTIVE: This article presents nuances to be considered when applying the PRECIS-2 framework to describe pragmatic clinical trials. Trials must ensure that patient-centered treatment flexibility does not compromise delivery of interventions as designed, such that measurement and analysis of treatment effects is reliable.

Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis.

BACKGROUND: Over the past two decades, pragmatic and implementation science clinical trial research methods have advanced substantially. Pragmatic and implementation studies have natural areas of overlap, particularly relating to the goal of using clinical trial data to leverage health care system policy changes. Few investigations have addressed pragmatic and implementation science randomized trial methods development while also considering policy impact. METHODS: The investigation used the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) and PRECIS-2-Provider Strategies (PRECIS-2-PS) tools to evaluate the design of two multisite randomized clinical trials that targeted patient-level effectiveness outcomes, provider-level practice changes and health care system policy. Seven raters received PRECIS-2 training and applied the tools in the coding of the two trials. Descriptive statistics were produced for both trials, and PRECIS-2 wheel diagrams were constructed. Interrater agreement was assessed with the Intraclass Correlation (ICC) and Kappa statistics. The Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) qualitative approach was applied to understanding integrative themes derived from the PRECIS-2 ratings and an end-of-study policy summit. RESULTS: The ICCs for the composite ratings across the patient and provider-focused PRECIS-2 domains ranged from 0.77 to 0.87, and the Kappa values ranged from 0.25 to 0.37, reflecting overall fair-to-good interrater agreement for both trials. All four PRECIS-2 wheels were rated more pragmatic than explanatory, with composite mean and median scores ≥ 4. Across trials, the primary intent-to-treat analysis domain was consistently rated most pragmatic (mean = 5.0, SD = 0), while the follow-up/data collection domain was rated most explanatory (mean range = 3.14-3.43, SD range = 0.49-0.69). RAPICE field notes identified themes related to potential PRECIS-2 training improvements, as well as policy themes related to using trial data to inform US trauma care system practice change; the policy themes were not captured by the PRECIS-2 ratings. CONCLUSIONS: The investigation documents that the PRECIS-2 and PRECIS-2-PS can be simultaneously used to feasibly and reliably characterize clinical trials with patient and provider-level targets. The integration of pragmatic and implementation science clinical trial research methods can be furthered by using common metrics such as the PRECIS-2 and PRECIS-2-PS. Future study could focus on clinical trial policy research methods development. TRIAL REGISTRATION: DO-SBIS ClinicalTrials.gov NCT00607620. registered on January 29, 2008. TSOS ClinicalTrials.gov NCT02655354, registered on July 27, 2015.

Assessing the Pragmatic Nature of Mobile Health Interventions Promoting Physical Activity: Systematic Review and Meta-analysis.

BACKGROUND: Mobile health (mHealth) apps can promote physical activity; however, the pragmatic nature (ie, how well research translates into real-world settings) of these studies is unknown. The impact of study design choices, for example, intervention duration, on intervention effect sizes is also understudied. OBJECTIVE: This review and meta-analysis aims to describe the pragmatic nature of recent mHealth interventions for promoting physical activity and examine the associations between study effect size and pragmatic study design choices. METHODS: The PubMed, Scopus, Web of Science, and PsycINFO databases were searched until April 2020. Studies were eligible if they incorporated apps as the primary intervention, were conducted in health promotion or preventive care settings, included a device-based physical activity outcome, and used randomized study designs. Studies were assessed using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) and Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) frameworks. Study effect sizes were summarized using random effect models, and meta-regression was used to examine treatment effect heterogeneity by study characteristics. RESULTS: Overall, 3555 participants were included across 22 interventions, with sample sizes ranging from 27 to 833 (mean 161.6, SD 193.9, median 93) participants. The study populations' mean age ranged from 10.6 to 61.5 (mean 39.6, SD 6.5) years, and the proportion of males included across all studies was 42.8% (1521/3555). Additionally, intervention lengths varied from 2 weeks to 6 months (mean 60.9, SD 34.9 days). The primary app- or device-based physical activity outcome differed among interventions: most interventions (17/22, 77%) used activity monitors or fitness trackers, whereas the rest (5/22, 23%) used app-based accelerometry measures. Data reporting across the RE-AIM framework was low (5.64/31, 18%) and varied within specific dimensions (Reach=44%; Effectiveness=52%; Adoption=3%; Implementation=10%; Maintenance=12.4%). PRECIS-2 results indicated that most study designs (14/22, 63%) were equally explanatory and pragmatic, with an overall PRECIS-2 score across all interventions of 2.93/5 (SD 0.54). The most pragmatic dimension was flexibility (adherence), with an average score of 3.73 (SD 0.92), whereas follow-up, organization, and flexibility (delivery) appeared more explanatory with means of 2.18 (SD 0.75), 2.36 (SD 1.07), and 2.41 (SD 0.72), respectively. An overall positive treatment effect was observed (Cohen d=0.29, 95% CI 0.13-0.46). Meta-regression analyses revealed that more pragmatic studies (-0.81, 95% CI -1.36 to -0.25) were associated with smaller increases in physical activity. Treatment effect sizes were homogenous across study duration, participants' age and gender, and RE-AIM scores. CONCLUSIONS: App-based mHealth physical activity studies continue to underreport several key study characteristics and have limited pragmatic use and generalizability. In addition, more pragmatic interventions observe smaller treatment effects, whereas study duration appears to be unrelated to the effect size. Future app-based studies should more comprehensively report real-world applicability, and more pragmatic approaches are needed for maximal population health impacts. TRIAL REGISTRATION: PROSPERO CRD42020169102; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=169102.

How pragmatic are randomized trials of remdesivir and favipiravir for in-hospital treatment of COVID-19: a descriptive methodological review of trial design using the PRECIS-2 framework.

OBJECTIVES: To review the pragmatism of published randomized trials of remdesivir and favipiravir based on the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) framework. STUDY DESIGN AND SETTING: Ten eligible trials were identified from an existing comprehensive living review and were evaluated across the nine PRECIS-2 domains by two independent reviewers. RESULTS: All 10 trials had mostly pragmatic design characteristics. Four of the domains (i.e., recruitment, setting, organization, and primary analysis) were found to be pragmatic with most trials scoring four or five across the two interventions. In comparison scores for four other design domains (i.e., eligibility, follow-up, flexibility of delivery, and primary outcome) varied across the trials with some design choices being more explanatory. CONCLUSION: In our descriptive review of randomized controlled trails for two drugs for patients infected with COVID-19 early in the pandemic, we found that most trials had more pragmatic than explanatory characteristics. Some design choices for some of the trials, however, were not consistent with the urgent goal of informing clinical decision making in an epidemic. PRECIS-2 should be used as a guide by trialists, to help them match their trial design choices to the intended purpose of their trial.

The relationship between pragmatism, timing, and study size on impact of randomized trials: a qualitative, hypothesis generating study of trials of systemic corticosteroids for COVID-19.

OBJECTIVE: To explore qualitatively the relationship between selected trial design choices and proxies for a scientific and clinical uptake in a cohort of published randomized controlled trials (RCTs) of corticosteroids for COVID-19, to identify design characteristics that may result in trials with potential to eliminate equipoise, achieve uptake, and help reduce research waste. STUDY DESIGN AND SETTING: A systematic literature search and qualitative, narrative review of published RCTs (up to April 13, 2021) evaluating the effectiveness of systemic corticosteroids in treatment of COVID-19. We extracted information on sample size, number of centers, single-country or multi-country conduct, dates of initiation and of publication, risk of bias and pragmatism scores, and also on an impact measured by citation in scientific literature and in clinical guidelines. We qualitatively compared design features of the highest impact vs. other trials. RESULTS: Randomised Evaluation of COVID-19 Therapy (RECOVERY) was by the most impactful of the seven eligible RCTs as it was 10 times more frequently cited in peer-reviewed literature and influenced all the selected COVID-19 treatment guidelines. All trials started recruiting from similar dates. RECOVERY was a single-country, multicentre platform trial at low risk of bias, features which also fail to distinguish it from the other trials. RECOVERY was distinguished by more strongly pragmatic design features, more centers, and more rapid recruitment resulting in a larger sample size and early publication. CONCLUSION: Higher pragmatism scores may contribute to recruiting more centers and more rapid recruitment of patients at each center, leading to larger size, earlier publication, and greater scientific and guideline uptake. By eliminating equipoise, RECOVERY rendered other simultaneous trials redundant. Further work is needed to confirm these findings in a larger quantitative study and to identify the individual contribution of each characteristic of pragmatism to conduct and impact of trials and their interaction in different national contexts. Until then, research waste might be reduced by designing trials with as many of the characteristics of RECOVERY as is feasible.

An application of PRECIS-2 to evaluate trial design in a pilot cluster randomised controlled trial of a community-based smoking cessation intervention for women living in disadvantaged areas of Ireland.

BACKGROUND: "We Can Quit2" (WCQ2) was a pilot cluster randomised controlled trial with an embedded process evaluation assessing the feasibility and acceptability of 'We Can Quit' (WCQ, a peer-delivered community-based stop-smoking programme for women in disadvantaged communities. The control group comprised 'enhanced usual care' offered by the Irish Health Service Executive (HSE). The PRagmatic Explanatory Continuum Indicator Summary (PRECIS-2) is a tool to assess whether a trial design is more explanatory (working under ideal conditions) or pragmatic (working under 'real-world' conditions). The aim of this paper was to retrospectively evaluate the WCQ2 pilot trial using PRECIS-2 to inform the decision-making process on progression to a future definitive trial (DT). METHODS: The WCQ2 trial protocol and HSE standard stop-smoking service were described across the nine PRECIS-2 domains: eligibility, recruitment, setting, organisation, flexibility-delivery, flexibility-adherence, follow-up and primary outcome. Team members scored the domains as pragmatic or explanatory for each arm in a half-day workshop. RESULTS: Seven team members (practitioners and researchers) assessed the overall trial design as more explanatory than pragmatic. Important differences emerged between the two arms. WCQ targeted adult women from disadvantaged communities whereas HSE run a limited enhanced service for all quitters. Trial recruitment was challenging, intense efforts were needed as the trial proceeded. WCQ was delivered in a non-clinical community setting, HSE services in a clinical setting. WCQ organisation was co-designed with community partners and comprises peer-to-peer group support delivered by trained lay community facilitators, whereas HSE one-to-one support is delivered by Smoking Cessation Officers with a clinical background. Only WCQ allowed flexibility in delivery and adherence. Follow-up was more intensive in WCQ. Greater efforts to improve participant retention will be required in a future DT. CONCLUSIONS: PRECIS-2 allowed the reflection of practitioners and researchers on similarities and differences between intervention and control arms. Results will inform the decision on progression to an effectiveness DT, which will require more a pragmatic and less explanatory design. This novel use of PRECIS-2 to retrospectively evaluate a complex community-based pilot trial in advance of a full DT will also support learning for those undertaking hybrid trials of implementation and effectiveness. TRIAL REGISTRATION: This trial is registered with the ISRCTN registry ( No. 74721694 ).

Ethics and regulatory complexities posed by a pragmatic clinical trial: a case study from Lilongwe, Malawi.

Background: Pragmatic clinical trials generally rely on real world data and have the potential to generate real world evidence. This approach arose from concerns that many trial results did not adequately inform real world practice. However, maintaining the real world setting during the conduct of a trial and ensuring adequate protection for research participants can be challenging. Best practices in research oversight for pragmatic clinical trials are nascent and underdeveloped, especially in developing countries. Methods: We use the PRECIS-2 tool to present a case study from Lilongwe in Malawi to describe ethical and regulatory challenges encountered during the conduct of a pragmatic trial and suggest possible solutions. Results: In this article, we highlight the following six issues: (1) one public facility hosting several pragmatic trials within the same period; (2) research participants refusing financial incentives; (3) inadequate infrastructure and high workload to conduct research; (4) silos among partner organisations involved in delivery of health care; (5) individuals influencing the implementation of revised national guidelines; (6) difficulties with access to electronic medical records. Conclusion: Multiple stakeholder engagement is critical to the conduct of pragmatic trials, and even with careful stakeholder engagement, continuous monitoring by gatekeepers is essential. In the Malawian context, active engagement of the district research committees can complement the work of the research ethics committees (RECs).

Design characteristics of comparative effectiveness trials for the relief of symptomatic dyspepsia: A systematic review.

BACKGROUND: Dyspepsia represents a symptom domain rather than a diagnostic condition and covers a wide range of complex, underlying pathophysiologies that are not well understood. The review explores comparative effectiveness interventions for the treatment of symptomatic dyspepsia along a pragmatic-explanatory continuum. The aim is to identify relevant design characteristics applicable to future upper gastrointestinal comparative effectiveness research employing integrative medicine. METHODS: Medline, CINAHL, Scopus, Cochrane Central Register of Controlled Trials (CENTRAL) and WHO Clinical Trials were systematically searched until January 2019. Included articles were original research with two or more comparative intervention arms for the primary outcome; relief of symptomatic dyspepsia. Evaluation of the studies was conducted using the pragmatic-explanatory continuum indicator summary (PRECIS-2) tool. RESULTS: Thirty-six articles were included in the review. A total of 68 Patient Reported Outcome Measurements (PROMs), utilizing 50 different formats were deployed across the studies. The appraisal process revealed eligibility, flexibility in adherence, flexibility in delivery and organization domains further aligned towards an explanatory design. CONCLUSION: This review identified three design characteristics relevant for future comparative effectiveness research for the treatment of upper gastrointestinal disorders in a community setting. Extensive exclusion eligibility criteria limited the generalization of comparative effectiveness study results by removing sub-groups of the target populations more at risk of dyspeptic symptoms. The requirement for entry endoscopy was found to be common and not always pragmatically justifiable. Development of validated PROMs appropriate for a generic application to upper gastrointestinal disorders would be advantageous for future comparative effectiveness research within integrative medicine.