Is this study feasible? Facilitating management of pragmatic trial planning milestones under a phased award funding mechanism.

BACKGROUND: Improving efficiencies in clinical research is crucial to translation of findings into practice and delivery of effective, patient-centered health care. This paper describes a project that monitored pragmatic clinical trials by working with investigators to track achievement of early phase milestones. The National Institutes of Health (NIH) Pragmatic Trials Collaborative Project supported scientifically diverse, low-cost, randomized, controlled, pragmatic clinical intervention trials. Funds were available through a cooperative agreement award mechanism, with the initial phase supporting trial planning and the subsequent 4-year awards funding trial implementation. A coordinating center provided evaluation and administrative support, which included capturing progress toward achieving milestones. METHODS: Six funded trials participated in monthly calls throughout the first year to identify and demonstrate metrics and deliverables for each milestone in the Notice of Grant Award. Interviews were conducted with investigators, trial team members, and NIH program officers/project scientists to discuss their perceptions of the impact and value of the management strategy. RESULTS: Five of six trials transitioned to the implementation phase with milestones ranging from 6 to 15 and quantifiable metrics ranging from 15 to 33, for a total of 121 deliverables. One third of the metrics (42, 35%) were trial-specific. Trial teams reported that the oversight was onerous but complemented their management strategies; program officers/project scientists found that documentation submitted for review was sufficient to assess trial feasibility; and investigators reported advantages to the phased award mechanism, such as leverage to secure commitments from stakeholders and collaborators, help with task prioritization, and earlier consultation with key members of the trial team. CONCLUSIONS: Implementing systematic approaches to identify milestones and track metrics can strengthen the evidence base regarding time and effort to plan and conduct pragmatic clinical trials. Investigators were unaccustomed to producing evidence of performance, and it was challenging to determine what documentation to provide. Efforts to standardize expectations regarding milestones that mark a significant change or stage in trial development or that represent minimum success criteria may provide guidance for more effective and efficient trial management. A framework with clearly specified metrics is especially critical for transparency, particularly when funding decisions are contingent on both merit and feasibility.

Framing the conversation: use of PRECIS-2 ratings to advance understanding of pragmatic trial design domains.

BACKGROUND: There continues to be debate about what constitutes a pragmatic trial and how it is distinguished from more traditional explanatory trials. The NIH Pragmatic Trials Collaborative Project, which includes five trials and a coordinating unit, has adopted the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) instrument. The purpose of the study was to collect PRECIS-2 ratings at two points in time to assess whether the tool was sensitive to change in trial design, and to explore with investigators the rationale for rating shifts. METHODS: A mixed-methods design included sequential collection and analysis of quantitative data (PRECIS-2 ratings) and qualitative data. Ratings were collected at two annual, in-person project meetings, and subsequent interviews conducted with investigators were recorded, transcribed, and coded using NVivo 11 Pro for Windows. Rating shifts were coded as either (1) actual change (reflects a change in procedure or protocol), (2) primarily a rating shift reflecting rater variability, or (3) themes that reflect important concepts about the tool and/or pragmatic trial design. RESULTS: Based on PRECIS-2 ratings, each trial was highly pragmatic at the planning phase and remained so 1 year later in the early phases of trial implementation. Over half of the 45 paired ratings for the nine PRECIS-2 domains indicated a rating change from Time 1 to Time 2 (N = 24, 53%). Of the 24 rating changes, only three represented a true change in the design of the trial. Analysis of rationales for rating shifts identified critical themes associated with the tool or pragmatic trial design more generally. Each trial contributed one or more relevant comments, with Eligibility, Flexibility of Adherence, and Follow-up each accounting for more than one. CONCLUSIONS: PRECIS-2 has proved useful for "framing the conversation" about trial design among members of the Pragmatic Trials Collaborative Project. Our findings suggest that design elements assessed by the PRECIS-2 tool may represent mostly stable decisions. Overall, there has been a positive response to using PRECIS-2 to guide conversations around trial design, and the project's focus on the use of the tool by this group of early adopters has provided valuable feedback to inform future trainings on the tool.