Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory.

BACKGROUND: The clinical research enterprise is not producing the evidence decision makers arguably need in a timely and cost effective manner; research currently involves the use of labor-intensive parallel systems that are separate from clinical care. The emergence of pragmatic clinical trials (PCTs) poses a possible solution: these large-scale trials are embedded within routine clinical care and often involve cluster randomization of hospitals, clinics, primary care providers, etc. Interventions can be implemented by health system personnel through usual communication channels and quality improvement infrastructure, and data collected as part of routine clinical care. However, experience with these trials is nascent and best practices regarding design operational, analytic, and reporting methodologies are undeveloped. METHODS: To strengthen the national capacity to implement cost-effective, large-scale PCTs, the Common Fund of the National Institutes of Health created the Health Care Systems Research Collaboratory (Collaboratory) to support the design, execution, and dissemination of a series of demonstration projects using a pragmatic research design. RESULTS: In this article, we will describe the Collaboratory, highlight some of the challenges encountered and solutions developed thus far, and discuss remaining barriers and opportunities for large-scale evidence generation using PCTs. CONCLUSION: A planning phase is critical, and even with careful planning, new challenges arise during execution; comparisons between arms can be complicated by unanticipated changes. Early and ongoing engagement with both health care system leaders and front-line clinicians is critical for success. There is also marked uncertainty when applying existing ethical and regulatory frameworks to PCTS, and using existing electronic health records for data capture adds complexity.

Evaluating automated electronic case report form data entry from electronic health records.

Background: Many clinical trials leverage real-world data. Typically, these data are manually abstracted from electronic health records (EHRs) and entered into electronic case report forms (CRFs), a time and labor-intensive process that is also error-prone and may miss information. Automated transfer of data from EHRs to eCRFs has the potential to reduce data abstraction and entry burden as well as improve data quality and safety. Methods: We conducted a test of automated EHR-to-CRF data transfer for 40 participants in a clinical trial of hospitalized COVID-19 patients. We determined which coordinator-entered data could be automated from the EHR (coverage), and the frequency with which the values from the automated EHR feed and values entered by study personnel for the actual study matched exactly (concordance). Results: The automated EHR feed populated 10,081/11,952 (84%) coordinator-completed values. For fields where both the automation and study personnel provided data, the values matched exactly 89% of the time. Highest concordance was for daily lab results (94%), which also required the most personnel resources (30 minutes per participant). In a detailed analysis of 196 instances where personnel and automation entered values differed, both a study coordinator and a data analyst agreed that 152 (78%) instances were a result of data entry error. Conclusions: An automated EHR feed has the potential to significantly decrease study personnel effort while improving the accuracy of CRF data.

Stakeholder perspectives regarding pragmatic clinical trial collateral findings.

CONTEXT: Pragmatic clinical trials (PCTs), which are becoming widespread since they are relatively inexpensive and offer important benefits for healthcare decision-making, can also present practical, ethical, and legal challenges. One such challenge involves managing "pragmatic clinical trial collateral findings" (PCT-CFs), or information emerging in a PCT that is unrelated to the primary research question(s), yet may have implications for individual patients, clinicians, or health care systems from whom or within which data were collected. The expansion of PCTs makes it likely healthcare systems will increasingly encounter PCT-CFs, yet little guidance exists regarding their appropriate management. METHODS: We conducted semi-structured interviews with key stakeholders experienced in the conduct or oversight of PCTs and those in health system leadership. Interviews explored respondents' experience with PCTs and PCT-CFs, and actual or hypothetical reactions to PCT-CF management. We used standard methods of qualitative analysis to identify key themes. FINDINGS: Forty-one stakeholders participated. Four key themes emerged. First, discussions of PCT-CFs are complicated by layers of ambiguity related to both the nature of PCTs themselves, and unanticipated results that emanate from them. Second, management of PCT-CFs is context-specific, and not amenable to a "one-size-fits-all" approach. Third, there was a wide diversity of attitudes regarding the scope of researcher responsibilities in PCTs. Fourth, PCT-CFs had generally not been previously considered by respondents, but there was widespread belief in the importance of prospective planning to anticipate such issues in future PCTs. CONCLUSIONS: PCT-CFs are likely to increase, yet those charged with PCT-CF decision-making and their disclosure are unlikely to have experience with these issues. Further deliberation about the ethical obligations and implementation processes regarding PCT-CFs is needed. To enhance the likelihood of developing sound policies and practices, such deliberations should include the input and perspectives of key stakeholders in PCTs, including professionals, policy makers, and patients.

Determining the Suitability of Registries for Embedding Clinical Trials in the United States: A Project of the Clinical Trials Transformation Initiative.

BACKGROUND: Patient registries are organized systems that use observational methods to collect uniform data on specified outcomes in a population defined by a particular disease, condition, or exposure. Data collected in registries often coincide with data that could support clinical trials. Integrating clinical trials within registries to create registry-embedded clinical trials offers opportunities to reduce duplicative data collection, identify and recruit patients more efficiently, decrease time to database lock, accelerate time to regulatory decision-making, and reduce clinical trial costs. This article describes a project of the Clinical Trials Transformation Initiative (CTTI) intended to help clinical trials researchers determine when a registry could potentially serve as the platform for the conduct of a clinical trial. METHODS: Through a review of registry-embedded clinical trials and commentaries, semi-structured interviews with experts, and a multi-stakeholder expert meeting, the project team addressed how to identify and describe essential registry characteristics, practices, and processes required to for conducting embedded clinical trials intended for regulatory submissions in the United States. RESULTS: Recommendations, suggested practices, and decision trees that facilitate the assessment of whether a registry is suitable for embedding clinical trials were developed, as well as considerations for the design of new registries. Essential registry characteristics include relevancy, robustness, reliability, and assurance of patient protections. CONCLUSIONS: The project identifies a clear role for registries in creating a sustainable and reusable infrastructure to conduct clinical trials. Adoption of these recommendations will facilitate the ability to perform high-quality and efficient prospective registry-based clinical trials.