Potential bias and lack of generalizability in electronic health record data: reflections on health equity from the National Institutes of Health Pragmatic Trials Collaboratory.

Embedded pragmatic clinical trials (ePCTs) play a vital role in addressing current population health problems, and their use of electronic health record (EHR) systems promises efficiencies that will increase the speed and volume of relevant and generalizable research. However, as the number of ePCTs using EHR-derived data grows, so does the risk that research will become more vulnerable to biases due to differences in data capture and access to care for different subsets of the population, thereby propagating inequities in health and the healthcare system. We identify 3 challenges-incomplete and variable capture of data on social determinants of health, lack of representation of vulnerable populations that do not access or receive treatment, and data loss due to variable use of technology-that exacerbate bias when working with EHR data and offer recommendations and examples of ways to actively mitigate bias.

Addressing guideline and policy changes during pragmatic clinical trials.

While conducting a set of large-scale multi-site pragmatic clinical trials involving high-impact public health issues such as end-stage renal disease, opioid use, and colorectal cancer, there were substantial changes to both policies and guidelines relevant to the trials. These external changes gave rise to unexpected challenges for the trials, including decisions regarding how to respond to new clinical practice guidelines, increased difficulty in implementing trial interventions, achieving separation between treatment groups, and differential responses across sites. In this article, we describe these challenges and the approaches used to address them. When deliberating appropriate action in the face of external changes during a pragmatic clinical trial, we recommend considering the well-being of the participants, clinical equipoise, and the strength and quality of the evidence associated with the change; involving those charged with data and safety monitoring; and where possible, planning for potential external changes as the trial is being designed. Any solution must balance the primary obligation to protect the well-being of participants with the secondary obligation to protect the integrity of the trial in order to gain meaningful answers to important public health questions.

Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory.

BACKGROUND: The clinical research enterprise is not producing the evidence decision makers arguably need in a timely and cost effective manner; research currently involves the use of labor-intensive parallel systems that are separate from clinical care. The emergence of pragmatic clinical trials (PCTs) poses a possible solution: these large-scale trials are embedded within routine clinical care and often involve cluster randomization of hospitals, clinics, primary care providers, etc. Interventions can be implemented by health system personnel through usual communication channels and quality improvement infrastructure, and data collected as part of routine clinical care. However, experience with these trials is nascent and best practices regarding design operational, analytic, and reporting methodologies are undeveloped. METHODS: To strengthen the national capacity to implement cost-effective, large-scale PCTs, the Common Fund of the National Institutes of Health created the Health Care Systems Research Collaboratory (Collaboratory) to support the design, execution, and dissemination of a series of demonstration projects using a pragmatic research design. RESULTS: In this article, we will describe the Collaboratory, highlight some of the challenges encountered and solutions developed thus far, and discuss remaining barriers and opportunities for large-scale evidence generation using PCTs. CONCLUSION: A planning phase is critical, and even with careful planning, new challenges arise during execution; comparisons between arms can be complicated by unanticipated changes. Early and ongoing engagement with both health care system leaders and front-line clinicians is critical for success. There is also marked uncertainty when applying existing ethical and regulatory frameworks to PCTS, and using existing electronic health records for data capture adds complexity.