Learning health system benefits: Development and initial validation of a framework.

Introduction: Implementation of research findings in clinical practice often is not realized or only partially achieved, and if so, with a significant delay. Learning health systems (LHSs) hold promise to overcome this problem by embedding clinical research and evidence-based best practices into care delivery, enabling innovation and continuous improvement. Implementing an LHS is a complex process that requires participation and resources of a wide range of stakeholders, including healthcare leaders, clinical providers, patients and families, payers, and researchers. Engaging these stakeholders requires communicating clear, tangible value propositions. Existing models identify broad categories of benefits but do not explicate the full range of benefits or ways they can manifest in different organizations. Methods: To develop such a framework, a working group with representatives from six Clinical and Translational Science Award (CTSA) hubs reviewed existing literature on LHS characteristics, models, and goals; solicited expert input; and applied the framework to their local LHS experiences. Results: The Framework of LHS Benefits includes six categories of benefits (quality, safety, equity, patient satisfaction, reputation, and value) relevant for a range of stakeholders and defines key concepts within each benefit. Applying the framework to five LHS case examples indicated preliminary face validity across varied LHS approaches and revealed three dimensions in which the framework is relevant: defining goals of individual LHS projects, facilitating collaboration based on shared values, and establishing guiding tenets of an LHS program or mission. Conclusion: The framework can be used to communicate the value of an LHS to different stakeholders across varied contexts and purposes, and to identify future organizational priorities. Further validation will contribute to the framework's evolution and support its potential to inform the development of tools to evaluate LHS impact.

Implementation of a bundle to improve HIV testing during hospitalization for people who inject drugs.

Background: Increased HIV testing is essential to ending the HIV epidemic. People who inject drugs (PWID) are among the highest risk for HIV infection. Previous research at Tufts Medical Center identified low HIV testing rates in hospitalized PWID. Our research team aimed to identify and overcome barriers to inpatient HIV screening of PWID using implementation science methods. Methods: Stakeholders were engaged to gather perspectives on barriers and facilitators of HIV testing. A PWID care bundle was developed and implemented, which included (1) HIV screening; (2) hepatitis A, B, and C testing and vaccination; (3) medications for opioid use disorder; and (4) naloxone prescription. Strategies from all nine Expert Recommendations for Implementing Change (ERIC) clusters guided the implementation plan. Stakeholder feedback was gathered throughout implementation, and implementation outcomes of acceptability and feasibility were assessed. Results: PWID overall felt comfortable with HIV testing being offered while hospitalized. Clinicians cited that the main barriers to HIV testing were discomfort and confusion around consenting requirements. Many resident physicians surveyed reported that, at times, they forgot HIV testing for PWID. Overall, though, resident physicians felt that the PWID bundle was useful and did not distract from other patient care responsibilities. Conclusions: Engagement of key stakeholders to increase HIV testing in an inpatient setting led to the implementation of a PWID bundle, which was feasible and acceptable. Bundling evidence-informed care elements for inpatient PWID should be investigated further.

People who inject drugs (PWID) are at an increased risk of contracting HIV. HIV testing is a key strategy to stop the spread of HIV. Our study created a bundle of services and tests to offer to all PWID who were admitted to the hospital. The bundle included HIV testing; hepatitis A, B, and C testing and vaccination; medications for opioid use disorder; and prescription for Narcan, a medication that can reverse opioid overdose. We then asked doctors and patients how they felt about the bundle and any barriers and facilitators that they predicted for expanding HIV testing to PWID while admitted to the hospital. Patients were accepting of expanding HIV testing, and resident physicians felt it was important as well and was a manageable addition to their list of responsibilities. However, the most likely part of the bundle to be forgotten was HIV testing. This study lays the groundwork for bundling services for PWID while they are hospitalized. We also highlight areas for future exploration.

Participant and research team perspectives on the conduct of a remote therapeutic COVID-19 clinical trial: A mixed methods approach.

Background: Responding to the need to investigate potential treatments of COVID-19, a research team employed a telehealth platform to determine whether niclosamide, an oral anthelmintic drug that had shown antiviral activity, reduced SARS-CoV-2 shedding and duration of symptoms in patients with mild-to-moderate symptoms of COVID-19. To encourage compliance with patient self-quarantine, this randomized placebo-controlled clinical trial was conducted utilizing a remote telehealth design to complete all study visits, monitor symptoms, and coordinate participant self-collected specimens. Methods: A mixed methods approach employing surveys and interviews of trial participants and interviews of research team members was used to collect their experiences with and perspectives on the acceptability of the remote clinical trial design and delivery. Results: Of the 67 eligible trial participants invited to take part in a study to evaluate the telehealth platform, 46% (n = 31) completed a post-participation survey. While 97% (n = 30) of respondents had not previously participated in a clinical trial, 77% (n = 24) reported they would consider taking part in a future remote research study. The majority of respondents were moderately or very comfortable (93%) with using the technology. Conclusions: The COVID-19 crisis was a call to action to expand understanding of the conduct of remote clinical trials, including the experiences of research participants. Our findings showed that this approach can be both effective for the conduct of research and positive for participants. Further research on the use of telehealth research platforms seems warranted in rural, underserved populations, and remote trials of prevention, screening, and treatment.

Lessons Learned from the Pathways to Community Health Study to Evaluate the Transition of Care from Jail to Community for Men with HIV.

Time spent in jail can provide opportunities to deliver comprehensive medical care, including screening and treatment for HIV; however, engagement in HIV care postrelease is often fragmented. Identifying ways to improve the transition of care from jail to community for people with HIV (PWH) may help with engagement in HIV care postrelease. We evaluated the current HIV care transition processes of one jail in Massachusetts and identified change ideas to facilitate improving the transition of care from the jail to the community for PWH. We conducted qualitative interviews in 2018-2019 with incarcerated men with HIV (n = 17), jail staff (n = 7), and community providers (n = 6) to understand the processes of HIV care prerelease from the jail and engagement in care on release. Data from these interviews and quality improvement tools were used to identify ways to improve the release process for PWH, such as using a release planning checklist, to help ensure that a 30-day supply of HIV medication and an appointment with a community provider within 30 days of release were provided. We identified communication process inefficiencies related to knowing release dates between the HIV care team and case managers that prevented providing HIV medications on release. We worked with jail administrators to find ways to improve the prerelease planning process, which is vital to the continuity of successful HIV care. The use of quality improvement methods generated a list of testable change ideas to improve the release planning process to better align with the Centers for Disease Control and Prevention guidelines, which has implications for PWH and public health.

Prioritizing studies of COVID-19 and lessons learned.

INTRODUCTION: COVID-19 altered research in Clinical and Translational Science Award (CTSA) hubs in an unprecedented manner, leading to adjustments for COVID-19 research. METHODS: CTSA members volunteered to conduct a review on the impact of CTSA network on COVID-19 pandemic with the assistance from NIH survey team in October 2020. The survey questions included the involvement of CTSAs in decision-making concerning the prioritization of COVID-19 studies. Descriptive and statistical analyses were conducted to analyze the survey data. RESULTS: 60 of the 64 CTSAs completed the survey. Most CTSAs lacked preparedness but promptly responded to the pandemic. Early disruption of research triggered, enhanced CTSA engagement, creation of dedicated research areas and triage for prioritization of COVID-19 studies. CTSAs involvement in decision-making were 16.75 times more likely to create dedicated diagnostic laboratories (95% confidence interval [CI] = 2.17-129.39; P < 0.01). Likewise, institutions with internal funding were 3.88 times more likely to establish COVID-19 dedicated research (95% CI = 1.12-13.40; P < 0.05). CTSAs were instrumental in securing funds and facilitating establishment of laboratory/clinical spaces for COVID-19 research. Workflow was modified to support contracting and IRB review at most institutions with CTSAs. To mitigate chaos generated by competing clinical trials, central feasibility committees were often formed for orderly review/prioritization. CONCLUSIONS: The lessons learned from the COVID-19 pandemic emphasize the pivotal role of CTSAs in prioritizing studies and establishing the necessary research infrastructure, and the importance of prompt and flexible research leadership with decision-making capacity to manage future pandemics.

Increased HIV testing in people who use drugs hospitalized in the first wave of the COVID-19 pandemic.

People who use drugs (PWUD) often experience barriers to preventative health care. During the COVID-19 pandemic, due to lapses in harm reduction services, several public health experts forecasted subsequent increases in diagnosis of HIV in PWUD. As many inpatient hospitals reworked patient flow during the COVID-19 surge, we hypothesized that HIV testing in PWUD would decrease. To answer this question, we compiled a deidentified list of hospitalized patients with electronic medical record indicators of substance use-a positive urine toxicology screen, prescribed medications to treat opioid use disorder, a positive CIWA score, or a positive CAGE score-admitted between January, 2020 and August, 2020. The outcome of interest was HIV test completion during inpatient hospitalization. The study used logistic regression to examine associations between type of substance use and receipt of HIV test. The study grouped substance use type into four groups (1) opioids (oxycodone, fentanyl, or other opiates) or opioid use disorder treatments (methadone, buprenorphine, naltrexone); (2) stimulant use (cocaine or amphetamines); (3) alcohol use (presence of a positive CAGE or CIWA score or alcohol present on toxicology screen); and (4) benzodiazepine use (benzodiazepines present on toxicology screen). The proportion of PWUD who were tested for HIV increased from 10.4% in January, 2020 to 28.2% in April, 2020 and back down to 12% in August. Notably, there was an inverse trend over time for number of people hospitalized with drug use, from 259 in January to a nadir of 85 in April, and then up to 217 in August, 2020. Contrary to our hypothesis, HIV testing increased during the COVID-19 pandemic, and we discuss explanations for this finding. The decrease in HIV testing post-pandemic peak is a reminder that we must work to develop interventions that lead to sustained high rates of HIV testing for all people, and especially for PWUD.

Stakeholder engagement in methodological research: Development of a clinical decision support tool.

INTRODUCTION: Shared patient-clinician decision-making is central to choosing between medical treatments. Decision support tools can have an important role to play in these decisions. We developed a decision support tool for deciding between nonsurgical treatment and surgical total knee replacement for patients with severe knee osteoarthritis. The tool aims to provide likely outcomes of alternative treatments based on predictive models using patient-specific characteristics. To make those models relevant to patients with knee osteoarthritis and their clinicians, we involved patients, family members, patient advocates, clinicians, and researchers as stakeholders in creating the models. METHODS: Stakeholders were recruited through local arthritis research, advocacy, and clinical organizations. After being provided with brief methodological education sessions, stakeholder views were solicited through quarterly patient or clinician stakeholder panel meetings and incorporated into all aspects of the project. RESULTS: Participating in each aspect of the research from determining the outcomes of interest to providing input on the design of the user interface displaying outcome predications, 86% (12/14) of stakeholders remained engaged throughout the project. Stakeholder engagement ensured that the prediction models that form the basis of the Knee Osteoarthritis Mathematical Equipoise Tool and its user interface were relevant for patient-clinician shared decision-making. CONCLUSIONS: Methodological research has the opportunity to benefit from stakeholder engagement by ensuring that the perspectives of those most impacted by the results are involved in study design and conduct. While additional planning and investments in maintaining stakeholder knowledge and trust may be needed, they are offset by the valuable insights gained.

Implementing Common Metrics across the NIH Clinical and Translational Science Awards (CTSA) consortium.

The Clinical and Translational Science Award (CTSA) Consortium and the National Center for Advancing Translational Science (NCATS) undertook a Common Metrics Initiative to improve research processes across the national CTSA Consortium. This was implemented by Tufts Clinical and Translational Science Institute at the 64 CTSA academic medical centers. Three metrics were collaboratively developed by NCATS staff, CTSA Consortium teams, and outside consultants for Institutional Review Board Review Duration, Careers in Clinical and Translational Research, and Pilot Award Publications and Subsequent Funding. The implementation program included training on the metric operational guidelines, data collection, data reporting system, and performance improvement framework. The implementation team provided small-group coaching and technical assistance. Collaborative learning sessions, driver diagrams, and change packages were used to disseminate best and promising practices. After 14 weeks, 84% of hubs had produced a value for one metric and about half had produced an initial improvement plan. Overall, hubs reported that the implementation activities facilitated their Common Metrics performance improvement process. Experiences implementing the first three metrics can inform future directions of the Common Metrics Initiative and other research groups implementing standardized metrics and performance improvement processes, potentially including other National Institutes of Health institutes and centers.

Pilot test of an accrual Common Metric for the NIH Clinical and Translational Science Awards (CTSA) Consortium: Metric feasibility and data quality.

Failure to accrue participants into clinical trials incurs economic costs, wastes resources, jeopardizes answering research questions meaningfully, and delays translating research discoveries into improved health. This paper reports the results of a pilot test of the Median Accrual Ratio (MAR) metric developed as a part of the Common Metrics Initiative of the NIH's National Center for Advancing Translational Science (NCATS) Clinical and Translational Science Award (CTSA) Consortium. Using the metric is intended to enhance the ability of the CTSA Consortium and its "hubs" to increase subject accrual into trials within expected timeframes. The pilot test was undertaken at Tufts Clinical and Translational Science Institute (CTSI) with eight CTSA Consortium hubs. We describe the pilot test methods, and results regarding feasibility of collecting metric data and the quality of data that was collected. Participating hubs welcomed the opportunity to assess accrual efforts, but experienced challenges in collecting accrual metric data due to insufficient infrastructure and inconsistent implementation of electronic data systems and lack of uniform data definitions. Also, the metric could not be constructed for all trial designs, particularly those using competitive enrollment strategies. We offer recommendations to address the identified challenges to facilitate progress to broad accrual metric data collection and use.

Pilot test of an accrual Common Metric for the NIH Clinical and Translational Science Awards (CTSA) Consortium: Metric usefulness.

The Common Metrics Initiative aims to develop and field metrics to improve research processes within the national Clinical and Translational Science Award (CTSA) Consortium. A Median Accrual Ratio (MAR) common metric was developed to assess the results of efforts to increase subject accrual into a set of clinical trials within the expected time period. A pilot test of the MAR was undertaken at Tufts Clinical and Translational Science Institute (CTSI) with eight CTSA Consortium hubs. Post-pilot interviews were conducted with 9 CTSA Principal Investigators (PIs) and 23 pilot team members. Over three-quarters (78%) of respondents reported that the MAR could be useful for performance improvement, but also described limitations or concerns. The most commonly cited barrier to MAR use for performance improvement was difficulty in interpreting the single value that is produced. Most respondents were interested in using the MAR to assess recruitment at an individual trial level. Majority of respondents (63%) had mixed opinions about aggregating metric results across the CTSA Consortium for comparison or benchmarking. Collecting data about additional contextual factors, and comparing accrual between subgroups, were cited as potentially helping address concerns about aggregation. Significant challenges remain in ensuring that the MAR can be sufficiently useful for collaborative process improvement. We offer recommendations to potentially improve metric usefulness.

The use of patient-specific equipoise to support shared decision-making for clinical care and enrollment into clinical trials.

BACKGROUND: To enhance enrollment into randomized clinical trials (RCTs), we proposed electronic health record-based clinical decision support for patient-clinician shared decision-making about care and RCT enrollment, based on "mathematical equipoise." OBJECTIVES: As an example, we created the Knee Osteoarthritis Mathematical Equipoise Tool (KOMET) to determine the presence of patient-specific equipoise between treatments for the choice between total knee replacement (TKR) and nonsurgical treatment of advanced knee osteoarthritis. METHODS: With input from patients and clinicians about important pain and physical function treatment outcomes, we created a database from non-RCT sources of knee osteoarthritis outcomes. We then developed multivariable linear regression models that predict 1-year individual-patient knee pain and physical function outcomes for TKR and for nonsurgical treatment. These predictions allowed detecting mathematical equipoise between these two options for patients eligible for TKR. Decision support software was developed to graphically illustrate, for a given patient, the degree of overlap of pain and functional outcomes between the treatments and was pilot tested for usability, responsiveness, and as support for shared decision-making. RESULTS: The KOMET predictive regression model for knee pain had four patient-specific variables, and an r 2 value of 0.32, and the model for physical functioning included six patient-specific variables, and an r 2 of 0.34. These models were incorporated into prototype KOMET decision support software and pilot tested in clinics, and were generally well received. CONCLUSIONS: Use of predictive models and mathematical equipoise may help discern patient-specific equipoise to support shared decision-making for selecting between alternative treatments and considering enrollment into an RCT.

Intervening to enhance collaboration in translational research: A relational coordination approach.

INTRODUCTION: A core challenge of a multidisciplinary and multi-organizational translational research enterprise such as a Clinical and Translational Research Award (CTSA) is coordinating and integrating the work of individuals, workgroups, and organizations accustomed to working independently and autonomously. Tufts Clinical and Translational Science Institute (CTSI) undertook and studied a multifacted intervention to address this challenge and to create a culture of systems thinking, process awareness, responsive to others' needs, and shared decision-making. INTERVENTION: The intervention, based on relational coordination, included 1) relational interventions, in three staff retreats and a diagnostic survey to provide feedback on the current quality of relational coordination, and 2) structural interventions, in the launching of five new cross-functional teams with regular meeting structures. METHODS: A mixed-methods evaluation yielded quantitative data via two types of team surveys and qualitative data via interviews and meeting observations. RESULTS: The findings suggest that interventions to improve relational coordination are feasible for CTSAs, including good fidelity to the model and staff/physician engagement. Survey and interview data suggest model improvements in coordination and alignment. Further research about their optimal design is warranted.

Applying Process Improvement Methods to Clinical and Translational Research: Conceptual Framework and Case Examples.

There is growing appreciation that process improvement holds promise for improving quality and efficiency across the translational research continuum but frameworks for such programs are not often described. The purpose of this paper is to present a framework and case examples of a Research Process Improvement Program implemented at Tufts CTSI. To promote research process improvement, we developed online training seminars, workshops, and in-person consultation models to describe core process improvement principles and methods, demonstrate the use of improvement tools, and illustrate the application of these methods in case examples. We implemented these methods, as well as relational coordination theory, with junior researchers, pilot funding awardees, our CTRC, and CTSI resource and service providers. The program focuses on capacity building to address common process problems and quality gaps that threaten the efficient, timely and successful completion of clinical and translational studies.

The Massachusetts Emergency Medical Service Stroke Quality Improvement Collaborative, 2009-2012.

INTRODUCTION: Quality improvement collaboratives are a popular model used to address gaps between evidence-based practice and patient care. Little is known about use of such collaboratives in emergency medical services, particularly for improving prehospital stroke care. To determine the feasibility of using this approach to improve prehospital stroke care, we conducted a pilot study of the Emergency Medical Services Stroke Quality Improvement Collaborative. METHODS: Seventeen Massachusetts emergency medical service agencies participated in the quality improvement collaborative pilot project. We identified 5 prehospital stroke performance measures to assess the quality of prehospital care, guide collaborative activities, and monitor change in performance over time. During learning sessions, participants were trained in quality improvement and performance measurement, analyzed performance measure results, and shared successes and challenges. Focus groups were conducted to understand participants' experiences with the collaborative. RESULTS: Participating emergency medical service agencies collected stroke performance measures on 3,009 stroke patients during the pilot study. Adherence to 4 of 5 performance measures increased significantly over time. Participants acknowledged that the collaborative provided them with an efficient and effective framework for stroke quality improvement and peer-learning opportunities. CONCLUSION: As evidenced in Massachusetts, quality improvement collaboratives can be an effective tool to improve prehospital stroke care. The data collected, improvements made, participation of emergency medical service agencies, and positive experiences within the collaborative support the continued use of this approach.

A new taxonomy for stakeholder engagement in patient-centered outcomes research.

Despite widespread agreement that stakeholder engagement is needed in patient-centered outcomes research (PCOR), no taxonomy exists to guide researchers and policy makers on how to address this need. We followed an iterative process, including several stages of stakeholder review, to address three questions: (1) Who are the stakeholders in PCOR? (2) What roles and responsibilities can stakeholders have in PCOR? (3) How can researchers start engaging stakeholders? We introduce a flexible taxonomy called the 7Ps of Stakeholder Engagement and Six Stages of Research for identifying stakeholders and developing engagement strategies across the full spectrum of research activities. The path toward engagement will not be uniform across every research program, but this taxonomy offers a common starting point and a flexible approach.

Improving use of prehospital 12-lead ECG for early identification and treatment of acute coronary syndrome and ST-elevation myocardial infarction.

BACKGROUND: Performance of prehospital ECGs expedites identification of ST-elevation myocardial infarction and reduces door-to-balloon times for patients receiving reperfusion therapy. To fully realize this benefit, emergency medical service performance must be measured and used in feedback reporting and quality improvement. METHODS AND RESULTS: This quasi-experimental design trial tested an approach to improving emergency medical service prehospital ECGs using feedback reporting and quality improvement interventions in 2 cities' emergency medical service agencies and receiving hospitals. All patients age > or =30 years, calling 9-1-1 with possible acute coronary syndrome, were included. In total, 6994 patients were included: 1589 patients in the baseline period without feedback and 5405 in the intervention period when there were feedback reports and quality improvement interventions. Mean age was 66+/-17 years, and women represented 51%. Feedback and quality improvement increased prehospital ECG performance for patients with acute coronary syndrome from 76% to 93% (P=<0.0001) and for patients with ST-elevation myocardial infarction from 77% to 99% (P=<0.0001). Aspirin administration increased from 75% to 82% (P=0.001), but the median total emergency medical service run time remained the same at 22 minutes. The proportion of patients with door-to-balloon times of < or =90 minutes increased from 27% to 67% (P=0.006). CONCLUSIONS: Feedback reports and quality improvement improved prehospital ECG performance for patients with acute coronary syndrome and ST-elevation myocardial infarction and increased aspirin administration without prehospital transport delays. Improvements in door-to-balloon times were also seen.

The potential of training to increase acceptance and use of computerized decision support systems for medical diagnosis.

OBJECTIVE: The goals of this study were to understand the reasons underlying the limited use of medical decision-support tools and to explore the potential of a computer-based tutorial to mitigate barriers to use. BACKGROUND: Medical decision-support tools such the Acute Cardiac Ischemia Time-Insensitive Predictive Instrument (ACI-TIPI) have demonstrated statistical validity and clinical impact for patient safety but have seen limited adoption and use. METHODS: The study developed a brief Web-based "demystifying" ACI-TIPI tutorial employing case-based training and evaluated the effectiveness of that tutorial in changing self-reported attitudes and behaviors. RESULTS: Clinicians using the tutorial reported greater understanding of how to use the ACI-TIPI score appropriately and increased confidence in the score. Case studies in the tutorial that provided examples of how to use the score for actual cases were rated as especially helpful. CONCLUSION: This study suggests that a primary barrier to the use of statistical decision support tools for patient diagnosis is lack of training or experience in combining a population-based numerical risk score with other diagnostic information about the individual patient's case that is not considered in that score. The results of this study indicate that there is a potential for a relatively brief tutorial to increase acceptance and use of decision support tools for medical diagnosis. APPLICATION: These findings have the potential for the identification of methods to help clinicians learn how to use statistical and probabilistic information to better assess risk and to promote integration of decision support tools into medical decision making for improvement of patient safety.

Medical error prevention in ED triage for ACS: use of cardiac care decision support and quality improvement feedback.

Medical errors in the care of patients who present with acute coronary syndrome (ACS)include errors in emergency department (ED) triage, such as the decision to send home a patient who presents with ACS or to hospitalize a patient who does not have ACS to the cardiac care unit (CCU), and errors in treatment, such as the failure to promptly use reperfusion therapy for patients who present with ST-elevation acute myocardial infarction(AMI). ECG-based acute cardiac ischemia time-insensitive predictive instrument(ACI-TIPI) and thrombolytic predictive instruments (TPIs), with a linked TIPI information system (TIPI-IS), provide real-time, concurrent, and retrospective decision support tools and feedback for the prevention of medical errors in the care of patients who present with ACS. In real-time, ACI-TIPI probabilities printed on the ECG header for the ED physician, provide an additional piece of information for triage decision making, and the ACI-TIPI Risk Management form reduces liability risk by prompting consideration and documentation of key clinical factors in the diagnosis of ACI. Also in real-time, the TPI increases overall coronary reperfusion therapy use. Concurrent flagging by TIPI-IS uses electronically acquired ECG and hospital data to provide concurrent alerts about potential misdiagnosis or mis-triage of patients with ACS. Retrospectively TIPI-IS-based feedback reports allow performance improvement. These examples of information technology tools integrated into ECG equipment already used in hospitals to deliver patient care demonstrate the potential to adapt other existing equipment or other patient care activities to enhance patient safety and error reduction.