Evaluating implementation of the World Health Organization's Strategic Approach to strengthening sexual and reproductive health policies and programs to address unintended pregnancy and unsafe abortion.

BACKGROUND: We conducted a process evaluation to assess how the World Health Organization's (WHO) Strategic Approach to strengthening sexual and reproductive health policies and programs ("the SA") was used in 15 countries that requested WHO's technical support in addressing unintended pregnancy and unsafe abortion. The SA is a three-stage planning, policy, and program implementation process. We used the social ecological model (SEM) to analyze the contextual factors that influenced SA implementation. METHODS: We used a two-phased sequential approach to data collection and analysis. In Phase A, we conducted a document and literature review and synthesized data thematically. In Phase B, we conducted interviews with stakeholders who used the SA in the countries of interest. We used a qualitative method triangulation technique to analyze and combine data from both phases to understand how the SA was implemented in each country. RESULTS: Data from 145 documents and 19 interviews described the SA process and activities in each country. All 15 countries completed Stage 1 activities. The activities of Stage 1 determined activities in subsequent stages and varied across countries. Following Stage 1, some countries focused on reforming policies to improve access to sexual and reproductive health (SRH) services whereas others focused on improving provider-level capacity to enhance SRH service quality and improving community-level SRH education. We identified factors across SEM levels that affected SA implementation, including individual- and community-level perceptions of using the SA and the recommendations that emerged from its use, organizational capacity to conduct SA activities, and how well these activities aligned with the existing political climate. Stakeholders perceived SA implementation to be country-driven and systematic in bringing attention to important SRH issues in their countries. CONCLUSION: We identified key success factors for influencing the individual, organization, and system change required for implementing the SA. These include sustaining stakeholder engagement for all SA stages, monitoring and reporting on activities, and leveraging activities and outputs from each SA stage to obtain technical and financial support for subsequent stages. Results may be used to optimize ongoing implementation efforts to improve access to and the quality of SRH services.

Navigating barriers: two-year follow up on recommendations to improve the use of maternal health guidelines in Kosovo.

BACKGROUND: Although there are a growing number of initiatives aimed at supporting guideline implementation in resource-constrained settings, few studies assess progress on achieving next steps and goals after the initial activities are completed and the initial funding period has ended. The aim of the current study was to conduct a qualitative process evaluation of progress, barriers, facilitators, and proposed solutions to operationalize nine recommendations to prepare Kosovo to implement the 2012 World Health Organization (WHO) prevention and treatment of postpartum haemorrhage guideline. METHODS/DESIGN: In 2012, we co-created nine recommendations designed to support implementing the WHO's guideline on the prevention and treatment of postpartum haemorrhage in Kosovo. The current study uses a realist evaluation approach to assess activities and progress two years after the recommendations were developed. The study involved conducting qualitative focus groups and one-on-one interviews with participants from the first meeting to evaluate the activities and progress on the nine recommendations. RESULTS: Forty-three participants provided insights into the barriers and opportunities experienced to date and proposed future directions. Although progress has been made towards implementation of a number of the recommendations, scaling up has been limited by barriers, such as lack of awareness, limited resources, and evaluation challenges. Participants proposed addressing these barriers by building within- and between-country partnerships to facilitate guideline implementation. In addition, participants reported less progress on implementing recommendations related to broader cultural changes, which indicates a need for specific and actionable recommendations to operationalize implementation efforts. CONCLUSIONS: In the two years since the initial meeting, there has been mixed progress on the recommendations. Based on participant feedback, we refined the recommendations so that they can be operationalized by health care system stakeholders in Kosovo to further support implementation efforts. It is beneficial to share these lessons learned throughout the implementation process to inform next steps in Kosovo and offer ideas for use in other settings.

Ready, Set, Change! Development and usability testing of an online readiness for change decision support tool for healthcare organizations.

BACKGROUND: To address challenges related to selecting a valid, reliable, and appropriate readiness assessment measure in practice, we developed an online decision support tool to aid frontline implementers in healthcare settings in this process. The focus of this paper is to describe a multi-step, end-user driven approach to developing this tool for use during the planning stages of implementation. METHODS: A multi-phase, end-user driven approach was used to develop and test the usability of a readiness decision support tool. First, readiness assessment measures that are valid, reliable, and appropriate for healthcare settings were identified from a systematic review. Second, a mapping exercise was performed to categorize individual items of included measures according to key readiness constructs from an existing framework. Third, a modified Delphi process was used to collect stakeholder ratings of the included measures on domains of feasibility, relevance, and likelihood to recommend. Fourth, two versions of a decision support tool prototype were developed and evaluated for usability. RESULTS: Nine valid and reliable readiness assessment measures were included in the decision support tool. The mapping exercise revealed that of the nine measures, most measures (78 %) focused on assessing readiness for change at the organizational versus the individual level, and that four measures (44 %) represented all constructs of organizational readiness. During the modified Delphi process, stakeholders rated most measures as feasible and relevant for use in practice, and reported that they would be likely to recommend use of most measures. Using data from the mapping exercise and stakeholder panel, an algorithm was developed to link users to a measure based on characteristics of their organizational setting and their readiness for change assessment priorities. Usability testing yielded recommendations that were used to refine the Ready, Set, Change! decision support tool . CONCLUSIONS: Ready, Set, Change! decision support tool is an implementation support that is designed to facilitate the routine incorporation of a readiness assessment as an early step in implementation. Use of this tool in practice may offer time and resource-saving implications for implementation.

Evaluation of the "Foundations in Knowledge Translation" training initiative: preparing end users to practice KT.

BACKGROUND: Current knowledge translation (KT) training initiatives are primarily focused on preparing researchers to conduct KT research rather than on teaching KT practice to end users. Furthermore, training initiatives that focus on KT practice have not been rigorously evaluated and have focused on assessing short-term outcomes and participant satisfaction only. Thus, there is a need for longitudinal training evaluations that assess the sustainability of training outcomes and contextual factors that may influence outcomes. METHODS: We evaluated the KT training initiative "Foundations in KT" using a mixed-methods longitudinal design. "Foundations in KT" provided training in KT practice and included three tailored in-person workshops, coaching, and an online platform for training materials and knowledge exchange. Two cohorts were included in the study (62 participants, including 46 "Foundations in KT" participants from 16 project teams and 16 decision-maker partners). Participants completed self-report questionnaires, focus groups, and interviews at baseline and at 6, 12, 18, and 24 months after the first workshop. RESULTS: Participant-level outcomes include survey results which indicated that participants' self-efficacy in evidence-based practice (F(1,8.9) = 23.7, p = 0.001, n = 45), KT activities (F(1,23.9) = 43.2, p < 0.001, n = 45), and using evidence to inform practice increased over time (F(1,11.0) = 6.0, p = 0.03, n = 45). Interviews and focus groups illustrated that participants' understanding of and confidence in using KT increased from baseline to 24 months after the workshop. Interviews and focus groups suggested that the training initiative helped participants achieve their KT project objectives, plan their projects, and solve problems over time. Contextual factors include teams with high self-reported organizational capacity and commitment to implement at the start of their project had buy-in from upper management that resulted in secured funding and resources for their project. Training initiative outcomes include participants who applied the KT knowledge and skills they learned to other projects by sharing their knowledge informally with coworkers. Sustained spread of KT practice was observed with five teams at 24 months. CONCLUSIONS: We completed a longitudinal evaluation of a KT training initiative. Positive participant outcomes were sustained until 24 months after the initial workshop. Given the emphasis on implementing evidence and the need to train implementers, these findings are promising for future KT training.

Improving opioid safety practices in primary care: protocol for the development and evaluation of a multifaceted, theory-informed pilot intervention for healthcare providers.

INTRODUCTION: In North America, drug overdose deaths are reaching unprecedented levels, largely driven by increasing prescription opioid-related deaths. Despite the development of several opioid guidelines, prescribing behaviours still contribute to poor patient outcomes and societal harm. Factors at the provider and system level may hinder or facilitate the application of evidence-based guidelines; interventions designed to address such factors are needed. METHODS AND ANALYSIS: Using implementation science and behaviour change theory, we have planned the development and evaluation of a comprehensive Opioid Self-Assessment Package, designed to increase adherence to the Canadian Opioid Guideline among family physicians. The intervention uses practical educational and self-assessment tools to provide prescribers with feedback on their current knowledge and practices, and resources to improve their practice. The evaluation approach uses a pretest and post-test design and includes both quantitative and qualitative methods at baseline and 6 months. We will recruit a purposive sample of approximately 10 family physicians in Ontario from diverse practice settings, who currently treat patients with long-term opioid therapy for chronic pain. Quantitative data will be analysed using basic descriptive statistics, and qualitative data will be analysed using the Framework Method. ETHICS AND DISSEMINATION: The University Health Network Research Ethics Board approved this study. Dissemination plan includes publications, conference presentations and brief stakeholder reports. This evidence-informed, theory-driven intervention has implications for national application of opioid quality improvement tools in primary care settings. We are engaging experts and end users in advisory and stakeholder roles throughout our project to increase its national relevance, application and sustainability. The performance measures could be used as the basis for health system quality improvement indicators to monitor opioid prescribing. Additionally, the methods and approach used in this study could be adapted for other opioid guidelines, or applied to other areas of preventive healthcare and clinical guideline implementation processes.

Low- and middle-income countries face many common barriers to implementation of maternal health evidence products.

OBJECTIVES: To explore similarities and differences in challenges to maternal health and evidence implementation in general across several low- and middle-income countries (LMICs) and to identify common and unique themes representing barriers to and facilitators of evidence implementation in LMIC health care settings. STUDY DESIGN: Secondary analysis of qualitative data. SETTING: Meeting reports and articles describing projects undertaken by the authors in five LMICs on three continents were analyzed. Projects focused on identifying barriers to and facilitators of implementation of evidence products: five World Health Organization maternal health guidelines, and a knowledge translation strategy to improve adherence to tuberculosis treatment. Data were analyzed using thematic content analysis. RESULTS: Among identified barriers to evidence implementation, a high degree of commonality was found across countries and clinical areas, with lack of financial, material, and human resources most prominent. In contrast, few facilitators were identified varied substantially across countries and evidence implementation products. CONCLUSION: By identifying common barriers and areas requiring additional attention to ensure capture of unique barriers and facilitators, these findings provide a starting point for development of a framework to guide the assessment of barriers to and facilitators of maternal health and potentially to evidence implementation more generally in LMICs.

Barriers, Facilitators and Priorities for Implementation of WHO Maternal and Perinatal Health Guidelines in Four Lower-Income Countries: A GREAT Network Research Activity.

BACKGROUND: Health systems often fail to use evidence in clinical practice. In maternal and perinatal health, the majority of maternal, fetal and newborn mortality is preventable through implementing effective interventions. To meet this challenge, WHO's Department of Reproductive Health and Research partnered with the Knowledge Translation Program at St. Michael's Hospital (SMH), University of Toronto, Canada to establish a collaboration on knowledge translation (KT) in maternal and perinatal health, called the GREAT Network (Guideline-driven, Research priorities, Evidence synthesis, Application of evidence, and Transfer of knowledge). We applied a systematic approach incorporating evidence and theory to identifying barriers and facilitators to implementation of WHO maternal heath recommendations in four lower-income countries and to identifying implementation strategies to address these. METHODS: We conducted a mixed-methods study in Myanmar, Uganda, Tanzania and Ethiopia. In each country, stakeholder surveys, focus group discussions and prioritization exercises were used, involving multiple groups of health system stakeholders (including administrators, policymakers, NGOs, professional associations, frontline healthcare providers and researchers). RESULTS: Despite differences in guideline priorities and contexts, barriers identified across countries were often similar. Health system level factors, including health workforce shortages, and need for strengthened drug and equipment procurement, distribution and management systems, were consistently highlighted as limiting the capacity of providers to deliver high-quality care. Evidence-based health policies to support implementation, and improve the knowledge and skills of healthcare providers were also identified. Stakeholders identified a range of tailored strategies to address local barriers and leverage facilitators. CONCLUSION: This approach to identifying barriers, facilitators and potential strategies for improving implementation proved feasible in these four lower-income country settings. Further evaluation of the impact of implementing these strategies is needed.

The development of an online decision support tool for organizational readiness for change.

BACKGROUND: Much importance has been placed on assessing readiness for change as one of the earliest steps of implementation, but measuring it can be a complex and daunting task. Organizations and individuals struggle with how to reliably and accurately measure readiness for change. Several measures have been developed to help organizations assess readiness, but these are often underused due to the difficulty of selecting the right measure. In response to this challenge, we will develop and test a prototype of a decision support tool that is designed to guide individuals interested in implementation in the selection of an appropriate readiness assessment measure for their setting. METHODS: A multi-phase approach will be used to develop the decision support tool. First, we will identify key measures for assessing organizational readiness for change from a recently completed systematic review. Included measures will be those developed for healthcare settings (e.g., acute care, public health, mental health) and that have been deemed valid and reliable. Second, study investigators and field experts will engage in a mapping exercise to categorize individual items of included measures according to key readiness constructs from an existing framework. Third, a stakeholder panel will be recruited and consulted to determine the feasibility and relevance of the selected measures using a modified Delphi process. Fourth, findings from the mapping exercise and stakeholder consultation will inform the development of a decision support tool that will guide users in appropriately selecting change readiness measures. Fifth, the tool will undergo usability testing. DISCUSSION: Our proposed decision support tool will address current challenges in the field of organizational change readiness by aiding individuals in selecting a valid and reliable assessment measure that is relevant to user needs and practice settings. We anticipate that implementers and researchers who use our tool will be more likely to conduct readiness for change assessments in their settings when planning for implementation. This, in turn, may contribute to more successful implementation outcomes. We will test this tool in a future study to determine its efficacy and impact on implementation processes.