Organizing the dissemination and implementation field: who are we, what are we doing, and how should we do it?

Two decades into its tenure as a field, dissemination and implementation (D&I) scientists have begun a process of self-reflection, illuminating a missed opportunity to bridge the gap between research and practice-one of the field's foundational objectives. In this paper, we, the authors, assert the research-to-practice gap has persisted, in part due to an inadequate characterization of roles, functions, and processes within D&I. We aim to address this issue, and the rising tension between D&I researchers and practitioners, by proposing a community-centered path forward that is grounded in equity.We identify key players within the field and characterize their unique roles using the translational science spectrum, a model originally developed in the biomedical sciences to help streamline the research-to-practice process, as a guide. We argue that the full translational science spectrum, from basic science research, or "T0," to translation to community, or "T4," readily applies within D&I and that in using this framework to clarify roles, functions, and processes within the field, we can facilitate greater collaboration and respect across the entire D&I research-to-practice continuum. We also highlight distinct opportunities (e.g., changes to D&I scientific conference structures) to increase regular communication and engagement between individuals whose work sits at different points along the D&I translational science spectrum that can accelerate our efforts to close the research-to-practice gap and achieve the field's foundational objectives.

The Pathway Building Technique in Implementation Research Using Mixed Methods Design.

BACKGROUND: Data integration refers to combining quantitative and qualitative data in mixed methods. It can be achieved through several integration procedures. The building integration procedure can be used for developing quantitative instruments by integrating data from the qualitative phase. There are limited examples of data integration using the building procedure in mixed methods and implementation science. PURPOSE: The purpose of this article is to illustrate how the pathway building technique can be used to integrate data in mixed methods research through concurrent use of implementation science models and frameworks. METHODS: This two pathway building technique was developed based on a mixed methods implementation project of developing implementation strategies to promote compassionate nursing care of complex patients. RESULTS: The first pathway is the integration of qualitative data from the first phase of mixed methods study with implementation models and frameworks to create a quantitative instrument (i.e., a Q-sort survey) for the subsequent phase. The second pathway is the operationalization of the Q-sort survey results (i.e., implementation strategies) using an implementation science specification framework. CONCLUSION: The pathway technique is valuable for mixed methods research and implementation science as it offers a theory-based innovative method to tackle integration challenge.

Corrigendum to "Exploring social movement concepts and actions in a knowledge uptake and sustainability context: A concept analysis" [Int J Nurs Sci 9/4 (2022) 411-421].

[This corrects the article DOI: 10.1016/j.ijnss.2022.08.003.].

Exploring social movement concepts and actions in a knowledge uptake and sustainability context: A concept analysis.

Objectives: To share a concept analysis of social movement aimed at advancing its application to evidence uptake and sustainability in health-care. Methods: We applied Walker and Avant method to clarify the concept of social movement in the context of knowledge uptake and sustainability. Peer-reviewed and grey literature databases were systematically searched for relevant reports that described how social movement action led to evidence-based practice changes in health and community settings. Titles, abstracts and full texts were reviewed independently and in duplicate, resulting in 38 included articles. Results: Social movement action for knowledge uptake and sustainability can be defined as individuals, groups, or organizations that, as voluntary and intrinsically motivated change agents, mobilize around a common cause to improve outcomes through knowledge uptake and sustainability. The 10 defining attributes, three antecedents and three consequences that we identified are dynamic and interrelated, often mutually reinforcing each other to fortify various aspects of the social movement. Examples of defining attributes include an urgent need for action, collective action and collective identity. The concept analysis resulted in the development of the Social Movement Action Framework. Conclusions: Social movement action can provide a lens through which we view implementation science. Collective action and collective identity - concepts less frequently canvassed in implementation science literature - can lend insight into grassroots approaches to uptake and sustainability. Findings can also inform providers and change leaders on the practicalities of harnessing social movement action for real-world change initiatives. By mobilizing individuals, groups, or organizations through social movement approaches, they can engage as powered change agents and teams that impact the individual, organizational and health systems levels to facilitate knowledge uptake and sustainability.

Selecting implementation models, theories, and frameworks in which to integrate intersectional approaches.

BACKGROUND: Models, theories, and frameworks (MTFs) provide the foundation for a cumulative science of implementation, reflecting a shared, evolving understanding of various facets of implementation. One under-represented aspect in implementation MTFs is how intersecting social factors and systems of power and oppression can shape implementation. There is value in enhancing how MTFs in implementation research and practice account for these intersecting factors. Given the large number of MTFs, we sought to identify exemplar MTFs that represent key implementation phases within which to embed an intersectional perspective. METHODS: We used a five-step process to prioritize MTFs for enhancement with an intersectional lens. We mapped 160 MTFs to three previously prioritized phases of the Knowledge-to-Action (KTA) framework. Next, 17 implementation researchers/practitioners, MTF experts, and intersectionality experts agreed on criteria for prioritizing MTFs within each KTA phase. The experts used a modified Delphi process to agree on an exemplar MTF for each of the three prioritized KTA framework phases. Finally, we reached consensus on the final MTFs and contacted the original MTF developers to confirm MTF versions and explore additional insights. RESULTS: We agreed on three criteria when prioritizing MTFs: acceptability (mean = 3.20, SD = 0.75), applicability (mean = 3.82, SD = 0.72), and usability (median = 4.00, mean = 3.89, SD = 0.31) of the MTF. The top-rated MTFs were the Iowa Model of Evidence-Based Practice to Promote Quality Care for the 'Identify the problem' phase (mean = 4.57, SD = 2.31), the Consolidated Framework for Implementation Research for the 'Assess barriers/facilitators to knowledge use' phase (mean = 5.79, SD = 1.12), and the Behaviour Change Wheel for the 'Select, tailor, implement interventions' phase (mean = 6.36, SD = 1.08). CONCLUSIONS: Our interdisciplinary team engaged in a rigorous process to reach consensus on MTFs reflecting specific phases of the implementation process and prioritized each to serve as an exemplar in which to embed intersectional approaches. The resulting MTFs correspond with specific phases of the KTA framework, which itself may be useful for those seeking particular MTFs for particular KTA phases. This approach also provides a template for how other implementation MTFs could be similarly considered in the future. TRIAL REGISTRATION: Open Science Framework Registration: osf.io/qgh64.

Development and usability testing of tools to facilitate incorporating intersectionality in knowledge translation.

BACKGROUND: The field of knowledge translation (KT) has been criticized for neglecting contextual and social considerations that influence health equity. Intersectionality, a concept introduced by Black feminist scholars, emphasizes how human experience is shaped by combinations of social factors (e.g., ethnicity, gender) embedded in systemic power structures. Its use has the potential to advance equity considerations in KT. Our objective was to develop and conduct usability testing of tools to support integrating intersectionality in KT through three key phases of KT: identifying the gap; assessing barriers to knowledge use; and selecting, tailoring, and implementing interventions. METHODS: We used an integrated KT approach and assembled an interdisciplinary development committee who drafted tools. We used a mixed methods approach for usability testing with KT intervention developers that included semi-structured interviews and the System Usability Scale (SUS). We calculated an average SUS score for each tool. We coded interview data using the framework method focusing on actionable feedback. The development committee used the feedback to revise tools, which were formatted by a graphic designer. RESULTS: Nine people working in Canada joined the development committee. They drafted an intersectionality primer and one tool that included recommendations, activities, reflection prompts, and resources for each of the three implementation phases. Thirty-one KT intervention developers from three countries participated in usability testing. They suggested the tools to be shorter, contain more visualizations, and use less jargon. Average SUS scores of the draft tools ranged between 60 and 78/100. The development committee revised and shortened all tools, and added two, one-page summary documents. The final toolkit included six documents. CONCLUSIONS: We developed and evaluated tools to help embed intersectionality considerations in KT. These tools go beyond recommending the use of intersectionality to providing practical guidance on how to do this. Future work should develop guidance for enhancing social justice in intersectionality-enhanced KT.

Promoting Social and Emotional Learning in Middle School: Intervention Effects of Facing History and Ourselves.

There are fewer evidence-based social and emotional learning programs for middle school students compared to younger grades. This randomized controlled trial tests the effectiveness of Facing History and Ourselves (hereafter, Facing History) with a sample of 694 (Facing History n = 437; Comparison n = 257) students from a low-resourced school district. Youth self-identified as female (59%), Black/African American (61%), Hispanic/Latinx (18%), White (2%), and multi-racial or some other race/ethnicity (19%). Intervention students perceived their classrooms as more caring and democratic than students in the comparison group. They reported higher levels of empathy, prosocial behavior, and stronger participatory citizenship beliefs. This study expands the evidence-based of effective SEL programs available to schools.

Virtual learning collaboratives to improve urine culturing and antibiotic prescribing in long-term care: controlled before-and-after study.

BACKGROUND: Urine culturing practices are highly variable in long-term care and contribute to overprescribing of antibiotics for presumed urinary tract infections. The purpose of this study was to evaluate the use of virtual learning collaboratives to support long-term care homes in implementing a quality improvement programme focused on reducing unnecessary urine culturing and antibiotic overprescribing. METHODS: Over a 4-month period (May 2018-August 2018), 45 long-term care homes were self-selected from five regions to participate in virtual learning collaborative sessions, which provided an orientation to a quality improvement programme and guidance for implementation. A process evaluation complemented the use of a controlled before-and-after study with a propensity score matched control group (n=127) and a difference-in-difference analysis. Primary outcomes included rates of urine cultures performed and urinary antibiotic prescriptions. Secondary outcomes included rates of emergency department visits, hospital admission and mortality. An 18-month baseline period was compared with a 16-month postimplementation period with the use of administrative data sources. RESULTS: Rates of urine culturing and urinary antibiotic prescriptions per 1000 resident days decreased significantly more among long-term care homes that participated in learning collaboratives compared with matched controls (differential reductions of 19% and 13%, respectively, p<0.0001). There was no statistically significant changes to rates of emergency department visits, hospital admissions or mortality. These outcomes were observed with moderate adherence to the programme model. CONCLUSIONS: Rates of urine culturing and urinary antibiotic prescriptions declined among long-term care homes that participated in a virtual learning collaborative to support implementation of a quality improvement programme. The results of this study have refined a model to scale this programme in long-term care.

Towards a comprehensive model for understanding adaptations' impact: the model for adaptation design and impact (MADI).

BACKGROUND: Implementation science is shifting from qualifying adaptations as good or bad towards understanding adaptations and their impact. Existing adaptation classification frameworks are largely descriptive (e.g., who made the adaptation) and geared towards researchers. They do not help practitioners in decision-making around adaptations (e.g., is an adaptation likely to have negative impacts? Should it be pursued?). Moreover, they lack constructs to consider "ripple effects" of adaptations (i.e., both intended and unintended impacts on outcomes, recognizing that an adaptation designed to have a positive impact on one outcome may have unintended impacts on other outcomes). Finally, they do not specify relationships between adaptations and outcomes, including mediating and moderating relationships. The objective of our research was to promote systematic assessment of intended and unintended impacts of adaptations by using existing frameworks to create a model that proposes relationships among constructs. MATERIALS AND METHODS: We reviewed, consolidated, and refined constructs from two adaptation frameworks and one intervention-implementation outcome framework. Using the consolidated and refined constructs, we coded qualitative descriptions of 14 adaptations made to an existing evidence-based intervention; the 14 adaptations were designed in prior research by a stakeholder panel using a modified Delphi approach. Each of the 14 adaptations had detailed descriptions, including the nature of the adaptation, who made it, and its goal and reason. Using coded data, we arranged constructs from existing frameworks into a model, the Model for Adaptation Design and Impact (MADI), that identifies adaptation characteristics, their intended and unintended impacts (i.e., ripple effects), and potential mediators and moderators of adaptations' impact on outcomes. We also developed a decision aid and website ( MADIguide.org ) to help implementation scientists apply MADI in their work. RESULTS AND CONCLUSIONS: Our model and associated decision aids build on existing frameworks by comprehensively characterizing adaptations, proposing how adaptations impact outcomes, and offering practical guidance for designing adaptations. MADI encourages researchers to think about potential causal pathways of adaptations (e.g., mediators and moderators) and adaptations' intended and unintended impacts on outcomes. MADI encourages practitioners to design adaptations in a way that anticipates intended and unintended impacts and leverages best practice from research.

Applying an intersectionality lens to the theoretical domains framework: a tool for thinking about how intersecting social identities and structures of power influence behaviour.

BACKGROUND: A key component of the implementation process is identifying potential barriers and facilitators that need to be addressed. The Theoretical Domains Framework (TDF) is one of the most commonly used frameworks for this purpose. When applying the TDF, it is critical to understand the context in which behaviours occur. Intersectionality, which accounts for the interface between social identity factors (e.g. age, gender) and structures of power (e.g. ageism, sexism), offers a novel approach to understanding how context shapes individual decision-making and behaviour. We aimed to develop a tool to be used alongside applications of the TDF to incorporate an intersectionality lens when identifying implementation barriers and enablers. METHODS: An interdisciplinary Framework Committee (n = 17) prioritized the TDF as one of three models, theories, and frameworks (MTFs) to enhance with an intersectional lens through a modified Delphi approach. In collaboration with the wider Framework Committee, a subgroup considered all 14 TDF domains and iteratively developed recommendations for incorporating intersectionality considerations within the TDF and its domains. An iterative approach aimed at building consensus was used to finalize recommendations. RESULTS: Consensus on how to apply an intersectionality lens to the TDF was achieved after 12 rounds of revision. Two overarching considerations for using the intersectionality alongside the TDF were developed by the group as well as two to four prompts for each TDF domain to guide interview topic guides. Considerations and prompts were designed to assist users to reflect on how individual identities and structures of power may play a role in barriers and facilitators to behaviour change and subsequent intervention implementation. CONCLUSIONS: Through an expert-consensus approach, we developed a tool for applying an intersectionality lens alongside the TDF. Considering the role of intersecting social factors when identifying barriers and facilitators to implementing research evidence may result in more targeted and effective interventions that better reflect the realities of those involved.

Scaling-up implementation in community hospitals: a multisite interrupted time series design of the Mobilization of Vulnerable Elders (MOVE) program in Alberta.

BACKGROUND: As the population ages, older hospitalized patients are at increased risk for hospital-acquired morbidity. The Mobilization of Vulnerable Elders (MOVE) program is an evidence-informed early mobilization intervention that was previously evaluated in Ontario, Canada. The program was effective at improving mobilization rates and decreasing length of stay in academic hospitals. The aim of this study was to scale-up the program and conduct a replication study evaluating the impact of the evidence-informed mobilization intervention on various units in community hospitals within a different Canadian province. METHODS: The MOVE program was tailored to the local context at four community hospitals in Alberta, Canada. The study population was patients aged 65 years and older who were admitted to medicine, surgery, rehabilitation and intensive care units between July 2015 and July 2016. The primary outcome was patient mobilization measured by conducting visual audits twice a week, three times a day. The secondary outcomes included hospital length of stay obtained from hospital administrative data, and perceptions of the intervention assessed through a qualitative assessment. Using an interrupted time series design, the intervention was evaluated over three time periods (pre-intervention, during, and post-intervention). RESULTS: A total of 3601 patients [mean age 80.1 years (SD = 8.4 years)] were included in the overall analysis. There was a significant increase in mobilization at the end of the intervention period compared to pre-intervention, with 6% more patients out of bed (95% confidence interval (CI) 1, 11; p-value = 0.0173). A decreasing trend in median length of stay was observed, where patients on average stayed an estimated 3.59 fewer days (95%CI -15.06, 7.88) during the intervention compared to pre-intervention period. CONCLUSIONS: MOVE is a low-cost, effective and adaptable intervention that improves mobilization in older hospitalized patients. This intervention has been replicated and scaled up across various units and hospital settings.

How do researchers conceptualize and plan for the sustainability of their NIH R01 implementation projects?

BACKGROUND: Inadequate sustainability of implementation of evidence-based interventions has led to calls for research on how sustainability can be optimized. To advance our understanding of intervention sustainability, we explored how implementation researchers conceptualized and planned for the sustainability of their implemented interventions with studies funded by the United States (US) National Institutes of Health (NIH). METHODS: We used sequential, mixed methods to explore how researchers conceptualized and planned for the sustainability of the health interventions using (1) a document review of all active and completed US NIH R01 Grants and Equivalents reviewed within the Dissemination and Implementation Research in Health (DIRH) Study Section between 2004 and 2016 and (2) a qualitative content analysis of semi-structured interviews with NIH R01 DIRH grant recipients. RESULTS: We found 277 R01 profiles within the DIRH study section listed on the US NIH RePORTER website including 84 that were eligible for screening. Of the 84 unique projects, 76 (90.5%) had primary implementation outcomes. Of the 76 implementation project profiles, 51 (67.1%) made references to sustainability and none referred to sustainability planning. In both profiles and interviews, researchers conceptualized sustainability primarily as the continued delivery of interventions, programs, or implementation strategies. Few researchers referenced frameworks with sustainability constructs and offered limited information on how they operationalized frameworks. Researchers described broad categories of approaches and strategies to promote sustainability and key factors that may influence researchers to plan for sustainability, such as personal beliefs, self-efficacy, perception of their role, and the challenges of the grant funding system. CONCLUSIONS: We explored how US NIH R01 DIRH grant recipients conceptualized and planned for the sustainability of their interventions. Our results identified the need to test, consolidate, and provide guidance on how to operationalize sustainability frameworks, and to develop strategies on how funders and researchers can advance sustainability research.

Can the effects of the mobilization of vulnerable elders in Ontario (MOVE ON) implementation be replicated in new settings: an interrupted time series design.

BACKGROUND: Bed rest for older hospitalized patients places them at risk for hospital-acquired morbidity. We previously evaluated an early mobilization intervention and found it to be effective at improving mobilization rates and decreasing length of stay on internal medicine units. The aim of this study was to conduct a replication study evaluating the impact of the evidence-informed mobilization intervention on surgery, psychiatry, medicine, and cardiology inpatient units. METHODS: A multi-component early mobilization intervention was tailored to the local context at seven hospitals in Ontario, Canada. The primary outcome was patient mobilization measured by conducting visual audits twice a week, three times a day. Secondary outcomes were hospital length of stay and discharge destination, which were obtained from hospital decision support data. The study population was patients aged 65 years and older who were admitted to surgery, psychiatry, medicine, and cardiology inpatient units between March and August 2014. Using an interrupted time series design, the intervention was evaluated over three time periods-pre-intervention, during, and post-intervention. RESULTS: A total of 3098 patients [mean age 78.46 years (SD 8.38)] were included in the overall analysis. There was a significant increase in mobility immediately after the intervention period compared to pre-intervention with a slope change of 1.91 (95% confidence interval [CI] 0.74-3.08, P-value = 0.0014). A decreasing trend in median length of stay was observed in the majority of the participating sites. Overall, a median length of stay of 26.24 days (95% CI 23.67-28.80) was observed pre-intervention compared to 23.81 days (95% CI 20.13-27.49) during the intervention and 24.69 days (95% CI 22.43-26.95) post-intervention. The overall decrease in median length of stay was associated with the increase in mobility across the sites. CONCLUSIONS: MOVE increased mobilization and these results were replicated across surgery, psychiatry, medicine, and cardiology inpatient units.

A recipe for antimicrobial stewardship success: Using intervention mapping to develop a program to reduce antibiotic overuse in long-term care.

OBJECTIVE: To better understand barriers and facilitators that contribute to antibiotic overuse in long-term care and to use this information to inform an evidence and theory-informed program. METHODS: Information on barriers and facilitators associated with the assessment and management of urinary tract infections were identified from a mixed-methods survey and from focus groups with stakeholders working in long-term care. Each barrier or facilitator was mapped to corresponding determinants of behavior change, as described by the theoretical domains framework (TDF). The Rx for Change database was used to identify strategies to address the key determinants of behavior change. RESULTS: In total, 19 distinct barriers and facilitators were mapped to 8 domains from the TDF: knowledge, skills, environmental context and resources, professional role or identity, beliefs about consequences, social influences, emotions, and reinforcements. The assessment of barriers and facilitators informed the need for a multifaceted approach with the inclusion of strategies (1) to establish buy-in for the changes; (2) to align organizational policies and procedures; (3) to provide education and ongoing coaching support to staff; (4) to provide information and education to residents and families; (5) to establish process surveillance with feedback to staff; and (6) to deliver reminders. CONCLUSIONS: The use of a stepped approach was valuable to ensure that locally relevant barriers and facilitators to practice change were addressed in the development of a regional program to help long-term care facilities minimize antibiotic prescribing for asymptomatic bacteriuria. This stepped approach provides considerable opportunity to advance the design and impact of antimicrobial stewardship programs.

Sustaining knowledge translation interventions for chronic disease management in older adults: protocol for a systematic review and network meta-analysis.

BACKGROUND: Failure to sustain knowledge translation (KT) interventions impacts patients and health systems, diminishing confidence in future implementation. Sustaining KT interventions used to implement chronic disease management (CDM) interventions is of critical importance given the proportion of older adults with chronic diseases and their need for ongoing care. Our objectives are to (1) complete a systematic review and network meta-analysis of the effectiveness and cost-effectiveness of sustainability of KT interventions that target CDM for end-users including older patients, clinicians, public health officials, health services managers and policy-makers on health care outcomes beyond 1 year after implementation or the termination of initial project funding and (2) use the results of this review to complete an economic analysis of the interventions identified to be effective. METHODS: For objective 1, comprehensive searches of relevant electronic databases (e.g. MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials), websites of health care provider organisations and funding agencies will be conducted. We will include randomised controlled trials (RCTs) examining the impact of a KT intervention targeting CDM in adults aged 65 years and older. To examine cost, economic studies (e.g. cost, cost-effectiveness analyses) will be included. Our primary outcome will be the sustainability of the delivery of the KT intervention beyond 1 year after implementation or termination of study funding. Secondary outcomes will include behaviour changes at the level of the patient (e.g. symptom management) and clinician (e.g. physician test ordering) and health system (e.g. cost, hospital admissions). Article screening, data abstraction and risk of bias assessment will be completed independently by two reviewers. Using established methods, if the assumption of transitivity is valid and the evidence forms a connected network, Bayesian random-effects pairwise and network meta-analysis will be conducted. For objective 2, we will build a decision analytic model comparing effective interventions to estimate an incremental cost-effectiveness ratio. DISCUSSION: Our results will inform knowledge users (e.g. patients, clinicians, policy-makers) regarding the sustainability of KT interventions for CDM. Dissemination plan of our results will be tailored to end-users and include passive (e.g. publications, website posting) and interactive (e.g. knowledge exchange events with stakeholders) strategies. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42018084810.

Longitudinal evaluation of a course to build core competencies in implementation practice.

BACKGROUND: Few training opportunities are available for implementation practitioners; we designed the Practicing Knowledge Translation (PKT) to address this gap. The goal of PKT is to train practitioners to use evidence and apply implementation science in healthcare settings. The aim of this study was to describe PKT and evaluate participant use of implementation science theories, models, and frameworks (TMFs), knowledge, self-efficacy, and satisfaction and feedback on the course. METHODS: PKT was delivered to implementation practitioners between September 2015 and February 2016 through a 3-day workshop, 11 webinars. We assessed PKT using an uncontrolled before and after study design, using convergent parallel mixed methods. The primary outcome was use of TMFs in implementation projects. Secondary outcomes were knowledge and self-efficacy across six core competencies, factors related to each of the outcomes, and satisfaction with the course. Participants completed online surveys and semi-structured interviews at baseline, 3, 6, and 12 months. RESULTS: Participants (n = 15) reported an increase in their use of implementation TMFs (mean = 2.11; estimate = 2.11; standard error (SE) = 0.4; p = 0.03). There was a significant increase in participants' knowledge of developing an evidence-informed, theory-driven program (ETP) (estimate = 4.10; SE = 0.37; p = 0.002); evidence implementation (estimate = 2.68; SE = 0.42; p < 0.001); evaluation (estimate = 4.43; SE = 0.36; p < 0.001); sustainability, scale, and spread (estimate = 2.55; SE = 0.34; p < 0.001); and context assessment (estimate = 3.86; SE = 0.32; p < 0.001). There was a significant increase in participants' self-efficacy in developing an ETP (estimate = 3.81; SE = 0.34; p < 0.001); implementation (estimate = 3.01; SE = 0.36; p < 0.001); evaluation (estimate = 3.83; SE = 0.39; p = 0.002); sustainability, scale, and spread (estimate = 3.06; SE = 0.46; p = 0.003); and context assessment (estimate = 4.05; SE = 0.38; p = 0.016). CONCLUSION: Process and outcome measures collected indicated that PKT participants increased use of, knowledge of, self-efficacy in KT. Our findings highlight the importance of longitudinal evaluations of training initiatives to inform how to build capacity for implementers.

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study.

BACKGROUND: Systematic reviews are infrequently used by health care managers (HCMs) and policy-makers (PMs) in decision-making. HCMs and PMs co-developed and tested novel systematic review of effects formats to increase their use. METHODS: A three-phased approach was used to evaluate the determinants to uptake of systematic reviews of effects and the usability of an innovative and a traditional systematic review of effects format. In phase 1, survey and interviews were conducted with HCMs and PMs in four Canadian provinces to determine perceptions of a traditional systematic review format. In phase 2, systematic review format prototypes were created by HCMs and PMs via Conceptboard©. In phase 3, prototypes underwent usability testing by HCMs and PMs. RESULTS: Two hundred two participants (80 HCMs, 122 PMs) completed the phase 1 survey. Respondents reported that inadequate format (Mdn = 4; IQR = 4; range = 1-7) and content (Mdn = 4; IQR = 3; range = 1-7) influenced their use of systematic reviews. Most respondents (76%; n = 136/180) reported they would be more likely to use systematic reviews if the format was modified. Findings from 11 interviews (5 HCMs, 6 PMs) revealed that participants preferred systematic reviews of effects that were easy to access and read and provided more information on intervention effectiveness and less information on review methodology. The mean System Usability Scale (SUS) score was 55.7 (standard deviation [SD] 17.2) for the traditional format; a SUS score < 68 is below average usability. In phase 2, 14 HCMs and 20 PMs co-created prototypes, one for HCMs and one for PMs. HCMs preferred a traditional information order (i.e., methods, study flow diagram, forest plots) whereas PMs preferred an alternative order (i.e., background and key messages on one page; methods and limitations on another). In phase 3, the prototypes underwent usability testing with 5 HCMs and 7 PMs, 11 out of 12 participants co-created the prototypes (mean SUS score 86 [SD 9.3]). CONCLUSIONS: HCMs and PMs co-created prototypes for systematic review of effects formats based on their needs. The prototypes will be compared to a traditional format in a randomized trial.

Organizational Factors Contributing to Incivility at an Academic Medical Center and Systems-Based Solutions: A Qualitative Study.

PURPOSE: A rise in incivility has been documented in medicine, with implications for patient care, organizational effectiveness, and costs. This study explored organizational factors that may contribute to incivility at one academic medical center and potential systems-level solutions to combat it. METHOD: The authors completed semistructured individual interviews with full-time faculty members of the Department of Medicine (DOM) at the University of Toronto Faculty of Medicine, Toronto, Ontario, Canada, with clinical appointments at six affiliated hospitals, between June and September 2016. They asked about participants' experiences with incivility, potential contributing factors, and possible solutions. Two analysts independently coded a portion of the transcripts until a framework was developed with excellent agreement within the research team, as signified by the Kappa coefficient. A single coder completed analysis of the remaining transcripts. RESULTS: Forty-nine interviews with physicians from all university ranks and academic position descriptions were completed. All participants had collegial relationships with colleagues but had observed, heard of, or been personally affected by uncivil behavior. Incivility occurred furtively, face-to-face, or online. The participants identified several organizational factors that bred incivility including physician nonemployee status in hospitals, silos within the DOM, poor leadership, a culture of silence, and the existence of power cliques. They offered many systems-level solutions to combat incivility through prevention, improved reporting, and clearer consequences. CONCLUSIONS: Existing strategies to combat incivility have focused on modifying individual behavior, but opportunities may exist to reduce incivility through a greater understanding of the role of health care organizations in shaping workplace culture.

Supporting the implementation of stroke quality-based procedures (QBPs): a mixed methods evaluation to identify knowledge translation activities, knowledge translation interventions, and determinants of implementation across Ontario.

BACKGROUND: In 2013, Health Quality Ontario introduced stroke quality-based procedures (QBPs) to promote use of evidence-based practices for patients with stroke in Ontario hospitals. The study purpose was to: (a) describe the knowledge translation (KT) interventions used to support stroke QBP implementation, (b) assess differences in the planned and reported KT interventions by region, and (c) explore determinants perceived to have affected outcomes. METHODS: A mixed methods approach was used to evaluate: activities, KT interventions, and determinants of stroke QBP implementation. In Phase 1, a document review of regional stroke network work plans was conducted to capture the types of KT activities planned at a regional level; these were mapped to the knowledge to action framework. In Phase 2, we surveyed Ontario hospital staff to identify the KT interventions used to support QBP implementation at an organizational level. Phase 3 involved qualitative interviews with staff to elucidate deeper understanding of survey findings. RESULTS: Of the 446 activities identified in the document review, the most common were 'dissemination' (24.2%; n = 108), 'implementation' (22.6%; n = 101), 'implementation planning' (15.0%; n = 67), and 'knowledge tools' (10.5%; n = 47). Based on survey data (n = 489), commonly reported KT interventions included: staff educational meetings (43.1%; n = 154), champions (41.5%; n = 148), and staff educational materials (40.6%; n = 145). Survey participants perceived stroke QBP implementation to be successful (median = 5/7; interquartile range = 4-6; range = 1-7; n = 335). Forty-four people (e.g., managers, senior leaders, regional stroke network representatives, and frontline staff) participated in interviews/focus groups. Perceived facilitators to QBP implementation included networks and collaborations with external organizations, leadership engagement, and hospital prioritization of stroke QBP. Perceived barriers included lack of funding, size of the hospital (i.e., too small), lack of resources (i.e., staff and time), and simultaneous implementation of other QBPs. CONCLUSIONS: Information on the types of activities and KT interventions used to support stroke QBP implementation and the key determinants influencing uptake of stroke QBPs can be used to inform future activities including the development and evaluation of interventions to address barriers and leverage facilitators.

Evaluation of the "Foundations in Knowledge Translation" training initiative: preparing end users to practice KT.

BACKGROUND: Current knowledge translation (KT) training initiatives are primarily focused on preparing researchers to conduct KT research rather than on teaching KT practice to end users. Furthermore, training initiatives that focus on KT practice have not been rigorously evaluated and have focused on assessing short-term outcomes and participant satisfaction only. Thus, there is a need for longitudinal training evaluations that assess the sustainability of training outcomes and contextual factors that may influence outcomes. METHODS: We evaluated the KT training initiative "Foundations in KT" using a mixed-methods longitudinal design. "Foundations in KT" provided training in KT practice and included three tailored in-person workshops, coaching, and an online platform for training materials and knowledge exchange. Two cohorts were included in the study (62 participants, including 46 "Foundations in KT" participants from 16 project teams and 16 decision-maker partners). Participants completed self-report questionnaires, focus groups, and interviews at baseline and at 6, 12, 18, and 24 months after the first workshop. RESULTS: Participant-level outcomes include survey results which indicated that participants' self-efficacy in evidence-based practice (F(1,8.9) = 23.7, p = 0.001, n = 45), KT activities (F(1,23.9) = 43.2, p < 0.001, n = 45), and using evidence to inform practice increased over time (F(1,11.0) = 6.0, p = 0.03, n = 45). Interviews and focus groups illustrated that participants' understanding of and confidence in using KT increased from baseline to 24 months after the workshop. Interviews and focus groups suggested that the training initiative helped participants achieve their KT project objectives, plan their projects, and solve problems over time. Contextual factors include teams with high self-reported organizational capacity and commitment to implement at the start of their project had buy-in from upper management that resulted in secured funding and resources for their project. Training initiative outcomes include participants who applied the KT knowledge and skills they learned to other projects by sharing their knowledge informally with coworkers. Sustained spread of KT practice was observed with five teams at 24 months. CONCLUSIONS: We completed a longitudinal evaluation of a KT training initiative. Positive participant outcomes were sustained until 24 months after the initial workshop. Given the emphasis on implementing evidence and the need to train implementers, these findings are promising for future KT training.