Selecting implementation models, theories, and frameworks in which to integrate intersectional approaches.

BACKGROUND: Models, theories, and frameworks (MTFs) provide the foundation for a cumulative science of implementation, reflecting a shared, evolving understanding of various facets of implementation. One under-represented aspect in implementation MTFs is how intersecting social factors and systems of power and oppression can shape implementation. There is value in enhancing how MTFs in implementation research and practice account for these intersecting factors. Given the large number of MTFs, we sought to identify exemplar MTFs that represent key implementation phases within which to embed an intersectional perspective. METHODS: We used a five-step process to prioritize MTFs for enhancement with an intersectional lens. We mapped 160 MTFs to three previously prioritized phases of the Knowledge-to-Action (KTA) framework. Next, 17 implementation researchers/practitioners, MTF experts, and intersectionality experts agreed on criteria for prioritizing MTFs within each KTA phase. The experts used a modified Delphi process to agree on an exemplar MTF for each of the three prioritized KTA framework phases. Finally, we reached consensus on the final MTFs and contacted the original MTF developers to confirm MTF versions and explore additional insights. RESULTS: We agreed on three criteria when prioritizing MTFs: acceptability (mean = 3.20, SD = 0.75), applicability (mean = 3.82, SD = 0.72), and usability (median = 4.00, mean = 3.89, SD = 0.31) of the MTF. The top-rated MTFs were the Iowa Model of Evidence-Based Practice to Promote Quality Care for the 'Identify the problem' phase (mean = 4.57, SD = 2.31), the Consolidated Framework for Implementation Research for the 'Assess barriers/facilitators to knowledge use' phase (mean = 5.79, SD = 1.12), and the Behaviour Change Wheel for the 'Select, tailor, implement interventions' phase (mean = 6.36, SD = 1.08). CONCLUSIONS: Our interdisciplinary team engaged in a rigorous process to reach consensus on MTFs reflecting specific phases of the implementation process and prioritized each to serve as an exemplar in which to embed intersectional approaches. The resulting MTFs correspond with specific phases of the KTA framework, which itself may be useful for those seeking particular MTFs for particular KTA phases. This approach also provides a template for how other implementation MTFs could be similarly considered in the future. TRIAL REGISTRATION: Open Science Framework Registration: osf.io/qgh64.

Development and usability testing of tools to facilitate incorporating intersectionality in knowledge translation.

BACKGROUND: The field of knowledge translation (KT) has been criticized for neglecting contextual and social considerations that influence health equity. Intersectionality, a concept introduced by Black feminist scholars, emphasizes how human experience is shaped by combinations of social factors (e.g., ethnicity, gender) embedded in systemic power structures. Its use has the potential to advance equity considerations in KT. Our objective was to develop and conduct usability testing of tools to support integrating intersectionality in KT through three key phases of KT: identifying the gap; assessing barriers to knowledge use; and selecting, tailoring, and implementing interventions. METHODS: We used an integrated KT approach and assembled an interdisciplinary development committee who drafted tools. We used a mixed methods approach for usability testing with KT intervention developers that included semi-structured interviews and the System Usability Scale (SUS). We calculated an average SUS score for each tool. We coded interview data using the framework method focusing on actionable feedback. The development committee used the feedback to revise tools, which were formatted by a graphic designer. RESULTS: Nine people working in Canada joined the development committee. They drafted an intersectionality primer and one tool that included recommendations, activities, reflection prompts, and resources for each of the three implementation phases. Thirty-one KT intervention developers from three countries participated in usability testing. They suggested the tools to be shorter, contain more visualizations, and use less jargon. Average SUS scores of the draft tools ranged between 60 and 78/100. The development committee revised and shortened all tools, and added two, one-page summary documents. The final toolkit included six documents. CONCLUSIONS: We developed and evaluated tools to help embed intersectionality considerations in KT. These tools go beyond recommending the use of intersectionality to providing practical guidance on how to do this. Future work should develop guidance for enhancing social justice in intersectionality-enhanced KT.

Promoting Social and Emotional Learning in Middle School: Intervention Effects of Facing History and Ourselves.

There are fewer evidence-based social and emotional learning programs for middle school students compared to younger grades. This randomized controlled trial tests the effectiveness of Facing History and Ourselves (hereafter, Facing History) with a sample of 694 (Facing History n = 437; Comparison n = 257) students from a low-resourced school district. Youth self-identified as female (59%), Black/African American (61%), Hispanic/Latinx (18%), White (2%), and multi-racial or some other race/ethnicity (19%). Intervention students perceived their classrooms as more caring and democratic than students in the comparison group. They reported higher levels of empathy, prosocial behavior, and stronger participatory citizenship beliefs. This study expands the evidence-based of effective SEL programs available to schools.

Applying an intersectionality lens to the theoretical domains framework: a tool for thinking about how intersecting social identities and structures of power influence behaviour.

BACKGROUND: A key component of the implementation process is identifying potential barriers and facilitators that need to be addressed. The Theoretical Domains Framework (TDF) is one of the most commonly used frameworks for this purpose. When applying the TDF, it is critical to understand the context in which behaviours occur. Intersectionality, which accounts for the interface between social identity factors (e.g. age, gender) and structures of power (e.g. ageism, sexism), offers a novel approach to understanding how context shapes individual decision-making and behaviour. We aimed to develop a tool to be used alongside applications of the TDF to incorporate an intersectionality lens when identifying implementation barriers and enablers. METHODS: An interdisciplinary Framework Committee (n = 17) prioritized the TDF as one of three models, theories, and frameworks (MTFs) to enhance with an intersectional lens through a modified Delphi approach. In collaboration with the wider Framework Committee, a subgroup considered all 14 TDF domains and iteratively developed recommendations for incorporating intersectionality considerations within the TDF and its domains. An iterative approach aimed at building consensus was used to finalize recommendations. RESULTS: Consensus on how to apply an intersectionality lens to the TDF was achieved after 12 rounds of revision. Two overarching considerations for using the intersectionality alongside the TDF were developed by the group as well as two to four prompts for each TDF domain to guide interview topic guides. Considerations and prompts were designed to assist users to reflect on how individual identities and structures of power may play a role in barriers and facilitators to behaviour change and subsequent intervention implementation. CONCLUSIONS: Through an expert-consensus approach, we developed a tool for applying an intersectionality lens alongside the TDF. Considering the role of intersecting social factors when identifying barriers and facilitators to implementing research evidence may result in more targeted and effective interventions that better reflect the realities of those involved.

Can the effects of the mobilization of vulnerable elders in Ontario (MOVE ON) implementation be replicated in new settings: an interrupted time series design.

BACKGROUND: Bed rest for older hospitalized patients places them at risk for hospital-acquired morbidity. We previously evaluated an early mobilization intervention and found it to be effective at improving mobilization rates and decreasing length of stay on internal medicine units. The aim of this study was to conduct a replication study evaluating the impact of the evidence-informed mobilization intervention on surgery, psychiatry, medicine, and cardiology inpatient units. METHODS: A multi-component early mobilization intervention was tailored to the local context at seven hospitals in Ontario, Canada. The primary outcome was patient mobilization measured by conducting visual audits twice a week, three times a day. Secondary outcomes were hospital length of stay and discharge destination, which were obtained from hospital decision support data. The study population was patients aged 65 years and older who were admitted to surgery, psychiatry, medicine, and cardiology inpatient units between March and August 2014. Using an interrupted time series design, the intervention was evaluated over three time periods-pre-intervention, during, and post-intervention. RESULTS: A total of 3098 patients [mean age 78.46 years (SD 8.38)] were included in the overall analysis. There was a significant increase in mobility immediately after the intervention period compared to pre-intervention with a slope change of 1.91 (95% confidence interval [CI] 0.74-3.08, P-value = 0.0014). A decreasing trend in median length of stay was observed in the majority of the participating sites. Overall, a median length of stay of 26.24 days (95% CI 23.67-28.80) was observed pre-intervention compared to 23.81 days (95% CI 20.13-27.49) during the intervention and 24.69 days (95% CI 22.43-26.95) post-intervention. The overall decrease in median length of stay was associated with the increase in mobility across the sites. CONCLUSIONS: MOVE increased mobilization and these results were replicated across surgery, psychiatry, medicine, and cardiology inpatient units.

Scaling-up implementation in community hospitals: a multisite interrupted time series design of the Mobilization of Vulnerable Elders (MOVE) program in Alberta.

BACKGROUND: As the population ages, older hospitalized patients are at increased risk for hospital-acquired morbidity. The Mobilization of Vulnerable Elders (MOVE) program is an evidence-informed early mobilization intervention that was previously evaluated in Ontario, Canada. The program was effective at improving mobilization rates and decreasing length of stay in academic hospitals. The aim of this study was to scale-up the program and conduct a replication study evaluating the impact of the evidence-informed mobilization intervention on various units in community hospitals within a different Canadian province. METHODS: The MOVE program was tailored to the local context at four community hospitals in Alberta, Canada. The study population was patients aged 65 years and older who were admitted to medicine, surgery, rehabilitation and intensive care units between July 2015 and July 2016. The primary outcome was patient mobilization measured by conducting visual audits twice a week, three times a day. The secondary outcomes included hospital length of stay obtained from hospital administrative data, and perceptions of the intervention assessed through a qualitative assessment. Using an interrupted time series design, the intervention was evaluated over three time periods (pre-intervention, during, and post-intervention). RESULTS: A total of 3601 patients [mean age 80.1 years (SD = 8.4 years)] were included in the overall analysis. There was a significant increase in mobilization at the end of the intervention period compared to pre-intervention, with 6% more patients out of bed (95% confidence interval (CI) 1, 11; p-value = 0.0173). A decreasing trend in median length of stay was observed, where patients on average stayed an estimated 3.59 fewer days (95%CI -15.06, 7.88) during the intervention compared to pre-intervention period. CONCLUSIONS: MOVE is a low-cost, effective and adaptable intervention that improves mobilization in older hospitalized patients. This intervention has been replicated and scaled up across various units and hospital settings.

The perceived organizational impact of the gender gap across a Canadian department of medicine and proposed strategies to combat it: a qualitative study.

BACKGROUND: Despite the gender parity existing in medical schools for over three decades, women remain underrepresented in academic medical centers, particularly in senior ranks and in leadership roles. This has consequences for patient care, education, research, and workplace culture within healthcare organizations. This study was undertaken to explore the perspectives of faculty members at a single department of medicine on the impact of the existing gender gap on organizational effectiveness and workplace culture, and to identify systems-based strategies to mitigate the gap. METHODS: The study took place at a large university department of medicine in Toronto, Canada, with six affiliated hospitals. In this qualitative study, semi-structured individual interviews were conducted between May and September 2016 with full-time faculty members who held clinical and university-based appointments. Transcripts of the interviews were analyzed using thematic analysis. Three authors independently reviewed the transcripts to determine a preliminary list of codes and establish a coding framework. A modified audit consensus coding approach was applied; a single analyst reviewed all the transcripts and a second analyst audited 20% of the transcripts in each round of coding. Following each round, inter-rater reliability was determined, discrepancies were resolved through discussion, and modifications were made as needed to the coding framework. The analysis revealed faculty members' perceptions of the gender gap, potential contributing factors, organizational impacts, and possible solutions to bridge the gap. RESULTS: Of the 43 full-time faculty members who participated in the survey (29 of whom self-identified as female), most participants were aware of the existing gender gap within academic medicine. Participants described social exclusion, reinforced stereotypes, and unprofessional behaviors as consequences of the gap on organizational effectiveness and culture. They suggested improvements in (1) the processes for recruitment, hiring, and promotion; (2) inclusiveness of the work environment; (3) structures for mentorship; and (4) ongoing monitoring of the gap. CONCLUSION: The existing gender gap in academic medicine may have negative consequences for organizational effectiveness and workplace culture but many systems-based strategies to mitigate the gap exist. Although these solutions warrant rigorous evaluation, they are feasible to institute within most healthcare organizations immediately.

Outcomes of Mobilisation of Vulnerable Elders in Ontario (MOVE ON): a multisite interrupted time series evaluation of an implementation intervention to increase patient mobilisation.

Background: older patients admitted to hospitals are at risk for hospital-acquired morbidity related to immobility. The aim of this study was to implement and evaluate an evidence-based intervention targeting staff to promote early mobilisation in older patients admitted to general medical inpatient units. Methods: the early mobilisation implementation intervention for staff was multi-component and tailored to local context at 14 academic hospitals in Ontario, Canada. The primary outcome was patient mobilisation. Secondary outcomes included length of stay (LOS), discharge destination, falls and functional status. The targeted patients were aged ≥ 65 years and admitted between January 2012 and December 2013. The intervention was evaluated over three time periods-pre-intervention, during and post-intervention using an interrupted time series design. Results: in total, 12,490 patients (mean age 80.0 years [standard deviation 8.36]) were included in the overall analysis. An increase in mobilisation was observed post-intervention, where significantly more patients were out of bed daily (intercept difference = 10.56%, 95% CI: [4.94, 16.18]; P < 0.001) post-intervention compared to pre-intervention. Hospital median LOS was significantly shorter during the intervention period (intercept difference = -3.45 days, 95% CI: [-6.67,-0.23], P = 0.0356) compared to pre-intervention. It continued to decrease post-intervention with significantly fewer days in hospital (intercept difference= -6.1, 95% CI: [-11,-1.2]; P = 0.015) in the post-intervention period compared to pre-intervention. Conclusions: this is a large-scale study evaluating an implementation strategy for early mobilisation in older, general medical inpatients. The positive outcome of this simple intervention on an important functional goal of getting more patients out of bed is a striking success for improving care for hospitalised older patients.

Supporting the implementation of stroke quality-based procedures (QBPs): a mixed methods evaluation to identify knowledge translation activities, knowledge translation interventions, and determinants of implementation across Ontario.

BACKGROUND: In 2013, Health Quality Ontario introduced stroke quality-based procedures (QBPs) to promote use of evidence-based practices for patients with stroke in Ontario hospitals. The study purpose was to: (a) describe the knowledge translation (KT) interventions used to support stroke QBP implementation, (b) assess differences in the planned and reported KT interventions by region, and (c) explore determinants perceived to have affected outcomes. METHODS: A mixed methods approach was used to evaluate: activities, KT interventions, and determinants of stroke QBP implementation. In Phase 1, a document review of regional stroke network work plans was conducted to capture the types of KT activities planned at a regional level; these were mapped to the knowledge to action framework. In Phase 2, we surveyed Ontario hospital staff to identify the KT interventions used to support QBP implementation at an organizational level. Phase 3 involved qualitative interviews with staff to elucidate deeper understanding of survey findings. RESULTS: Of the 446 activities identified in the document review, the most common were 'dissemination' (24.2%; n = 108), 'implementation' (22.6%; n = 101), 'implementation planning' (15.0%; n = 67), and 'knowledge tools' (10.5%; n = 47). Based on survey data (n = 489), commonly reported KT interventions included: staff educational meetings (43.1%; n = 154), champions (41.5%; n = 148), and staff educational materials (40.6%; n = 145). Survey participants perceived stroke QBP implementation to be successful (median = 5/7; interquartile range = 4-6; range = 1-7; n = 335). Forty-four people (e.g., managers, senior leaders, regional stroke network representatives, and frontline staff) participated in interviews/focus groups. Perceived facilitators to QBP implementation included networks and collaborations with external organizations, leadership engagement, and hospital prioritization of stroke QBP. Perceived barriers included lack of funding, size of the hospital (i.e., too small), lack of resources (i.e., staff and time), and simultaneous implementation of other QBPs. CONCLUSIONS: Information on the types of activities and KT interventions used to support stroke QBP implementation and the key determinants influencing uptake of stroke QBPs can be used to inform future activities including the development and evaluation of interventions to address barriers and leverage facilitators.

Evaluating implementation of the World Health Organization's Strategic Approach to strengthening sexual and reproductive health policies and programs to address unintended pregnancy and unsafe abortion.

BACKGROUND: We conducted a process evaluation to assess how the World Health Organization's (WHO) Strategic Approach to strengthening sexual and reproductive health policies and programs ("the SA") was used in 15 countries that requested WHO's technical support in addressing unintended pregnancy and unsafe abortion. The SA is a three-stage planning, policy, and program implementation process. We used the social ecological model (SEM) to analyze the contextual factors that influenced SA implementation. METHODS: We used a two-phased sequential approach to data collection and analysis. In Phase A, we conducted a document and literature review and synthesized data thematically. In Phase B, we conducted interviews with stakeholders who used the SA in the countries of interest. We used a qualitative method triangulation technique to analyze and combine data from both phases to understand how the SA was implemented in each country. RESULTS: Data from 145 documents and 19 interviews described the SA process and activities in each country. All 15 countries completed Stage 1 activities. The activities of Stage 1 determined activities in subsequent stages and varied across countries. Following Stage 1, some countries focused on reforming policies to improve access to sexual and reproductive health (SRH) services whereas others focused on improving provider-level capacity to enhance SRH service quality and improving community-level SRH education. We identified factors across SEM levels that affected SA implementation, including individual- and community-level perceptions of using the SA and the recommendations that emerged from its use, organizational capacity to conduct SA activities, and how well these activities aligned with the existing political climate. Stakeholders perceived SA implementation to be country-driven and systematic in bringing attention to important SRH issues in their countries. CONCLUSION: We identified key success factors for influencing the individual, organization, and system change required for implementing the SA. These include sustaining stakeholder engagement for all SA stages, monitoring and reporting on activities, and leveraging activities and outputs from each SA stage to obtain technical and financial support for subsequent stages. Results may be used to optimize ongoing implementation efforts to improve access to and the quality of SRH services.

Factors that may be influencing the rise in prescription testosterone replacement therapy in adult men: a qualitative study.

OBJECTIVE: To explore and describe the factors that may be influencing the rise of prescribing and use of testosterone replacement therapy (TRT) in adult men. DESIGN: A rapid qualitative research design using semi-structured interviews with providers and patients. SETTING: Ontario, Canada. PARTICIPANTS: Nine men who have used TRT (referred to as "patients"), and six primary care clinicians and seven specialists (collectively referred to as "providers") who prescribed or administered TRT. METHOD: Patients' and providers' perspectives were investigated through semi-structured interviews. A purposive sampling approach was used to recruit all participants. We conducted qualitative analysis using the framework approach for applied health research. MAIN FINDINGS: Participants perceived the following factors to have influenced TRT prescriptions and use in adult men: provider factors (diagnostic ambiguity of age-related hypogonadism and beliefs about appropriateness of TRT) and patient factors (access to information on TRT and drug seeking behavior). They perceived that these factors have perpetuated a rise in prescription in the absence of clear clinical guidelines and unclear research evidence on the safety and efficacy of TRT. CONCLUSION: The findings of this study highlight that much work still needs to be done to improve diagnostic accuracy and encourage appropriate TRT prescription in adult men. In addition, both patients and providers need more information about the risks and long-term effects of TRT in men.

Navigating barriers: two-year follow up on recommendations to improve the use of maternal health guidelines in Kosovo.

BACKGROUND: Although there are a growing number of initiatives aimed at supporting guideline implementation in resource-constrained settings, few studies assess progress on achieving next steps and goals after the initial activities are completed and the initial funding period has ended. The aim of the current study was to conduct a qualitative process evaluation of progress, barriers, facilitators, and proposed solutions to operationalize nine recommendations to prepare Kosovo to implement the 2012 World Health Organization (WHO) prevention and treatment of postpartum haemorrhage guideline. METHODS/DESIGN: In 2012, we co-created nine recommendations designed to support implementing the WHO's guideline on the prevention and treatment of postpartum haemorrhage in Kosovo. The current study uses a realist evaluation approach to assess activities and progress two years after the recommendations were developed. The study involved conducting qualitative focus groups and one-on-one interviews with participants from the first meeting to evaluate the activities and progress on the nine recommendations. RESULTS: Forty-three participants provided insights into the barriers and opportunities experienced to date and proposed future directions. Although progress has been made towards implementation of a number of the recommendations, scaling up has been limited by barriers, such as lack of awareness, limited resources, and evaluation challenges. Participants proposed addressing these barriers by building within- and between-country partnerships to facilitate guideline implementation. In addition, participants reported less progress on implementing recommendations related to broader cultural changes, which indicates a need for specific and actionable recommendations to operationalize implementation efforts. CONCLUSIONS: In the two years since the initial meeting, there has been mixed progress on the recommendations. Based on participant feedback, we refined the recommendations so that they can be operationalized by health care system stakeholders in Kosovo to further support implementation efforts. It is beneficial to share these lessons learned throughout the implementation process to inform next steps in Kosovo and offer ideas for use in other settings.

Ready, Set, Change! Development and usability testing of an online readiness for change decision support tool for healthcare organizations.

BACKGROUND: To address challenges related to selecting a valid, reliable, and appropriate readiness assessment measure in practice, we developed an online decision support tool to aid frontline implementers in healthcare settings in this process. The focus of this paper is to describe a multi-step, end-user driven approach to developing this tool for use during the planning stages of implementation. METHODS: A multi-phase, end-user driven approach was used to develop and test the usability of a readiness decision support tool. First, readiness assessment measures that are valid, reliable, and appropriate for healthcare settings were identified from a systematic review. Second, a mapping exercise was performed to categorize individual items of included measures according to key readiness constructs from an existing framework. Third, a modified Delphi process was used to collect stakeholder ratings of the included measures on domains of feasibility, relevance, and likelihood to recommend. Fourth, two versions of a decision support tool prototype were developed and evaluated for usability. RESULTS: Nine valid and reliable readiness assessment measures were included in the decision support tool. The mapping exercise revealed that of the nine measures, most measures (78 %) focused on assessing readiness for change at the organizational versus the individual level, and that four measures (44 %) represented all constructs of organizational readiness. During the modified Delphi process, stakeholders rated most measures as feasible and relevant for use in practice, and reported that they would be likely to recommend use of most measures. Using data from the mapping exercise and stakeholder panel, an algorithm was developed to link users to a measure based on characteristics of their organizational setting and their readiness for change assessment priorities. Usability testing yielded recommendations that were used to refine the Ready, Set, Change! decision support tool . CONCLUSIONS: Ready, Set, Change! decision support tool is an implementation support that is designed to facilitate the routine incorporation of a readiness assessment as an early step in implementation. Use of this tool in practice may offer time and resource-saving implications for implementation.

Access to Triptans for Acute Episodic Migraine: A Qualitative Study.

OBJECTIVE: Our study aims to examine factors related to access of triptans among multiple stakeholder groups. BACKGROUND: Triptans are a cornerstone of pain management for the acute treatment of migraine, but actual utilization of triptans is lower than ideal. Initial and continued access to triptans may be an important clinical issue in the acute treatment of migraines, but factors affecting access at the patient, provider, and health-care system levels have not been comprehensively explored. METHODS: A qualitative study was conducted in Ontario, Canada, between August 2013 and January 2014. Three participant groups were recruited to the qualitative study: (1) migraineurs who have experience accessing triptans; (2) physicians, including primary care physicians (PCPs) and neurologists, who have prescribed triptans; and (3) pharmacists who have dispensed triptans. Qualitative data were collected through one-on-one, semi-structured telephone interviews. The framework approach was used for data collection and analysis. RESULTS: Data collected from 19 migraineurs, 6 physicians, and 8 pharmacists were included in the analysis. Study participants discussed various factors that facilitate or hinder access to triptans, which were synthesized into four themes that emerged at the patient, provider, and health-care systems levels: (1) awareness; (2) apathy; (3) advocacy; and (4) affordability. Across all participant groups, awareness of available treatments and coverage policies for those treatments were potential factors relating to timely drug provision. Participants describe apathy in terms of patients' health-seeking behaviors and physicians' lack of concern toward migraine, which were seen as factors that could delay diagnosis and provision of appropriate treatment. Patients engaging in self-advocacy enhanced their ability to seek timely and appropriate provision of triptans at the patient level. At the health-care provider level, pharmacists were identified by patients as advocates for receiving more effective treatments for their migraines; pharmacists also self-identified with the advocate role. The affordability of triptans was a key concern impacting access at the systems level, but coverage limitations (eg, quantity limits) were also described to influence the appropriateness of prescribed migraine treatment. CONCLUSION: This study fills a gap in knowledge about access to triptans and how this may be impacted by patient, provider, and health-care systems barriers. Overall, our study sheds light on the experiences of prescribing, dispensing, and accessing triptans for migraine treatment, and unveils important information that can impact how patients access these drugs.

Sustaining evidence-based prevention programs: correlates in a large-scale dissemination initiative.

Over the past four decades, significant strides have been made in the science of preventing youth problem behaviors. Subsequently, policymakers and funders have begun to insist on the use of evidence-based programs (EBPs) as a requirement for funding. However, unless practitioners are able to sustain these programs beyond initial seed funding, they are unlikely to achieve their ultimate goal of broad public health impact. Despite its obvious importance, sustainability has received relatively little attention in prevention science until recently. Moreover, there have been few opportunities to study the correlates of sustainability in large-scale, multi-year initiatives involving multiple programs. The present study examined rates of sustainment of a wide range of proven-effective prevention and intervention programs; identified factors related to organizational support and readiness, program and implementer characteristics, and sustainability planning that distinguished sustained programs; and examined variability in these associations across classroom-based, community/mentoring, family-focused prevention, and family treatment program types within the context of a state-wide EBP dissemination initiative in Pennsylvania over 4 years. The majority of EBPs sustained functioning 2 years or more beyond their initial funding. In general, sustained programs reported greater community coalition functioning, communication to key stakeholders, knowledge of the program's logical model, communication with the trainer or program developer, and sustainability planning. In addition to these universal correlates, important program-specific correlates emerged as well. Implications for the technical assistance and support necessary to promote the sustainability of EBPs in nonresearch contexts are also discussed.

Organizing the dissemination and implementation field: who are we, what are we doing, and how should we do it?

Two decades into its tenure as a field, dissemination and implementation (D&I) scientists have begun a process of self-reflection, illuminating a missed opportunity to bridge the gap between research and practice-one of the field's foundational objectives. In this paper, we, the authors, assert the research-to-practice gap has persisted, in part due to an inadequate characterization of roles, functions, and processes within D&I. We aim to address this issue, and the rising tension between D&I researchers and practitioners, by proposing a community-centered path forward that is grounded in equity.We identify key players within the field and characterize their unique roles using the translational science spectrum, a model originally developed in the biomedical sciences to help streamline the research-to-practice process, as a guide. We argue that the full translational science spectrum, from basic science research, or "T0," to translation to community, or "T4," readily applies within D&I and that in using this framework to clarify roles, functions, and processes within the field, we can facilitate greater collaboration and respect across the entire D&I research-to-practice continuum. We also highlight distinct opportunities (e.g., changes to D&I scientific conference structures) to increase regular communication and engagement between individuals whose work sits at different points along the D&I translational science spectrum that can accelerate our efforts to close the research-to-practice gap and achieve the field's foundational objectives.

The Pathway Building Technique in Implementation Research Using Mixed Methods Design.

BACKGROUND: Data integration refers to combining quantitative and qualitative data in mixed methods. It can be achieved through several integration procedures. The building integration procedure can be used for developing quantitative instruments by integrating data from the qualitative phase. There are limited examples of data integration using the building procedure in mixed methods and implementation science. PURPOSE: The purpose of this article is to illustrate how the pathway building technique can be used to integrate data in mixed methods research through concurrent use of implementation science models and frameworks. METHODS: This two pathway building technique was developed based on a mixed methods implementation project of developing implementation strategies to promote compassionate nursing care of complex patients. RESULTS: The first pathway is the integration of qualitative data from the first phase of mixed methods study with implementation models and frameworks to create a quantitative instrument (i.e., a Q-sort survey) for the subsequent phase. The second pathway is the operationalization of the Q-sort survey results (i.e., implementation strategies) using an implementation science specification framework. CONCLUSION: The pathway technique is valuable for mixed methods research and implementation science as it offers a theory-based innovative method to tackle integration challenge.

Understanding real-world implementation quality and "active ingredients" of PBIS.

Programs delivered in the "real world" often look substantially different from what was originally intended by program developers. Depending on which components of a program are being trimmed or altered, such modifications may seriously undermine the effectiveness of a program. In the present study, these issues are explored within a widely used school-based, non-curricular intervention, Positive Behavioral Intervention and Supports. The present study takes advantage of a uniquely large dataset to gain a better understanding of the "real-world" implementation quality of PBIS and to take a first step toward identifying the components of PBIS that "matter most" for student outcomes. Data from 27,689 students and 166 public primary and secondary schools across seven states included school and student demographics, indices of PBIS implementation quality, and reports of problem behaviors for any student who received an office discipline referral during the 2007-2008 school year. Results of the present study identify three key components of PBIS that many schools are failing to implement properly, three program components that were most related to lower rates of problem behavior (i.e., three "active ingredients" of PBIS), and several school characteristics that help to account for differences across schools in the quality of PBIS implementation. Overall, findings highlight the importance of assessing implementation quality in "real-world" settings, and the need to continue improving understanding of how and why programs work. Findings are discussed in terms of their implications for policy.

Drawing causal inferences using propensity scores: a practical guide for community psychologists.

Confounding present in observational data impede community psychologists' ability to draw causal inferences. This paper describes propensity score methods as a conceptually straightforward approach to drawing causal inferences from observational data. A step-by-step demonstration of three propensity score methods-weighting, matching, and subclassification-is presented in the context of an empirical examination of the causal effect of preschool experiences (Head Start vs. parental care) on reading development in kindergarten. Although the unadjusted population estimate indicated that children with parental care had substantially higher reading scores than children who attended Head Start, all propensity score adjustments reduce the size of this overall causal effect by more than half. The causal effect was also defined and estimated among children who attended Head Start. Results provide no evidence for improved reading if those children had instead received parental care. We carefully define different causal effects and discuss their respective policy implications, summarize advantages and limitations of each propensity score method, and provide SAS and R syntax so that community psychologists may conduct causal inference in their own research.

Examining adaptations of evidence-based programs in natural contexts.

When evidence-based programs (EBPs) are scaled up in natural, or non-research, settings, adaptations are commonly made. Given the fidelity-versus-adaptation debate, theoretical rationales have been provided for the pros and cons of adaptations. Yet the basis of this debate is theoretical; thus, empirical evidence is needed to understand the types of adaptations made in natural settings. In the present study, we introduce a taxonomy for understanding adaptations. This taxonomy addresses several aspects of adaptations made to programs including the fit (philosophical or logistical), timing (proactive or reactive), and valence, or the degree to which the adaptations align with the program's goals and theory, (positive, negative, or neutral). Self-reported qualitative data from communities delivering one of ten state-funded EBPs were coded based on the taxonomy constructs; additionally, quantitative data were used to examine the types and reasons for making adaptations under natural conditions. Forty-four percent of respondents reported making adaptations. Adaptations to the procedures, dosage, and content were cited most often. Lack of time, limited resources, and difficulty retaining participants were listed as the most common reasons for making adaptations. Most adaptations were made reactively, as a result of issues of logistical fit, and were not aligned with, or deviated from, the program's goals and theory.