Adapting Evidence-Based Falls Prevention Programs for Remote Delivery - Implementation Insights through the RE-AIM Evaluation Framework to Promote Health Equity.

COVID-19 disproportionally impacted the health and well-being of older adults-many of whom live with chronic conditions-due to their higher risk of dying and being hospitalized. It also created several secondary pandemics, including increased falls risk, sedentary behavior, social isolation, and physical inactivity due to limitations in mobility from lock-down policies. With falls as the leading cause of preventable death and hospitalizations, it became vital for in-person evidence-based falls prevention programs (EBFPPs) to pivot to remote delivery. In Spring 2020, many EBFPP administrators began re-designing programs for remote delivery to accommodate physical distancing guidelines necessitated by the pandemic. Transition to remote delivery was essential for older adults and persons with disabilities to access EBFPPs for staying healthy, falls and injury free, out of hospitals, and also keeping them socially engaged. We collaborated with the Administration on Community Living (ACL), the National Council on Aging (NCOA), and the National Falls Prevention Resource Center (NFPRC), for an in-depth implementation evaluation of remotely delivered EBFPPs. We examined the process of adapting and implementing four EBFPPs for remote delivery, best practices for implementing the programs remotely within the RE-AIM evaluation framework. This enhances NFPRC's ongoing work supporting dissemination, implementation, and sustainability of EBFPPs. We purposively sampled organizations for maximum variation in organization and provider type, geographic location, and reach of underserved older populations (Black, Indigenous, or other People of Color (BIPOC), rural, disabilities). This qualitative evaluation includes provider-level data from semi-structured interviews (N = 22) with program administrators, staff, and leaders. The interview guide included what, why, and how adaptations were made to EBFPP interventions and implementation strategies using Wiltsey-Stirman (2019) adaptations framework (FRAME), reach, and implementation outcomes (acceptability, feasibility, fidelity, and costs; Proctor et al., 2011), focusing on equity to learn for whom these programs were working and opportunities to address inequities. Findings demonstrate remote EBFPPs made planned and fidelity-consistent adaptations to remote delivery in partnership with researchers and community organizations, focusing on participant safety both in program content and delivery. Supports using and accessing technology were needed for delivery sites and leaders to facilitate engagement, and improved over time. While remote EBFPP delivery has increased access to EBFPPs for some populations from the perspective of program administrator, leaders, and staff (e.g., caregivers, rural-dwellers, persons with physical disabilities), the digital divide remains a barrier in access to and comfort using technology. Remote-delivered EBFPPs were acceptable and feasible to delivery organizations and leaders, were able to be delivered with fidelity using adaptations from program developers, but were more resource intensive and costly to implement compared to in-person. This work has important implications beyond the pandemic. Remote delivery has expanded access to groups traditionally underserved by in-person programming, particularly disability communities. This work will help answer important questions about reach, accessibility, feasibility, and cost of program delivery for older adults and people with disabilities at risk for falls, those living with chronic conditions, and communities most vulnerable to disparities in access to health care, health promotion programming, and health outcomes. It will also provide critical information to funders about elements required to adapt EBFPPs proven effective in in-person settings for remote delivery with fidelity to achieve comparable outcomes.

Experiences With and Access to Evidence-Based Health Promotion Programs for Older American Indian, Alaska Native, and Native Hawaiian Peoples.

Evidence-based programs (EBPs) work effectively for participants whose characteristics match those of the EBP research participants. However, EBPs have been almost exclusively developed and evaluated for the general U.S. population with limited American Indian, Alaska Native, and Native Hawaiian (AI/AN/NH) community engagement. Thus, an AI/AN/NH Evidence-Based Program Advisory Council sought to identify AI/AN/NH peoples' experiences with and access to EBPs. We held 20 listening sessions with AI/AN/NH Elder services program staff (n = 118) and with AI/AN/NH Elders (n = 82) and conducted a self-administered online survey with Title VI Directors (n = 63). The six themes that emerged from the listening sessions with staff included misunderstanding community engagement, valuing fidelity over flexibility, lack of cultural awareness, assumptions about available infrastructure, unrealistic implementation timelines, and funding restrictions. Listening session themes with Elders included definitions of aging well, participation motivators, preferred activities, participation barriers, and unmet needs. Survey data indicated that programming of greatest interest for Elders as identified by Title VI Directors and staff included and/or addressed cultural activities (81%), socialization (75%), diabetes (73%), caregiving (68%), and nutrition (68%). Seventy-six percent of survey respondents had heard of EBPs and 24% indicated that EBPs were not being implemented in their community. The Advisory Council developed specific action steps with the goal of improving AI/AN/NH communities' access to culturally appropriate and feasible EBPs. The steps require collective action from federal agencies, national partners, EBP program developers and administrators, local organizations, and Elders to ensure EBPs are accessible and culturally appropriate for AI/AN/NH Elders.

Study Protocol: Multi-level Determinants of Implementation and Sustainment in the Education Sector.

Evidence-based programs (EBPs) delivered in elementary schools show great promise in reducing risk for emotional and behavioral disorders (EBDs). However, efforts to sustain EBPs in school face barriers. Improving EBP sustainment thus represents a priority, but little research exists to inform the development of sustainment strategies. To address this gap, the Sustaining Evidenced-Based Innovations through Multi-level Implementation Constructs (SEISMIC) project will: (a) Determine if malleable individual, intervention, and organizational factors predict EBP treatment fidelity and modifications during implementation, sustainment, or both; (b) Assess the impact of EBP fidelity and modifications on child outcomes during implementation and sustainment; and (c) Explore the mechanisms through which individual, intervention, and organizational factors influence sustainment outcomes. This protocol paper describes SEISMIC, which builds upon a federally-funded RCT evaluating BEST in CLASS, a teacher-delivered program for K-3rd grade children at risk for EBDs. The sample will include 96 teachers, 384 children, and 12 elementary schools. A multi-level, interrupted time series design will be used to examine the relationship between baseline factors, treatment fidelity, modifications, and child outcomes, followed by a mixed-method approach to elucidate the mechanisms that influence sustainment outcomes. Findings will be used to create a strategy to improve EBP sustainment in schools.

The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs.

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the USA is early childhood home visiting. While there is substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting's evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Visitors' Assessment and Utilization of Evidence-Based Program Resources (EBPR) Websites.

Background: Evidence-based program resources (EBPR) websites for behavioral health are a potentially useful tool to assist decision-makers and practitioners in deciding which behavioral health interventions to implement. EBPR websites apply rigorous research standards to assess the effectiveness of behavioral healthcare programs, models, and clinical practices. Method: Visitors to a convenience sample of six EBPR websites (N=369, excluding students) were recruited for telephone interviews primarily by means of a pop-up invitation on the sites. Results: The visitors view the EBPR sites as important sources of information to support the identification and adoption of evidence-based programs/practices (EBPs) in behavioral healthcare, which aligns with the primary mission of EBPRs. For repeat visitors, there was some indication that the information obtained helped effect certain changes in their agencies' programs and policies. However, increased or improved guidance on EBP implementation was also requested. Conclusion: EBPR websites should be better publicized to the behavioral healthcare field.

Enhancing the Adoption of Evidence-Based Health Marketing and Promotion Strategies in Local Communities: Building a Communication Dissemination and Support System for the National Diabetes Prevention Program.

The Centers for Disease Control and Prevention (CDC) created a health communication marketing and promotion support system (support system) to help 10 CDC-funded national organizations (recipients) grow enrollment of underserved populations in the National Diabetes Prevention Program. This article describes the creation of a successful support system to increase the use of effective marketing approaches and key messaging. The support system was developed using a systematic approach. It included a needs assessment, audience research, marketing strategy identification, expert panel review, materials development, and dissemination guidance. Hands-on, individualized, and group end-user training and technical assistance was also included. Recipients received culturally and linguistically tailored marketing materials to support their specific priority audiences, as well as corresponding training on recommended dissemination methods. In in-depth key-informant interviews, staff from six recipients reported increased knowledge of local communities and audiences, efficacy and skills to conduct media interviews, capacity to identify and train champions and influencers, and greater community partner investments. With marketing support, 90% of recipients reported increased enrollment, of which 40% exceeded self-set targets and another 40% doubled or tripled their enrollment numbers. These findings indicate that a customized strategic health communication marketing and promotion support system presents a significant opportunity to help recipients increase enrollment in evidence-based interventions. Practitioners disseminating evidence-based interventions may consider a support system to increase program uptake.

The United Nations Office on Drugs and Crime's Efforts to Strengthen a Culture of Prevention in Low- and Middle-Income Countries.

This article discusses how decision-makers can be supported to strengthen a culture of prevention. This article presents an example of the United Nations Office on Drugs and Crime's (UNODC) work to engage with decision-makers to create readiness, demand, and capacity for evidence-based prevention programming among them, particularly in low- and middle-income countries. First, we utilized two of the UNODC's data sources to describe the context where the UNODC's prevention efforts take place. Analysis of the first dataset on prevention activities implemented globally revealed a gap in translating evidence into practice on a global scale. The second dataset consisted of UNODC policy documents mandating and guiding global action to address substance use. The analysis showed that at the level of political frameworks, prevention is gradually gaining more attention but is still frequently left in the shadow of health- and law enforcement-related issues. In addition, these guiding documents did not reflect fully the current scientific understanding of what constitutes an effective prevention response. Against this background, the feasibility of the UNODC's efforts to bridge the science-practice gap in the field of prevention was discussed by presenting the results from the UNODC's regional capacity-building seminars focused on the role of monitoring and evaluation in prevention programming. The results showed potential of this capacity building to affect the attitudes and knowledge of targeted decision-makers. Such efforts to increase decision-makers' readiness and ultimately their endorsement, adoption, and ongoing support of evidence-based preventive interventions should be continued and intensified.

Community Stakeholders' Perspectives on Introducing Human Papillomavirus Vaccination and Biobanking Evidence-Based Programs Within Medically Underserved Communities: A Community-Engaged Approach.

PURPOSE: We explored the perspectives of program coordinators and community leaders from Community-based-organizations (CBOs) and Faith-based-organizations (FBOs) to determine how information on human papillomavirus (HPV) vaccination and biobanking is understood and supported within medically underserved communities, and identified strategies for introducing evidence-based programs (EBP) on HPV vaccination and biobanking within underserved communities. METHODS: The EPIS framework guided discussions in four focus groups with program coordinators (n = 27) and one-on-one interviews with community leaders (n = 15) from CBOs and FBOs. RESULTS: Participants reported that community members were aware of HPV vaccination but did not know of the link between HPV infection and cancers. Awareness for biobanking was low. HPV vaccination and biobanking were not priority health concerns among community members due to other health issues. However, HPV vaccination and biobanking were considered sensitive health topics. For HPV vaccination, sensitivity was due to concerns that HPV vaccination promoted sex among adolescents, while for biobanking, sensitivity was due to historical abuses of people of color by the medical community. Participants reported that program coordinators' awareness of HPV vaccination and biobanking depended on their organizations' mission. Neither were considered a priority health concern due to lack of funding. Few EBP were available on HPV vaccination and biobanking. Recommendations on culturally-appropriate strategies for engaging community members on HPV vaccination and biobanking EBP are discussed. CONCLUSION: Given the promise of HPV vaccination and the Precision Medicine Initiative to reduce cancer disparities, findings elucidate factors to be considered when implementing EBP on HPV vaccination and biobanking into medically underserved communities.

Wellness project implementation within Houston's Faith and Diabetes initiative: a mixed methods study.

BACKGROUND: Faith-based health promotion has shown promise for supporting healthy lifestyles, but has limited evidence of reaching scale or sustainability. In one recent such effort, volunteers from a diverse range of faith organizations were trained as peer educators to implement diabetes self-management education (DSME) classes within their communities. The purpose of this study was to identify factors associated with provision of these classes within six months of peer-educator training. METHODS: This study used the Consolidated Framework for Implementation Research (CFIR) to identify patterns from interviews, observations, attendance records, and organizational background information. Two research team members thematically coded interview transcripts and observation memos to identify patterns distinguishing faith organizations that did, versus did not, conduct DSME classes within six months of peer-educator training. Bivariate statistics were also used to identify faith organizational characteristics associated with DSME class completion within this time frame. RESULTS: Volunteers from 24 faith organizations received peer-educator training. Of these, 15 led a DSME class within six months, graduating a total of 132 participants. Thematic analyses yielded two challenges experienced disproportionately by organizations unable to complete DSME within six months: [1] Their peer educators experienced DSME as complex, despite substantial planning efforts at simplification, and [2] the process of engaging peer educators and leadership within their organizations was often more difficult than anticipated, despite initial communication by Faith and Diabetes organizers intended to secure informed commitments by both groups. Many peer educators were overwhelmed by training content, the responsibility required to start and sustain DSME classes, and other time commitments. Other priorities competed for time in participants' lives and on organizational calendars, and scheduling processes could be slow. In an apparent dynamic of "crowding out," coordination was particularly difficult in larger organizations, which were less likely than smaller organizations to complete DSME classes despite their more substantial resources. CONCLUSIONS: Initial commitment from faith organizations' leadership and volunteers may not suffice to implement even relatively short and low cost health promotion programs. Faith organizations might benefit from realistic previews about just how challenging it is to make these programs a sufficiently high organizational and individual priority.

The Nurse-Family Partnership in Colorado: Supporting High-Quality Programming With Implementation Science.

PURPOSE: The purpose of this article is to describe how the Nurse-Family Partnership (NFP) has been scaled up and supported in Colorado. As an intermediary, Invest in Kids (IIK) provides implementation support for the NFP in Colorado using a generalizable implementation framework, the Active Implementation Frameworks (AIF). ORGANIZING CONSTRUCT: An overlay of the AIF and the clinical nursing-informed implementation support that IIK offers to NFP providers across Colorado is explored, and relevant examples are highlighted. CONCLUSIONS: Without the use of the AIF in combination with clinical nursing expertise to support high fidelity use of the NFP throughout Colorado, promised NFP program outcomes may likely not be realized and sustained. CLINICAL RELEVANCE: Further understanding of how to utilize implementation frameworks to support evidence-based clinical nursing programs and interventions may allow for results found in research studies to be more widely attained and maintained across practice settings.

Rigorous Research on Existing Child Maltreatment Prevention Programs: Introduction to the Special Section.

Awareness of child maltreatment as a major public health problem in the US has increased in recent years. In response, major public initiatives have been launched to fund the delivery of evidence-based programs, such as home visiting, in an effort to promote child and family functioning and health and prevent maltreatment. While promising, the number of families served by these programs remains small relative to need. Further, many families across the US are served by community-designed and supported programs for which rigorous outcome evidence has never been collected. To broaden the evidence-base on child maltreatment prevention programs, and to encourage the use of rigorous research designs in community settings, the Children's Bureau sponsored four randomized controlled trials of established programs that had a limited or no evidence base. In this introduction to a special section of Prevention Science on the prevention of child maltreatment, an overview is provided on the epidemiology of maltreatment and the funding initiative that sponsored the four trials, and a call is made for further rigorous research by prevention scientists.

Cost Analysis of a Randomized Trial of Getting to Outcomes Implementation Support of CHOICE in Boys and Girls Clubs in Southern California.

Costs of supporting prevention program implementation are not well known. This study estimates the societal costs of implementing CHOICE, a voluntary after-school alcohol and other drug prevention program for adolescents, in Boys and Girls Clubs (BGCs) across Southern California with and without an implementation support system called Getting To Outcomes© (GTO). This article uses micro-costing methods to estimate the cost of the CHOICE program and GTO support. Labor and expense data were obtained from logs kept by the BGC staff and by the GTO technical assistance (TA) staff, and staff time was valued based on Bureau of Labor Statistics estimates. From the societal perspective, the cost of implementing CHOICE at BGCs over the 2-year study period was $27 per attendee when CHOICE was offered by itself (all costs incurred by the BGCs) and $177 per attendee when CHOICE was offered with GTO implementation support ($67 cost to the BGCs; $110 to the entity funding GTO). These results were most sensitive to assumptions as to the number of times CHOICE was offered per year. Adding GTO implementation support to CHOICE increased the cost per attendee by approximately $150. For this additional cost, there was evidence that the CHOICE program was offered with more fidelity and offered more often after the 2-year intervention ended. If the long-term benefits of this better and continued implementation are found to exceed these additional costs, GTO could be an attractive structure to support evidence-based substance misuse prevention programs. Trial Registration. This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was registered May 12, 2014.

Cost analysis of a randomized trial of Getting to Outcomes implementation support for a teen pregnancy prevention program offered in Boys and Girls Clubs in Alabama and Georgia.

Implementation support can improve outcomes of evidence-based programs (EBP) for adolescents, but with a cost. To assist in determining whether this cost is worthwhile, this study estimated the cost of adding Getting To Outcomes© (GTO) implementation support to a teen pregnancy and sexually transmitted infection prevention EBP called Making Proud Choices (MPC) in 32 Boys and Girls Clubs (BGCs) in Alabama and Georgia. Enhancing Quality Interventions Promoting Healthy Sexuality (EQUIPS) was a 2-year, cluster-randomized controlled trial comparing MPC with MPC + GTO. We used micro-costing to estimate costs and captured MPC and GTO time from activity logs completed by GTO staff. Key resource use and cost components were compared between the randomized groups, years, and states (to capture different community site circumstances) using 2-sample t tests. There were no significant differences between randomized groups in attendees per site, resource use, or costs for either year. However, there were significant differences between states. Adding GTO to MPC increased the societal costs per attendee from $67 to $144 (2015 US dollars) in Georgia and from $106 to $314 in Alabama. The higher Alabama cost was due to longer travel distances and to more BGC staff time spent on GTO in that state. GTO also improved adherence, classroom delivery, and condom-use intentions more in Alabama youth. Thus, Alabama's GTO-related BGC staff time costs may be better estimates of effective GTO. If teen childbearing costs taxpayers approximately $20,000 per teen birth, adding GTO to MPC would be worthwhile to society if it prevented one more teen birth per 140 attendees than MPC alone.Trial registration. ClinicalTrials.gov , NCT01818791. Registered March 26, 2013, https://clinicaltrials.gov/ct2/show/NCT01818791?term=NCT01818791&draw=2&rank=1.

Community capacity coach: Embedded support to implement evidenced-based prevention.

Given the pervasive nature of child abuse and neglect, multilevel public health approaches are necessary. Implementation science can help in understanding the most effective ways to build infrastructure and support delivery of such approaches. In this theoretical paper, we describe the implementation of the Positive Parenting Program (Triple P), an evidence-based population-level parenting program in South Carolina. While implementation science has informed how to move population-level efforts to scale, we discuss challenges that persist in practice, such as when there is a need for multiple stakeholders to understand, support, and apply implementation best practices in a systematic and consistent way. To address this challenge, we introduce the role of a Community Capacity Coach, who lives in the local community and works towards the implementation of Triple P. The Coach is responsible for bridging gaps between the local community and statewide support systems. We detail the ways in which the Coach's role aligns with key intermediary functions, and how the Coach is embedded within the larger Triple P statewide support system. We then discuss the assessment of the Coach's impact; and conclude with future directions and next steps for this role within Triple P South Carolina.

Sustained effects of a school readiness intervention: 5th grade outcomes of the Head Start REDI program.

This study examined the sustained effects of the Head Start Research-based Developmentally Informed (REDI) program, which enriched preschools with evidence-based programming targeting social-emotional and language/emergent literacy skills. 44 Head Start classrooms were randomly assigned to intervention or a usual practice control group, and 356 4-year-olds (25% African American, 17% Latino, 54% female) were followed through the end of 5th grade. Growth curve analyses revealed that significant intervention effects on teacher-rated social adjustment, academic engagement, and parent involvement identified at the end of the Head Start year were sustained throughout elementary school. These findings demonstrate that evidence-based curricula combined with professional development support can enhance preschool programming and promote the elementary school adjustment of children living in poverty.

Participatory development and pilot testing of iChoose: an adaptation of an evidence-based paediatric weight management program for community implementation.

BACKGROUND: To describe the identification, adaptation, and testing of an evidence-based pediatric weight management program for a health disparate community. METHODS: A community advisory board (CAB) of decision-makers and staff from local health care, public health, and recreation organizations engaged with academic partners to select an evidence-based program (EBP) for local implementation. Three EBPs were identified (Traffic Light, Bright Bodies, Golan and colleagues Home Environmental Model) and each EBP was rated on program characteristics, implementation and adaptation, and adoptability. Following selection of the EBP that was rated highest, the POPS-CAB made adaptations based on the program principles described in peer-reviewed publications. The adapted intervention, iChoose, was then pilot tested in 3 iterative phases delivered initially by research partners, then co-delivered by research and community partners, then delivered by community partners. The RE-AIM framework was used to plan and evaluate the iChoose intervention across all waves with assessments at baseline, post program (3 months), and follow-up (6 months). RESULTS: Bright Bodies rated highest on program characteristics and adoptability (p's < 0.05), while Home Environmental Model rated highest on implementation factors (p < 0.05). Qualitatively, the selection focused on important program characteristics and on matching those characteristics to the potential to fit within the community partner services. The adapted program-iChoose-had 18% reach and with participants that were representative of the target population on age, gender, ethnicity, and race. Effectiveness was demonstrated by modest, but significant reductions in BMI z-scores at post-program compared to baseline (MΔ = - 0.047; t = - 2.11, p = 0.046). This decrease returned to values similar to baseline 3 months (MΔ = 0.009) after the program was completed. Implementation fidelity was high and implementation fidelity did not differ between community or research delivery agents. CONCLUSION: The process to help organizations identify and select evidence-based programs appropriate for their community led to consensus on a single EBP. While iChoose was successful in initiating changes in BMI z-scores, could be implemented in a low resource community with fidelity, it was insufficient to lead to sustained child BMI z-scores. In response to these data, maintenance of program effects and delivery are the current focus of the CBPR team.

Public Health Benefits 16 Years After a Statewide Policy Change: Communities That Care in Pennsylvania.

Communities That Care (CTC), an evidence-based prevention system, has been installed outside of a research context in over 500 communities worldwide. Yet, its effectiveness in a non-research context is unknown. Using a repeated cross-sectional design with propensity score weighting at the school district-level, the purpose of this study was to examine the effect of widespread diffusion of CTC across Pennsylvania on adolescent substance use, delinquency, and depression. Anonymous youth survey data were collected from 6th, 8th, 10th, and 12th grade students every other year from 2001 to 2011. Three-hundred eighty-eight school districts participated in one to six waves of data collection during that time, resulting in a total of 470,798 student-reported observations. The intervention school districts received programming provided by CTC coalitions. Outcome measures were lifetime and past 30-day alcohol, tobacco, marijuana, and other drug use. Lifetime and past year participation in delinquency and current depressive symptoms were also analyzed. Analyses revealed that CTC school districts had significantly lower levels of adolescent substance use, delinquency, and depression. This effect was small to moderate, depending on the particular outcome studied. Overall effects became stronger after accounting for use of evidence-based programs; there are likely differences in implementation quality and other factors that contribute to the observed overall small effect size. Future research needs to unpack these factors.

Commentary on Scaling-Up Evidence-Based Interventions in Public Systems.

This commentary reviews the findings and recommendations from the Society for Prevention Research (SPR) Mapping Advances in Prevention Science (MAPS) IV Task Force as detailed in the article by Fagan et al. (Prevention Science, 2019). In addition to highlighting similarities and differences with the prior MAPS II Task Force findings, discussion focuses on four extension recommendations: the importance of attending to equitable implementation as a pathway to evidence-based intervention (EBI) uptake by communities, the value of broadening conceptualization of data monitoring and evaluation capacity beyond EBI-specific data, the opportunity to more precisely guide EBI-referrals from public systems, and the importance of EBI developers and purveyors proactively and deliberately operationalizing fidelity measurement using actionable constructs.

Scaling up Evidence-Based Interventions in US Public Systems to Prevent Behavioral Health Problems: Challenges and Opportunities.

A number of programs, policies, and practices have been tested using rigorous scientific methods and shown to prevent behavioral health problems (Catalano et al., Lancet 379:1653-1664, 2012; National Research Council and Institute of Medicine, 2009). Yet these evidence-based interventions (EBIs) are not widely used in public systems, and they have limited reach (Glasgow et al., American Journal of Public Health 102:1274-1281, 2012; National Research Council and Institute of Medicine 2009; Prinz and Sanders, Clinical Psychology Review 27:739-749, 2007). To address this challenge and improve public health and well-being at a population level, the Society for Prevention Research (SPR) formed the Mapping Advances in Prevention Science (MAPS) IV Translation Research Task Force, which considered ways to scale up EBIs in five public systems: behavioral health, child welfare, education, juvenile justice, and public health. After reviewing other efforts to scale up EBIs in public systems, a common set of factors were identified as affecting scale-up in all five systems. The most important factor was the degree to which these systems enacted public policies (i.e., statutes, regulations, and guidance) requiring or recommending EBIs and provided public funds for EBIs. Across systems, other facilitators of scale-up were creating EBIs that are ready for scale-up, public awareness of and support for EBIs, community engagement and capacity to implement EBIs, leadership support for EBIs, a skilled workforce capable of delivering EBIs, and data monitoring and evaluation capacity. It was concluded that the following actions are needed to significantly increase EBI scale-up in public systems: (1) provide more public policies and funding to support the creation, testing, and scaling up of EBIs; (2) develop and evaluate specific frameworks that address systems level barriers impeding EBI scale-up; and (3) promote public support for EBIs, community capacity to implement EBIs at scale, and partnerships between community stakeholders, policy makers, practitioners, and scientists within and across systems.

Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings.

The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the "New Beginnings Program" serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.