A Scoping Review of the Influence of Evidence-Based Program Resources (EBPR) Websites for Behavioral Health.

Evidence-based program resources (EBPR) websites evaluate behavioral health programs, practices or policies (i.e., interventions) according to a predetermined set of research criteria and standards, usually resulting in a summary rating of the strength of an intervention's evidence base. This study is a mixed-methods analysis of the peer-reviewed academic literature relating to the influence of EBPRs on clinical practice and policy in the behavioral health field. Using an existing framework for a scoping review, we searched for research articles in PubMed, Web of Science, SCOPUS, and ProQuest that were published between January 2002 and March 2022, referenced an EBPR or multiple EBPRs, and presented data showing the influence of one or more EBPRs on behavioral health. A total of 210 articles met the inclusion criteria and were classified into five distinct categories of influence, the most important of which was showing the direct impact of one or more EBPRs on behavioral health (8.1% of articles), defined as documenting observable changes in interventions or organizations that are at least partly due to information obtained from EBPR(s). These included impacts at the state legislative and policy-making level, at the community intervention level, provider agency level, and individual practitioner level. The majority of influences identified in the study were indirect demonstrations of how EBPRs are used in various ways. However, more studies are needed to learn about the direct impact of information from EBPRs on the behavioral health field, including impact on clinician practice and treatment outcomes for consumers.

Visitors' Assessment and Utilization of Evidence-Based Program Resources (EBPR) Websites.

Background: Evidence-based program resources (EBPR) websites for behavioral health are a potentially useful tool to assist decision-makers and practitioners in deciding which behavioral health interventions to implement. EBPR websites apply rigorous research standards to assess the effectiveness of behavioral healthcare programs, models, and clinical practices. Method: Visitors to a convenience sample of six EBPR websites (N=369, excluding students) were recruited for telephone interviews primarily by means of a pop-up invitation on the sites. Results: The visitors view the EBPR sites as important sources of information to support the identification and adoption of evidence-based programs/practices (EBPs) in behavioral healthcare, which aligns with the primary mission of EBPRs. For repeat visitors, there was some indication that the information obtained helped effect certain changes in their agencies' programs and policies. However, increased or improved guidance on EBP implementation was also requested. Conclusion: EBPR websites should be better publicized to the behavioral healthcare field.

From Research Training to Scientific Advancement-Contributions from the Implementation Research Institute: An Introduction to the Special Issue.

The special series is designed to provide examples of funded implementation research conducted by alumni of the first four cohorts of the Implementation Research Institute (IRI). The introduction links the six substantive papers to the conceptual and methodological challenges laid out in a 2009 publication in this journal which led to the IRI training program in the emerging science of implementation with a special focus on behavior health settings. The 7th paper in the series illustrates an innovative evaluative approach to design and measurement of IRI fellow publications and grants informed by the training program such as bibliometrics. The introduction also notes some elements identified in the 2009 foundational paper not represented in these papers such as costs as well as important developments and foci in the decade since 2009 such as de-implementation, sustainability, dynamic adaptation processes, and hybrid designs that need to be an integral part of training programs in implementation research.

Evaluation of the Implementation Research Institute: Trainees' Publications and Grant Productivity.

With growth in the field of dissemination and implementation (D&I) research, there has been growth in capacity building, with many training opportunities. As such, it is important to continue to evaluate D&I research training programs. This paper reports the results of an evaluation of the Implementation Research Institute (IRI), a R25 funded by the National Institute of Mental Health with additional funding by the Department of Veterans Affairs (VA). The fourth cohort also had a supplement from the National Institute on Drug Abuse. Using bibliometrics data, we report on a quasi-experimental retrospective cohort study assessing whether the rates of scholarly productivity in D&I science of IRI fellows (those who applied and were accepted to the training) were greater than those who applied but were not accepted to IRI. Our findings show that Selected Applicants' odds of publishing in implementation science were higher for earlier alumni, starting at 12% 1 year out and increasing to 94% for those who were 4 years out from starting training. Chances for Non-Selected Applicants remained relatively stable, starting at 47% at 1 year and going to 33% at 4 years since their application, a pattern that was stable even after controlling for demographic characteristics. These results support the hypothesis that IRI is increasing the D&I research productivity of those selected to the program, and that our fellows are advancing the field of D&I compared to those investigators not selected to our institute. Our finding also indicates the importance of a 2-year training.

Factors Associated with Community-Partnered School Behavioral Health Clinicians' Adoption and Implementation of Evidence-Based Practices.

Community-partnered school behavioral health (CP-SBH) is a model whereby schools partner with local community agencies to deliver services. This mixed-methods study examined 80 CP-SBH clinicians' adoption and implementation of evidence-based practice (EBP) approaches following mandated training. Forty-four clinicians were randomly assigned to one of two training conditions for a modular common elements approach to EBPs; 36 clinicians were preselected for training in a non-modular EBP. EBP knowledge improved for all training conditions at 8-month follow-up and practice element familiarity improved for modular approach training conditions, but the modular condition including ongoing consultation did not yield better results. Qualitative interviews (N = 17) highlighted multi-level influences of the CP-SBH service system and individual clinician characteristics on adoption and implementation.

An Overview of Research and Evaluation Designs for Dissemination and Implementation.

The wide variety of dissemination and implementation designs now being used to evaluate and improve health systems and outcomes warrants review of the scope, features, and limitations of these designs. This article is one product of a design workgroup that was formed in 2013 by the National Institutes of Health to address dissemination and implementation research, and whose members represented diverse methodologic backgrounds, content focus areas, and health sectors. These experts integrated their collective knowledge on dissemination and implementation designs with searches of published evaluations strategies. This article emphasizes randomized and nonrandomized designs for the traditional translational research continuum or pipeline, which builds on existing efficacy and effectiveness trials to examine how one or more evidence-based clinical/prevention interventions are adopted, scaled up, and sustained in community or service delivery systems. We also mention other designs, including hybrid designs that combine effectiveness and implementation research, quality improvement designs for local knowledge, and designs that use simulation modeling.

Homelessness and Aging Out of Foster Care: A National Comparison of Child Welfare-Involved Adolescents.

The present study represents the first large-scale, prospective comparison to test whether aging out of foster care contributes to homelessness risk in emerging adulthood. A nationally representative sample of adolescents investigated by the child welfare system in 2008 to 2009 from the second cohort of the National Survey of Child and Adolescent Well-being Study (NSCAW II) reported experiences of housing problems at 18- and 36-month follow-ups. Latent class analyses identified subtypes of housing problems, including literal homelessness, housing instability, and stable housing. Regressions predicted subgroup membership based on aging out experiences, receipt of foster care services, and youth and county characteristics. Youth who reunified after out-of-home placement in adolescence exhibited the lowest probability of literal homelessness, while youth who aged out experienced similar rates of literal homelessness as youth investigated by child welfare but never placed out of home. No differences existed between groups on prevalence of unstable housing. Exposure to independent living services and extended foster care did not relate with homelessness prevention. Findings emphasize the developmental importance of families in promoting housing stability in the transition to adulthood, while questioning child welfare current focus on preparing foster youth to live.

Socio-contextual Determinants of Research Evidence Use in Public-Youth Systems of Care.

Recent efforts have been devoted to understanding the conditions by which research evidence use (REU) is facilitated from the perspective of system leaders in the context of implementing evidence-based child mental health interventions. However, we have limited understanding of the extent to which outer contextual factors influence REU. Outer contextual factors for 37 counties in California were gathered from public records in 2008; and child welfare, juvenile justice, and mental health system leaders' perceptions of their REU were measured via a web-based survey from 2010 to 2012. Results showed that leaders with higher educational attainment and in counties with lower expenditures on inpatient mental health services were significantly associated with higher REU. Positive relationships between gathering research evidence and racial minority concentration and poverty at the county level were also detected. Results underscore the need to identify the organizational and socio-political factors by which mental health services and resources meet client demands that influence REU, and to recruit and retain providers with a graduate degree to negotiate work demands and interpret research evidence.

Effects of the KEEP Foster Parent Intervention on Child and Sibling Behavior Problems and Parental Stress During a Randomized Implementation Trial.

Children in foster care are at risk for externalizing behavior problems, which can in turn increase the risk of changes in foster care placement. The KEEP (Keeping Foster Parents Trained and Supported) foster parent training intervention was designed to equip foster parents with strategies for managing externalizing behavior problems. The primary goals of this investigation were to (a) examine the effectiveness of the KEEP intervention in reducing child behavior problems, as delivered by a community agency; (b) determine if the effects of the KEEP intervention generalize to more than one child in the same home; and (c) examine the effectiveness of the KEEP intervention in reducing parental stress associated with child behavior problems. The data from 335 foster and kinship families with children between the ages of 5 and 12 years were analyzed to address these objectives. Families were randomly assigned to the intervention or control condition. The results indicated that the KEEP intervention was effective in reducing child behavior problems when delivered by a community agency. These results expanded prior research on the KEEP intervention, revealing that the intervention was effective in reducing the behavior problems of more than one child in the same household and in reducing parental stress levels associated with the behavioral issues of the focal child. Thus, the KEEP intervention model holds promise for reducing the behavior problems of children in foster care and reducing stress levels of foster and kinship caregivers as it is disseminated and implemented within similar child welfare settings.

Interorganizational relationships among family support organizations and child mental health agencies.

This study examined: (1) qualitative aspects of close working relationships between family support organizations and child mental health agencies, including effective and ineffective characteristics of the relationship and aspects that they would change, and (2) the impact of the working relationship upon the family support organization. Semi-structured interviews were conducted with 40 directors of family support organizations characterized as having a close working relationship with a child mental health agency. Three main themes emerged regarding the quality of the working relationship: (a) interactional factors, including shared trust, communication, collaboration and service coordination; (b) aspects of the inner context of the family support organization, mental health agency, or both, including alignment of goals and values and perceptions of mental health services; and (c) outer contextual factors external to the organizations, such as financial and county regulations. Responses to the perceived impact of the relationship was divided into two themes: positive impacts (e.g. gained respect, influence and visibility), and negative impacts (e.g. lack of trust). This study lays the foundation for future research to better understand the mechanisms underlying interorganizational relationships in communities among different types of providers to create a more seamless continuum of services for families of children with mental health conditions.

The Cost of Implementing New Strategies (COINS): A Method for Mapping Implementation Resources Using the Stages of Implementation Completion.

OBJECTIVE: Illustrate the value of a strategy used for measuring the costs and resources used in the implementation process over and above the costs of the intervention itself in the context of a two-arm randomized controlled trial. METHODS: Counties in California and Ohio (sites) were invited to implement Multidimensional Treatment Foster Care (MTFC), an alternative to congregate care for youth. Participating sites (n=53) were randomized to one of two implementation = sites share information and move through the implementation process as a cohort facilitated by an MTFC purveyor or (2) Individual Implementation (IND: "as usual") where sites work individually with the MTFC purveyor. The implementations were monitored using the Stages of Implementation Completion (SIC) measure of a number of observable activities, developed as part of the trial to segment the implementation process into 8 stages of implementation. Resource data gathered from the implementation purveyors and site participants were used to map costs onto each of the 8 stages to generate total cost measures stratified by type of resource and stage of implementation for each of the study arms. RESULTS: The SIC provided a feasible costing template to map costs onto observable activities and to enable the examination of important differences in implementation strategies for an evidence-based practice. The average total implementation cost prior to program start-up of CDT was $133,106; IND cost $118,699. While CDT cost more in a number of stages, it resulted in fewer county staff hours being used and shorter mean times to implementation than IND. In cases where rapidity of implementation of reducing staff time required for implementation is valued, then CDT would be the preferable implementation approach. CONCLUSIONS: The SIC is a useful tool for determining implementation resources needed for new evidence-based practice programs for youth and particularly for comparing different implementation strategies that might be tried in pilot programs.

Cost calculator methods for estimating casework time in child welfare services: A promising approach for use in implementation of evidence-based practices and other service innovations.

Estimating costs in child welfare services is critical as new service models are incorporated into routine practice. This paper describes a unit costing estimation system developed in England (cost calculator) together with a pilot test of its utility in the United States where unit costs are routinely available for health services but not for child welfare services. The cost calculator approach uses a unified conceptual model that focuses on eight core child welfare processes. Comparison of these core processes in England and in four counties in the United States suggests that the underlying child welfare processes generated from England were perceived as very similar by child welfare staff in California county systems with some exceptions in the review and legal processes. Overall, the adaptation of the cost calculator for use in the United States child welfare systems appears promising. The paper also compares the cost calculator approach to the workload approach widely used in the United States and concludes that there are distinct differences between the two approaches with some possible advantages to the use of the cost calculator approach, especially in the use of this method for estimating child welfare costs in relation to the incorporation of evidence-based interventions into routine practice.

Explaining variations in state foster care maintenance rates and the implications for implementing new evidence-based programs.

BACKGROUND: U.S. Child Welfare systems are involved in the lives of millions of children, and total spending exceeds $26 billion annually. Out-of-home foster care is a critical and expensive Child Welfare service, a major component of which is the maintenance rate paid to support housing and caring for a foster child. Maintenance rates vary widely across states and over time, but reasons for this variation are not well understood. As evidence-based programs are disseminated to state Child Welfare systems, it is important to understand what may be the important drivers in the uptake of these practices including state spending on core system areas. DATA AND METHODS: We assembled a unique, longitudinal, state-level panel dataset (1990-2008) for all 50 states with annual data on foster care maintenance rates and measures of child population in need, poverty, employment, urbanicity, proportion minority, political party control of the state legislature and governorship, federal funding, and lawsuits involving state foster care systems. All monetary values were expressed in per-capita terms and inflation adjusted to 2008 dollars. We used longitudinal panel regressions with robust standard errors and state and year fixed effects to estimate the relationship between state foster care maintenance rates and the other factors in our dataset, lagging all factors by one year to mitigate the possibility that maintenance rates influenced their predictors. Exploratory analyses related maintenance rates to Child Welfare outcomes. FINDINGS: State foster care maintenance rates have increased in nominal terms, but in many states, have not kept pace with inflation, leading to lower real rates in 2008 compared to those in 1991 for 54% of states for 2 year-olds, 58% for 9 year-olds, and 65% for 16 year-olds. In multivariate analyses including socioeconomic, demographic, and political factors, monthly foster care maintenance rates declined $15 for each 1% increase in state unemployment and declined $40 if a state's governorship and legislature became Republican, though significance was marginal. In analyses also examining state revenue, federal funding, and legal challenges, maintenance rates increased as the federal share of maximum TANF payments increased. However, >50% of variation in foster care maintenance rates was explained by unobserved state-level factors as measured by state fixed effects. These factors did not appear to be strongly related to 2008 Child Welfare outcomes like foster care placement stability and maltreatment which were also not correlated with foster care maintenance rates. CONCLUSIONS: Despite being part of a social safety net, foster care maintenance rates have declined in real terms since 1991 in many states, and there is no strong evidence that they increase in response to harsher economic climates or to federal programs or legal reviews. State variation in maintenance rates was not related to Child Welfare outcomes, though further analysis of this important relationship is needed. Variability in state foster care maintenance rates appears highly idiosyncratic, an important contextual factor to consider when designing and disseminating evidence-based services.

Exploration and Adoption of Evidence-based Practice by US Child Welfare Agencies.

OBJECTIVE: To examine the extent to which child welfare agencies adopt new practices and to determine the barriers to and facilitators of adoption of new practices. METHODS: Data came from telephone interviews with the directors of the 92 public child welfare agencies that constituted the probability sample for the first National Survey of Child and Adolescent Well-being (NSCAWI). In a semi-structured 40 minute interview administered by a trained Research Associate, agency directors were asked about agency demographics, knowledge of evidence-based practices, use of technical assistance and actual use of evidence-based practices.. Of the 92 agencies, 83 or 90% agreed to be interviewed. RESULTS: Agencies reported that the majority of staff had a BA degree (53.45%) and that they either paid for (52.6%) or provided (80.7%) continuing education. Although agencies routinely collect standardized child outcomes (90%) they much less frequently collect measures of child functioning (30.9%). Almost all agencies (94%) had started a new program or practice but only 24.8% were evidence-based and strategies used to explore new programs or practices usually involved local or state contracts. Factors that were associated with program success included internal support for the innovation (27.3%), and an existing evidence base (23.5%). CONCLUSIONS: Directors of child welfare agencies frequently institute new programs or practices but they are not often evidence-based. Because virtually all agencies provide some continuing education adding discussions of evidence-based programs/practices may spur adaption. Reliance on local and state colleagues to explore new programs and practices suggests that developing well informed social networks may be a way to increase the spread of evidence0based practices.

Inadequate housing among families under investigation for child abuse and neglect: prevalence from a national probability sample.

This study aimed to estimate the prevalence of inadequate housing that threaten out-of-home placement among families under investigation by child welfare. Data came from the National Survey of Child and Adolescent Well-Being, a nationally representative longitudinal survey of child welfare-involved families. Child protective services caseworkers as well as caregivers provided information on families whose child remained in the home after initial investigation (N = 3,867). Multilevel latent class analyses tested the presence of inadequately housed subgroups using 4 housing problem indicators at baseline. Logistic regressions assessed convergent and predictive validity. A two class latent solution best fit the data. Findings indicated that inadequate housing contributed to risk for out-of-home placement in approximately 16 % of intact families under investigation by child protective services. These families were 4 times more likely to need housing services 12 months later. Federal legislation emphasizes integration of social services as necessary to end homelessness. This study demonstrates overlap across public agencies. Enhanced coordination of child welfare and housing services facilitates interventions to prevent and mitigate homelessness.

Assessment of Adoption and Early Implementation Barriers and Facilitators of the Safe Environment for Every Kid (SEEK) Model.

OBJECTIVE: Although the use of interventions for screening for social determinants of health of families in pediatric primary care clinics has increased in the past decade, research on the barriers and facilitators of implementing such interventions has been limited. We explored barriers, facilitators, and the mechanisms clarifying their roles in the adoption and implementation of the Safe Environment for Every Kid (SEEK) model, an approach for strengthening families, promoting children's health and development, and preventing child maltreatment. METHODS: A total of 28 semistructured interviews were completed with 9 practice champions, 11 primary care professionals, 5 behavioral health professionals, and 3 nursing/administrative staff representing 12 pediatric primary care practices participating in a larger randomized control trial of implementing SEEK. RESULTS: We identified several barriers and facilitators in the stages of SEEK's adoption and early implementation. Barriers associated with outer and inner setting determinants and poor innovation-organization fit declined in importance over time, while facilitators associated with SEEK characteristics increased in importance based on participants' responses. Barriers and facilitators were linked by mechanisms of comparison and contrast of burdens and benefits, and problem-solving to address limited capacity with available resources. CONCLUSIONS: Any screening for and addressing social determinants of health demands greater attention to adoption and implementation mechanisms and the processes by which primary care professionals assess and utilize facilitators to address barriers. This occurs in a context defined by perceived burdens and benefits of innovation adoption and implementation, the capacity of the practice, and changes in perception with experiencing the innovation.

The influence of evidence-based program registry websites for dissemination of evidence-based interventions in behavioral healthcare.

PURPOSE: Evidence-based program registries (EBPRs) are web-based databases of evaluation studies that summarize the available evidence for the effectiveness of behavioral healthcare programs, including programs addressing substance misuse, mental health, child welfare, or offender rehabilitation. The study determined the extent to which visitors to selected EBPRs accomplished the objectives of their visits and how often those visits resulted in the adoption of new or improved evidence-based interventions (EBIs). METHOD: A follow-up telephone survey was conducted with 216 visitors to a convenience sample of six EBPRs an average of six months after the visitors' incident visit to the EBPR. RESULTS: The most frequent objective was to identify evidence-based programs/services, curricula or assessments, followed by finding resources to implement or improve the preceding and writing a grant proposal including to comply with funding requirements; 71% of such objectives were achieved across the full set of objectives. Implementation of an EBI was completely achieved for 31% of relevant objectives and some progress on EBI implementation occurred for 19% of relevant objectives. CONCLUSIONS: This is the first study to document the usage of EBPRs as a modality to increase the utilization of EBIs in the actual practice of behavioral healthcare. The results support the continued use of web-based EBPRs for disseminating information on evidence-based interventions for behavioral healthcare.

Practitioner review: Children in foster care--vulnerabilities and evidence-based interventions that promote resilience processes.

BACKGROUND: An increasing number of children are placed in foster care (i.e., a kin or nonkin family home other than the biological parent) due to experiences of physical, sexual, emotional, or psychological abuse, and/or neglect. Children in foster care are at increased risk for a host of negative outcomes encompassing emotional, behavioral, neurobiological, and social realms. METHODS: Areas of risk and vulnerability among foster children are described, including emotional and behavioral deficits, impaired neurobiological development, and social relationship deficits. Evidence suggesting the significance of family placement changes and prenatal exposure to substances as contributing mechanisms is presented. Based on a systematic search of the PsycINFO database (to March 2012), eight efficacious evidence-based interventions for foster families are summarized. FINDINGS: Although the development of evidence-based interventions that improve outcomes for foster children has lagged behind the delivery of interventions in other service sectors (e.g., mental health and educational sectors), several interventions across childhood and adolescence offer promise. Service system constraints offer both challenges and opportunities for more routine implementation of evidence-based interventions. CONCLUSIONS: Given the increased likelihood of poor outcomes for foster children, increased efforts to understand the pathways to vulnerability and to implement interventions shown to be effective in remediating risks and improving outcomes for this population are indicated. Evaluation of efficacious interventions in countries outside of the United States is also needed.

Evaluating child welfare policies with decision-analytic simulation models.

The objective was to demonstrate decision-analytic modeling in support of Child Welfare policymakers considering implementing evidence-based interventions. Outcomes included permanency (e.g., adoptions) and stability (e.g., foster placement changes). Analyses of a randomized trial of KEEP-a foster parenting intervention-and NSCAW-1 estimated placement change rates and KEEP's effects. A microsimulation model generalized these findings to other Child Welfare systems. The model projected that KEEP could increase permanency and stability, identifying strategies targeting higher-risk children and geographical regions that achieve benefits efficiently. Decision-analytic models enable planners to gauge the value of potential implementations.

State Department and Provider Agency Utilization of Evidence-Based Program Registries in Behavioral Healthcare and Child Welfare.

Evidence-based program registries (EBPRs) are web-based compilations of behavioral healthcare programs/interventions that rely on research-based criteria to rate program efficacy or effectiveness for support of programmatic decision-making. The objective was to determine the extent to which behavioral health decision-makers access EBPRs and to understand whether and exactly how they use the information obtained from EPBRs. Single State Authorities (SSAs) and service provider agencies in the areas of behavioral health and child welfare were recruited nationally. Senior staff (n = 375) responsible for the selection and implementation of programs and/or policies were interviewed by telephone concerning their visits (if any) to 28 relevant EBPRs, the types of information they were seeking, whether they found it, and how they may have used that information to effect changes in their organizations. At least one EBPR was visited by 80% of the respondents, with a median of three different registers being visited. Most visitors (55%) found all the information they were seeking; those who did not desired more guidance or tools for individual program implementation or were unable to locate the program or practice that they were seeking. Most visitors (65%) related using the information obtained to make changes in their organizations, in particular to select, start or change a program, or to support the adoption or improvement of evidence-based clinical practices. EBPRs were shown to be important resources for dissemination of research-based program effectiveness data, leading to increased use of evidence-based practices in the field, but the study also identified needs for greater awareness of EBPRs generally and for more attention to implementation of specific recommended programs and practices.