Examining implementation determinants of a culturally grounded, school-based prevention curriculum in rural Hawai'i: A test development and validation study.

Background: This study examined the implementation determinants of a culturally grounded, school-based drug prevention curriculum in rural Hawai'i. Test development and validation procedures were used to examine the impact of implementation barriers and facilitators of the curriculum in public or charter middle/intermediate schools on Hawai'i Island. Method: A five-phase, mixed-methods approach toward test development and validation was used. These phases included item generation (Phase 1), item refinement and selection (Phase 2), item reduction (Phase 3), reliability testing (Phase 4), and validity testing (Phase 5). Educational administrators, teachers, and staff employed by the Hawai'i State Department of Education (HIDOE) participated in the study. Results: Phases 1 and 2 yielded 50 implementation barriers and 27 implementation facilitators that were evaluated by 204 HIDOE administrators, teachers, and staff. Factor analysis of the barrier items indicated a four-factor solution: (1) Innovation Barriers, (2) HIDOE State-Level Barriers, (3) Teacher-Level Barriers, and (4) Administrator-Level Barriers. Mean comparisons indicated that several barrier and facilitator items differentiated teachers from administrators in the sample. Conclusions: This study contributes to the implementation measurement literature, specifically in the areas of mental health and substance use. It also highlights the importance of addressing multiple contextual levels in the implementation of culturally focused prevention interventions. Plain Language Summary Title: Examining Implementation Barriers and Facilitators for School-Based Prevention in Hawai'i. Plain Language Summary Compared with other major ethnic groups, Native Hawaiian and Pacific Islander (NHPI) youth have substantially higher rates of substance use and higher adverse mental and physical health effects related to their use. Despite these disparities, educational and community-based practitioners have long struggled with implementing and sustaining promising substance use interventions for NHPI youth. This study contributes to our understanding of measuring implementation barriers and facilitators for substance use prevention for NHPI youth in rural school settings. It contributes to the field of health disparities and health equity promotion, by addressing calls for research to understand factors affecting successful implementation of prevention programs. This is critical toward achieving health equity for underrepresented and vulnerable populations, such as NHPI and rural youth.

Exploring Native Hawaiian and Pacific Islander Youths' E-Cigarette Resistance Strategies: Implications for Tobacco Product Use Prevention.

This study examined the e-cigarette and vaping resistance strategies used by Native Hawaiian and Pacific Islander (NHPI) youths in rural Hawai'i. Focus groups (N = 17) were conducted in eight geographically dispersed elementary, middle/intermediate, and multilevel schools in low-income communities on Hawai'i Island. Sixty-nine youths (67% NHPI, Mage = 12.5 years) participated in this study. The resistance strategies discussed across the greatest number of groups were "refuse" (saying no), "explain" (providing reasons for vaping refusal), "avoid" (avoiding people or places where e-cigarettes were used), and "leave" (walking away from a situation where e-cigarettes were being used). Participants described the challenges in using these strategies within contexts characterized by widespread peer and family vaping and strong social demands to use e-cigarettes. The findings suggest the need for multi-level interventions based on youths' resistance strategies to meaningfully reduce youth vaping use in rural and/or NHPI communities.

30 years of youth system of care lessons learned - a qualitative study of Hawai'i's partnership with the Substance Abuse and Mental Health Services Administration.

BACKGROUND: The Hawai'i State Department of Health, Child and Adolescent Mental Health Division (CAMHD) has maintained a longstanding partnership with Substance Abuse and Mental Health Services Administration (SAMHSA) to enhance capacity and quality of community-based mental health services. The current study explored CAMHD's history of SAMHSA system of care (SOC) awards and identified common themes, lessons learned, and recommendations for future funding. METHODS: Employing a two-phase qualitative approach, the study first conducted content analysis on seven final project reports, identifying themes and lessons learned based on SOC values and principles. Subsequently, interviews were conducted with 11 system leaders in grant projects and SOC award projects within the state. All data from project reports and interview transcripts were independently coded and analyzed using rapid qualitative analysis techniques. RESULTS: Content validation and interview coding unveiled two content themes, interagency collaboration and youth and family voice, as areas that required long-term and consistent efforts across multiple projects. In addition, two general process themes, connection and continuity, emerged as essential approaches to system improvement work. The first emphasizes the importance of fostering connections in family, community, and culture, as well as within workforce members and child-serving agencies. The second highlights the importance of nurturing continuity throughout the system, from interagency collaboration to individual treatment. CONCLUSIONS: The study provides deeper understanding of system of care evaluations, offering guidance to enhance and innovate youth mental health systems. The findings suggest that aligning state policies with federal guidelines and implementing longer funding mechanisms may alleviate administrative burdens.

Ho'ouna Pono implementation: applying concept mapping to a culturally grounded substance use prevention curriculum in rural Hawai'i schools.

BACKGROUND: Despite their potential to ameliorate health disparities and address youth substance use, prevention programs have been poorly disseminated and implemented across Hawai'i, which begs the question: Why are effective prevention programs not being used in communities most in need of them? Implementing and sustaining culturally grounded prevention programs is critical to address equitable healthcare and minimize health disparities in communities. The field of implementation science provides frameworks, theories, and methods to examine factors associated with community adoption of these programs. METHOD: Our project applies concept mapping methods to a culturally grounded youth drug prevention program with state level educational leadership in rural Hawai'i schools. The goal is to integrate barrier and facilitator salience collected through teacher and school staff surveys and specific implementation strategies to regionally tailored implementation plans on Hawai'i island. This protocol paper describes the concept mapping steps and how they will be applied in public and public-charter schools. DISCUSSION: Improving prevention program implementation in rural schools can result in sustained support for populations that need it most. The project will integrate implementation science and culturally grounded methods in rural Hawai'i, where most youth are of Native Hawaiian and Pacific Islander descent. This project addresses health disparities among Native Hawaiian and Pacific Islander youth and provides actionable plans for rural Hawai'i communities to implement effective prevention programming.

Metrics to evaluate implementation scientists in the USA: what matters most?

BACKGROUND: Implementation science has grown rapidly as a discipline over the past two decades. An examination of how publication patterns and other scholarly activities of implementation scientists are weighted in the tenure and promotion process is needed given the unique and applied focus of the field. METHODS: We surveyed implementation scientists (mostly from the USA) to understand their perspectives on the following matters: (1) factors weighted in tenure and promotion for implementation scientists, (2) how important these factors are for success as an implementation scientist, (3) how impact is defined for implementation scientists, (4) top journals in implementation science, and (5) how these journals are perceived with regard to their prestige. We calculated univariate descriptive statistics for all quantitative data, and we used Wilcoxon signed-rank tests to compare the participants' ratings of various factors. We analyzed open-ended qualitative responses using content analysis. RESULTS: One hundred thirty-two implementation scientists completed the survey (response rate = 28.9%). Four factors were rated as more important for tenure and promotion decisions: number of publications, quality of publication outlets, success in obtaining external funding, and record of excellence in teaching. Six factors were rated as more important for overall success as an implementation scientist: presentations at professional meetings, involvement in professional service, impact of the implementation scientist's scholarship on the local community and/or state, impact of the implementation scientist's scholarship on the research community, the number and quality of the implementation scientist's community partnerships, and the implementation scientist's ability to disseminate their work to non-research audiences. Participants most frequently defined and described impact as changing practice and/or policy. This expert cohort identified Implementation Science as the top journal in the field. CONCLUSIONS: Overall, there was a significant mismatch between the factors experts identified as being important to academic success (e.g., tenure and promotion) and the factors needed to be a successful implementation scientist. Findings have important implications for capacity building, although they are largely reflective of the promotion and tenure process in the USA.

Knowledge of Evidence-Based Services Questionnaire: Development and Validation of a Short Form.

The Knowledge of Evidence-Based Services Questionnaire (KEBSQ) is an objective measure of therapist knowledge of practices derived from the evidence base for the treatment of youth psychopathology. However, the length of this measure (i.e., 40 items) and respondent demands associated with each item makes it burdensome for researchers and clinicians. This study developed and validated a Short Form of the KEBSQ using Item Response Theory measurement models. The Short Form consists of 17 items and generates two separate scores: Correct Endorsements and Correct Rejections. The Short Form was found to correlate highly with and perform similarly to the Full Form, providing preliminary validity evidence.

The implementation of a culturally grounded, school-based, drug prevention curriculum in rural Hawai'i.

This article describes the process of infusing implementation strategies in the development of a school-based drug prevention curriculum for rural Native Hawaiian youth. The curriculum (Ho'ouna Pono) is a video-enhanced, teacher-implemented curriculum developed using a culturally grounded and community-based participatory research approach. Throughout the development of the curriculum, strategies reflective of the domains of the Consolidated Framework for Implementation Research (CFIR) were integrated into the teacher training manual, to promote the implementation, adoption, and sustainability of the curriculum in rural Hawai'i. These strategies were validated through qualitative data across two interrelated studies with community stakeholders in rural Hawai'i. Implications for prevention, community, and educational practices are described in this article.

A History of Child and Adolescent Treatment Through a Distillation Lens: Looking Back to Move Forward.

Youth psychosocial treatments have proliferated over many decades and the time is ripe to examine the unique contribution of these treatments. Six hundred eighty-nine randomized clinical trials were examined from the PracticeWise Evidence-Based Services Database, an ongoing data source of youth psychosocial literature. A number of studies, treatment protocols, and practice elements across 11 youth problem areas (e.g., anxiety) from 1966 to 2016 are provided. Using distillation methodology, we compared the relationship between new treatment protocols and the identification of new practice elements within each problem area over time. Anxiety, disruptive behavior, attention/hyperactivity, and depression were the most studied problem areas, whereas suicide, eating disorders, and mania were least studied. While identification of new practice elements leveled off, studies continued to increase over time. For some problem areas, unique practice elements have not been identified for over a decade. Future studies examining treatment innovation and implications for behavioral health are discussed.

Interactions Between Clinician and Organizational Characteristics to Predict Cognitive-Behavioral and Psychodynamic Therapy Use.

Conceptual models of implementation posit contextual factors and their associations with evidence-based practice (EBP) use at multiple levels and suggest these factors exhibit complex cross-level interactions. Little empirical work has examined these interactions, which is critical to advancing causal implementation theory and optimizing implementation strategy design. Mixed effects regression examined cross-level interactions between clinician (knowledge, attitudes) and organizational characteristics (culture, climate) to predict cognitive-behavioral and psychodynamic therapy use with youth (N = 247 clinicians across 28 agencies). Results indicated several interactions, highlighting the importance of attending to interactions between variables at multiple levels to advance multilevel implementation theory and strategies.

Therapist and Youth Predictors of Specific Practices Derived from the Evidence-Base in Community Mental Health.

Over the past several years, youth treatment research has moved toward understanding the dissemination and implementation of evidence-based practices (EBPs). As a result, studies have focused on identifying predictors that aid in successful adoption and sustainment of EBPs. Theories of behavior change posit that therapist knowledge and attitudes play a fundamental role in EBP adoption; however, studies have produced mixed findings, which may be an artifact of broad definitions of both EBP knowledge and EBP itself. The current study was an examination of 46 youth community therapists and the extent to which varying types of knowledge and attitudes as well as youth characteristics predicted specific practices derived from the evidence-base. Results suggested that specific EBP knowledge predicted specific practices, highlighting the need for more specificity when examining predictors of EBP use. Therapists' attitudes, demographic characteristics, and youth characteristics were also significant predictors of EBP use. Future research should consider examining discrete and specific practices to better understand and predict therapists' future behavior.

Understanding the Landscape of Behavioral Health Pre-service Training to Inform Evidence-Based Intervention Implementation.

Reports on the behavioral health workforce highlight the need to enhance evidence-based capacity; evidence-based interventions incorporated into pre-service graduate curricula (coursework and fieldwork) are needed to meet this goal. Improving educational practices across pre-service settings will require understanding of and careful attention to the contextual factors that exert pressure on curricula. The authors believe efforts to change educational practices can be enhanced by application of implementation science principles. This Open Forum delineates the key contextual factors that influence pre-service education, highlights gaps in the literature, and proposes an agenda for future research at the intersection of behavioral health workforce development and implementation science.

Insights in Public Health: The Help Your Keiki Website: Increasing Youth and Caregiver Awareness of Youth Psychosocial Mental Health Treatment.

Increasing evidence-based practice (EBP) use in community mental health is a national priority, especially given that one in five youth will suffer from mental health concerns before adulthood. Implementation science offers a unique lens for understanding EBP use that identifies barriers and facilitators of successful adoption. Consumer engagement is often overlooked as an EBP implementation strategy. In this article, we describe the State of Hawai'i Child and Adolescent Mental Health Division's innovative effort to target consumer EBP demand via the Help Your Keiki Website. Feedback from community stakeholders and website analytics converge to suggest that the most helpful content is related to finding help, normalizing concerns, and questions to ask therapists. Future outreach efforts as well as ongoing improvement and enhancement of the website are discussed.

Furthering Our Understanding of Therapist Knowledge and Attitudinal Measurement in Youth Community Mental Health.

Examining therapist evidence-based practice (EBP) knowledge seems an important step for supporting successful implementation. Advances in implementation science suggest a distinction between practice specific (i.e., knowing which practices are derived from the evidence base) and EBP process (i.e., integrating research evidence, clinical experience, client characteristics, and monitoring outcomes) knowledge. An examination of how these knowledge types are measured and relate to attitudes appears warranted. In our sample of 58 youth community therapists, both practice specific and EBP process knowledge accounted for EBP attitude scores, which varied by therapist demographic variables. Implications for measurement of therapist constructs and future research in identifying therapist predictors of EBP use and youth clinical improvement are discussed.

Predictors of clinician use of exposure therapy in community mental health settings.

Exposure therapy is recognized as the key component of cognitive-behavioral treatment for anxiety. However, exposure is the least used evidence-based treatment in community mental health settings and is the most challenging technique for clinicians to adopt within the context of effectiveness and implementation trials. Little work has examined clinician and organizational characteristics that predict use of exposure, which is important for identifying implementation strategies that may increase its use. In a large sample of community health clinicians (N=335) across 31 clinical practice sites, this study characterized clinician and organizational predictors of exposure use and relaxation for anxiety. Mixed effects regression analyses indicated that both clinician attitudes and an organization's implementation climate may be important levers for interventions seeking to increase clinician exposure use. Greater clinician use of relaxation strategies was also associated with less exposure use. Results point to important implications for implementing cognitive-behavioral therapy for anxiety, including de-emphasizing relaxation and attending to organizational climate.

Evidence-Based Practice Implementation Within a Theory of Planned Behavior Framework.

Although significant progress has been made in the identification of youth evidence-based practices, the adoption of these interventions into community-based mental health care remains limited. Dissemination and implementation (DI) research has the potential to bridge this science-practice gap in clinical psychology. The theory of planned behavior (TPB) offers a useful conceptualization of individual behavior change including behavioral intention as defined by attitudes, subjective norms, and perceived behavioral control. To facilitate application of this model to DI efforts, the current study explores perspectives about using evidence-based practice from stakeholders in the field of youth mental health (including clinical supervisors, case managers, administrators at the departments of health and education, and direct service providers in clinic-based, school-based, and intensive in-home settings) within the TPB framework. A set of instrument items was created from this rich qualitative data using a rigorous mixed-method content validation approach. Instrument items are provided for future use in DI research.

Therapist Attitudes Towards Evidence-Based Practice: A Joint Factor Analysis.

Despite the accumulated research support for the use of evidence-based practices (EBPs) with youth, these treatment approaches remain underutilized in community settings. Therapist attitudes towards EBPs play a pivotal role in their adoption and implementation of these practices. The present investigation employs joint exploratory factor analysis to evaluate the structure of two measures of therapist attitudes, the Evidence-Based Practices Attitudes Scale and the Modified Practice Attitude Scale. Results suggest three factors including (a) importance of clinical experience over EBPs, (b) clinician openness to change, and (c) problems with EBPs. Recommendations are provided for future evaluation of therapist attitudes and associated characteristics.

The Price per Prospective Consumer of Providing Therapist Training and Consultation in Seven Evidence-Based Treatments within a Large Public Behavioral Health System: An Example Cost-Analysis Metric.

OBJECTIVE: Public-sector behavioral health systems seeking to implement evidence-based treatments (EBTs) may face challenges selecting EBTs given their limited resources. This study describes and illustrates one method to calculate cost related to training and consultation to assist system-level decisions about which EBTs to select. METHODS: Training, consultation, and indirect labor costs were calculated for seven commonly implemented EBTs. Using extant literature, we then estimated the diagnoses and populations for which each EBT was indicated. Diagnostic and demographic information from Medicaid claims data were obtained from a large behavioral health payer organization and used to estimate the number of covered people with whom the EBT could be used and to calculate implementation-associated costs per consumer. RESULTS: Findings suggest substantial cost to therapists and service systems related to EBT training and consultation. Training and consultation costs varied by EBT, from Dialectical Behavior Therapy at $238.07 to Cognitive Behavioral Therapy at $0.18 per potential consumer served. Total cost did not correspond with the number of prospective consumers served by an EBT. CONCLUSION: A cost-metric that accounts for the prospective recipients of a given EBT within a given population may provide insight into how systems should prioritize training efforts. Future policy should consider the financial burden of EBT implementation in relation to the context of the population being served and begin a dialog in creating incentives for EBT use.

An Exploratory Factor Analysis of the Knowledge of Evidence-Based Services Questionnaire.

Recent developments for disseminating and implementing evidence-based practices (EBPs) have emphasized the importance of increasing therapists' knowledge of such techniques. However, systematic efforts to measure knowledge in therapists serving youth have mostly relied on poorly researched study-specific measures. This study investigated the structure of EBP knowledge in a large sample of youth community therapists (N = 240) via a therapist report instrument emphasizing therapeutic commonalities across various types of treatment approaches. Findings supported a three-factor structure of knowledge with scores on factors varying by therapists' primary practice setting and education level. The relationships between therapist knowledge of and attitudes towards EBPs are also discussed.

Engineering youth service system infrastructure: Hawaii's continued efforts at large-scale implementation through knowledge management strategies.

Hawaii's Child and Adolescent Mental Health Division provides a unique illustration of a youth public mental health system with a long and successful history of large-scale quality improvement initiatives. Many advances are linked to flexibly organizing and applying knowledge gained from the scientific literature and move beyond installing a limited number of brand-named treatment approaches that might be directly relevant only to a small handful of system youth. This article takes a knowledge-to-action perspective and outlines five knowledge management strategies currently under way in Hawaii. Each strategy represents one component of a larger coordinated effort at engineering a service system focused on delivering both brand-named treatment approaches and complimentary strategies informed by the evidence base. The five knowledge management examples are (a) a set of modular-based professional training activities for currently practicing therapists, (b) an outreach initiative for supporting youth evidence-based practices training at Hawaii's mental health-related professional programs, (c) an effort to increase consumer knowledge of and demand for youth evidence-based practices, (d) a practice and progress agency performance feedback system, and (e) a sampling of system-level research studies focused on understanding treatment as usual. We end by outlining a small set of lessons learned and a longer term vision for embedding these efforts into the system's infrastructure.