Research Community Collaboration in Observational Implementation Research: Complementary Motivations and Concerns in Engaging in the Study of Implementation as Usual.

Implementation research is dominated by studies of investigator-driven implementation of evidence-based practices (EBPs) in community settings. However, systems of care have increasingly driven the scale-up of EBPs through policy and fiscal interventions. Research community partnerships (RCPs) are essential to generating knowledge from these efforts. Interviews were conducted with community stakeholders (system leaders, program managers, therapists) involved in a study of a system-driven implementation of multiple EBPs in children's mental health services. Findings suggest novel considerations in initial engagement phases of an RCP, given the unique set of potentially competing and complementary interests of different stakeholder groups in implementation as usual.

Agency responses to a system-driven implementation of multiple evidence-based practices in children's mental health services.

BACKGROUND: Large mental health systems are increasingly using fiscal policies to encourage the implementation of multiple evidence-based practices (EBPs). Although many implementation strategies have been identified, little is known about the types and impacts of strategies that are used by organizations within implementation as usual. This study examined organizational-level responses to a fiscally-driven, rapid, and large scale EBP implementation in children's mental health within the Los Angeles County Department of Mental Health. METHODS: Qualitative methods using the principles of grounded theory were used to characterize the responses of 83 community-based agencies to the implementation effort using documentation from site visits conducted 2 years post reform. RESULTS: Findings indicated that agencies perceived the rapid system-driven implementation to have both positive and negative organizational impacts. Identified challenges were primarily related to system implementation requirements rather than to characteristics of specific EBPs. Agencies employed a variety of implementation strategies in response to the system-driven implementation, with agency size associated with implementation strategies used. Moderate- and large-sized agencies were more likely than small agencies to have employed systematic strategies at multiple levels (i.e., organization, therapist, client) to support implementation. CONCLUSIONS: These findings are among the first to characterize organizational variability in response to system-driven implementation and suggest ways that implementation interventions might be tailored by organizational characteristics.

Measuring Sustainment of Multiple Practices Fiscally Mandated in Children's Mental Health Services.

Examining sustainment of multiple evidence-based practices is crucial to understanding the processes and outcomes of system-driven implementation efforts that are increasingly common. This study used administrative claims data to characterize volume and penetration of six practices over 19 fiscal quarters following initial implementation within the context of a system-driven, fiscally mandated implementation effort in Los Angeles County Department of Mental Health's Prevention and Early Intervention services. Patterns of volume changes over time revealed an overall ramp up of practice claims over time, but variability in patterns of volume and penetration for each practice. Findings varied by the methods used to index and analyze volume and penetration. Furthermore, a number of client case-mix and therapist characteristics were associated with the volume of therapists' claims for each practice relative to their claims for the other practices.

Using Survival Analysis to Understand Patterns of Sustainment within a System-Driven Implementation of Multiple Evidence-Based Practices for Children's Mental Health Services.

Evidence-based practice (EBP) implementation requires substantial resources in workforce training; yet, failure to achieve long-term sustainment can result in poor return on investment. There is limited research on EBP sustainment in mental health services long after implementation. This study examined therapists' continued vs. discontinued practice delivery based on administrative claims for reimbursement for six EBPs [Cognitive Behavioral Interventions for Trauma in Schools (CBITS), Child-Parent Psychotherapy, Managing and Adapting Practices (MAP), Seeking Safety (SS), Trauma-Focused Cognitive Behavior Therapy (TF-CBT), and Positive Parenting Program] adopted in a system-driven implementation effort in public mental health services for children. Our goal was to identify agency and therapist factors associated with a sustained EBP delivery. Survival analysis (i.e., Kaplan-Meier survival functions, log-rank tests, and Cox regressions) was used to analyze 19 fiscal quarters (i.e., approximately 57 months) of claims data from the Prevention and Early Intervention Transformation within the Los Angeles County Department of Mental Health. These data comprised 2,322,389 claims made by 6,873 therapists across 88 agencies. Survival time was represented by the time elapsed from therapists' first to final claims for each practice and for any of the six EBPs. Results indicate that therapists continued to deliver at least one EBP for a mean survival time of 21.73 months (median = 18.70). When compared to a survival curve of the five other EBPs, CBITS, SS, and TP demonstrated a higher risk of delivery discontinuation, whereas MAP and TF-CBT demonstrated a lower risk of delivery discontinuation. A multivariate Cox regression model revealed that agency (centralization and service setting) and therapist (demographics, discipline, and case-mix characteristics) characteristics were significantly associated with risk of delivery discontinuation for any of the six EBPs. This study illustrates a novel application of survival analysis to administrative claims data in system-driven implementation of multiple EBPs. Findings reveal variability in the long-term continuation of therapist-level delivery of EBPs and highlight the importance of both agency and workforce characteristics in the sustained delivery of EBPs. Findings direct the field to potential targets of sustainment interventions (e.g., strategic assignment of therapists to EBP training and strategic selection of EBPs by agencies).