Synchronizing Watches: The Challenge of Aligning Implementation Science and Public Systems.

This special issue of Administration and Policy in Mental Health explores the complexities of the outer system context in implementation science research. In this commentary, we highlight areas of asynchrony between implementation science research and policy realities of public systems. Timing is a critical factor for many aspects of system-level implementation including when and how evidence-based practice initiatives are launched, short and inconsistent timeframes for funding and support, need for early indicators of success and demonstrating return on investment. Greater consideration for the timing that drives change in public systems will strengthen efforts to implement and sustain EBPs in community settings.

Applying the Policy Ecology Framework to Philadelphia's Behavioral Health Transformation Efforts.

Raghavan et al. (Implement Sci 3(26):1-9, 2008) proposed that effective implementation of evidence-based practices requires implementation strategies deployed at multiple levels of the "policy ecology," including the organizational, regulatory or purchaser agency, political, and social levels. However, much of implementation research and practice targets providers without accounting for contextual factors that may influence provider behavior. This paper examines Philadelphia's efforts to work toward an evidence-based and recovery-oriented behavioral health system, and uses the policy ecology framework to illustrate how multifaceted, multilevel implementation strategies can facilitate the widespread implementation of evidence-based practices. Ongoing challenges and implications for research and practice are discussed.

A Multi-Level Examination of Stakeholder Perspectives of Implementation of Evidence-Based Practices in a Large Urban Publicly-Funded Mental Health System.

Our goal was to identify barriers and facilitators to the implementation of evidence-based practices from the perspectives of multiple stakeholders in a large publicly funded mental health system. We completed 56 interviews with three stakeholder groups: treatment developers (n = 7), agency administrators (n = 33), and system leadership (n = 16). The three stakeholder groups converged on the importance of inner (e.g., agency competing resources and demands, therapist educational background) and outer context (e.g., funding) factors as barriers to implementation. Potential threats to implementation and sustainability included the fiscal landscape of community mental health clinics and an evolving workforce. Intervention characteristics were rarely endorsed as barriers. Inner context, outer context, and intervention characteristics were all seen as important facilitators. All stakeholders endorsed the importance of coordinated collaboration across stakeholder groups within the system to successfully implement evidence-based practices.

A Prospective Examination of Clinician and Supervisor Turnover Within the Context of Implementation of Evidence-Based Practices in a Publicly-Funded Mental Health System.

Staff turnover rates in publicly-funded mental health settings are high. We investigated staff and organizational predictors of turnover in a sample of individuals working in an urban public mental health system that has engaged in a system-level effort to implement evidence-based practices. Additionally, we interviewed staff to understand reasons for turnover. Greater staff burnout predicted increased turnover, more openness toward new practices predicted retention, and more professional recognition predicted increased turnover. Staff reported leaving their organizations because of personal, organizational, and financial reasons; just over half of staff that left their organization stayed in the public mental health sector. Implications include an imperative to focus on turnover, with a particular emphasis on ameliorating staff burnout.

A pilot randomized control trial: testing a transitional care model for acute psychiatric conditions.

OBJECTIVE: People with multiple and persistent mental and physical health problems have high rates of transition failures when transferring from a hospital level of care to home. The transitional care model (TCM) is evidence-based and demonstrated to improve posthospital outcomes for elderly with physical health conditions, but it has not been studied in the population with serious mental illness. METHOD: Using a randomized controlled design, 40 inpatients from two general hospital psychiatric units were recruited and randomly assigned to an intervention group (n = 20) that received the TCM intervention that was delivered by a psychiatric nurse practitioner for 90 days posthospitalization, or a control group (n = 20) that received usual care. Outcomes were as follows: service utilization, health-related quality of life, and continuity of care. RESULTS: The intervention group showed higher medical and psychiatric rehospitalization than the control group (p = .054). Emergency room use was lower for intervention group but not statistically significant. Continuity of care with primary care appointments were significantly higher for the intervention group (p = .023). The intervention group's general health improved but was not statistically significant compared with controls. CONCLUSIONS: A transitional care intervention is recommended; however, the model needs to be modified from a single nurse to a multidisciplinary team with expertise from a psychiatric nurse practitioner, a social worker, and a peer support specialist. A team approach can best manage the complex physical/mental health conditions and complicated social needs of the population with serious mental illness.

Implementing an E-prescribing system in outpatient mental health programs.

This study describes the implementation and evaluation of an electronic prescription ordering system and feedback report in three community-based mental health outpatient agencies and the usefulness of the system in improving psychiatrists' prescribing behavior. Using the e-prescribing system as a data collection tool, feedback on evidence based prescribing practices for patients diagnosed with schizophrenia spectrum disorder or major affective disorder was provided to agency directors and prescribers via a monthly report. The results of the project were that e-prescribing tools can be installed at a reasonable cost with a short start up period. Although the feedback intervention did not show a significant reduction in questionable prescribing patterns, we should continue to investigate how to best use HIT to improve safety, reduce costs, and enhance the quality of healthcare. A better understanding of what prescribers find useful and the reasons why they are prescribing non-evidenced based medications is needed if interventions of this type are to be effective. Given the availability of administrative claims data and electronic prescribing technology, considerable potential exists to provide useful information for monitoring and clinical decision making in public mental health systems.

Human Services and Behavioral Health Integration: A Model for Whole-Person Medicaid Managed Care.

A comprehensive, whole-person approach to individuals' health care can be achieved by aligning, integrating, and coordinating health services with other human services. HealthChoices, Pennsylvania's managed Medicaid program, delegates responsibility for Medicaid-funded behavioral health service management to individual counties or multicounty collaboratives. County administrators' programmatic and fiscal oversight of Medicaid-funded services allows them to create synergies between behavioral health and other human service delivery systems and to set priorities on the basis of local needs. This model supports access to community-based care, integration of general medical and behavioral health services, and programs that address social determinants of health.

Care Management Intervention to Address Determinants of Health for Individuals With Multiple Behavioral Health Readmission.

PURPOSE OF STUDY: To examine the effectiveness of a care management intervention to decrease readmissions and to better understand clinical and social determinants associated with readmission. PRIMARY PRACTICE SETTING: Inpatient mental health (MH) and substance use disorder (SUD) facilities, nonhospital SUD withdrawal management and rehabilitation facilities. METHODOLOGY AND SAMPLE: The authors identified 3,950 Medicaid-enrolled individuals who received the intervention from licensed clinical staff of a behavioral health managed care organization; 2,182 individuals were eligible but did not receive the intervention, for treatment as usual (TAU). We used logistic regression to examine factors associated with readmission. Determinants of readmission were summarized through descriptive tests. RESULTS: The intervention was associated with lower readmissions to SUD facilities compared with TAU (6.0% vs. 8.6%, p = .0002) and better follow-up to aftercare. Controlling for clinical differences between groups, regression results found increased odds of readmission for male gender (odds ratio [OR]: 1.33; 95% confidence interval [CI]: 1.16-1.52, p < .0001) and dual MH and SUD diagnoses (OR: 1.52; CI: 1.29-1.79, p < .0001). Prior inpatient and case management services were also associated with increased odds for readmission. In the regression model, the intervention was not associated with decreased odds for readmission. Individuals with readmission (n = 796) were more likely to report being prescribed psychotropic medication and having housing difficulties and less likely to report having a recovery plan than those without readmission. IMPLICATIONS FOR CASE MANAGEMENT PRACTICE: Characteristics of Medicaid populations with hospitalization may contribute to readmission, which may be mitigated through care management intervention.

Increased likelihood of psychiatric readmission with Medicaid expansion vs legacy coverage.

OBJECTIVES: To compare patterns of psychiatric hospitalization and readmission within 30 days for Medicaid expansion (expansion) vs previously insured (legacy) samples. STUDY DESIGN: Retrospective analysis using Medicaid behavioral health service claims. METHODS: We identified 24,044 individuals with hospitalizations in calendar years 2017 and 2018 within the network of a behavioral health managed care organization in Pennsylvania. Logistic regression was used to examine factors associated with readmission. RESULTS: Individuals covered under expansion (n = 7747) vs legacy (n = 16,297) were older and more likely to be male and European American, with higher rates of cooccurring mental health (MH) and substance use disorder (SUD) diagnoses, as well as lower rates of MH and SUD services in the 30 days prior and any prior MH hospitalization. A higher proportion of individuals with expansion vs legacy status were readmitted (11.3% vs 9.0%; P < .0001). Controlling for factors associated with readmission, regression showed an increased likelihood of readmission for expansion vs legacy status (adjusted odds ratio [AOR], 1.23; 95% CI, 1.12-1.35; P < .0001). Increased risk for readmission was also found across populations for male patients (AOR, 1.12; 95% CI, 1.02-1.22; P = .0124), those with prior MH hospitalizations (AOR, 1.65; 95% CI, 1.51-1.81; P < .0001) or other behavioral health services (AOR, 1.14; 95% CI, 1.03-1.26; P = .0142), those with longer hospitalization episodes (AOR, 1.01; 95% CI, 1.00-1.01; P < .0001), and those with cooccurring SUD (AOR, 1.58; 95% CI, 1.44-1.74; P < .0001). CONCLUSIONS: Individuals with coverage through Medicaid expansion compared with legacy coverage have an increased risk of psychiatric readmission and may warrant targeted interventions that also address service utilization and cooccurring SUD.

Payer-Provider Partnership to Identify Successful Retention Strategies for the Behavioral Health Workforce.

BACKGROUND: High staff turnover rates are a burden for behavioral health providers because they may negatively impact staff morale, quality of care, and clinical outcomes as well as increase costs. The Staff Assessment and Retention (STAR) Project is a partnership between a behavioral health managed care organization and community-based providers designed to: 1) share information on research-based approaches, 2) identify strategies that providers find successful yet feasible, and 3) develop a learning community around research-based, community-informed strategies to increase staff retention. METHODS: Participants from 87 community-based behavioral health providers completed a survey about successful strategies and barriers to staff retention, current retention and turnover rates, and ratings of commonly used strategies supported by research. Results were shared and discussed across the partnership through a learning community including a webinar co-facilitated by two participating providers. RESULTS: Successfully demonstrated but less utilized strategies included use of exit and stay interviews, training in best practices, availability of electronic records and other technology, and flexible work schedules. Providers reported the type of employment offered (full or part time, benefits, competitive wage; reported in 17% of responses), ability to offer trainings and staff development (13%) and using staff feedback (10%) were most important to retention. CONCLUSIONS: The partnership demonstrated that providers desire a vehicle for sharing ideas and problem-solving issues related to the behavioral health workforce. The endorsement by several community-based providers of lower-utilized strategies known to improve retention would not be realized across the community without the collaboration of the payer-provider partnership.

Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study.

BACKGROUND: Examining the role of modifiable barriers and facilitators is a necessary step toward developing effective implementation strategies. This study examines whether both general (organizational culture, organizational climate, and transformational leadership) and strategic (implementation climate and implementation leadership) organizational-level factors predict therapist-level determinants of implementation (knowledge of and attitudes toward evidence-based practices). METHODS: Within the context of a system-wide effort to increase the use of evidence-based practices (EBPs) and recovery-oriented care, we conducted an observational, cross-sectional study of 19 child-serving agencies in the City of Philadelphia, including 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. Organizational variables included characteristics such as EBP initiative participation, program size, and proportion of independent contractor therapists; general factors such as organizational culture and climate (Organizational Social Context Measurement System) and transformational leadership (Multifactor Leadership Questionnaire); and strategic factors such as implementation climate (Implementation Climate Scale) and implementation leadership (Implementation Leadership Scale). Therapist-level variables included demographics, attitudes toward EBPs (Evidence-Based Practice Attitudes Scale), and knowledge of EBPs (Knowledge of Evidence-Based Services Questionnaire). We used linear mixed-effects regression models to estimate the associations between the predictor (organizational characteristics, general and strategic factors) and dependent (knowledge of and attitudes toward EBPs) variables. RESULTS: Several variables were associated with therapists' knowledge of EBPs. Clinicians in organizations with more proficient cultures or higher levels of transformational leadership (idealized influence) had greater knowledge of EBPs; conversely, clinicians in organizations with more resistant cultures, more functional organizational climates, and implementation climates characterized by higher levels of financial reward for EBPs had less knowledge of EBPs. A number of organizational factors were associated with the therapists' attitudes toward EBPs. For example, more engaged organizational cultures, implementation climates characterized by higher levels of educational support, and more proactive implementation leadership were all associated with more positive attitudes toward EBPs. CONCLUSIONS: This study provides evidence for the importance of both general and strategic organizational determinants as predictors of knowledge of and attitudes toward EBPs. The findings highlight the need for longitudinal and mixed-methods studies that examine the influence of organizational factors on implementation.

Quality of Diabetes Care Among Adult Medicaid Enrollees With Mental Disorders.

OBJECTIVE: In light of the national trend toward integrating mental and general medical care, this study examined disparities in diabetes treatment among Medicaid recipients in a nonintegrated, managed care behavioral health carve-out system. METHODS: A retrospective study of Medicaid claims (July 2009-June 2010) compared quality of diabetes treatment among 21,015 patients with and without mental disorders. RESULTS: Presence of a mental disorder was associated with higher use of outpatient and primary care services for diabetes, lower rates of hospitalizations for diabetes, and higher odds of receiving three or more quality measures for diabetes care. Patients with serious mental illness had better diabetes care compared with patients with other mental disorders and patients with no mental disorders. CONCLUSIONS: Findings suggest that managed care behavioral health carve-out systems should be considered among the range of approaches for improving treatment for diabetes among persons with comorbid mental disorders, particularly serious mental disorders.

The Perfect Storm: Collision of the Business of Mental Health and the Implementation of Evidence-Based Practices.

Financing has been hypothesized to be an important driver of the implementation of evidence-based practices (EBPs), yet there has been little systematic investigation of financing as a factor in EBP implementation. This column presents findings from a qualitative study of the effects of financial factors on the implementation of EBPs in a large urban publicly funded mental health system. Interviews with 33 agency leaders and 16 policy makers identified financial distress in community mental health agencies, leading to concerns about complex and expensive implementation of EBPs. Stakeholders agreed that the cost of EBP implementation should be shared between the agencies and the system; however, the stakeholders did not agree on how EBPs should be financed.

Therapist- and Client-Level Predictors of Use of Therapy Techniques During Implementation in a Large Public Mental Health System.

OBJECTIVE: Many youths receiving community mental health treatment do not receive evidence-based interventions. Research suggests that community mental health therapists use a broad range of therapeutic techniques at low intensities. This study examined the relationship between therapist- and client-level predictors of community-based therapists' report of cognitive, behavioral, psychodynamic, and family techniques within the context of implementation efforts. METHODS: A total of 130 therapists participated from 23 organizations in an urban, publicly funded behavioral health system implementing evidence-based practices. Therapist-level predictors included age, gender, clinical experience, licensure status, and participation in evidence-based practice initiatives. Child-level predictors included therapist-reported child primary disorder (externalizing, internalizing, or other) and child age. Therapists completed the Therapist Procedures Checklist-Family Revised, a self-report measure of therapeutic techniques used. RESULTS: Unlicensed therapists were more likely than licensed therapists to report using psychodynamic and behavioral techniques. Therapists who did not participate in an evidence-based practice initiative were less likely to report use of cognitive techniques. Those with clients with externalizing disorders were more likely to report use of behavioral and family techniques. Therapists with the youngest clients (ages three to seven years) were most likely to report use of behavioral techniques and less likely to report use of cognitive and psychodynamic techniques. CONCLUSIONS: Results suggest that both therapist and client factors predict self-reported use of therapy techniques. Participating in an evidence-based practice initiative was associated with increased reports of using cognitive techniques. Therapists reported using behavioral and family techniques more than other techniques when working with youths with externalizing disorders and using fewer cognitive and psychodynamic techniques with young clients.

Independent Contractors in Public Mental Health Clinics: Implications for Use of Evidence-Based Practices.

OBJECTIVES: Community mental health clinics are increasingly utilizing independent contractors to provide clinical services. At the same time, many organizations are participating in initiatives intended to increase implementation of evidence-based practices (EBPs). The primary aim of this study was to understand the associations of utilizing independent contractors with clinician knowledge and attitudes toward EBPs and organizational culture and climate. The study also sought to understand the potential impact of using independent contractors on mental health services delivery from the perspective of organizational leadership. METHODS: Quantitative data were collected from 130 therapists in 23 organizations; qualitative data were collected from executive administrators in nine of the 16 organizations participating in EBP initiatives sponsored by the City of Philadelphia. Regression with random effects was used to estimate the associations between worker status (contractor or employee) and clinician attitudes toward EBPs, knowledge of EBPs, and organizational culture and climate. Qualitative inquiry was used to understand the impact of reliance on independent contractors on organizational participation in EBP initiatives. RESULTS: Independent contractors endorsed less positive attitudes toward EBPs and scored lower on knowledge of EBPs. Interviews revealed four main themes: reasons for using independent contractors, general consequences of using independent contractors, specific impact of independent contractors on participation in EBP initiatives, and suggestions for alternatives. CONCLUSIONS: A growing number of community mental health clinics rely on independent contractors. There may be consequences of this shift that deserve exploration.

Predictors of community therapists' use of therapy techniques in a large public mental health system.

IMPORTANCE: Few studies have examined the effects of individual and organizational characteristics on the use of evidence-based practices in mental health care. Improved understanding of these factors could guide future implementation efforts to ensure effective adoption, implementation, and sustainment of evidence-based practices. OBJECTIVE: To estimate the relative contribution of individual and organizational factors on therapist self-reported use of cognitive-behavioral, family, and psychodynamic therapy techniques within the context of a large-scale effort to increase use of evidence-based practices in an urban public mental health system serving youth and families. DESIGN, SETTING, AND PARTICIPANTS: In this observational, cross-sectional study of 23 organizations, data were collected from March 1 through July 25, 2013. We used purposive sampling to recruit the 29 largest child-serving agencies, which together serve approximately 80% of youth receiving publically funded mental health care. The final sample included 19 agencies with 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. MAIN OUTCOMES AND MEASURES: Therapist self-reported use of cognitive-behavioral, family, and psychodynamic therapy techniques, as measured by the Therapist Procedures Checklist-Family Revised. RESULTS: Individual factors accounted for the following percentages of the overall variation: cognitive-behavioral therapy techniques, 16%; family therapy techniques, 7%; and psychodynamic therapy techniques, 20%. Organizational factors accounted for the following percentages of the overall variation: cognitive-behavioral therapy techniques, 23%; family therapy techniques, 19%; and psychodynamic therapy techniques, 7%. Older therapists and therapists with more open attitudes were more likely to endorse use of cognitive-behavioral therapy techniques, as were those in organizations that had spent fewer years participating in evidence-based practice initiatives, had more resistant cultures, and had more functional climates. Women were more likely to endorse use of family therapy techniques, as were those in organizations employing more fee-for-service staff and with more stressful climates. Therapists with more divergent attitudes and less knowledge about evidence-based practices were more likely to use psychodynamic therapy techniques. CONCLUSIONS AND RELEVANCE: This study suggests that individual and organizational factors are important in explaining therapist behavior and use of evidence-based practices, but the relative importance varies by therapeutic technique.

A complementary marriage of perspectives: understanding organizational social context using mixed methods.

BACKGROUND: Organizational factors impact the delivery of mental health services in community settings. Mixed-methods analytic approaches have been recommended, though little research within implementation science has explicitly compared inductive and deductive perspectives to understand their relative value in understanding the same constructs. The purpose of our study is to use two different paradigmatic approaches to deepen our understanding of organizational social context. We accomplish this by using a mixed-methods approach in an investigation of organizational social context in community mental health clinics. METHODS: Nineteen agencies, representing 23 sites, participated. Enrolled participants included 130 therapists, 36 supervisors, and 22 executive administrators. Quantitative data was obtained via the Organizational Social Context (OSC) measure. Qualitative data, comprised of direct observation with spot sampling generated from agency visits, was coded using content analysis and grounded theory. The present study examined elements of organizational social context that would have been missed if only quantitative data had been obtained and utilized mixed methods to investigate if stratifying observations based on quantitative ratings from the OSC resulted in the emergence of differential themes. RESULTS: Four of the six OSC constructs were commonly observed in field observations (i.e., proficiency, rigidity, functionality, stress), while the remaining two constructs were not frequently observed (i.e., resistance, engagement). Constructs emerged related to organizational social context that may have been missed if only quantitative measurement was employed, including those around the physical environment, commentary about evidence-based practice initiatives, leadership, cultural diversity, distrust, and affect. Stratifying agencies by "best," "average," and "worst" organizational social context impacted interpretation for three constructs (affect, stress, and leadership). CONCLUSIONS: Results support the additive value of integrating inductive and deductive perspectives in implementation science research. This synthesis of approaches facilitated a more comprehensive understanding and interpretation of the findings than would have been possible if either methodology had been employed in isolation.