Pilot evaluation of the Fiscal Mapping Process for sustainable financing of evidence-based youth mental health treatments: A comparative case study analysis.

Background: Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies. Supports are needed that can guide service agencies in accessing sustainable funding for EBTs. We conducted a pilot evaluation of the Fiscal Mapping Process, an Excel-based strategic planning tool that helps service agency leaders identify and coordinate financing strategies for their EBT programs. Method: Pilot testing of the Fiscal Mapping Process was completed with 10 youth mental health service agencies over a 12-month period, using trauma-focused cognitive-behavioral therapy or parent-child interaction therapy programs. Service agency representatives received initial training and monthly coaching in using the tool. We used case study methods to synthesize all available data (surveys, focus groups, coaching notes, document review) and contrast agency experiences to identify key findings through explanation building. Results: Key evaluation findings related to the process and outcomes of using the Fiscal Mapping Process, as well as contextual influences. Process evaluation findings helped clarify the primary use case for the tool and identified the importance-and challenges-of engaging external collaborators. Outcome evaluation findings documented the impacts of the Fiscal Mapping Process on agency-reported sustainment capacities (strategic planning, funding stability), which fully explained reported improvements in outcomes (extent and likelihood)-although these impacts were incremental. Findings on contextual factors documented the influence of environmental and organizational capacities on engagement with the tool and concerns about equitable impacts, but also the view that the process could usefully generalize to other EBTs. Conclusions: Our pilot evaluation of the Fiscal Mapping Process was promising. In future work, we plan to integrate the tool into EBT implementation initiatives and test its impact on long-term sustainment outcomes across various EBTs, while increasing attention to equity considerations.

Pilot-Testing a Tool for Planning the Sustainable Financing of Youth Mental Health Treatments that Work Plain Language Summary Youth mental health treatments that work must be consistently available to improve youth mental health in our communities, but funding for these treatments is often limited and hard to access. Youth mental health service agencies need tools that can help guide them in accessing sustainable funding for evidence-based treatments. We developed the Fiscal Mapping Process, an Excel-based strategic planning tool for planning sustainable financing of youth mental health treatment programs, and conducted a 1-year pilot-testing evaluation with 10 youth mental health service agencies. We used case study methods to compare and contrast agency experiences with using the tool, related to the process, outcomes, and contextual influences on using the Fiscal Mapping Process. Key findings included clarification of the ideal characteristics of contributors and treatment programs for using the tool; initial confirmation that the tool can improve agency-reported capacities for sustaining treatments that work and long-term sustainment outlooks, although these impacts were incremental; and documentation of the influence of environmental and organizational capacities on engagement with the tool, concerns about equitable impacts, and user views that the process could be applied to a wide range of treatment models. In summary, our pilot evaluation of the Fiscal Mapping Process showed that this tool is promising for supporting the financial sustainment of treatments that work in youth mental health services. In future research, we plan to incorporate the tool into real-world training initiatives with mental health service agencies, test its impact on long-term sustainment across a variety of treatment models, and incorporate attention to equity considerations.

Lessons From an Implementation Evaluation of a Real-World Multi-City Initiative to Address COVID-19 Vaccination Inequities.

Background: The COVID-19 pandemic shed light on stark racial and ethnic inequities in access to care and accurate health information in the U.S. When COVID-19 vaccines became available, communities of color faced multiple barriers that contributed to low vaccine rates. To address this gap, the Equity-First Vaccination Initiative supported community organizations in five demonstration cities to plan and implement hyper-local strategies to increase COVID-19 vaccine access and uptake among communities of color.Purpose: To draw learnings from the experiences of the participating organizations, we applied a framework that integrated implementation science and health equity principles.Design and sample: In this commentary, we describe how we used this framework to guide qualitative interviews with community organizations, focusing on insights across five implementation elements (reach, design, implementation, adaptation, implementation outcomes).Conclusions: Learnings from this evaluation may help guide future implementation of similarly complex initiatives involving multiple organizations and sites to advance health equity during a public health crisis.

Comparing organization-focused and state-focused financing strategies on provider-level reach of a youth substance use treatment model: a mixed-method study.

BACKGROUND: Financial barriers in substance use disorder service systems have limited the widespread adoption-i.e., provider-level reach-of evidence-based practices (EBPs) for youth substance use disorders. Reach is essential to maximizing the population-level impact of EBPs. One promising, but rarely studied, type of implementation strategy for overcoming barriers to EBP reach is financing strategies, which direct financial resources in various ways to support implementation. We evaluated financing strategies for the Adolescent Community Reinforcement Approach (A-CRA) EBP by comparing two US federal grant mechanisms, organization-focused and state-focused grants, on organization-level A-CRA reach outcomes. METHOD: A-CRA implementation took place through organization-focused and state-focused grantee cohorts from 2006 to 2021. We used a quasi-experimental, mixed-method design to compare reach between treatment organizations funded by organization-focused versus state-focused grants (164 organizations, 35 states). Using administrative training records, we calculated reach as the per-organization proportion of trained individuals who received certification in A-CRA clinical delivery and/or supervision by the end of grant funding. We tested differences in certification rate by grant type using multivariable linear regression models that controlled for key covariates (e.g., time), and tested threats to internal validity from our quasi-experimental design through a series of sensitivity analyses. We also drew on interviews and surveys collected from the treatment organizations and (when relevant) interviews with state administrators to identify factors that influenced reach. RESULTS: The overall certification rates were 27 percentage points lower in state-focused versus organization-focused grants (p = .01). Sensitivity analyses suggested these findings were not explained by confounding temporal trends nor by organizational or state characteristics. We did not identify significant quantitative moderators of reach outcomes, but qualitative findings suggested certain facilitating factors were more influential for organization-focused grants (e.g., strategic planning) and certain barrier factors were more impactful for state-focused grants (e.g., states finding it difficult to execute grant activities). DISCUSSION: As the first published comparison of EBP reach outcomes between financing strategies, our findings can help guide state and federal policy related to financing strategies for implementing EBPs that reduce youth substance use. Future work should explore contextual conditions under which different financing strategies can support the widespread implementation of EBPs for substance use disorder treatment.

Perspectives on Financing Strategies for Evidence-Based Treatment Implementation in Youth Mental Health Systems.

BACKGROUND: Evidence-based treatments (EBTs) are critical to effectively address mental health problems among children and adolescents, but costly for mental health service agencies to implement and sustain. Financing strategies help agencies overcome cost-related barriers by obtaining financial resources to support EBT implementation and/or sustainment. AIMS: We sought to (i) understand how youth mental health system decision-makers involved with EBT implementation and sustainment view key features (e.g., relevance, feasibility) that inform financing strategy selection and (ii) compare service agency, funding agency, and intermediary representative perspectives. METHOD: Two surveys were disseminated to 48 representatives across U.S. youth mental health service agencies, funding agencies, and intermediaries who were participating in a larger study of financing strategies. Quantitative and qualitative data were gathered on 23 financing strategies through quantitative ratings and open-ended responses. Data were analyzed using descriptive statistics and rapid content analysis. RESULTS: The financing strategies rated as most relevant include braided funding streams, contracts for EBTs, credentialing/rostering providers, fee-for-service reimbursement (regular and increased), and grant funding. All other strategies were unfamiliar to 1/3 to 1/2 of participants. The six strategies were rated between somewhat and quite available, feasible, and effective for EBT sustainment. For sustaining different EBT components (e.g., delivery, materials), the mix of financing strategies was rated as somewhat adequate. Qualitative analysis revealed challenges with strategies being non-recurring or unavailable in representatives' regions. Ratings were largely similar across participant roles, though funding agency representatives were the most familiar with financing strategies. DISCUSSION: Despite the breadth of innovative financing strategies, expert representatives within the youth mental health services ecosystem had limited knowledge of most options. Experts relied on strategies that were familiar but often did not adequately support EBT implementation or sustainment. These findings underscore more fundamental issues with under-resourced mental health systems in the U.S.; financing strategies can help agencies navigate EBT use but must be accompanied by larger-scale system reforms. Limitations include difficulties generalizing results due to using a small sample familiar with EBTs, high agreement as a potential function of snowball recruiting, and limited responses to the open-ended survey questions. IMPLICATIONS FOR HEALTH CARE PROVISION AND USE: Although EBTs have been found to effectively address mental health problems in children and adolescents, available strategies for financing their implementation and sustainment in mental health systems are insufficient. This constraint prevents many children and adolescents from receiving high-quality services. IMPLICATIONS FOR HEALTH POLICIES: Financing strategies alone cannot solve systematic issues that prevent youth mental health service agencies from providing EBTs. Policy changes may be required, such as increased financial investment from the U.S. government into mental health services to support basic infrastructure (e.g., facility operations, measuring outcomes). IMPLICATIONS FOR FURTHER RESEARCH: Future work should examine expert perspectives on EBT financing strategies in different contexts (e.g., substance use services), gathering targeted feedback on financing strategies that are less well known, and exploring topics such as strategic planning, funding stability, and collaborative decision-making as they relate to EBT implementation and sustainment.

Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study.

BACKGROUND: Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures. METHOD: We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment. DISCUSSION: The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.

Coordination of sustainable financing for evidence-based youth mental health treatments: protocol for development and evaluation of the fiscal mapping process.

BACKGROUND: Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health and economic impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies (hereafter, "service agencies"). Strategic planning tools are needed that can guide these service agencies in their coordination of sustainable funding for EBTs. This protocol describes a mixed-methods research project designed to (1) develop and (2) evaluate our novel fiscal mapping process that guides strategic planning efforts to finance the sustainment of EBTs in youth mental health services. METHOD: Participants will be 48 expert stakeholder participants, including representatives from ten service agencies and their partners from funding agencies (various public and private sources) and intermediary organizations (which provide guidance and support on the delivery of specific EBTs). Aim 1 is to develop the fiscal mapping process: a multi-step, structured tool that guides service agencies in selecting the optimal combination of strategies for financing their EBT sustainment efforts. We will adapt the fiscal mapping process from an established intervention mapping process and will incorporate an existing compilation of 23 financing strategies. We will then engage participants in a modified Delphi exercise to achieve consensus on the fiscal mapping process steps and gather information that can inform the selection of strategies. Aim 2 is to evaluate preliminary impacts of the fiscal mapping process on service agencies' EBT sustainment capacities (i.e., structures and processes that support sustainment) and outcomes (e.g., intentions to sustain). The ten agencies will pilot test the fiscal mapping process. We will evaluate how the fiscal mapping process impacts EBT sustainment capacities and outcomes using a comparative case study approach, incorporating data from focus groups and document review. After pilot testing, the stakeholder participants will conceptualize the process and outcomes of fiscal mapping in a participatory modeling exercise to help inform future use and evaluation of the tool. DISCUSSION: This project will generate the fiscal mapping process, which will facilitate the coordination of an array of financing strategies to sustain EBTs in community youth mental health services. This tool will promote the sustainment of youth-focused EBTs.

Value of family involvement in substance use disorder treatment: Aligning clinical and financing priorities.

BACKGROUND: Family members' support (e.g., informational, tangible, emotional) has important and lasting impacts on individuals' recovery from substance use disorders (SUDs). Unfortunately, SUD services in the United States do not consistently incorporate patients' family members effectively. One barrier to family involvement in SUD services is the mechanisms through which SUD services in the United States are commonly financed. METHOD: Using our recent experiences with developing a group intervention for support persons of patients with opioid use disorder, we illustrate how gaps in feasible financing models limit SUD service systems from effectively implementing and sustaining family services for individuals receiving SUD treatment. DISCUSSION: Long-term availability of family-inclusive interventions will require collaboration with payors and health systems. We offer two sets of recommendations for funding family involvement in SUD services; one set of immediately implementable recommendations and other longer-term goals requiring structural changes in SUD service delivery and financing.

Moving beyond Aim Three: a need for a transdisciplinary approach to build capacity for economic evaluations in implementation science.

BACKGROUND: Understanding the costs and economic benefits of implementation has been identified by policymakers and researchers as critical to increase the uptake and sustainment of evidence-based practices, but this topic remains relatively understudied. Conducting team science with health economists has been proposed as a solution to increase economic evaluation in implementation science; however, these recommendations ignore the differences in goals and perspectives in these two fields. Our recent qualitative research identified that implementation researchers predominantly approach health economists to examine costs, whereas the majority of health economists expressed limited interest in conducting economic evaluations and a desire to be more integrated within implementation science initiatives. These interviews pointed to challenges in establishing fruitful partnerships when health economists are relegated to the "Third Aim" (i.e., lowest-priority research objective) in implementation science projects by their research partners. DISCUSSION: In this debate paper, we argue that implementation researchers and health economists need to focus on team science research principles to expand capacity to address pressing research questions that cut across the two fields. Specifically, we use the four-phase model of transdisciplinary research to outline the goals and processes needed to build capacity in this area (Hall et al., Transl Behav Med 2:415-30, 2012). The first phase focuses on the development of transdisciplinary research teams, including identifying appropriate partners (e.g., considering policy or public health researchers in addition to health economists) and building trust. The conceptual phase focuses on strategies to consider when developing joint research questions and methodology across fields. In the implementation phase, we outline the effective processes for conducting research projects, such as team learning. Finally, in the translation phase, we highlight how a transdisciplinary approach between health economists and implementation researchers can impact real-world practice and policy. The importance of investigating the economic impact of evidence-based practice implementation is widely recognized, but efforts have been limited due to the challenges in conducting team science across disciplines. Training in team science can help advance transdisciplinary efforts, which has the potential to increase the rigor and impact of economic evaluations in implementation science while expanding the roles taken by health economists.

Translating economic evaluations into financing strategies for implementing evidence-based practices.

BACKGROUND: Implementation researchers are increasingly using economic evaluation to explore the benefits produced by implementing evidence-based practices (EBPs) in healthcare settings. However, the findings of typical economic evaluations (e.g., based on clinical trials) are rarely sufficient to inform decisions about how health service organizations and policymakers should finance investments in EBPs. This paper describes how economic evaluations can be translated into policy and practice through complementary research on financing strategies that support EBP implementation and sustainment. MAIN BODY: We provide an overview of EBP implementation financing, which outlines key financing and health service delivery system stakeholders and their points of decision-making. We then illustrate how economic evaluations have informed decisions about EBP implementation and sustainment with three case examples: (1) use of Pay-for-Success financing to implement multisystemic therapy in underserved areas of Colorado, USA, based in part on the strength of evidence from economic evaluations; (2) an alternative payment model to sustain evidence-based oncology care, developed by the US Centers for Medicare and Medicaid Services through simulations of economic impact; and (3) use of a recently developed fiscal mapping process to collaboratively match financing strategies and needs during a pragmatic clinical trial for a newly adapted family support intervention for opioid use disorder. CONCLUSIONS: EBP financing strategies can help overcome cost-related barriers to implementing and sustaining EBPs by translating economic evaluation results into policy and practice. We present a research agenda to advance understanding of financing strategies in five key areas raised by our case examples: (1) maximize the relevance of economic evaluations for real-world EBP implementation; (2) study ongoing changes in financing systems as part of economic evaluations; (3) identify the conditions under which a given financing strategy is most beneficial; (4) explore the use and impacts of financing strategies across pre-implementation, active implementation, and sustainment phases; and (5) advance research efforts through strong partnerships with stakeholder groups while attending to issues of power imbalance and transparency. Attention to these research areas will develop a robust body of scholarship around EBP financing strategies and, ultimately, enable greater public health impacts of EBPs.

Collaborating with health economists to advance implementation science: a qualitative study.

BACKGROUND: Implementation research infrequently addresses economic factors, despite the importance of understanding the costs of implementing evidence-based practices (EBPs). Though partnerships with health economists have the potential to increase attention to economic factors within implementation science, barriers to forming these collaborations have been noted. This study investigated the experiences of health economists and implementation researchers who have partnered across disciplines to inform strategies to increase such collaborations. METHODS: A purposeful sampling approach was used to identify eight health economists and eight implementation researchers with experience participating in cross-disciplinary research. We used semi-structured interviews to gather information about participants' experiences with collaborative research. Thematic analysis was conducted to identify core themes related to facilitators and barriers to collaborations. RESULTS: Health economists and implementation researchers voiced different perspectives on collaborative research, highlighting the importance of increasing cross-disciplinary understanding. Implementation researchers described a need to measure costs in implementation studies, whereas many health economists described that they seek to collaborate on projects that extend beyond conducting cost analyses. Researchers in both disciplines articulated motivations for collaborative research and identified strategies that promote successful collaboration, with varying degrees of convergence across these themes. Shared motivations included improving methodological rigor of research and making a real-world impact. Strategies to improve collaboration included starting partnerships early in the study design period, having a shared interest, and including health economists in the larger scope of the research. CONCLUSIONS: Health economists and implementation researchers both conduct research with significant policy implications and have the potential to inform one another's work in ways that might more rapidly advance the uptake of EBPs. Collaborative research between health economists and implementation science has the potential to advance the field; however, researchers will need to work to bridge disciplinary differences. By beginning to develop strong working relationships; increasing their understanding of one another's disciplinary culture, methodology, and language; and increasing the role economists have within research design and execution, both implementation researchers and health economists can support successful collaborations and robust and informative research.

Estimating Downstream Budget Impacts in Implementation Research.

Health care decision makers often request information showing how a new treatment or intervention will affect their budget (i.e., a budget impact analysis; BIA). In this article, we present key topics for considering how to measure downstream health care costs, a key component of the BIA, when implementing an evidence-based program designed to reduce a quality gap. Tracking health care utilization can be done with administrative or self-reported data, but estimating costs for these utilization data raises 2 issues that are often overlooked in implementation science. The first issue has to do with applicability: are the cost estimates applicable to the health care system that is implementing the quality improvement program? We often use national cost estimates or average payments, without considering whether these cost estimates are appropriate. Second, we need to determine the decision maker's time horizon to identify the costs that vary in that time horizon. If the BIA takes a short-term time horizon, then we should focus on costs that vary in the short run and exclude costs that are fixed over this time. BIA is an increasingly popular tool for health care decision makers interested in understanding the financial effect of implementing an evidence-based program. Without careful consideration of some key conceptual issues, we run the risk of misleading decision makers when presenting results from implementation studies.

Designing the Future of Children's Mental Health Services.

Advancements in evidence-based psychosocial interventions, digital technologies, and implementation strategies (i.e., health services research products) for youth mental health services have yet to yield significant improvement in public health outcomes. Achieving such impact will require that these research products are easy to use, useful, and contextually appropriate. This paper describes how human-centered design (HCD), an approach that aligns product development with the needs of the people and settings that use those products, can be leveraged to improve youth mental health services. We articulate how HCD can advance accessibility, effectiveness, and equity, with specific consideration of unique aspects of youth mental health services.

A scoping review of strategies for financing the implementation of evidence-based practices in behavioral health systems: State of the literature and future directions.

Background: Increased availability of evidence-based practices (EBPs) is essential to alleviating the negative public health and societal effects of behavioral health problems. A major challenge to implementing and sustaining EBPs broadly is the limited and fragmented nature of available funding. Method: We conducted a scoping review that assessed the current state of evidence on EBP financing strategies for behavioral health based on recent literature (i.e., post-Affordable Care Act). We defined financing strategies as techniques that secure and direct financial resources to support EBP implementation. This article introduces a conceptualization of financing strategies and then presents a compilation of identified strategies, following established reporting guidelines for the implementation strategies. We also describe the reported level of use for each financing strategy in the research literature. Results: Of 23 financing strategies, 13 were reported as being used within behavioral health services, 4 had potential for use, 5 had conceptual use only, and 1 was potentially contraindicated. Examples of strategies reported being used include increased fee-for-service reimbursement, grants, cost sharing, and pay-for-success contracts. No strategies had been evaluated in ways that allowed for strong conclusions about their impact on EBP implementation outcomes. Conclusion: The existing literature on EBP financing strategies in behavioral health raises far more questions than answers. Therefore, we propose a research agenda that will help better understand these financing strategies. We also discuss the implications of our findings for behavioral health professionals, system leaders, and policymakers who want to develop robust, sustainable financing for EBP implementation in behavioral health systems. Plain language abstract: Organizations that treat behavioral health problems (mental health and substance use) often seek to adopt and use evidence-based practices (EBPs). A challenge to adopting EBPs broadly is the limited funding available, often from various sources that are poorly coordinated with one another. To help organizations plan effectively to adopt EBPs, we conducted a review of recent evidence (i.e., since the passage of the 2010 Affordable Care Act) on strategies for financing EBP adoption in behavioral health systems. We present definitions of 23 identified strategies and describe each strategy's reported (in the research literature) level of use to fund EBP adoption in behavioral health services. Of the 23 financing strategies, 13 strategies had evidence of use, 4 had potential for use, 5 had conceptual use only, and 1 was potentially contraindicated. Examples of strategies with evidence of use include increased fee-for-service reimbursement, grants, cost sharing, and pay-for-success contracts. This comprehensive list of EBP financing strategies may help guide decision-making by behavioral health professionals, system leaders, and policymakers. The article also presents a research agenda for building on the current research literature by (1) advancing methods to evaluate financing strategies' effects, (2) partnering with stakeholders and decision-makers to examine promising financing strategies, (3) focusing on strategies and service systems with the greatest needs, (4) improving methods to guide the selection of financing strategies, and (5) paying greater attention to sustainable long-term financing of EBPs.

Qualitative Analysis of Administrator, Provider, and Stakeholder Views on the Costs and Benefits of a Treatment for Problematic Sexual Behavior of Youth.

Evidence-based treatment for youth with problematic sexual behavior (PSB) has the potential for a broad range of costs and benefits, raising the importance of querying its public health impact. This qualitative study examined the impact of treatment for youth with PSB through content analysis of interviews (N = 57) with service agency administrators, treatment providers, and professional stakeholders in communities where recent implementation of interventions for youth with PSB had occurred. Interviewees emphasized multi-level impacts of the program on families (e.g., knowledge, well-being), communities (e.g., public safety, education), and public agencies (e.g., caseloads, stress). Implications for comprehensively evaluating the impact of PSB interventions are discussed.

Economic evaluation in implementation science: Making the business case for implementation strategies.

Implementation researchers have made notable progress in developing and testing implementation strategies (i.e., highly-specified methods used to help providers improve uptake of mental health evidence-based practices: EBPs). Yet, implementation strategies are not widely applied in healthcare organizations to improve delivery of EBPs. Economic considerations are a key factor influencing the use of implementation strategies to deliver and sustain mental health evidence-based practices, in part because many health care leaders are reluctant to invest in ongoing implementation strategy support without knowing the return-on-investment. Comparative economic evaluation of implementation strategies provides critical information for payers, policymakers, and providers to make informed decisions if specific strategies are an efficient use of scarce organizational resources. Currently, few implementation studies include implementation cost data and even fewer conduct comparative economic analyses of implementation strategies. This summary will introduce clinicians, researchers and other health professionals to the economic evaluation in implementation science. We provide an overview of different economic evaluation methods, discuss differences between economic evaluation in health services and implementation science. We also highlight approaches and frameworks to guide economic evaluation of implementation, provide an example for a cognitive-behavioral therapy program and discuss recommendations.

Economic value of community-based services for problematic sexual behaviors in youth: A mixed-method cost-effectiveness analysis.

BACKGROUND: Problematic sexual behavior in youth represents a significant public health problem in need of evidence-based treatments. Unfortunately, such treatments are not available in most communities. OBJECTIVE: This study used a mixed quantitative-qualitative approach to investigate the economics of the implementation of Problematic Sexual Behavior - Cognitive-Behavioral Therapy (PSB-CBT), an evidence-based treatment for problem sexual behaviors in youth. PARTICIPANTS AND SETTING: Youth (N = 413) participated in PSB-CBT at six program sites in youth service agencies across the United States. METHOD: We used cost-effectiveness ratios (CERs) to compare the direct and indirect costs of PSB-CBT to self- and caregiver-reported youth clinical outcomes (i.e., problem sexual behavior as well as secondary behavioral health problems). CERs represented the cost of achieving one standard unit of change on a measure (i.e., d = 1.0). The design and interpretation of those quantitative analyses were informed by qualitative themes about program costs and benefits that were derived from interviews with 59 therapists, administrators, and stakeholders. RESULTS: CERs (i.e., $ per SD) were $1,772 per youth for problem sexual behavior and ranged from $2,867 to $4,899 per youth for secondary outcomes. These quantitative results, considered alongside the qualitative perspectives of interviewees, suggested that the implementation of PSB-CBT was cost-effective. The results were robust to uncertainty in key parameters under most, but not all, conditions. CONCLUSIONS: The results have important implications for decisions made by administrators, policymakers, and therapists regarding use of community-based approaches to address problematic sexual behavior of youth.

Moving Upstream to Improve Children's Mental Health Through Community and Policy Change.

Using a social determinants of health framework, we argue that the majority of evidence-based interventions focused on child and adolescent mental health are limited by their focus on individual youth (and sometimes families). While necessary, these interventions are insufficient for addressing the midstream- and upstream/macro-level determinants of mental health in society. We illustrate our perspective through four examples from youth mental health and related services, in which midstream and upstream interventions-i.e., at the community and public policy levels-need to be prioritized along with downstream treatments to improve population mental health and reduce social inequalities in mental health outcomes.

The Potential of Pay-for-Success as a Financing Strategy for Evidence-Based Practices: An Illustration with Multisystemic Therapy.

There is a critical need to identify strategies for financing the implementation of evidence-based practices. We illustrate the potential of pay-for-success financing (PFS)-a strategy in which private investors fund implementation and receive a return on investment from a government payer-using multisystemic therapy as an example. We argue that standard multisystemic therapy (for serious juvenile offenders) and several of its adaptations (for other complex behavioral problems in youth) would be good candidates for PFS in the right contexts. Despite some challenges for policymakers and administrators, PFS has significant potential as a financing strategy for evidence-based practices.

Mixed-method approaches to strengthen economic evaluations in implementation research.

BACKGROUND: Guidance from economic evaluations on which implementation strategies represent the best return on investment will be critical to advancing the Triple Aim of health care: improving patient care and population health while minimizing per-capita cost. The results of traditional (quantitative) economic evaluations are limited by a remaining "qualitative residual" of contextual information and stakeholders perspectives, which cannot be captured by monetary values alone and is particularly prevalent in implementation science research. The emergence of qualitative methods for economic evaluation offers a promising solution. MAIN BODY: To maximize the contributions of economic evaluations to implementation science, we recommend that researchers embrace a mixed-methods research agenda that merges traditional quantitative approaches with innovative, contextually grounded qualitative methods. Such studies are exceedingly rare at present. To assist implementation scientists in making use of mixed methods in this research context, we present an adapted taxonomy of mixed-method studies relevant to economic evaluation. We then illustrate the application of mixed methods in a recently completed cost-effectiveness evaluation, making use of an adapted version of reporting standards for economic evaluations. CONCLUSIONS: By incorporating qualitative methods, implementation researchers can enrich their economic evaluations with detailed, context-specific information that tells the full story of the costs and impacts of implementation. We end by providing suggestions for building a research agenda in mixed-method economic evaluation, along with more resources and training to support investigators who wish to answer our call to action.

Economic Impact of the Statewide Implementation of an Evidence-Based Treatment: Multisystemic Therapy in New Mexico.

Several states have made considerable investments into large-scale implementation of evidence-based treatments (EBTs), yet little is known about key success indicators for these implementation efforts such as cost and sustainability. To that end, the present study examined the economic impact of statewide implementation of multisystemic therapy (MST; Henggeler, Schoenwald, Borduin, Rowland, & Cuningham, 2009), a family- and community-based behavioral EBT for serious juvenile offenders in New Mexico. Participants were 1,869 youth who received MST across 23 teams during the study period. We conducted a cost-benefit analysis using metrics from state data sources that compared the cost of MST to its benefits (i.e., avoided expenses from pre- to posttreatment) in two domains: (a) behavioral health services (i.e., Medicaid claims) and (b) juvenile crime (i.e., taxpayer expenses, tangible and intangible expenses to crime victims). MST costs were based on Medicaid claims, which were reimbursed at an enhanced billing rate that was intended to cover expenses for both clinical and implementation (e.g., training, quality assurance) activities. Results suggest that implementation of MST in New Mexico over the 7-year study period may have produced net benefits, through 2 years posttreatment, of more than $4,643 per youth in avoided behavioral health claims and $15,019 per youth through reductions in juvenile crime. Stated differently, every dollar that New Mexico spent on MST appeared to have returned $3.34 for a total benefit of $64.2 million over the course of the study. We discuss implications of these findings for policymakers, administrators, and researchers who are interested in increasing the sustainability of complex EBTs in community settings.