Evaluation Design for the Department of the Air Force's True North Program: Development of a Logic Model and Program Measures.

Developing strong resiliency and care solutions for airmen and guardians is key to human capital development and force readiness. The True North program is one of the Department of the Air Force's (DAF's) most significant recent investments in promoting the resiliency of its people. Assessing the program's level of success, justifying funding, and informing decisions about the program's future will require a rigorous evaluation. The authors of this study (1) identify desired outcomes for members participating in the program, (2) define appropriate measures of effectiveness that could be used in evaluating the True North program, and (3) make recommendations for ongoing internal evaluation of the program. The True North program encompasses selected installation welcome centers, embedded religious support teams (RSTs), and embedded mental health (EMH) teams. To determine how DAF might evaluate this program and its components, the authors reviewed relevant policies and procedures and literature relevant to the program components and conducted interviews with 17 True North program personnel and 21 group and squadron commanders. They present a program logic model to determine potential evaluation measures.

Multi-level alignment processes in the sustainment of a youth substance use treatment model following a federal implementation initiative: A mixed method study.

INTRODUCTION: Government agencies have identified evidence-based practice (EBP) dissemination as a pathway to high-quality behavioral health care for youth. However, gaps remain about how to best sustain EBPs in treatment organizations in the U.S., especially in resource-constrained settings like publicly-funded youth substance use services. One important, but understudied, determinant of EBP sustainment is alignment: the extent to which multi-level factors that influence sustainment processes and outcomes are congruent, consistent, and/or coordinated. This study examined the role of alignment in U.S. states' efforts to sustain the Adolescent Community Reinforcement Approach (A-CRA), an EBP for youth substance use disorders, during the COVID-19 pandemic. METHODS: In this mixed methods study, the qualitative investigation preceded and informed the quantitative investigation. We interviewed state administrators and providers (i.e., supervisors and clinicians) from 15 states that had completed a federal A-CRA implementation grant; providers also completed surveys. The sample included 50 providers from 35 treatment organizations that reported sustaining A-CRA when the COVID-19 pandemic began, and 20 state administrators. In qualitative thematic analyses, we applied the EPIS (Exploration, Preparation, Implementation, Sustainment) framework to characterize alignment processes that interviewees described as influential on sustainment. We then used survey items to quantitatively explore the associations described in qualitative themes, using bivariate linear regressions. RESULTS: At the time of interview, staff from 80 % of the treatment organizations (n = 28), reported sustaining A-CRA. Providers from both sustainer and non-sustainer organizations, as well as state administrators, described major sources of misalignment when state agencies ceased technical assistance post-grant, and because limited staff capacity conflicted with A-CRA's training model, which was perceived as time-intensive. Participants described the pandemic as exacerbating preexisting challenges, including capacity issues. Sustainer organizations reported seeking new funding to help sustain A-CRA. Quantitative associations between self-rated extent of sustainment and other survey items largely followed the pattern predicted from the qualitative findings. CONCLUSIONS: The COVID-19 pandemic amplified longstanding A-CRA sustainment challenges, but treatment organizations already successfully sustaining A-CRA pre-pandemic largely continued. There are missed opportunities for state-level actors to coordinate with providers on the shared goal of EBP sustainment. A greater focus on alignment processes in research and practice could help states and providers strengthen sustainability planning.

How the Dobbs Decision Could Affect U.S. National Security.

Women are an integral part of the military, comprising 17.2 percent of the active-duty force. They are the fastest-growing subpopulation in the military. In recent years, the Department of Defense (DoD) and military services have been deliberately recruiting women because they represent a higher percentage of the recruitable population than their male counterparts. Service women and their civilian counterparts have been and are serving in roles that are essential to military readiness. The Dobbs v. Jackson Supreme Court ruling will limit service women's and DoD civilian women's access to reproductive health care and affect the health of these critical populations. In this article, the authors use publicly available data to provide an estimate of the scope of the effects of the decision on the health and readiness of the U.S. armed forces. They estimate how many military-employed women's reproductive health options have been or might soon be limited and identify force readiness-related concerns, such as effects on the military health care, education, and child care systems, as well as on military recruiting and retention.

An Evaluation of a Multisite, Health Systems-Based Direct Care Worker Retention Program: Key Findings and Recommendations.

The U.S. direct care workforce employs nearly 4.6 million people and represents one of the fastest growing occupations in the United States. Direct care workers, or "caregivers," include nursing assistants, home care workers, and residential care aides, all of whom provide basic care to older adults and individuals with disabilities in various health care settings. Despite a growing need for caregivers, supply has not kept up with demand due to high turnover and low wages. In addition, caregivers often face high levels of workplace stress, limited training and growth opportunities, and personal stressors. Ranging from 35 to 90 percent, depending on the health care setting, the turnover rates of direct care workers pose a major challenge for health systems, as well as care recipients and workers themselves. In 2019, the Ralph C. Wilson Jr. Foundation funded three health systems to support the implementation of a new program: Transformational Healthcare Readiness through Innovative Vocational Education (THRIVE). This 12-month program was designed to help address barriers that entry-level caregivers experience and reduce turnover through a comprehensive risk assessment, training, and one-on-one coaching. Researchers from RAND conducted a process and outcome evaluation to determine whether THRIVE was meeting its goals of improving retention and achieving a positive return on investment (ROI). They also examined potential areas for program improvement.

Virtual Behavioral Health for Army Soldiers: Soldier Perspectives and Patterns of Treatment.

Delivery of high-quality behavioral health (BH) care is essential to supporting the readiness of the U.S. armed forces and their families. The coronavirus disease 2019 (COVID-19) pandemic led to a dramatic expansion of virtual behavioral health (VBH) care: remote patient access to BH care using technology such as a computer or cellular phone. The U.S. Army asked RAND Arroyo Center to examine the use of VBH to inform recommendations on the role of VBH care in the future of BH care in the Military Health System. The authors analyzed administrative data on VBH and in-person BH care from prior to the pandemic through March 2022 and surveyed soldiers who received BH care to assess their perceptions of VBH care. Administrative data analyses showed that direct care providers were less likely to deliver VBH care than private-sector providers and relied heavily on audio rather than video VBH. In addition, soldiers who received VBH care typically received a mix of VBH and in-person visits. Survey respondents who used VBH care had similar perceptions of the quality of their care and more-positive views of VBH than respondents who did not use VBH care. Few respondents had declined VBH care in favor of in-person care. Using these findings, the authors make recommendations on the role of VBH care in overall BH delivered by the military.

Comparing organization-focused and state-focused financing strategies on provider-level reach of a youth substance use treatment model: a mixed-method study.

BACKGROUND: Financial barriers in substance use disorder service systems have limited the widespread adoption-i.e., provider-level reach-of evidence-based practices (EBPs) for youth substance use disorders. Reach is essential to maximizing the population-level impact of EBPs. One promising, but rarely studied, type of implementation strategy for overcoming barriers to EBP reach is financing strategies, which direct financial resources in various ways to support implementation. We evaluated financing strategies for the Adolescent Community Reinforcement Approach (A-CRA) EBP by comparing two US federal grant mechanisms, organization-focused and state-focused grants, on organization-level A-CRA reach outcomes. METHOD: A-CRA implementation took place through organization-focused and state-focused grantee cohorts from 2006 to 2021. We used a quasi-experimental, mixed-method design to compare reach between treatment organizations funded by organization-focused versus state-focused grants (164 organizations, 35 states). Using administrative training records, we calculated reach as the per-organization proportion of trained individuals who received certification in A-CRA clinical delivery and/or supervision by the end of grant funding. We tested differences in certification rate by grant type using multivariable linear regression models that controlled for key covariates (e.g., time), and tested threats to internal validity from our quasi-experimental design through a series of sensitivity analyses. We also drew on interviews and surveys collected from the treatment organizations and (when relevant) interviews with state administrators to identify factors that influenced reach. RESULTS: The overall certification rates were 27 percentage points lower in state-focused versus organization-focused grants (p = .01). Sensitivity analyses suggested these findings were not explained by confounding temporal trends nor by organizational or state characteristics. We did not identify significant quantitative moderators of reach outcomes, but qualitative findings suggested certain facilitating factors were more influential for organization-focused grants (e.g., strategic planning) and certain barrier factors were more impactful for state-focused grants (e.g., states finding it difficult to execute grant activities). DISCUSSION: As the first published comparison of EBP reach outcomes between financing strategies, our findings can help guide state and federal policy related to financing strategies for implementing EBPs that reduce youth substance use. Future work should explore contextual conditions under which different financing strategies can support the widespread implementation of EBPs for substance use disorder treatment.

Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study.

BACKGROUND: Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures. METHOD: We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment. DISCUSSION: The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.