Action planning for building public health program sustainability: results from a group-randomized trial.

BACKGROUND: Public health programs are charged with implementing evidence-based interventions to support public health improvement; however, to achieve long-term population-based benefits, these interventions must be sustained. Empirical evidence suggests that program sustainability can be improved through training and technical assistance, but few resources are available to support public health programs in building capacity for sustainability. METHODS: This study sought to build capacity for sustainability among state tobacco control programs through a multiyear, group-randomized trial that developed, tested, and evaluated a novel Program Sustainability Action Planning Model and Training Curricula. Using Kolb's experiential learning theory, we developed this action-oriented training model to address the program-related domains proven to impact capacity for sustainability as outlined in the Program Sustainability Framework. We evaluated the intervention using a longitudinal mixed-effects model using Program Sustainability Assessment (PSAT) scores from three time points. The main predictors in our model included group (control vs intervention) and type of dosage (active and passive). Covariates included state-level American Lung Association Score (proxy for tobacco control policy environment) and percent of CDC-recommended funding (proxy for program resources). RESULTS: Twenty-three of the 24 state tobacco control programs were included in the analyses: 11 received the training intervention and 12 were control. Results of the longitudinal mixed-effects linear regression model, where the annual PSAT score was the outcome, showed that states in the intervention condition reported significantly higher PSAT scores. The effects of CDC-recommended funding and American Lung Association smoke-free scores (proxy for policy environment) were small but statistically significant. CONCLUSION: This study found that the Program Sustainability Action Planning Model and Training Curricula was effective in building capacity for sustainability. The training was most beneficial for programs that had made less policy progress than others, implying that tailored training may be most appropriate for programs possibly struggling to make progress. Finally, while funding had a small, statistically significant effect on our model, it virtually made no difference for the average program in our study. This suggests that other factors may be more or equally important as the level of funding a program receives. CLINICALTRIALS: gov, NCT03598114. Registered on July 26, 2018.

A conceptual model for building program sustainability in public health settings: Learning from the implementation of the program sustainability action planning model and training curricula.

Background: The emergence of implementation science has driven an increase in research examining the implementation of evidence-based programs and policies. However, there has been less attention through program sustainability. To achieve the full benefit of investment in program development and implementation, there must be an understanding of the factors that relate to sustainability; additionally, there is a need for a robust set of tools and trainings to support strategic long-term program sustainability. This paper presents results of our sustainability training intervention and a new conceptual model of sustainability. The proposed conceptual model builds upon the intervention design, further specifying the implementation strategy, strategy-mechanism linkages, and effect modifiers. Methods: This research is part of the larger randomized control trial evaluating the effectiveness of the Program Sustainability Action Planning Model and Training Curricula. Specifically, this multimethod study establishes a conceptual model for program sustainability and related capacity-building interventions. The training intervention was delivered through workshops and technical assistance to 11 state tobacco control programs, principally entailing the development and implementation of a sustainability action plan. We utilize descriptive statistics and participant perspectives to evaluate the training intervention and propose an empirically-grounded conceptual model for sustainability capacity-building interventions in public health settings. Results: Participants found intervention components (workshop, workbook, instructor and resources) to be effective. Overall, participants found the intervention improved their ability to develop sustainability action plans and assess their program and partners. Throughout the study, program managers emphasized the importance of the workshop in providing direction for their sustainability work and the value of robust, ongoing technical assistance. Program managers identified several factors that interfered with intervention reception including staff turnover, competing priorities, partnership challenges, and the COVID-19 pandemic. Conclusion: The present study documents the development and implementation of a novel Program Sustainability Action Planning Model and Training Curricula, one of the first interventions designed to improve program sustainability. In addition, we present an empirically-grounded conceptual model for program sustainability. Considering the paucity of research in this understudied and undefined topic area, this is an important contribution that can serve as a framework for similar intervention designs and implementation efforts. Clinical Trail Registration: ClinicalTrails.gov identification number is NCT03598114.

Implementation of Flexibilities to the National School Lunch and Breakfast Programs and Their Impact on Schools in Missouri.

BACKGROUND: In 2018, the United States Department of Agriculture (USDA) issued flexibilities to the National School Lunch and Breakfast Programs, relaxing the nutrition standards for milk, whole grains, and sodium. This study examines the implementation decision-making among Missouri school food services and the impact of implementing these flexibilities on the meals served. METHODS: We developed a survey using the Consolidated Framework of Implementation to determine schools' implementation of the flexibilities and factors related to implementation. To determine how the implementation of flexibilities affected participation, we merged the survey results with school-level meal county data from the Missouri Department of Elementary and Secondary Education. We used ordinary least squares regression to examine how flexibility adoption related to the number of meals served. RESULTS: Most schools implemented the wheat, milk, and sodium flexibilities. Common reasons for implementation were increasing participation, meeting students' preferences, expanding menu variety, and saving money. The implementation of flexibilities was associated with more lunches and breakfasts being served per month, particularly among free and reduced-price meals. CONCLUSIONS: Continued research is needed to determine how the increased uptake of school meals that do not fully meet dietary guidelines by low-income students results in inequities in health outcomes. The findings can inform the design and implementation of future policies, especially as new rules related to flexibility design are determined.

Action Planning for Building Program Sustainability: Results from a Group-Randomized Trial.

Background: Public health programs are charged with implementing evidence-based interventions to support public health improvement, however, to achieve long term population based benefit these interventions must be sustained. Empirical evidence suggests that program sustainability can be improved through training and technical assistance, but few resources are available to support public health programs in building capacity for sustainability. Methods: This study sought to build capacity for sustainability among state tobacco control programs through a multiyear, group-randomized trial that developed, tested, and evaluated a novel Program Sustainability Action Planning Model and Training Curricula. Using Kolb's experiential learning theory, we developed this action-oriented training model to address the program-related domains proven to impact capacity for sustainability as outlined in the Program Sustainability Framework. We evaluated the intervention using a longitudinal mixed-effects model using Program Sustainability Assessment (PSAT) scores from three time points. The main predictors in our model included group (control vs intervention) and type of dosage (active and passive). Covariates included state level American Lung Association score (proxy for tobacco control policy environment) and percent of CDC-recommended funding (proxy for program resources). Results: Twenty-three of the 24 state tobacco control programs were included in the analyses: 11 received the training intervention and 12 were control. Results of the longitudinal mixed-effects linear regression model, where the annual PSAT score was the outcome, showed that states in the intervention condition reported significantly higher PSAT scores. The effects for CDC-recommended funding and American Lung Association smoke-free scores (proxy for policy environment) were small but statistically significant. Conclusion: This study found that the Program Sustainability Action Planning Model and Training Curricula was effective in building capacity for sustainability. The training was most beneficial for programs that had made less policy progress than others, implying that tailored training may be most appropriate for programs possibly struggling to make progress. Finally, while funding had a small, statistically significant effect in our model, it virtually made no difference for the average program in our study. This suggests that other factors may be more or equally important as the level of funding a program receives. Trial registration: NCT03598114, Registered on July 26, 2018 (clnicaltrials.gov/NCT03598114).

Quantitative measures used in empirical evaluations of mental health policy implementation: A systematic review.

Background: Mental health is a critical component of wellness. Public policies present an opportunity for large-scale mental health impact, but policy implementation is complex and can vary significantly across contexts, making it crucial to evaluate implementation. The objective of this study was to (1) identify quantitative measurement tools used to evaluate the implementation of public mental health policies; (2) describe implementation determinants and outcomes assessed in the measures; and (3) assess the pragmatic and psychometric quality of identified measures. Method: Guided by the Consolidated Framework for Implementation Research, Policy Implementation Determinants Framework, and Implementation Outcomes Framework, we conducted a systematic review of peer-reviewed journal articles published in 1995-2020. Data extracted included study characteristics, measure development and testing, implementation determinants and outcomes, and measure quality using the Psychometric and Pragmatic Evidence Rating Scale. Results: We identified 34 tools from 25 articles, which were designed for mental health policies or used to evaluate constructs that impact implementation. Many measures lacked information regarding measurement development and testing. The most assessed implementation determinants were readiness for implementation, which encompassed training (n = 20, 57%) and other resources (n = 12, 34%), actor relationships/networks (n = 15, 43%), and organizational culture and climate (n = 11, 31%). Fidelity was the most prevalent implementation outcome (n = 9, 26%), followed by penetration (n = 8, 23%) and acceptability (n = 7, 20%). Apart from internal consistency and sample norms, psychometric properties were frequently unreported. Most measures were accessible and brief, though minimal information was provided regarding interpreting scores, handling missing data, or training needed to administer tools. Conclusions: This work contributes to the nascent field of policy-focused implementation science by providing an overview of existing measurement tools used to evaluate mental health policy implementation and recommendations for measure development and refinement. To advance this field, more valid, reliable, and pragmatic measures are needed to evaluate policy implementation and close the policy-to-practice gap. Plain Language Summary: Mental health is a critical component of wellness, and public policies present an opportunity to improve mental health on a large scale. Policy implementation is complex because it involves action by multiple entities at several levels of society. Policy implementation is also challenging because it can be impacted by many factors, such as political will, stakeholder relationships, and resources available for implementation. Because of these factors, implementation can vary between locations, such as states or countries. It is crucial to evaluate policy implementation, thus we conducted a systematic review to identify and evaluate the quality of measurement tools used in mental health policy implementation studies. Our search and screening procedures resulted in 34 measurement tools. We rated their quality to determine if these tools were practical to use and would yield consistent (i.e., reliable) and accurate (i.e., valid) data. These tools most frequently assessed whether implementing organizations complied with policy mandates and whether organizations had the training and other resources required to implement a policy. Though many were relatively brief and available at little-to-no cost, these findings highlight that more reliable, valid, and practical measurement tools are needed to assess and inform mental health policy implementation. Findings from this review can guide future efforts to select or develop policy implementation measures.