Prioritizing Workforce Development Training in State Health Departments Using TRAIN: Challenges and Opportunities.

OBJECTIVE: The objectives of this study were (1) to obtain data on the current status of public health workforce training and the use of the Training Finder Real-Time Affiliate Network (TRAIN), a public health learning management platform, in state health departments, and (2) to use the data to identify organizational features that might be affecting training and to determine barriers to and opportunities for improving training. METHODS: We conducted structured interviews in 2014 with TRAIN administrators and performance improvement managers (n = 14) from 7 state health departments that were using TRAIN to determine training practices and barriers to training. We determined key organizational features of the 7 agencies, including training structure, required training, TRAIN administrators' employment status (full time or part time), barriers to the use and tracking of core competencies in TRAIN, training needs assessment methods, leadership support of training and staff development, and agency interest in applying for Public Health Accreditation Board accreditation. RESULTS: We identified 4 common elements among TRAIN-affiliated state health departments: (1) underuse of TRAIN as a training tool, (2) inadequate ownership of training within the organization, (3) insufficient valuation of and budgeting for training, and (4) emerging collaboration and changing perceptions about training stimulated by agency preparation for accreditation. CONCLUSIONS: Public health leaders can increase buy-in to the importance of training by giving responsibility for training to a person, centralizing training, and setting expectations for the newly responsible training leader to update training policy and require the use of TRAIN to develop, implement, evaluate, monitor, and report on agency-wide training.

Evaluating Structural Change Approaches to Health Promotion: An Exploratory Scoping Review of a Decade of U.S. Progress.

Structural change approaches-also known as policy and environmental changes-are becoming increasingly common in health promotion, yet our understanding of how to evaluate them is still limited. An exploratory scoping review of the literature was conducted to understand approaches and methods used to evaluate structural change interventions in health promotion and public health literature. Two analysts-along with health sciences librarian consultation-searched PubMed, Web of Science, and EMBASE for peer-reviewed U.S.-based, English language studies published between 2005 and 2016. Data were extracted on the use of evaluation frameworks, study designs, duration of evaluations, measurement levels, and measurement types. Forty-five articles were included for the review. Notably, the majority (73%) of studies did not report application of a specific evaluation framework. Studies used a wide range of designs, including process evaluations, quasi- or nonexperimental designs, and purely descriptive approaches. In addition, 15.6% of studies only measured outcomes at the individual level. Last, 60% of studies combined more than one measurement type (e.g., site observation + focus groups) to evaluate interventions. Future directions for evaluating structural change approaches to health promotion include more widespread use and reporting of evaluation frameworks, developing validated tools that measure structural change, and shifting the focus to health-directed approaches, including an expanded consideration for evaluation designs that address health inequities.

Public Health Practice of Population-Based Birth Defects Surveillance Programs in the United States.

CONTEXT: Birth defects remain a leading cause of infant mortality in the United States and contribute substantially to health care costs and lifelong disabilities. State population-based surveillance systems have been established to monitor birth defects, yet no recent systematic examination of their efforts in the United States has been conducted. OBJECTIVE: To understand the current population-based birth defects surveillance practices in the United States. DESIGN: The National Birth Defects Prevention Network conducted a survey of US population-based birth defects activities that included questions about operational status, case ascertainment methodology, program infrastructure, data collection and utilization, as well as priorities and challenges for surveillance programs. Birth defects contacts in the United States, including District of Columbia and Puerto Rico, received the survey via e-mail; follow-up reminders via e-mails and telephone were used to ensure a 100% response rate. RESULTS: Forty-three states perform population-based surveillance for birth defects, covering approximately 80% of the live births in the United States. Seventeen primarily use an active case-finding approach and 26 use a passive case-finding approach. These programs all monitor major structural malformations; however, passive case-finding programs more often monitor a broader list of conditions, including developmental conditions and newborn screening conditions. Active case-finding programs more often use clinical reviewers, cover broader pregnancy outcomes, and collect more extensive information, such as family history. More than half of the programs (24 of 43) reported an ability to conduct follow-up studies of children with birth defects. CONCLUSIONS: The breadth and depth of information collected at a population level by birth defects surveillance programs in the United States serve as an important data source to guide public health action. Collaborative efforts at the state and national levels can help harmonize data collection and increase utility of birth defects programs.

Assessing the Practices of Population-Based Birth Defects Surveillance Programs Using the CDC Strategic Framework, 2012.

OBJECTIVE: We assessed the practices of U.S. population-based birth defects surveillance programs in addressing current and emergent public health needs. METHODS: Using the CDC Strategic Framework considerations for public health surveillance (i.e., lexicon and standards, legal authority, technological advances, workforce, and analytic capacity), during 2012 and 2013, we conducted a survey of all U.S. operational birth defects programs (n=43) soliciting information on legal authorities, case definition and clinical information collected, types of data sources, and workforce staffing. In addition, we conducted semi-structured interviews with nine program directors to further understand how programs are addressing current and emergent needs. RESULTS: Three-quarters of birth defects surveillance programs used national guidelines for case definition. Most birth defects surveillance programs (86%) had a legislative mandate to conduct surveillance, and many relied on a range of prenatal, postnatal, public health, and pediatric data sources for case ascertainment. Programs reported that the transition from paper to electronic formats was altering the information collected, offering an opportunity for remote access to improve timeliness for case review and verification. Programs also reported the growth of pooled, multistate data collaborations as a positive development. Needs identified included ongoing workforce development to improve information technology and analytic skills, more emphasis on data utility and birth defects-specific standards for health information exchange, and support to develop channels for sharing ideas on data interpretation and dissemination. CONCLUSION: The CDC Strategic Framework provided a useful tool to determine the birth defects surveillance areas with positive developments, such as multi-state collaborative epidemiologic studies, and areas for improvement, such as preparation for health information exchanges and workforce database and analytic skills. Our findings may inform strategic deliberations for enhancing the effectiveness of birth defects surveillance programs.

Policy, systems, and environmentally oriented school-based obesity prevention: opportunities and challenges.

Public health is increasingly emphasizing policy, systems, and environmental (PSE) change as a key strategy for population-level health promotion and disease prevention. When applied to childhood obesity, this strategy typically involves school systems, since children spend large portions of their days in school and are heavily influenced by this environment. While most school systems have implemented nutrition education and physical activity programs for some time, their understanding and use of PSE approaches to obesity prevention is accelerating based on several large federally funded initiatives. As part of one initiative's evaluation, key informant interviews reveal the specific obesity prevention PSE strategies schools are attempting and the corresponding barriers and facilitators to their implementation. These evaluation findings raise several fundamental issues regarding school-based obesity prevention, including the potential role of school personnel, the influence of grant funding on school health initiatives, and the fit between public health and educational priorities.

Evaluating programs that address ideological issues: ethical and practical considerations for practitioners and evaluators.

There are important practical and ethical considerations for organizations in conducting their own, or commissioning external, evaluations and for both practitioners and evaluators, when assessing programs built on strongly held ideological or philosophical approaches. Assessing whether programs "work" has strong political, financial, and/or moral implications, particularly when expending public dollars, and may challenge objectivity about a particular program or approach. Using a case study of the evaluation of a school-based abstinence-until-marriage program, this article discusses the challenges, lessons learned, and ethical responsibilities regarding decisions about evaluation, specifically associated with ideologically driven programs. Organizations should consider various stakeholders and views associated with their program to help identify potential pitfalls in evaluation. Once identified, the program or agency needs to carefully consider its answers to two key questions: Do they want the answer and are they willing to modify the program? Having decided to evaluate, the choice of evaluator is critical to assuring that ethical principles are maintained and potential skepticism or criticism of findings can be addressed appropriately. The relationship between program and evaluator, including agreements about ownership and eventual publication and/or promotion of data, should be addressed at the outset. Programs and organizations should consider, at the outset, their ethical responsibility when findings are not expected or desired. Ultimately, agencies, organizations, and programs have an ethical responsibility to use their data to provide health promotion programs, whether ideologically founded or not, that appropriately and effectively address the problems they seek to solve.

Improving water quality communications at beaches: input from stakeholders.

OBJECTIVES: Water quality communication practices vary widely and stakeholder input has not played a role in defining acceptable levels of risk. Although the 2012 Recreational Water Quality Criteria (RWQC) emphasize the importance of promptly notifying the public about hazardous conditions, little is known about the public's understanding of notifications, or about levels of risk deemed acceptable. We sought to address these gaps. METHODS: A mixed methods approach was used. Focus groups (FGs) provided qualitative data regarding the understanding of surface water quality, awareness, and use, of currently available water quality information, and acceptability of risk. Intercept interviews (INTs) at recreation sites provided quantitative data. RESULTS: INTs of 374 people and 15 FG sessions were conducted. Participants had limited awareness about water quality information posted at beaches, even during swim bans and advisories. Participants indicated that communication content should be current, from a trusted source, and describe health consequences. Communicating via mobile electronics should be useful for segments of the population. Risk acceptability is lower with greater outcome severity, or if children are impacted. CONCLUSIONS: Current water quality communications approaches must be enhanced to make notification programs more effective. Further work should build on this initial effort to evaluate risk acceptability among US beachgoers.

Evaluating social media's capacity to develop engaged audiences in health promotion settings: use of Twitter metrics as a case study.

Use of social media in health promotion and public health continues to grow in popularity, though most of what is reported in literature represents one-way messaging devoid of attributes associated with engagement, a core attribute, if not the central purpose, of social media. This article defines engagement, describes its value in maximizing the potential of social media in health promotion, proposes an evaluation hierarchy for social media engagement, and uses Twitter as a case study to illustrate how the hierarchy might function in practice. Partnership and participation are proposed as culminating outcomes for social media use in health promotion. As use of social media in health promotion moves toward this end, evaluation metrics that verify progress and inform subsequent strategies will become increasingly important.

Advocacy evaluation: challenges and emerging trends.

Devising, promoting, and implementing changes in policies and regulations are important components of population-level health promotion. Whether advocating for changes in school meal nutrition standards or restrictions on secondhand smoke, policy change can create environments conducive to healthier choices. Such policy changes often result from complex advocacy efforts that do not lend themselves to traditional evaluation approaches. In a challenging fiscal environment, allocating scarce resources to policy advocacy may be particularly difficult. A well-designed evaluation that moves beyond inventorying advocacy activities can help make the case for funding advocacy and policy change efforts. Although it is one thing to catalog meetings held, position papers drafted, and pamphlets distributed, it is quite another to demonstrate that these outputs resulted in useful policy change outcomes. This is where the emerging field of advocacy evaluation fits in by assessing (among other things) strategic learning, capacity building, and community organizing. Based on recent developments, this article highlights several challenges advocacy evaluators are currently facing and provides new resources for addressing them.

Use of social media in health promotion: purposes, key performance indicators, and evaluation metrics.

Despite the expanding use of social media, little has been published about its appropriate role in health promotion, and even less has been written about evaluation. The purpose of this article is threefold: (a) outline purposes for social media in health promotion, (b) identify potential key performance indicators associated with these purposes, and (c) propose evaluation metrics for social media related to the key performance indicators. Process evaluation is presented in this article as an overarching evaluation strategy for social media.

Developmental evaluation: building innovations in complex environments.

Developmental evaluation is an emerging approach to program evaluation that emphasizes innovation and learning. It is particularly well suited to evaluating innovative programs in their earliest stages of development and adapting existing programs to complex or changing environments. Key features of the developmental evaluation approach include a tight integration between evaluators and program staff and the use of data for continuous program improvement. This article presents developmental evaluation as a complementary approach to the traditional formative-summative evaluation cycle, especially when used for preformative evaluation. To illustrate this emerging approach, the article features a case example from the Illinois Caucus for Adolescent Health's evaluation of its school board sexuality education policy change project. The article concludes by suggesting ways that developmental evaluation can be useful in health promotion practice.

Basic priority rating model 2.0: current applications for priority setting in health promotion practice.

Priority setting is an important component of systematic planning in health promotion and also factors into the development of a comprehensive evaluation plan. The basic priority rating (BPR) model was introduced more than 50 years ago and includes criteria that should be considered in any priority setting approach (i.e., use of predetermined criteria, standardized comparisons, and a rubric that controls bias). Although the BPR model has provided basic direction in priority setting, it does not represent the broad array of data currently available to decision makers. Elements in the model also give more weight to the impact of communicable diseases compared with chronic diseases. For these reasons, several modifications are recommended to improve the BPR model and to better assist health promotion practitioners in the priority setting process. The authors also suggest a new name, BPR 2.0, to represent this revised model.

Advocacy evaluation: what it is and where to find out more about it.

Advocacy and policy change have become increasingly important strategies in health promotion efforts. As a result, advocacy evaluation is emerging as a necessary competency for health promotion practitioners. This article introduces the growing field of advocacy evaluation by describing its typical features, which include using prospective evaluation designs, emphasizing real-time data collection and use, monitoring and responding to changing environmental conditions, and assessing both individual and organizational capacity building. A number of freely available resources for learning more about advocacy evaluation's key concepts and methods are highlighted.

Sustaining a school-based prevention program: results from the Aban Aya Sustainability Project.

Sustaining effective school-based prevention programs is critical to improving youth and population-based health. This article reports on results from the Aban Aya Sustainability Project, an effort to sustain a school-based prevention program that was tested via a randomized trial and targeted violence, drug use, and risky sex-related behaviors among a cohort of 5th-grade African American children followed through 10th grade. Sustainability project health educators trained parent educators to deliver the Aban Aya prevention curriculum in five schools, and project researchers studied the resultant curricular implementation and relations between the research and school-based teams. Study results showed uneven implementation across the five schools that we largely attributed to parent educator preparation and parent educator-health educator relations. These and related results are discussed to answer the study's primary research question: How viable was the sustainability project's parent-centered approach to sustaining a school-based prevention program?