Evaluating Cross-Cutting Approaches to Chronic Disease Prevention and Management: Developing a Comprehensive Evaluation.

We provide an overview of the comprehensive evaluation of State Public Health Actions to Prevent and Control Diabetes, Heart Disease, Obesity and Associated Risk Factors and Promote School Health (State Public Health Actions). State Public Health Actions is a program funded by the Centers for Disease Control and Prevention to support the statewide implementation of cross-cutting approaches to promote health and prevent and control chronic diseases. The evaluation addresses the relevance, quality, and impact of the program by using 4 components: a national evaluation, performance measures, state evaluations, and evaluation technical assistance to states. Challenges of the evaluation included assessing the extent to which the program contributed to changes in the outcomes of interest and the variability in the states' capacity to conduct evaluations and track performance measures. Given the investment in implementing collaborative approaches at both the state and national level, achieving meaningful findings from the evaluation is critical.

Impacts of a farmers' market incentive programme on fruit and vegetable access, purchase and consumption.

OBJECTIVE: The present study examines the impact of Health Bucks, a farmers' market incentive programme, on awareness of and access to farmers' markets, and fruit and vegetable purchase and consumption in low-income New York City neighbourhoods. DESIGN: The evaluation used two primary data collection methods: (i) an on-site point-of-purchase survey of farmers' market shoppers; and (ii) a random-digit-dial telephone survey of residents in neighbourhoods where the programme operates. Additionally, we conducted a quasi-experimental analysis examining differential time trends in consumption before and after programme introduction using secondary Community Health Survey (CHS) data. SETTING: New York City farmers' markets and communities. SUBJECTS: Farmers' market shoppers (n 2287) completing point-of-purchase surveys in a representative sample of New York City farmers' markets in 2010; residents (n 1025) completing random-digit-dial telephone survey interviews in 2010; and respondents (n 35 606) completing CHS interviews in 2002, 2004, 2008 and 2009. RESULTS: Greater Health Bucks exposure was associated with: (i) greater awareness of farmers' markets; (ii) increased frequency and amount of farmers' market purchases; and (iii) greater likelihood of a self-reported year-over-year increase in fruit and vegetable consumption. However, our CHS analysis did not detect impacts on consumption. CONCLUSIONS: While our study provides promising evidence that use of farmers' market incentives is associated with increased awareness and use of farmers' markets, additional research is needed to better understand impacts on fruit and vegetable consumption.

Training and technical assistance for compliance with beverage and physical activity components of New York City's regulations for early child care centers.

INTRODUCTION: In 2006, the New York City Department of Health and Mental Hygiene (DOHMH) passed regulations for child care centers that established standards for beverages provided to children and set a minimum amount of time for daily physical activity. DOHMH offered several types of training and technical assistance to support compliance with the regulations. This article analyzes the association between training and technical assistance provided and compliance with the regulations in a sample of 174 group child care centers. METHODS: Compliance was measured by using a site inventory of beverages stored on premises and a survey of centers' teachers regarding the amount of physical activity provided. Training and technical assistance measures were based on the DOHMH records of training and technical assistance provided to the centers in the sample and on a survey of center directors. Ordinal logistic regression was used to assess the association between training and technical assistance measures and compliance with the regulations. RESULTS: Measures of training related to physical activity the center received: the number of staff members who participated in Sport, Play and Active Recreation for Kids (SPARK) and other training programs in which a center participated were associated with better compliance with the physical activity regulations. Neither training nor technical assistance were associated with compliance with the regulations related to beverages. CONCLUSION: Increased compliance with regulations pertaining to physical activity was not related to compliance with beverage regulations. Future trainings should be targeted to the specific regulation requirements to increase compliance.

Relationship between child care centers' compliance with physical activity regulations and children's physical activity, New York City, 2010.

INTRODUCTION: Physical activity may protect against overweight and obesity among preschoolers, and the policies and characteristics of group child care centers influence the physical activity levels of children who attend them. We examined whether children in New York City group child care centers that are compliant with the city's regulations on child physical activity engage in more activity than children in centers who do not comply. METHODS: A sample of 1,352 children (mean age, 3.39 years) served by 110 group child care centers in low-income neighborhoods participated. Children's anthropometric data were collected and accelerometers were used to measure duration and intensity of physical activity. Multilevel generalized linear regression modeling techniques were used to assess the effect of center- and child-level factors on child-level physical activity. RESULTS: Centers' compliance with the regulation of obtaining at least 60 minutes of total physical activity per day was positively associated with children's levels of moderate to vigorous physical activity (MVPA); compliance with the regulation of obtaining at least 30 minutes of structured activity was not associated with increased levels of MVPA. Children in centers with a dedicated outdoor play space available also spent more time in MVPA. Boys spent more time in MVPA than girls, and non-Hispanic black children spent more time in MVPA than Hispanic children. CONCLUSION: To increase children's level of MVPA in child care, both time and type of activity should be considered. Further examination of the role of play space availability and its effect on opportunities for engaging in physical activity is needed.

Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations.

INTRODUCTION: New York City Article 47 regulations, implemented in 2007, require licensed child care centers to improve the nutrition, physical activity, and television-viewing behaviors of enrolled children. To supplement an evaluation of the Article 47 regulations, we conducted an exploratory ecologic study to examine changes in childhood obesity prevalence among low-income preschool children enrolled in the Nutrition Program for Women, Infants, and Children (WIC) in New York City neighborhoods with or without a district public health office. We conducted the study 3 years before (from 2004 through 2006) and after (from 2008 through 2010) the implementation of the regulations in 2007. METHODS: We used an ecologic, time-trend analysis to compare 3-year cumulative obesity prevalence among WIC-enrolled preschool children during 2004 to 2006 and 2008 to 2010. Outcome data were obtained from the New York State component of the Centers for Disease Control and Prevention's Pediatric Nutrition Surveillance System. RESULTS: Early childhood obesity prevalence declined in all study neighborhoods from 2004-2006 to 2008-2010. The greatest decline occurred in Manhattan high-risk neighborhoods where obesity prevalence decreased from 18.6% in 2004-2006 to 15.3% in 2008-2010. The results showed a narrowing of the gap in obesity prevalence between high-risk and low-risk neighborhoods in Manhattan and the Bronx, but not in Brooklyn. CONCLUSION: The reductions in early childhood obesity prevalence in some high-risk and low-risk neighborhoods in New York City suggest that progress was made in reducing health disparities during the years just before and after implementation of the 2007 regulations. Future research should consider the built environment and markers of differential exposure to known interventions and policies related to childhood obesity prevention.

Evaluation design of New York City's regulations on nutrition, physical activity, and screen time in early child care centers.

This article describes the multi-method cross-sectional design used to evaluate New York City Department of Health and Mental Hygiene's regulations of nutrition, physical activity, and screen time for children aged 3 years or older in licensed group child care centers. The Center Evaluation Component collected data from a stratified random sample of 176 licensed group child care centers in New York City. Compliance with the regulations was measured through a review of center records, a facility inventory, and interviews of center directors, lead teachers, and food service staff. The Classroom Evaluation Component included an observational and biometric study of a sample of approximately 1,400 children aged 3 or 4 years attending 110 child care centers and was designed to complement the center component at the classroom and child level. The study methodology detailed in this paper may aid researchers in designing policy evaluation studies that can inform other jurisdictions considering similar policies.

Measurement of compliance with New York City's regulations on beverages, physical activity, and screen time in early child care centers.

INTRODUCTION: Policy interventions designed to change the nutrition environment and increase physical activity in child care centers are becoming more common, but an understanding of the implementation of these interventions is yet to be developed. The objective of this study was to explore the extent and consistency of compliance with a policy intervention designed to promote nutrition and physical activity among licensed child care centers in New York City. METHODS: We used a multimethod cross-sectional approach and 2 independent components of data collection (Center Evaluation Component and Classroom Evaluation Component). The methods were designed to evaluate the impact of regulations on beverages served, physical activity, and screen time at child care centers. We calculated compliance scores for each evaluation component and each regulation and percentage agreement between compliance in the center and classroom components. RESULTS: Compliance with certain requirements of the beverage regulations was high and fairly consistent between components, whereas compliance with the physical activity regulation varied according to the data collection component. Compliance with the regulation on amount and content of screen time was high and consistent. CONCLUSION: Compliance with the physical activity regulation may be a more fluid, day-to-day issue, whereas compliance with the regulations on beverages and television viewing may be easier to control at the center level. Multiple indicators over multiple time points may provide a more complete picture of compliance - especially in the assessment of compliance with physical activity policies.

Evaluating public health resources: what happens when funding disappears?

INTRODUCTION: Although various factors affect the sustainability of public health programs, funding levels can influence many aspects of program continuity. Program evaluation in public health typically does not assess the progress of initiatives after discontinuation of funding. The objective of this study was to describe the effect of funding loss following expiration of a 5-year federal grant awarded to state health departments for development of statewide obesity prevention partnerships. METHODS: The study used qualitative methods involving semistructured key informant interviews with state health departments. Data were analyzed using thematic analysis for effect of funding loss on staffing, programs, partnerships, and implementation of state plans. RESULTS: Many of the programs that continued to run after the grant expired operated at reduced capacity, either reaching fewer people or conducting fewer program activities for the same population. Although many states were able to leverage funding from other sources, this shift in funding source often resulted in priorities changing to meet new funding requirements. Evaluation capacity suffered in all states. Nearly all states reported losing infrastructure and capacity to communicate widely with partners. All states reported a severe or complete loss of their ability to provide training and technical assistance to partners. Despite these reduced capacities, states reported several key resources that facilitated continued work on the state plan. CONCLUSIONS: Decisions regarding continuation of funding are often dependent on budget constraints, evidence of success, and perceived ability to succeed in the future. Evaluating public health funding decisions may help guide development of best practice strategies for supporting long-term program success.

Developing stories from the field to highlight policy, systems, and environmental approaches in obesity prevention.

As obesity prevention initiatives increasingly shift toward approaches focused on policy, systems, and environmental change, opportunities to share experiences from the field and lessons learned are growing. Stories are a tool to illustrate processes and outcomes of initiatives that can complement quantitative results. The use of stories, however, is not widely recognized, and the methods and tools available to develop stories are limited. Therefore, we describe the methods used to collect, develop, and disseminate stories featuring comprehensive obesity prevention efforts that various state health departments are planning and implementing. We also discuss potential challenges and provide recommendations that public health practitioners may consider when developing similar stories.

Implementing a farmers' market incentive program: perspectives on the New York City Health Bucks Program.

INTRODUCTION: One strategy for lowering the prevalence of obesity is to increase access to and affordability of fruits and vegetables through farmers' markets. However, little has been documented in the literature on the implementation of such efforts. To address this gap, the Division of Nutrition, Physical Activity, and Obesity (DNPAO) sponsored an evaluation of the New York City Health Bucks program, a farmers' market coupon incentive program intended to increase access to fresh fruits and vegetables in underserved neighborhoods while supporting local farmers. METHODS: We conducted a process evaluation of Health Bucks program implementation. We interviewed 6 farmer/vendors, 3 market managers, and 4 program administrators, and collected data on site at 86 farmers' markets, including surveys of 81 managers and 141 farmer/vendors on their perspectives on promotion and redemption of the incentive coupons; knowledge and attitudes regarding the program; experiences with markets and products; and facilitators and barriers to program participation. RESULTS: Results indicate that respondents view Health Bucks as a positive program model. Farmers' market incentive coupon programs like Health Bucks are one strategy to address the problem of obesity and were associated with higher fruit and vegetable access and purchases in low-income communities. CONCLUSIONS: This evaluation identified some areas for improving implementation of the Health Bucks program. Farmers' market incentive programs like Health Bucks may be one avenue to increase access to and affordability of fruits and vegetables among low-income persons. Further research is needed to assess the potential effects of these programs on access and health outcomes.

An evaluation framework for obesity prevention policy interventions.

As the emphasis on preventing obesity has grown, so have calls for interventions that extend beyond individual behaviors and address changes in environments and policies. Despite the need for policy action, little is known about policy approaches that are most effective at preventing obesity. The Centers for Disease Control and Prevention (CDC) and others are funding the implementation and evaluation of new obesity prevention policies, presenting a distinct opportunity to learn from these practice-based initiatives and build the body of evidence-based approaches. However, contributions from this policy activity are limited by the incomplete and inconsistent evaluation data collected on policy processes and outcomes. We present a framework developed by the CDC-funded Center of Excellence for Training and Research Translation that public health practitioners can use to evaluate policy interventions and identify the practice-based evidence needed to fill the gaps in effective policy approaches to obesity prevention.

Adapting logic models over time: the Washington State Heart Disease and Stroke Prevention Program experience.

Logic models are graphic representations of the relationship between program activities and their intended effects and are used for both program planning and evaluation. Logic models can provide an important foundation for program evaluation by identifying evaluation questions that most appropriately assess program processes and outcomes and by guiding measurement decisions. We demonstrate how logic models can be used to plan program evaluation by describing the adoption of logic modeling by the Washington State Heart Disease and Stroke Prevention Program (WaHDSPP) and by specifying the changes in process and use of logic models since the program's initial funding. Our paper describes how a logic model was used in generating the program evaluation plan for the WaHDSPP, including the identification of evaluation questions and development of indicators to track progress effectively. We describe the use of evaluation results, as well as steps state programs can take to use logic models in program evaluation.

Childhood Obesity Declines Project: Highlights of Community Strategies and Policies.

BACKGROUND: The social ecological model (SEM) is a framework for understanding the interactive effects of personal and environmental factors that determine behavior. The SEM has been used to examine childhood obesity interventions and identify factors at each level that impact behaviors. However, little is known about how those factors interact both within and across levels of the SEM. METHODS: The Childhood Obesity Declines (COBD) project was exploratory, attempting to capture retrospectively policies and programs that occurred in four communities that reported small declines in childhood obesity. It also examined contextual factors that may have influenced initiatives, programs, or policies. Data collection included policy and program assessments, key informant interviews, and document reviews. These data were aggregated by the COBD project team to form a site report for each community (available at www.nccor.org/projects/obesity-declines ). These reports were used to develop site summaries that illustrate how policies, programs, and activities worked to address childhood obesity in each study site. RESULTS/CONCLUSIONS: Site summaries for Anchorage, AK; Granville County, NC; Philadelphia, PA; and New York City, NY, describe those policies and programs implemented across the levels of the SEM to address childhood obesity and examine interactions both across and within levels of the model to better understand what factors appear important for implementation success.

Childhood Obesity Declines Project: A Different Methodology.

BACKGROUND: The evidence for and our understanding of community-level strategies such as policies, system, and environmental changes that support healthy eating and active living is growing. However, researchers and evaluation scientists alike are still not confident in what to recommend for preventing or sustaining declines in the prevalence of obesity. METHODS: The Systematic Screening and Assessment (SSA) methodology was adapted as a retrospective process to confirm obesity declines and to better understand what and how policies and programs or interventions may contribute as drivers. The Childhood Obesity Declines (COBD) project's adaptation of the SSA methodology consisted of the following components: (1) establishing and convening an external expert advisory panel; (2) identification and selection of sites reporting obesity declines; (3) confirmation and review of what strategies occurred and contextual factors were present during the period of the obesity decline; and (4) reporting the findings to sites and the field. RESULTS/DISCUSSION: The primary result of the COBD project is an in-depth examination of the question, "What happened and how did it happen in communities where the prevalence of obesity declined?" The primary aim of this article is to describe the project's methodology and present its limitations and strengths. CONCLUSIONS: Exploration of the natural experiments such that occurred in Anchorage, Granville County, New York City, and Philadelphia is the beginning of our understanding of the drivers and contextual factors that may affect childhood obesity. This retrospective examination allows us to: (1) describe targeted interventions; (2) examine the timeline and summarize intervention implementation; (3) document national, state, local, and institutional policies; and (4) examine the influence of the reach and potential multisector layering of interventions.

Childhood Obesity Declines Project: An Exploratory Study of Strategies Identified in Communities Reporting Declines.

BACKGROUND: Although childhood obesity rates have been high in the last few decades, recent national reports indicate a stabilization of rates among some subpopulations of children. This study examines the implementation of initiatives, policies, and programs (referred to as strategies) in four communities that experienced declines in childhood obesity between 2003 and 2012. METHODS AND RESULTS: The Childhood Obesity Decline project verified obesity declines and identified strategies that may have influenced and supported the decline in obesity. The project used an adaptation of the Systematic Screening and Assessment method to identify key informants in each site. Four settings were highlighted related to childhood: (1) communities, (2) schools, (3) early care and education, and (4) healthcare. The findings indicate that programs and policies were implemented across local settings (primarily in schools and early childhood settings) and at the state level, during a timeframe of supportive federal policies and initiatives. CONCLUSIONS: Multilevel approaches were aimed to improve the nutrition and physical activity environments where children spend most of their time. We hypothesized that other, more distal strategies amplified and reinforced the impact of the efforts that more directly targeted children. The simultaneous public health messaging and multilayered initiatives, supported by cross-sector partnerships and active, high-level champions, were identified as likely important contributors to success in attaining declines in rates of childhood obesity.

The Childhood Obesity Declines Project: A Review of Enacted Policies.

BACKGROUND: State- and local-level policies can influence children's diet quality and physical activity (PA) behaviors. The goal of this article is to understand the enacted state and local policy landscape in four communities reporting declines in childhood obesity. METHODS AND RESULTS: State-level policies were searched within the CDC's online Chronic Disease State Policy Tracking System. Local level policies were captured during key informant interviews in each of the sites. Policies were coded by setting [i.e., early care and education (ECE) also known as child care, school, community], jurisdictional level (i.e., state or local) and policy type (i.e., legislation or regulation). The time period for each site was unique, capturing enacted policies 5 years before the reported declines in childhood obesity in each of the communities. A total of 39 policies were captured across the 4 sites. The majority originated at the state level. Two policies pertaining to ECE, documented during key informant interviews, were found to be adopted at the local level. CONCLUSION: Similarities were noted between the four communities in the types of polices enacted. All four communities had state- and/or local-level policies that aimed to improve the nutrition environment and increase opportunities for PA in both the ECE and K-12 school settings. This article is a step in the process of determining what may have contributed to obesity declines in the selected communities.

The Childhood Obesity Declines Project: Implications for Research and Evaluation Approaches.

BACKGROUND: Childhood obesity remains prevalent and is increasing in some disadvantaged populations. Numerous research, policy and community initiatives are undertaken to impact this pandemic. Understudied are natural experiments. The need to learn from these efforts is paramount. Resulting evidence may not be readily available to inform future research, community initiatives, and policy development/implementation. METHODS: We discuss the implications of using an adaptation of the Systematic Screening and Assessment (SSA) method to evaluate the Childhood Obesity Declines (COBD) project. The project examined successful initiatives, programs and policies in four diverse communities which were concurrent with significant declines in child obesity. In the context of other research designs and evaluation schemas, rationale for use of SSA is presented. Evidence generated by this method is highlighted and guidance suggested for evaluation of future studies of community-based childhood obesity prevention initiatives. Support for the role of stakeholder collaboratives, in particular the National Collaborative on Childhood Obesity Research, as a synergistic vehicle to accelerate research on childhood obesity is discussed. RESULTS/DISCUSSION: SSA mapped active processes and provided contextual understanding of multi-level/component simultaneous efforts to reduce rates of childhood obesity in community settings. Initiatives, programs and policies were not necessarily coordinated. And although direct attribution of intervention/initiative/policy components could not be made, the what, by who, how, to whom was temporally associated with statistically significant reductions in childhood obesity. CONCLUSIONS: SSA provides evidence for context and processes which are not often evaluated in other data analytic methods. SSA provides an additional tool to layer with other evaluation approaches.

Childhood Obesity Declines Project: An Effort of the National Collaborative on Childhood Obesity Research to Explore Progress in Four Communities.

BACKGROUND: Recent findings show that national childhood obesity prevalence overall is improving among some age groups, but that disparities continue to persist, particularly among populations that have historically been at higher risk of obesity and overweight. Over the past several years, many jurisdictions at the city or county level across the nation have also reported declines. Little evaluation has focused on understanding the factors that influence the implementation of efforts to reduce childhood obesity rates. This article summarizes the rationale, aims, and overall design of the Childhood Obesity Declines Project (COBD), which was the first of its kind to systematically study and document the what, how, when, and where of community-based obesity strategies in four distinct communities across the nation. METHODS: COBD was initiated by the National Collaborative on Childhood Obesity Research (NCCOR), was led by a subset of NCCOR advisors and a research team at ICF, and was guided by external advisors made up of researchers, decision makers, and other key stakeholders. The research team used an adapted version of the Systematic Screening and Assessment method to review and collect retrospective implementation data in four communities. RESULTS: COBD found that sites implemented strategies across the many levels and environments that impact children's well being (akin to the social-ecological framework), building a Culture of Health in their communities. CONCLUSIONS: COBD demonstrates how collaboratives of major funders with the support of other experts and key stakeholders, can help to accelerate progress in identifying and disseminating strategies that promote healthy eating and physical activity.

Factors affecting older african american women's decisions to join the PLCO Cancer Screening Trial.

PURPOSE: The purpose of this study is to describe the factors associated with the decisions of older African American women to join the PLCO (Prostate, Lung, Colorectal and Ovarian) Cancer Screening Trial when recruited. METHODS: African American women between ages 55 and 74 years who were never diagnosed with a PLCO cancer were eligible for our study. Two methods of recruitment were used. First, mailings were sent to a random sample of women describing the PLCO followed by a telephone call to determine interest in the PLCO. If women were not interested in PLCO but consented to participate in our study, they were interviewed immediately. Second, we followed up with African American women who responded to mass mailings sent out before the start of our study by the Pittsburgh PLCO office. Women completed an interview about their cancer and clinical trial knowledge, attitudes, beliefs, and behaviors. The responses of women who joined the PLCO Trial are contrasted with the responses of women who did not join. RESULTS: Numerous factors were associated with the decision of older African American women to join the PLCO, including perceptions of cancer prevention and detection, the experience of having a loved one with cancer, knowledge of and experience with clinical trials, and beliefs regarding the benefits and risks of clinical trial participation. CONCLUSION: Minority recruitment to cancer clinical trials could be increased by designing interventions focused on individual, organizational, and community needs.

Evaluation of the Physical Activity and Public Health Course for Practitioners.

PURPOSE: From 1996 to 2013, a 6-day Physical Activity and Public Health Course for Practitioners has been offered yearly in the United States. An evaluation was conducted to assess the impact of the course on building public health capacity for physical activity and on shaping the physical activity and public health careers of fellows since taking the courses. METHOD: An evaluation quantified time that fellows spent in different course offerings and surveyed fellows. RESULTS: From 1996 to 2012, 410 fellows attended the course, and in 2013, 186 participated in the Web-based survey (56% response rate). The number of fellows attending the course ranged from 15 to 33 yearly. From 1996 to 2012, the course averaged 38 hr of instructional time that included topics on interventions and environment/policy work to increase physical activity, program evaluation, public health research, and health disparities. The course included consultations, collaborative work, and field-based experiences. Fellows who participated in the survey agreed that the course had a positive impact on the physical activity research or practice work they did (98%), met their expectations (96%), helped them with research/practice collaborations with other physical activity professionals (96%), assisted them in conducting higher-quality interventions/programs (95%), helped increase their professional networking in the field (93%), and had a positive impact on other work they did (91%). Following the course, 66% and 56% had further contact with faculty and other fellows, respectively. CONCLUSION: The Physical Activity and Public Health Course for Practitioners made important contributions toward building the capacity of physical activity and public health practitioners.