RESUMO
Public health researchers and practitioners are calling for greater focus on external validity, the ability to generalize findings of evidence-based interventions (EBIs) beyond the limited number of studies testing effectiveness. For public health, the goal is applicability: to translate, disseminate, and implement EBIs for an impact on population health. This article is a review of methods and how they might be combined to better assess external validity. The methods include (a) better description of EBIs and their contexts; (b) combining of statistical tools and logic to draw inferences about study samples; (c) sharper definition of the theory behind the intervention and core intervention components; and (d) more systematic consultation of practitioners. For population impact, studies should focus on context features that are likely to be both important (based on program theory) and frequently encountered by practitioners. Mixed-method programs of research will allow public health to expand causal generalizations.
Assuntos
Saúde Pública , Projetos de Pesquisa , Prática Clínica Baseada em Evidências , Humanos , PesquisaRESUMO
This article describes the evaluation of the Arkansas Act 1220 of 2003, a comprehensive legislative proposal to address the growing epidemic of childhood obesity through changes in the school environment. In addition, the article discusses specific components of the evaluation that may be applicable to other childhood obesity policy evaluation efforts. The conceptual framework for the evaluation, research questions, and evaluation design are described, along with data collection methods and analysis strategies. A mixed methods approach, including both quantitative (surveys, telephone interviews) and qualitative (key informant interviews, records reviews) approaches, was utilized to collect data from a range of informant groups including parents, adolescents, school principals, school district superintendents, and other stakeholders. Challenges encountered with the evaluation are discussed, as are strategies to overcome those challenges. Now in its 9th year, this evaluation has documented substantial changes to school policies and environments but fewer changes to student and family behaviors. The evaluation may inform the methods of other evaluations of childhood obesity prevention policies, as well as inform policymakers about how quickly they might expect implementation of such policies in their own states and localities and anticipate both positive and adverse outcomes.
Assuntos
Obesidade/prevenção & controle , Desenvolvimento de Programas/métodos , Adolescente , Arkansas , Criança , Intervalos de Confiança , Comportamento Alimentar , Política de Saúde/legislação & jurisprudência , Promoção da Saúde/legislação & jurisprudência , Humanos , Razão de Chances , Instituições Acadêmicas , Adulto JovemRESUMO
INTRODUCTION: Evidence-based technical assistance may be needed to implement recent federal policy to prevent childhood obesity through the schools. The Healthy Schools Program is the largest school-based obesity prevention program in the United States. The objectives of this study were to evaluate the role of the program's training and technical assistance and to explore other contributing factors in changing school policies, practices, and environments. METHODS: We analyzed interim progress of schools recruited during the 2007-2008 and 2008-2009 school years as of December 2010. Schools reported progress through an online inventory of policies, practices, and school environment. We compared baseline inventories to the most recent follow-up and tracked both training attendance and contact with national experts. To identify the factors associated with progress, we performed regression analysis on school level and demographics, number of months between baseline and follow-up, and technical assistance. RESULTS: The amount of training and technical assistance was significantly associated with school progress, controlling for school level and demographics, number of months between baseline and follow-up, and school status at baseline. Although all types of schools saw progress, schools in the South had the most progress and urban schools had the least progress. CONCLUSION: Evidence-based training and technical assistance were associated with school progress in changing policies, practices, and environment to prevent childhood obesity.
Assuntos
Programas Governamentais , Assistência Técnica ao Planejamento em Saúde , Obesidade/prevenção & controle , Avaliação de Programas e Projetos de Saúde , Serviços de Saúde Escolar , Criança , Promoção da Saúde , Humanos , Competência Profissional , Análise de Regressão , Estados UnidosRESUMO
INTRODUCTION: Federal and state policies identify schools as a setting to prevent childhood obesity, but schools need better health-promoting strategies. The objective of this study was to evaluate interim progress in schools receiving hands-on training from the Healthy Schools Program, the nation's largest school-based program aimed at preventing childhood obesity. The 4-year program targets schools with predominantly low-income, African American, or Hispanic students. METHODS: In 2010 we assessed schools that enrolled in the 2007-2008 and 2008-2009 school years. School representatives completed an inventory of 8 content areas: policy and systems, school meals, competitive foods and beverages, health education, physical education, physical activity outside of physical education, before- and after-school programs, and school employee wellness. Schools' baseline inventory was compared by t test with the most recent inventory available. RESULTS: Schools made significant changes in all content areas, and effect sizes were moderate to large. CONCLUSION: Participating schools improved environmental policies and practices to prevent childhood obesity. The program is a resource to implement recent federal and state policies.
Assuntos
Programas Governamentais , Assistência Técnica ao Planejamento em Saúde , Obesidade/prevenção & controle , Avaliação de Programas e Projetos de Saúde , Serviços de Saúde Escolar , Negro ou Afro-Americano , Criança , Promoção da Saúde , Hispânico ou Latino , Humanos , Pobreza , Competência Profissional , Análise de Regressão , Estados UnidosRESUMO
Evaluability assessment, also commonly known as exploratory evaluation, has assisted the field of public health to improve programs and to develop a pragmatic, practice-based research agenda. Evaluability assessment was originally developed as a low-cost pre-evaluation activity to prepare better for conventional evaluations of programs, practices, and some policies. For public health programs, however, it serves several other important purposes: (a) giving program staff rapid, constructive feedback about program operations; (b) assisting the core public health planning and assurance functions by helping to develop realistic objectives and providing low-cost, rapid feedback on implementation; (c) navigating federal performance measurement requirements; (d) translating research into practice by examining the feasibility, acceptability, and adaptation of evidence-based practices in new settings and populations; and (e) translating practice into research by identifying promising new approaches to achieve public health goals.
Assuntos
Avaliação de Programas e Projetos de Saúde/métodos , Prática de Saúde Pública/normas , Política Pública , Análise Custo-Benefício , Humanos , Modelos TeóricosRESUMO
PURPOSE: Food environments affect people's food-related decisions, consumption, and health outcomes. This paper critiques instruments available for assessing food environments to help researchers, practitioners, and community organizations select those that best meet their needs, resources, and level of expertise. Gaps and recommendations for future instrument development are discussed. METHOD: Instruments were selected by reviewing the literature and contacting researchers and practitioners. Using the social-ecological model, the instruments were classified according to the type of food environments they assess. Each instrument is rated according to criteria for ease of use, detail, resources required, and psychometric testing. RESULTS: Of the 48 instruments identified, only 39% were tested for validity or reliability. Observational tools were the largest category of instruments developed. Fifty-two percent of the instruments provide some level of technical assistance and 37% are considered appropriate for community members. CONCLUSION: Three distinct groups want to assess food environments: researchers, practitioners, and community organizations. These groups have different information needs and different capacities to undertake assessments. At this time there is a tradeoff between simplicity and low cost on the one hand, and detail and accuracy on the other. To choose the most suitable instrument, users should consider the pros and cons of each instrument and base their selection on purpose of their assessment, resources and expertise at hand, and validity, reliability, and ease of use of the instrument.
Assuntos
Planejamento Ambiental , Abastecimento de Alimentos , Psicometria/métodos , Comportamento de Escolha , Informação de Saúde ao Consumidor , Projetos de Pesquisa Epidemiológica , Indústria Alimentícia , Humanos , Meio SocialRESUMO
PURPOSE: Designing programs for mid-life to older adults whose sedentary behaviors are associated with increased health risks is crucial. The U.S. Task Force on Community Preventive Services strongly recommends individually adapted behavior change programs as one approach to increasing physical activity in communities. The purpose of this study is to report challenges organizations faced when translating two evidence-based programs in real-world settings, adaptations made, and whether or not fidelity was negatively impacted by these adaptations. DESIGN: A grounded theory approach to qualitative research was used. SETTING AND PARTICIPANTS: Nine community organizations across the country participated. Two organizations had more than one site participating, for a total of 12 sites from nine organizations. Within those organizations, 2796 participants were part of the program during the first 2 years. Participants were underactive (i.e., not meeting Centers for Disease Control and Prevention and American College of Sports Medicine recommendations) mid- to older-aged adults. METHODS: Community organizations participated in monthly conference calls, and program information was entered into an electronic database regularly. Data obtained from the calls and database were used for analyses. RESULTS: Challenges and adaptations emerged in three categories: (1) program logistics, (2) program theory, and (3) program philosophy. CONCLUSION: Challenges were present for community organizations; however, with some level of adaptation, the community organizations were able to effectively deliver and maintain fidelity in two evidence-based physical activity programs to a large and diverse group of mid- to older-aged adults.
Assuntos
Redes Comunitárias , Participação da Comunidade , Prática Clínica Baseada em Evidências , Promoção da Saúde/métodos , Atividade Motora , Idoso , Humanos , Masculino , Pessoa de Meia-Idade , Pesquisa Qualitativa , Estados UnidosRESUMO
BACKGROUND: Most evidence-based programs are never translated into community settings and thus never make a public health impact. DESIGN: Active for Life (AFL) was a 4-year translational initiative using a pre-post, quasi-experimental design. Data were collected from 2003 to 2007. Analyses were conducted in 2005 and 2008. SETTING/PARTICIPANTS: Nine lead organizations at 12 sites participated. Active Choices participants (n=2503) averaged 65.8 years (80% women, 41% non-Hispanic white). Active Living Every Day (ALED) participants (n=3388) averaged 70.6 years (83% women, 64% non-Hispanic white). INTERVENTION: In AFL, Active Choices was a 6-month telephone-based and ALED a 20-week group-based lifestyle behavior change program designed to increase physical activity, and both were grounded in social cognitive theory and the transtheoretical model. The interventions were evaluated in Years 1, 3, and 4. An adapted shortened ALED program was evaluated in Year 4. MAIN OUTCOME MEASURE: Moderate- to vigorous-intensity physical activity, assessed with the CHAMPS self-reported measure. RESULTS: Posttest survey response rates were 61% for Active Choices and 70% for ALED. Significant increases in moderate- to vigorous-intensity physical activity, total physical activity, and satisfaction with body appearance and function, and decreases in BMI were seen for both programs. Depressive symptoms and perceived stress, both low at pretest, also decreased over time in ALED. Results were generally consistent across years and sites. CONCLUSIONS: Active Choices and ALED were successfully translated across a range of real-world settings. Study samples were substantially larger, more ethnically and economically diverse, and more representative of older adult's health conditions than in efficacy studies, yet the magnitude of effect sizes were comparable.
Assuntos
Serviços de Saúde Comunitária/organização & administração , Promoção da Saúde/organização & administração , Atividades Cotidianas , Idoso , Análise de Variância , Terapia Comportamental , Distribuição de Qui-Quadrado , Comportamento de Escolha , Aconselhamento , Medicina Baseada em Evidências , Feminino , Humanos , Masculino , Pessoa de Meia-Idade , Avaliação de Resultados em Cuidados de Saúde , Satisfação Pessoal , Avaliação de Programas e Projetos de Saúde , Autoeficácia , Inquéritos e QuestionáriosRESUMO
BACKGROUND: Variations in local context bedevil the assessment of external validity: the ability to generalize about effects of treatments. For evaluation, the challenges of assessing external validity are intimately tied to the translation and spread of evidence-based interventions. This makes external validity a question for decision makers, who need to determine whether to endorse, fund, or adopt interventions that were found to be effective and how to ensure high quality once they spread. OBJECTIVE: To present the rationale for using theory to assess external validity and the value of more systematic interaction of theory and practice. METHODS: We review advances in external validity, program theory, practitioner expertise, and local adaptation. Examples are provided for program theory, its adaptation to diverse contexts, and generalizing to contexts that have not yet been studied. The often critical role of practitioner experience is illustrated in these examples. Work is described that the Robert Wood Johnson Foundation is supporting to study treatment variation and context more systematically. RESULTS: Researchers and developers generally see a limited range of contexts in which the intervention is implemented. Individual practitioners see a different and often a wider range of contexts, albeit not a systematic sample. Organized and taken together, however, practitioner experiences can inform external validity by challenging the developers and researchers to consider a wider range of contexts. Researchers have developed a variety of ways to adapt interventions in light of such challenges. CONCLUSIONS: In systematic programs of inquiry, as opposed to individual studies, the problems of context can be better addressed. Evaluators have advocated an interaction of theory and practice for many years, but the process can be made more systematic and useful. Systematic interaction can set priorities for assessment of external validity by examining the prevalence and importance of context features and treatment variations. Practitioner interaction with researchers and developers can assist in sharpening program theory, reducing uncertainty about treatment variations that are consistent or inconsistent with the theory, inductively ruling out the ones that are harmful or irrelevant, and helping set priorities for more rigorous study of context and treatment variation.
Assuntos
Modelos Teóricos , Reprodutibilidade dos Testes , Tomada de Decisões , Estudos de Avaliação como Assunto , Prática Clínica Baseada em Evidências , Formulação de PolíticasRESUMO
Quality improvement (QI) efforts affect a broader range of people than we often assume. These are the potential stakeholders for QI and its evaluation, and they have valuable perspectives to offer when they are consulted in planning, conducting and interpreting evaluations. QI practitioners are accustomed to consulting stakeholders to assess unintended consequences or assess patient experiences of care, but in many cases there are additional benefits to a broad inclusion of stakeholders. These benefits are better adherence to ethical standards, to assure that all legitimate interests take part, more useful and relevant evaluation information and better political buy-in to improve impact. Balancing various stakeholder needs for information requires skill for both politics and research management. These challenges have few pat answers, but several preferred practices, which are illustrated with practical examples.
Assuntos
Pessoal Administrativo , Comportamento Cooperativo , Pessoal de Saúde , Avaliação de Processos e Resultados em Cuidados de Saúde/organização & administração , Melhoria de Qualidade/organização & administração , Humanos , Avaliação de Processos e Resultados em Cuidados de Saúde/normas , Atenção Primária à Saúde/organização & administração , Melhoria de Qualidade/normasRESUMO
In many respects, obstetrics, perinatology, and neonatology have been at the forefront in the dissemination and uptake of evidence-based medicine. The work of the Cochrane Collaboration began largely with the need to systematize the evidence on these subjects. Among other organizations, the National Perinatal Information Service and the Vermont Oxford Network represent exemplary efforts to promote the uptake of evidence. With this background and support, the reader should be equipped to use the tools and examples provided in this article to promote evidence-based practice in his or her own practice setting.
Assuntos
Medicina Baseada em Evidências , Perinatologia , Garantia da Qualidade dos Cuidados de Saúde , Terapêutica , Difusão de Inovações , Humanos , Recém-Nascido , Abandono do Hábito de FumarRESUMO
BACKGROUND: Beginning in 2003, Active Living by Design (ALbD) established innovative approaches across 25 communities to increase physical activity through community design, public policies, programming, and communication strategies. PURPOSE: The complexity of the ALbD projects called for a mixed-methods evaluation to understand implementation as well as perceived and actual impacts of these efforts. METHODS: Six primary evaluation methods addressed three primary aims: (1) to assess impacts of physical projects and policy changes on community environments; (2) to document intervention strategies implemented, as well as intended and unintended consequences; and (3) to identify strengths and challenges in planning, developing, and implementing interventions. The ALbD evaluation included cross-site comparisons and more in-depth case studies. This article describes the methods used to address the three aims. RESULTS: Analysis of the strengths and challenges associated with the different methods, including partnership capacity surveys, Concept Mapping, an online progress reporting system, key informant interviews, focus groups, and photos and videos. Additional methods, including environmental audits and direct observation, were explored to specifically assess environmental changes. Several important challenges included the lack of baseline data, difficulty in evaluating natural experiments, the need for ongoing policy surveillance, and the need to capture longer-term endpoints. CONCLUSIONS: The mixed-methods evaluation of ALbD advances implementation and evaluation science related to community-based efforts for promoting active living through identification of methods and measures to capture multicomponent and complex interventions as well as translation of a range of approaches to create community change across a variety of populations and settings.
Assuntos
Exercício Físico , Promoção da Saúde/normas , Avaliação de Programas e Projetos de Saúde/métodos , Pesquisa Participativa Baseada na Comunidade , Grupos Focais , Política de Saúde , Promoção da Saúde/métodos , Promoção da Saúde/organização & administração , Humanos , Entrevistas como Assunto , Desenvolvimento de Programas , Características de ResidênciaRESUMO
BACKGROUND: From 2003 to 2008, a total of 25 cross-sector, multidisciplinary community partnerships funded through the Active Living by Design (ALbD) national program designed, planned, and implemented policy and environmental changes, with complementary programs and promotions. PURPOSE: This paper describes the use of concept mapping methods to gain insights into promising active living intervention strategies based on the collective experience of community representatives implementing ALbD initiatives. METHODS: Using Concept Systems software, community representatives (n=43) anonymously generated actions and changes in their communities to support active living (183 original statements, 79 condensed statements). Next, respondents (n=26, from 23 partnerships) sorted the 79 statements into self-created categories, or active living intervention approaches. Respondents then rated statements based on their perceptions of the most important strategies for creating community changes (n=25, from 22 partnerships) and increasing community rates of physical activity (n=23, from 20 partnerships). Cluster analysis and multidimensional scaling were used to describe data patterns. RESULTS: ALbD community partnerships identified three active living intervention approaches with the greatest perceived importance to create community change and increase population levels of physical activity: changes to the built and natural environment, partnership and collaboration efforts, and land-use and transportation policies. The relative importance of intervention approaches varied according to subgroups of partnerships working with different populations. CONCLUSIONS: Decision makers, practitioners, and community residents can incorporate what has been learned from the 25 community partnerships to prioritize active living policy, physical project, promotional, and programmatic strategies for work in different populations and settings.
Assuntos
Exercício Físico , Promoção da Saúde/organização & administração , Relações Comunidade-Instituição , Planejamento Ambiental , Política de Saúde , Prioridades em Saúde , Promoção da Saúde/métodos , Humanos , Atividade Motora , Desenvolvimento de Programas/métodos , Estados UnidosRESUMO
BACKGROUND: A US federal mandate that school districts devise and implement local wellness policies (LWPs) has potential widespread impact on the nutritional content of foods and beverages available in schools and on the amount of physical activity that students engage in; however, evidence concerning the mandate's effectiveness is limited. This study describes the content of LWPs of 6 US school districts and steps taken toward their implementation and evaluation. METHODS: During visits to 6 school districts, we interviewed 88 school and community representatives about the content of their district's LWPs and how the LWPs were being implemented and evaluated. RESULTS: The 6 LWPs were consistent with the federal mandate, although they varied in content and degree of specificity, and none had been fully implemented. All 6 districts were pursuing strategies to ensure that foods and beverages available at school met nutrition standards but did not offer nutrition education to all K-12 students. All 6 districts offered students only limited opportunities for physical activity, and all 6 collected data to monitor process and outcomes of their LWPs. CONCLUSIONS: Partial implementation of LWPs in the districts we visited resulted in significant improvement in the nutritional quality of foods available at district schools, but only slight improvement in students' opportunities for school-based physical activity. We provide recommendations for school districts on implementation and evaluation. Future research is needed to determine the impact of these LWPs on students' health.
Assuntos
Serviços de Alimentação/normas , Política de Saúde , Promoção da Saúde/métodos , Atividade Motora , Instituições Acadêmicas , Adolescente , Criança , Exercício Físico , Distribuidores Automáticos de Alimentos/normas , Guias como Assunto , Comportamentos Relacionados com a Saúde , Educação em Saúde/métodos , Política de Saúde/legislação & jurisprudência , Promoção da Saúde/organização & administração , Humanos , Entrevistas como Assunto , Inovação Organizacional , Estados UnidosAssuntos
Comportamento do Adolescente/psicologia , Delitos Sexuais/prevenção & controle , Violência/prevenção & controle , Adolescente , Criança , Feminino , Humanos , Relações Interpessoais , Masculino , Psicologia do Adolescente , Serviços de Saúde Escolar , Delitos Sexuais/psicologia , Violência/psicologia , Violência/estatística & dados numéricosAssuntos
Hispânico ou Latino , Obesidade/etnologia , Obesidade/prevenção & controle , Prevenção Primária/organização & administração , Criança , Dieta , Meio Ambiente , Exercício Físico , Fundações/organização & administração , Humanos , Disseminação de Informação , Pesquisa , Estados Unidos/epidemiologiaRESUMO
BACKGROUND: Understanding the process by which research is translated into practice is limited. This study sought to examine how interventions change during implementation. METHODS: Data were collected from July 2005 to September 2007. A real-time and cross-case comparison was conducted, examining ten interventions designed to improve health promotion in primary care practices in practice-based research networks. An iterative group process was used to analyze qualitative data (survey data, interviews, site visits, and project diary entries made by grantees approximately every 2 weeks) and to identify intervention adaptations reported during implementation. RESULTS: All interventions required changes as they were integrated into practice. Modifications differed by project and by practice, and were often unanticipated. Three broad categories of changes were identified and include modifications undertaken to accommodate practices' and patients' circumstances as well as personnel costs. In addition, research teams played a crucial role in fostering intervention uptake through their use of personal influence and by providing motivation, retraining, and instrumental assistance to practices. These efforts by the research teams, although rarely considered an essential component of the intervention, were an active ingredient in successful implementation and translation. CONCLUSIONS: Changes are common when interventions are implemented into practice settings. The translation of evidence into practice will be improved when research design and reporting standards are modified to help quality-improvement teams understand both these adaptations and the effort required to implement interventions in practice.