Good practice recommendations on implementation evaluation for policies targeting diet, physical activity, and sedentary behaviour.

Health policies aim to achieve specific health goals through system-level changes, unlike common health interventions that focus on promoting specific health behaviors on individual level. However, reliable data on the feasibility and implementation of policy actions across Europe are lacking. Moreover, no practice-oriented guidance exists for policy makers and implementers on how to evaluate policy implementation.As part of the Policy Evaluation Network, we aimed to synthesise knowledge on how to evaluate the implementation of policies promoting healthy diets, physical activity, and reducing sedentary behaviours. The multidisciplinary working group comprised 16 researchers and conducted two scoping reviews, three systematic reviews, two meta-reviews, two qualitative case studies and one quantitative case study over three years. The target populations included the general population, those at risk for obesity, and school children. Based on these reviews and case studies, this article summarises and presents the findings and lessons learned regarding the implementation evaluation of policies in nine case reports.Drawing on these experiences, three critical requirements for policy implementation evaluation were set: 1) conduct a comprehensive policy implementation evaluation from a multi-level perspective, 2) use implementation frameworks to address processes, determinants, and outcomes, and 3) engage relevant stakeholders in policy implementation evaluation. Finally, the consensus process resulted in 10 steps for the implementation evaluation of policies to promote physical activity and a healthy diet and to reduce sedentary behaviours, which adhere to the requirements and resources of the targeted policy.The findings of an implementation evaluation can lead to a better understanding of why policies work or not and can serve as a basis for developing solutions. This practice-oriented guidance outlines factors that should be considered in policy implementation evaluation to address its complexity. In this way, involved researchers and practitioners are empowered to engage in the evaluation process to close the knowledge gap regarding policy implementation.

Barriers and facilitators to implementation of direct fruit and vegetables provision interventions in kindergartens and schools: a qualitative systematic review applying the consolidated framework for implementation research (CFIR).

BACKGROUND: Although children's intake of fruit and vegetables has seen a recent rise, almost half of adolescents do not eat even one piece of fruit or vegetables per day. One way to address this problem is through interventions that provide fruit and vegetables directly to children in kindergartens and schools. For such interventions to meet their intended goals, what is important to consider in addition to impact is implementation. Our objective is to systematically review qualitative results reporting on the determinants (barriers and facilitators) to implementation of interventions that entail direct provision of fruit and vegetables in kindergarten and school settings and conduct a framework analysis of those results using the Consolidated Framework for Implementation Research (CFIR). METHODS: A systematic search was designed and run in November 2019 for: MEDLINE (Ovid), Embase (Ovid), PsychINFO (Ovid), ERIC (Ovid), Cochrane Library Reviews and Cochrane Library Trials. A keyword search of the journal Implementation Science was completed. Screening of titles and abstracts (n = 5427) and full texts (n = 227), led to 14 included articles. Coding and analysis were done using the framework method and CFIR. RESULTS: The following CFIR constructs were found relevant: 1) intervention characteristics domain: 'design quality and packaging', 'adaptability' 'cost'; 2) outer setting: 'cosmopolitanism', 'external policy and incentives' 'patients' needs and resources'; 3) inner setting: 'implementation climate', 'readiness for implementation' and 'structural characteristics'; 4) characteristics of individuals: 'individual stage of change', 'knowledge and beliefs about the intervention' 5) process: 'engaging', 'executing' and 'reflecting and evaluating'. The review stresses the dual role of parents as both supporting the implementation and targets of the intervention, which could have implications for the design and implementation of future fruit and vegetables interventions. Positive child perceptions of the value of the intervention and perceived behavior change due to the intervention were reported as relevant facilitators to implementation across several studies, and should be taken into consideration in future design efforts. CONCLUSIONS: CFIR offers a systematic way to identify and organize barriers and facilitators to implementation of interventions in the kindergarten and school setting. Revisions are encouraged to allow adequate space for perceptions of various implementation actors and the target group. PROSPERO REGISTRATION: CRD42020167697.

Frameworks for implementation of policies promoting healthy nutrition and physically active lifestyle: systematic review.

BACKGROUND: Policy frameworks focusing on policy implementation may vary in terms of their scope, included constructs, relationships between the constructs, and context factors. Although multiple policy implementation frameworks exist, the overarching synthesis characterizing differences between the frameworks is missing. This study investigated frameworks guiding implementation of policies aiming at healthy nutrition, physical activity promotion, and a reduction of sedentary behavior. In particular, we aimed at examining the scope of the frameworks and the content of included constructs (e.g., referring to implementation processes, determinants, or implementation evaluation), the level at which these constructs operate (e.g., the individual level, the organizational/community level), relationships between the constructs, and the inclusion of equity factors. METHODS: A systematic review (the PROSPERO registration no. CRD42019133251) was conducted using 9 databases and 8 stakeholder websites. The content of 38 policy implementation frameworks was coded and analyzed. RESULTS: Across the frameworks, 47.4% (18 in 38) addressed three aims: description of the process, determinants, and the evaluation of implementation. The majority of frameworks (65.8%; 25 in 38) accounted for constructs from three levels: individual, organizational/community, and the system level. System-level constructs were included less often (76.3%; 29 in 38) than individual-level or organizational/community-level constructs (86.8% [33 in 38 frameworks] and 94.7% [36 in 38 frameworks] respectively). The majority of frameworks (84.2%, 32 in 38) included at least some sections that were solely of descriptive character (a list of unassociated constructs); 50.0% (19 in 38) included sections of prescriptive character (general steps of implementation); 60.5% (23 in 38) accounted for explanatory sections (assuming bi- or uni-directorial associations). The complex system approach was accounted for only in 21.1% (8 in 38) of frameworks. More than half (55.3%; 21 in 38) of frameworks did not account for any equity constructs (e.g., socioeconomic status, culture). CONCLUSIONS: The majority of policy implementation frameworks have two or three aims (combining processes, determinants and/or the evaluation of implementation), include multi-level constructs (although the system-level determinants are less frequently included than those from the individual- or organizational/community-level), combine sections of purely descriptive character with sections accounting for prescriptive and/or explanatory associations, and are likely to include a little or no equity constructs. REGISTRATION: PROSPERO, #CRD42019133251.

Social, economic, political, and geographical context that counts: meta-review of implementation determinants for policies promoting healthy diet and physical activity.

BACKGROUND: This meta-review investigated the context-related implementation determinants from seven domains (geographical, epidemiological, sociocultural, economic, ethics-related, political, and legal) that were systematically indicated as occurring during the implementation of obesity prevention policies targeting a healthy diet and a physically active lifestyle. METHODS: Data from nine databases and documentation of nine major stakeholders were searched for the purpose of this preregistered meta-review (#CRD42019133341). Context-related determinants were considered strongly supported if they were indicated in ≥60% of the reviews/stakeholder documents. The ROBIS tool and the Methodological Quality Checklist-SP were used to assess the quality-related risk of bias. RESULTS: Published reviews (k = 25) and stakeholder documents that reviewed the evidence of policy implementation (k = 17) were included. Across documents, the following six determinants from three context domains received strong support: economic resources at the macro (66.7% of analyzed documents) and meso/micro levels (71.4%); sociocultural context determinants at the meso/micro level, references to knowledge/beliefs/abilities of target groups (69.0%) and implementers (73.8%); political context determinants (interrelated policies supported in 71.4% of analyzed reviews/documents; policies within organizations, 69.0%). CONCLUSIONS: These findings indicate that sociocultural, economic, and political contexts need to be accounted for when formulating plans for the implementation of a healthy diet and physical activity/sedentary behavior policies.

What we know about the actual implementation process of public physical activity policies: results from a scoping review.

BACKGROUND: Physical inactivity rates have remained high worldwide since 2001. Public policies are an essential upstream lever to target individual physical activity (PA) behaviour. However, implementers have different strategies and face implementation challenges that are poorly understood. The present study analyzes the implementation processes of public policies to promote PA in terms of: (i) the policies covered and their legal quality, (ii) the actors and stakeholders involved in the implementation process and (iii) the used implementation strategies (vertical, horizontal or a mix). METHODS: A scoping review was systematically conducted (registered Open Science Framework: osf.io/7w84q/), searching 10 databases and grey literature until March 2022. Of the 7741 titles and abstracts identified initially, 10 studies were included. RESULTS: The current evidence includes high-income countries (USA, n = 7; UK, New Zealand and Oman, n = 1 each). Policy areas covered are education (school sector) and PA promotion in general (national PA plans or city-wide approaches). The legal classification ranges from laws (school sector) to coordination and budgeting to non-legally binding recommendations. The jurisdictions covered were federal (n = 4), state (n = 1), county (n = 1), school district (n = 1) and city (n = 3). Implementation strategies for city-wide approaches are characterized by a coordinated approach with vertical and horizontal integration; federal PA policies by a mix of implementation strategies; and the school sector by a strict horizontal top-down integration without the involvement of other actors. CONCLUSION: Implementation strategies differ by policy field. Therefore, continuous evaluation of the implementation process is necessary to align policy implementation with policy goals to promote individual PA behaviour.

Sustainable Urban Mobility Plans: implementation process and indicators to evaluate effects on physical activity.

BACKGROUND: Active mobility and public transport increase physical activity (PA) levels. With varying intensity and effectiveness, European cities implement Sustainable Urban Mobility Plans (SUMPs) to spur transport-related PA. Therefore, we aim to examine drivers and barriers to SUMP implementation and assess its influence on PA across European cities. METHODS: We screened policy reports to gain insights into SUMP implementation in one Danish, two German and two Polish cities. Further, we conducted semi-structured interviews with SUMP stakeholders in these cities to explore their experiences with SUMP implementation. Thematic analysis of interview transcripts was applied to identify similarities and differences across cities. To assess the effect of SUMP implementation on PA, we searched for data on indicators of transport-related PA. RESULTS: All investigated cities are committed to sustainable mobility. Nonetheless, complex institutional structures, the dominant role of motorized traffic as well as complex regional and local policy integration hamper SUMP implementation. Danish, German and Polish cities face different contexts in terms of financing, national guidelines and the prominence of sustainability as a policy objective. Each city adopts unique indicators for monitoring the effects of SUMPs on transport-related PA. The variety of indicators and limited data availability impede a comparative evaluation across cities. Constrained by this restriction, we identified motorization rate, modal split and public transport ridership as suitable indicators. CONCLUSIONS: Local idiosyncrasies need to be accounted for when assessing the implementation of SUMPs. Nonetheless, consistent indicators and data transparency are essential for comparing the effectiveness of SUMPs and their impact on PA.

Acceptability of policies targeting dietary behaviours and physical activity: a systematic review of tools and outcomes.

BACKGROUND: Successful implementation of health policies require acceptance from the public and policy-makers. This review aimed to identify tools used to assess the acceptability of policies targeting physical activity and dietary behaviour, and examine if acceptability differs depending on characteristics of the policy and of the respondents. METHODS: A systematic review (PROSPERO: CRD42021232326) was conducted using three databases (Science Direct, PubMed and Web of Science). RESULTS: Of the initial 7780 hits, we included 48 eligible studies (n = 32 on dietary behaviour, n = 11 on physical activity and n = 5 on both), using qualitative and quantitative designs (n = 25 cross-sectional, quantitative; n = 15 qualitative; n = 5 randomized controlled trials; n = 3 mixed-methods design). Acceptability was analysed through online surveys (n = 24), interviews (n = 10), focus groups (n = 10), retrospective textual analysis (n = 3) and a taste-test experiment (n = 1). Notably, only 3 (out of 48) studies applied a theoretical foundation for their assessment. Less intrusive policies such as food labels and policies in a later stage of the implementation process received higher levels of acceptability. Women, older participants and respondents who rated policies as appropriate and effective showed the highest levels of acceptability. CONCLUSION: Highly intrusive policies such as taxations or restrictions are the least accepted when first implemented, but respondents' confidence in the relevance and effectiveness of the policy may boost acceptability over the course of implementation. Studies using validated tools and a theoretical foundation are needed to further examine opportunities to increase acceptability.

Physical activity through place attachment: Understanding perceptions of children and adolescents on urban places by using photovoice and walking interviews.

Public urban places and their environmental characteristics impact youth's physical activity (PA) through perceptions. The objective of this study was to use a qualitative participatory approach with children and adolescents to understand how their attachment to urban places perceived as PA-friendly or unfriendly is related to their PA behaviour. Ninety-three participants aged six to 17 from six neighbourhoods with varying objective walkability engaged in photovoice and walking interviews. Data were analysed by using the tripartite framework of place attachment (PPP model), which was adapted for application to PA behaviour and supplemented by photographs. Themes were identified for each (sub-)dimension of the PPP model with person, place and process factors influencing attachment. Further subdimensions (PA and other behaviours) and categories (travel mode, trip length and frequency of visits) were added to the PPP model. Urban design recommendations were derived by age and gender to promote PA through place attachment.

KIJANI App to Promote Physical Activity in Children and Adolescents: Protocol for a Mixed Method Evaluation.

BACKGROUND: The prevalence of physical inactivity among children and adolescents is alarmingly high despite the well-documented and comprehensive benefits of regular physical activity (PA). Therefore, PA promotion should start early in childhood and adolescence. Although reducing recreational screen time in children and adolescents is an urgent concern, digital approaches have the potential to make activity promotion attractive and age appropriate for the target group. KIJANI is a mobile app approach to promote PA in children and adolescents via gamification and augmented reality. OBJECTIVE: This study protocol aims to describe the KIJANI intervention in detail, as well as the evaluation approach. METHODS: KIJANI is based on the concept that virtual coins can be earned through PA, for example, in the form of a collected step count. With these coins, in turn, blocks can be bought, which can be used to create virtual buildings and integrate them into the player's real-world environment via augmented reality. PA of users is detected via accelerometers integrated into the smartphones. KIJANI can be played at predefined play locations that were comprehensively identified as safe, child-friendly, and attractive for PA by the target group in a partner project. The evaluation process will be divided into 2 different stages. The phase-I evaluation will be a mixed methods approach with one-on-one semistructured interviews and questionnaires to evaluate the user experience and receive feedback from the target group. After the implementation of results and feedback from the target group, the phase-II evaluation will proceed in the form of a 2-arm randomized controlled trial, in which the effectiveness of KIJANI will be assessed via objectively measured PA as well as questionnaires. RESULTS: The study received ethical approval from the ethical board of the Technical University of Munich. Participants for the phase-I evaluation are currently being recruited. CONCLUSIONS: The study will help to determine the efficacy, applicability, and user experience of a gamified activity promotion application in children and adolescents. Overall, digital health approaches provide easy and wide reachability at low cost and are age appropriate and attractive for the target group of adolescents. Strategies have to be developed to apply digital health approaches in the best possible way for activity promotion. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/55156.

Evaluation of the gamified application KIJANI to promote physical activity in children and adolescents: A multimethod study.

Objective: Digital approaches have the potential to make activity promotion attractive and age-appropriate for children and adolescents. KIJANI is a mobile application aiming to increase physical activity (PA) in youth via gamification and augmented reality. This study investigates the user experience with KIJANI through a multimethod approach. Approaches: KIJANI is based on the concept that virtual coins can be earned through PA, for example, in the form of collected step counts. With these coins, blocks can be bought, which can be used to create virtual buildings and landscapes and integrate these into the player's real-world environment via augmented reality. To evaluate the user experience, participants played KIJANI in groups of three for 25 min. Afterwards KIJANI was evaluated qualitatively with one-on-one semi-structured interviews as well as quantitatively with standardized questionnaires. Results: Overall, 22 participants (12.6 ± 1.7 years, 6 girls) were included in the study. The overall game concept and realization were well received by the target group. Study participants did have various and creative ideas for the further development of KIJANI. The majority (n = 16) thought that using KIJANI would increase their PA level. User experience based on the UEQ scale was (mean ± SD): attractiveness (1.78 ± 1.82), perspicuity (2.15 ± 0.680), efficiency (0.67 ± 1.25), dependability, (1.21 ± 0.93), stimulation (1.24 ± 1.78), and novelty (1.27 ± 1.34). Conclusion: With these insights, a further step has been taken in the participatory development of KIJANI. Apps like KIJANI appear to be suitable for PA promotion in children and adolescents.

Barriers and facilitators to the adoption of physical activity policies in elementary schools from the perspective of principals: An application of the consolidated framework for implementation research-A cross-sectional study.

Background: Studies have shown that policies to promote physical activity in schools can have a positive impact on children's physical activity behavior. However, a large research gap exists as to what determinants may influence the adoption of such policies. Applying the Consolidated Framework for Implementation Research (CFIR), we investigated barriers and facilitators to the adoption of physical activity policies in elementary schools in Baden-Wuerttemberg, Germany, from the perspective of school principals. Methods: A cross-sectional study was conducted between May and June 2021. School principals from elementary and special needs schools (n = 2,838) were invited to participate in the study. The online questionnaire used was developed based on the CFIR and included questions on school characteristics and constructs of the CFIR domains inner setting, characteristics of individuals, and process. Logistic regression analyses were performed to examine associations between policy adoption and school characteristics as well as CFIR determinants. Results: In total, 121 schools (4%) participated in the survey, of which 49 (40.5%) reported having adopted a policy to promote physical activity. Positive associations with policy adoption were found for general willingness among teaching staff [odds ratio (OR): 5.37, 95% confidence interval (CI): 1.92-15.05], available resources (OR: 2.15, 95% CI: 1.18-3.91), access to knowledge and information (OR: 2.11, 95% CI: 1.09-4.09), and stakeholder engagement (OR: 3.47, 95% CI: 1.24-9.75). Conclusions: This study provides a first insight into potential barriers and facilitators at the organizational level of schools that may be relevant to the adoption of physical activity policies, from the perspective of school principals. However, due to a low response rate, the results must be interpreted with caution. A strength of this study includes theoretical foundation through the use of the CFIR. The CFIR could be well-adapted to the school setting and provided valuable support for developing the questionnaire and interpreting the study results.

Meta-review of implementation determinants for policies promoting healthy diet and physically active lifestyle: application of the Consolidated Framework for Implementation Research.

BACKGROUND: Although multiple systematic reviews indicate that various determinants (barriers and facilitators) occur in the implementation processes of policies promoting healthy diet, physical activity (PA), and sedentary behavior (SB) reduction, the overarching synthesis of such reviews is missing. Applying the Consolidated Framework for Implementation Research (CFIR), this meta-review aims to (1) identify determinants that were systematically indicated as occurring during the implementation processes and (2) identify differences in the presence of determinants across reviews versus stakeholder documents on healthy diet/PA/SB policies, reviews/stakeholder documents addressing healthy diet policies versus PA/SB policies targeting any population/setting, and healthy diet/PA/SB policies focusing on school settings. METHODS: A meta-review of published systematic scoping or realist reviews (k = 25) and stakeholder documents (k = 17) was conducted. Data from nine bibliographic databases and documentation of nine major stakeholders were systematically searched. Included reviews (72%) and stakeholder documents (100%) provided qualitative synthesis of original research on implementation determinants of policies promoting healthy diet or PA or SB reduction, and 28% of reviews provided some quantitative synthesis. Determinants were considered strongly supported if they were indicated by ≥ 60.0% of included reviews/stakeholder documents. RESULTS: Across the 26 CFIR-based implementation determinants, seven were supported by 66.7-76.2% of reviews/stakeholder documents. These determinants were cost, networking with other organizations/communities, external policies, structural characteristics of the setting, implementation climate, readiness for implementation, and knowledge/beliefs of involved individuals. Most frequently, published reviews provided support for inner setting and individual determinants, whereas stakeholder documents supported outer and inner setting implementation determinants. Comparisons between policies promoting healthy diet with PA/SB policies revealed shared support for only three implementation determinants: cost, implementation climate, and knowledge/beliefs. In the case of healthy diet/PA/SB policies targeting school settings, 14 out of 26 implementation determinants were strongly supported. CONCLUSIONS: The strongly supported (i.e., systematically indicated) determinants may guide policymakers and researchers who need to prioritize potential implementation determinants when planning and monitoring the implementation of respective policies. Future research should quantitatively assess the importance or role of determinants and test investigate associations between determinants and progress of implementation processes. TRIAL REGISTRATION: PROSPERO, # CRD42019133341.