Estimating the impact of nutrition and physical activity policies with quasi-experimental methods and simulation modelling: an integrative review of methods, challenges and synergies.

BACKGROUND: The promotion of healthy lifestyles has high priority on the global public health agenda. Evidence on the real-world (cost-)effectiveness of policies addressing nutrition and physical activity is needed. To estimate short-term policy impacts, quasi-experimental methods using observational data are useful, while simulation models can estimate long-term impacts. We review the methods, challenges and potential synergies of both approaches for the evaluation of nutrition and physical activity policies. METHODS: We performed an integrative review applying purposive literature sampling techniques to synthesize original articles, systematic reviews and lessons learned from public international workshops conducted within the European Union Policy Evaluation Network. RESULTS: We highlight data requirements for policy evaluations, discuss the distinct assumptions of instrumental variable, difference-in-difference, and regression discontinuity designs and describe the necessary robustness and falsification analyses to test them. Further, we summarize the specific assumptions of comparative risk assessment and Markov state-transition simulation models, including their extension to microsimulation. We describe the advantages and limitations of these modelling approaches and discuss future directions, such as the adequate consideration of heterogeneous policy responses. Finally, we highlight how quasi-experimental and simulation modelling methods can be integrated into an evidence cycle for policy evaluation. CONCLUSIONS: Assumptions of quasi-experimental and simulation modelling methods in policy evaluations should be credible, rigorously tested and transparently communicated. Both approaches can be applied synergistically within a coherent framework to compare policy implementation scenarios and improve the estimation of nutrition and physical activity policy impacts, including their distribution across population sub-groups.

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies.

BACKGROUND: Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation. The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior. METHODS: A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as "guidelines," or systematic reviews used to inform policymaking were included. RESULTS: Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence. CONCLUSION: This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.

The impact of the French soda tax on prices and purchases. An ex post evaluation.

We estimate the price and consumption effects of the 2012 French tax on sweetened non-alcoholic drinks using a difference-in-difference approach. Our identification strategy exploits Italian data as a natural control group. We use French and Italian Consumer Price Indices, purchase prices and quantities from the 2011 and 2012 Kantar and GfK home-scan surveys for two French regions and two neighbouring Italian regions, and expenditure data from the 2011 and 2012 Italian Expenditure Survey. We check for the robustness of our results by applying the difference-in-difference models using only French data and considering water as the benchmark (control) good. We find that the tax is transmitted to the prices of taxed drinks, with full transmission for soft drinks and partial transmission for fruit juices. The evidence on purchase responses is mixed and less robust, indicating at most a very small reduction in soft drink purchases (about half a litre per capita per year), an impact which would be consistent with the low tax rate. We find suggestive evidence of a larger response by the sub-sample of heavy purchasers. Fruit juices and water do not seem to have been affected by the tax.

There is no such thing as a (gluten-)free lunch: Higher food prices and the cost for coeliac consumers.

A lifelong gluten-free diet is the only available treatment for coeliac disease at present. However, the high price of gluten-free substitute foods is likely to generate a welfare loss for consumers who drop gluten from their diet. Using original data on retail prices in four major UK supermarkets and consumption data from the UK Living Cost and Food Survey, we simulate the welfare change associated to a switch to the gluten-free diet. Within the "Bread and Cereals" category, retail price data show that the average price of gluten-free products is £1.12/100g relative to £ 0.59/100g of gluten-containing products. Our estimates indicate that on average in the UK coeliac consumers have to pay an extra £ 10 per week to maintain their utility levels prior to the dietary switch. This correspond to 29% of the weekly food budget. Results by income quartile are suggestive of regressive effects and the welfare loss for low-income consumers is estimated at 36% of their food budget compared to 24% of high-income consumers.

Estimation of unobservable selection effects in on-line surveys through propensity score matching: An application to public acceptance of healthy eating policies.

The use of model-based propensity scores as matching tools opens the way to the indirect estimation of mode-related measurement effects and selection effects in web surveys, including a component of selection that cannot be traced back to observable characteristics. By matching and comparing respondents from real independent surveys that use the same questionnaire, but different administration modes, it becomes possible to isolate the selection effect induced by unobservable (or unobserved) respondent characteristics. This study applies a stratification matching algorithm to compare a web survey from a proprietary panel with a computer-assisted telephone survey based on random digit-dialing. The experiment is run in two countries (UK and Italy) to check for consistencies across different cultures and different internet penetration rates. The application to the elicitation of support for healthy eating policies indicates large and significant measurement and selection effects. After controlling for differences in the observed characteristics of respondents and the intensity of internet use, findings suggest that web surveys record lower support and higher neutrality. Similarly, after controlling for administration mode and observed respondent characteristics, internet users are less likely to state support compared to non-users. This suggests that unobserved characteristics play a major role, and post-stratification weighting is not a sufficient countermeasure. As demonstrated by the cross-country comparison, rising internet penetration rates are not a guarantee against this type of error, as disparities in these unobserved characteristics are likely to increase at the same time.

Breaking Habits: The Effect of the French Vending Machine Ban on School Snacking and Sugar Intakes.

This paper estimates the effect of the 2005 vending machine ban in French secondary schools on nutrient intakes and on the frequency of morning snacking at school. Using data before and after the ban, and exploiting the discontinuity associated with the age-dependent exposure to the ban, we specify a difference-in-differences regression discontinuity design. Since the relationship between age-at-interview and school level is not precise, we introduce fuzziness in the model. We find that the ban has generated a 10-gram reduction in sugar intakes from morning snacks at school, and a significant reduction in the frequency of these morning snacks. However, we find no evidence that the intervention affects total daily intakes, and our results are suggestive of compensation effects.

An evaluation of the UK Food Standards Agency's salt campaign.

Excessive salt intake is linked to cardiovascular disease and several other health problems around the world. The UK Food Standards Agency initiated a campaign at the end of 2004 to reduce salt intake in the population. There is disagreement over whether the campaign was effective in curbing salt intake or not. We provide fresh evidence on the impact of the campaign, by using data on spot urinary sodium readings and socio-demographic variables from the Health Survey for England over 2003-2007 and combining it with food price information from the Expenditure and Food Survey. Aggregating the data into a pseudo-panel, we estimate fixed effects models to examine the trend in salt intake over the period and to deduce the heterogeneous effects of the policy on the intake of socio-demographic groups. Our results are consistent with a previous hypothesis that the campaign reduced salt intakes by approximately 10%. The impact is shown to be stronger among women than among men. Older cohorts of men show a larger response to the salt campaign compared to younger cohorts, while among women, younger cohorts respond more strongly than older cohorts.

Policies to promote healthy eating in Europe: a structured review of policies and their effectiveness.

This review provides a classification of public policies to promote healthier eating as well as a structured mapping of existing measures in Europe. Complete coverage of alternative policy types was ensured by complementing the review with a selection of major interventions from outside Europe. Under the auspices of the Seventh Framework Programme's Eatwell Project, funded by the European Commission, researchers from five countries reviewed a representative selection of policy actions based on scientific papers, policy documents, grey literature, government websites, other policy reviews, and interviews with policy-makers. This work resulted in a list of 129 policy interventions, 121 of which were in Europe. For each type of policy, a critical review of its effectiveness was conducted, based on the evidence currently available. The results of this review indicate a need exists for a more systematic and accurate evaluation of government-level interventions as well as for a stronger focus on actual behavioral change rather than changes in attitude or intentions alone. The currently available evidence is very heterogeneous across policy types and is often incomplete.

Assessment of evaluations made to healthy eating policies in Europe: a review within the EATWELL Project.

OBJECTIVE: To identify and assess healthy eating policies at national level which have been evaluated in terms of their impact on awareness of healthy eating, food consumption, health outcome or cost/benefit. DESIGN: Review of policy documents and their evaluations when available. SETTING: European Member States. SUBJECTS: One hundred and twenty-one policy documents revised, 107 retained. RESULTS: Of the 107 selected interventions, twenty-two had been evaluated for their impact on awareness or knowledge and twenty-seven for their impact on consumption. Furthermore sixteen interventions provided an evaluation of health impact, while three actions specifically measured any cost/benefit ratio. The indicators used in these evaluations were in most cases not comparable. Evaluation was more often found for public information campaigns, regulation of meals at schools/canteens and nutrition education programmes. CONCLUSIONS: The study highlights the need not only to develop harmonized and verifiable procedures but also indicators for measuring effectiveness and success and for comparing between interventions and countries. EU policies are recommended to provide a set of indicators that may be measured consistently and regularly in all countries. Furthermore, public information campaigns should be accompanied by other interventions, as evaluations may show an impact on awareness and intention, but rarely on consumption patterns and health outcome.

Five-a-day, a price to pay: an evaluation of the UK program impact accounting for market forces.

We provide an ex-post assessment of the UK 5-a-day information campaign, where the positive effects of information are disentangled from potentially conflicting price dynamics. Using 4 years of data from the Expenditure and Food Survey between 2002 and 2006, we estimate that the 5-a-day program has lifted fruit and vegetable consumption by 0.3 portions, on average. We also provide quantitative evidence of a differentiated impact by income group, ranging from 0.2 to 0.7 portions. All impacts are larger than those observed by simply comparing pre-policy and post-policy intakes.

Policy interventions to promote healthy eating: a review of what works, what does not, and what is promising.

Unhealthy diets can lead to various diseases, which in turn can translate into a bigger burden for the state in the form of health services and lost production. Obesity alone has enormous costs and claims thousands of lives every year. Although diet quality in the European Union has improved across countries, it still falls well short of conformity with the World Health Organization dietary guidelines. In this review, we classify types of policy interventions addressing healthy eating and identify through a literature review what specific policy interventions are better suited to improve diets. Policy interventions are classified into two broad categories: information measures and measures targeting the market environment. Using this classification, we summarize a number of previous systematic reviews, academic papers, and institutional reports and draw some conclusions about their effectiveness. Of the information measures, policy interventions aimed at reducing or banning unhealthy food advertisements generally have had a weak positive effect on improving diets, while public information campaigns have been successful in raising awareness of unhealthy eating but have failed to translate the message into action. Nutritional labeling allows for informed choice. However, informed choice is not necessarily healthier; knowing or being able to read and interpret nutritional labeling on food purchased does not necessarily result in consumption of healthier foods. Interventions targeting the market environment, such as fiscal measures and nutrient, food, and diet standards, are rarer and generally more effective, though more intrusive. Overall, we conclude that measures to support informed choice have a mixed and limited record of success. On the other hand, measures to target the market environment are more intrusive but may be more effective.