Causal Assessment of Income Inequality on Self-Rated Health and All-Cause Mortality: A Systematic Review and Meta-Analysis.

Policy Points Income is thought to impact a broad range of health outcomes. However, whether income inequality (how unequal the distribution of income is in a population) has an additional impact on health is extensively debated. Studies that use multilevel data, which have recently increased in popularity, are necessary to separate the contextual effects of income inequality on health from the effects of individual income on health. Our systematic review found only small associations between income inequality and poor self-rated health and all-cause mortality. The available evidence does not suggest causality, although it remains methodologically flawed and limited, with very few studies using natural experimental approaches or examining income inequality at the national level. CONTEXT: Whether income inequality has a direct effect on health or is only associated because of the effect of individual income has long been debated. We aimed to understand the association between income inequality and self-rated health (SRH) and all-cause mortality (mortality) and assess if these relationships are likely to be causal. METHODS: We searched Medline, ISI Web of Science, Embase, and EconLit (PROSPERO: CRD42021252791) for studies considering income inequality and SRH or mortality using multilevel data and adjusting for individual-level socioeconomic position. We calculated pooled odds ratios (ORs) for poor SRH and relative risk ratios (RRs) for mortality from random-effects meta-analyses. We critically appraised included studies using the Risk of Bias in Nonrandomized Studies - of Interventions tool. We assessed certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework and causality using Bradford Hill (BH) viewpoints. FINDINGS: The primary meta-analyses included 2,916,576 participants in 38 cross-sectional studies assessing SRH and 10,727,470 participants in 14 cohort studies of mortality. Per 0.05-unit increase in the Gini coefficient, a measure of income inequality, the ORs and RRs (95% confidence intervals) for SRH and mortality were 1.06 (1.03-1.08) and 1.02 (1.00-1.04), respectively. A total of 63.2% of SRH and 50.0% of mortality studies were at serious risk of bias (RoB), resulting in very low and low certainty ratings, respectively. For SRH and mortality, we did not identify relevant evidence to assess the specificity or, for SRH only, the experiment BH viewpoints; evidence for strength of association and dose-response gradient was inconclusive because of the high RoB; we found evidence in support of temporality and plausibility. CONCLUSIONS: Increased income inequality is only marginally associated with SRH and mortality, but the current evidence base is too methodologically limited to support a causal relationship. To address the gaps we identified, future research should focus on income inequality measured at the national level and addressing confounding with natural experiment approaches.

The effect of school summer holidays on inequalities in children and young people's mental health and cognitive ability in the UK using data from the millennium cohort study.

BACKGROUND: Summer learning loss has been the subject of longstanding concern among researchers, the public and policy makers. The aim of the current research was to investigate inequality changes in children's mental health and cognitive ability across the summer holidays. METHODS: We conducted linear and logistic regression analysis of mental health (borderline-abnormal total difficulty and prosocial scores on the strengths and difficulties questionnaire (SDQ)) and verbal cognitive ability (reading, verbal reasoning or vocabulary) at ages 7, 11 and 14, comparing UK Millennium Cohort Study members who were interviewed before and after the school summer holidays. Inequalities were assessed by including interaction terms in the outcome models between a discrete binary variable with values representing time periods and maternal academic qualifications. Coefficients of the interaction terms were interpreted as changes from the pre- to post-holiday period in the extent of inequality in the outcome between participants whose mothers had high or low educational qualifications. Separate models were fitted for each age group and outcome. We used inverse probability weights to allow for differences in the characteristics of cohort members assessed before and after the summer holidays. RESULTS: Mental health (borderline/abnormal SDQ total and prosocial scores) at ages 7 and 14 worsened and verbal cognitive ability scores at age 7 were lower among those surveyed after the summer holidays. Mental health inequalities were larger after the holidays at age 7 ([OR = 1.4; 95%CI (0.6, 3.2) and 14: [OR = 1.5; 95%CI (0.7, 3.2)], but changed little at age 11 (OR = 0.9; 95%CI (0.4, 2.6)]. There were differences in pro-social behaviours among those surveyed before/after the school holidays at age 14 [OR = 1.2; 95%CI (0.5, 3.5)] but not at age 7 or 11. There was little change in inequalities in verbal cognitive ability scores over the school holidays [Age 7: b = 1.3; 95%CI (- 3.3, 6.0); Age 11: b = - 0.7; 95%CI (- 4.3, 2.8); Age 14: b = - 0.3; 95%CI (- 1.0, 0.4)]. CONCLUSION: We found inequalities in mental health and cognitive ability according to maternal education, and some evidence or worsening mental health and mental health inequalities across school summer holidays. We found little evidence of widening inequalities in verbal cognitive ability. Widespread school closures during the COVID-19 restrictions have prompted concerns that prolonged closures may widen health and educational inequalities. Management of school closures should focus on preventing or mitigating inequalities that may arise from differences in the support for mental health and learning provided during closures by schools serving more or less disadvantaged children.

Regression Discontinuity Designs in Health: A Systematic Review.

BACKGROUND: Regression discontinuity designs are non-randomized study designs that permit strong causal inference with relatively weak assumptions. Interest in these designs is growing but there is limited knowledge of the extent of their application in health. We aimed to conduct a comprehensive systematic review of the use of regression discontinuity designs in health research. METHODS: We included studies that used regression discontinuity designs to investigate the physical or mental health outcomes of any interventions or exposures in any populations. We searched 32 health, social science, and gray literature databases (1 January 1960 to 1 January 2019). We critically appraised studies using eight criteria adapted from the What Works Clearinghouse Standards for regression discontinuity designs. We conducted a narrative synthesis, analyzing the forcing variables and threshold rules used in each study. RESULTS: The literature search retrieved 7658 records, producing 325 studies that met the inclusion criteria. A broad range of health topics was represented. The forcing variables used to implement the design were age, socioeconomic measures, date or time of exposure or implementation, environmental measures such as air quality, geographic location, and clinical measures that act as a threshold for treatment. Twelve percent of the studies fully met the eight quality appraisal criteria. Fifteen percent of studies reported a prespecified primary outcome or study protocol. CONCLUSIONS: This systematic review demonstrates that regression discontinuity designs have been widely applied in health research and could be used more widely still. Shortcomings in study quality and reporting suggest that the potential benefits of this method have not yet been fully realized.

Making the most of natural experiments: What can studies of the withdrawal of public health interventions offer?

Many interventions that may have large impacts on health and health inequalities, such as social and public health policies and health system reforms, are not amenable to evaluation using randomised controlled trials. The United Kingdom Medical Research Council's guidance on the evaluation of natural experiments draws attention to the need for ingenuity to identify interventions which can be robustly studied as they occur, and without experimental manipulation. Studies of intervention withdrawal may usefully widen the range of interventions that can be evaluated, allowing some interventions and policies, such as those that have developed piecemeal over a long period, to be evaluated for the first time. In particular, sudden removal may allow a more robust assessment of an intervention's long-term impact by minimising 'learning effects'. Interpreting changes that follow withdrawal as evidence of the impact of an intervention assumes that the effect is reversible and this assumption must be carefully justified. Otherwise, withdrawal-based studies suffer similar threats to validity as intervention studies. These threats should be addressed using recognised approaches, including appropriate choice of comparators, detailed understanding of the change processes at work, careful specification of research questions, and the use of falsification tests and other methods for strengthening causal attribution. Evaluating intervention withdrawal provides opportunities to answer important questions about effectiveness of population health interventions, and to study the social determinants of health. Researchers, policymakers and practitioners should be alert to the opportunities provided by the withdrawal of interventions, but also aware of the pitfalls.

Effectiveness of Mass Media Campaigns to Reduce Alcohol Consumption and Harm: A Systematic Review.

AIMS: To assess the effectiveness of mass media messages to reduce alcohol consumption and related harms using a systematic literature review. METHODS: Eight databases were searched along with reference lists of eligible studies. Studies of any design in any country were included, provided that they evaluated a mass media intervention targeting alcohol consumption or related behavioural, social cognitive or clinical outcomes. Drink driving interventions and college campus campaigns were ineligible. Studies quality were assessed, data were extracted and a narrative synthesis conducted. RESULTS: Searches produced 10,212 results and 24 studies were included in the review. Most campaigns used TV or radio in combination with other media channels were conducted in developed countries and were of weak quality. There was little evidence of reductions in alcohol consumption associated with exposure to campaigns based on 13 studies which measured consumption, although most did not state this as a specific aim of the campaign. There were some increases in treatment seeking and information seeking and mixed evidence of changes in intentions, motivation, beliefs and attitudes about alcohol. Campaigns were associated with increases in knowledge about alcohol consumption, especially where levels had initially been low. Recall of campaigns was high. CONCLUSION: Mass media health campaigns about alcohol are often recalled by individuals, have achieved changes in knowledge, attitudes and beliefs about alcohol but there is little evidence of reductions in alcohol consumption. SHORT SUMMARY: There is little evidence that mass media campaigns have reduced alcohol consumption although most did not state that they aimed to do so. Studies show recall of campaigns is high and that they can have an impact on knowledge, attitudes and beliefs about alcohol consumption.

The impact of participatory budgeting on health and wellbeing: a scoping review of evaluations.

BACKGROUND: Participatory budgeting (PB), citizens deliberating among themselves and with officials to decide how to allocate funds for public goods, has been increasingly implemented across Europe and worldwide. While PB is recommended as good practice by the World Bank and the United Nations, with potential to improve health and wellbeing, it is unclear what evaluations have been conducted on the impact of PB on health and wellbeing. METHODS: For this scoping review, we searched 21 databases with no restrictions on publication date or language. The search term 'participatory budget' was used as the relevant global label for the intervention of interest. Studies were included if they reported original analysis of health, social, political, or economic and budgetary outcomes of PB. We examined the study design, analysis, outcomes and location of included articles. Findings are reported narratively. RESULTS: From 1458 identified references, 37 studies were included. The majority of evaluations (n = 24) were of PB in South America, seven were in Europe. Most evaluations were case studies (n = 23) conducting ethnography and surveys, focussing on political outcomes such as participation in PB or impacts on political activities. All of the quantitative observational studies analysing population level data, except one in Russia, were conducted in South America. CONCLUSION: Despite increasing interest in PB, evaluations applying robust methods to analyse health and wellbeing outcomes are scarce, particularly beyond Brazil. Therefore, implementation of PB schemes should be accompanied by rigorous qualitative and quantitative evaluation to identify impacts and the processes by which they are realised.

The cost of school holidays for children from low income families.

School holidays can be stressful periods for children from low-income families. Poor provision of appropriate childcare, limited access to enrichment activities, and food insecurity mean that children's health and well-being can suffer and their learning stagnate or decline. This article examines and documents the evidence that has emerged on this topic and aims to raise its profile and the impact on children's lives. It makes the case for further academic scrutiny of this unexamined and neglected subject.

Lone parents, health, wellbeing and welfare to work: a systematic review of qualitative studies.

BACKGROUND: Lone parents and their children experience higher than average levels of adverse health and social outcomes, much of which are explained by high rates of poverty. Many high income countries have attempted to address high poverty rates by introducing employment requirements for lone parents in receipt of welfare benefits. However, there is evidence that employment may not reduce poverty or improve the health of lone parents and their children. METHODS: We conducted a systematic review of qualitative studies reporting lone parents' accounts of participation in welfare to work (WtW), to identify explanations and possible mechanisms for the impacts of WtW on health and wellbeing. Twenty one bibliographic databases were searched. Two reviewers independently screened references and assessed study quality. Studies from any high income country that met the criteria of focussing on lone parents, mandatory WtW interventions, and health or wellbeing were included. Thematic synthesis was used to investigate analytic themes between studies. RESULTS: Screening of the 4703 identified papers and quality assessment resulted in the inclusion of 16 qualitative studies of WtW in five high income countries, USA, Canada, UK, Australia, and New Zealand, covering a variety of welfare regimes. Our synthesis found that WtW requirements often conflicted with child care responsibilities. Available employment was often poorly paid and precarious. Adverse health impacts, such as increased stress, fatigue, and depression were commonly reported, though employment and appropriate training was linked to increased self-worth for some. WtW appeared to influence health through the pathways of conflict and control, analytical themes which emerged during synthesis. WtW reduced control over the nature of employment and care of children. Access to social support allowed some lone parents to manage the conflict associated with employment, and to increase control over their circumstances, with potentially beneficial health impacts. CONCLUSION: WtW can result in increased conflict and reduced control, which may lead to negative impacts on mental health. Availability of social support may mediate the negative health impacts of WtW.

Natural experiments for the evaluation of place-based public health interventions: a methodology scoping review.

Introduction: Place-based public health evaluations are increasingly making use of natural experiments. This scoping review aimed to provide an overview of the design and use of natural experiment evaluations (NEEs), and an assessment of the plausibility of the as-if randomization assumption. Methods: A systematic search of three bibliographic databases (Pubmed, Web of Science and Ovid-Medline) was conducted in January 2020 to capture publications that reported a natural experiment of a place-based public health intervention or outcome. For each, study design elements were extracted. An additional evaluation of as-if randomization was conducted by 12 of this paper's authors who evaluated the same set of 20 randomly selected studies and assessed 'as-if ' randomization for each. Results: 366 NEE studies of place-based public health interventions were identified. The most commonly used NEE approach was a Difference-in-Differences study design (25%), followed by before-after studies (23%) and regression analysis studies. 42% of NEEs had likely or probable as-if randomization of exposure (the intervention), while for 25% this was implausible. An inter-rater agreement exercise indicated poor reliability of as-if randomization assignment. Only about half of NEEs reported some form of sensitivity or falsification analysis to support inferences. Conclusion: NEEs are conducted using many different designs and statistical methods and encompass various definitions of a natural experiment, while it is questionable whether all evaluations reported as natural experiments should be considered as such. The likelihood of as-if randomization should be specifically reported, and primary analyses should be supported by sensitivity analyses and/or falsification tests. Transparent reporting of NEE designs and evaluation methods will contribute to the optimum use of place-based NEEs.

Cochrane's risk of bias tool for non-randomized studies (ROBINS-I) is frequently misapplied: A methodological systematic review.

OBJECTIVES: We aimed to review how 'Risk of Bias In Non-randomized Studies-of Interventions' (ROBINS-I), a Cochrane risk of bias assessment tool, has been used in recent systematic reviews. STUDY DESIGN AND SETTING: Database and citation searches were conducted in March 2020 to identify recently published reviews using ROBINS-I. Reported ROBINS-I assessments and data on how ROBINS-I was used were extracted from each review. Methodological quality of reviews was assessed using AMSTAR 2 ('A MeaSurement Tool to Assess systematic Reviews'). RESULTS: Of 181 hits, 124 reviews were included. Risk of bias was serious/critical in 54% of assessments on average, most commonly due to confounding. Quality of reviews was mostly low, and modifications and incorrect use of ROBINS-I were common, with 20% reviews modifying the rating scale, 20% understating overall risk of bias, and 19% including critical-risk of bias studies in evidence synthesis. Poorly conducted reviews were more likely to report low/moderate risk of bias (predicted probability 57% [95% CI: 47-67] in critically low-quality reviews, 31% [19-46] in high/moderate-quality reviews). CONCLUSION: Low-quality reviews frequently apply ROBINS-I incorrectly, and may thus inappropriately include or give too much weight to uncertain evidence. Readers should be aware that such problems can lead to incorrect conclusions in reviews.

ADAPT study: adaptation of evidence-informed complex population health interventions for implementation and/or re-evaluation in new contexts: protocol for a Delphi consensus exercise to develop guidance.

INTRODUCTION: Complex population health interventions that are effective in one context may not be effective elsewhere, and may even be harmful. As such, an intervention may require adaptation to ensure it fits with a new context. To date, there is no overarching guidance to help researchers to adapt and evaluate interventions in new contexts, and no criteria to support research funders or journals assess proposed or reported adaptations or evaluation. There is limited assistance for policy-makers and practitioners to decide if evidence-informed interventions are appropriate to their context, or if adaptation and further evaluation is needed. This Delphi exercise will contribute to the development of guidance for these communities to support the adaptation, implementation and/or re-evaluation of complex population health interventions in new contexts. METHODS: We will conduct a Delphi consensus exercise to gather expert opinion from researchers, research funders, journal editors and policy-makers. Expert opinion will be sought on: appropriate definitions and concepts, identifying key methodological considerations and establishing adaptations and processes to be undertaken during adaptation of complex population health interventions in new contexts. ETHICS AND DISSEMINATION: Ethics approval for the Delphi exercise has been obtained from the University of Glasgow and and the RAND institutional research board. Dissemination of the results of this study will be through peer-reviewed publications, workshops at national and international conferences, and a summary of the guidance developed for key organisations and stakeholders.

Lack of transparency in reporting narrative synthesis of quantitative data: a methodological assessment of systematic reviews.

OBJECTIVE: To assess the adequacy of reporting and conduct of narrative synthesis of quantitative data (NS) in reviews evaluating the effectiveness of public health interventions. STUDY DESIGN AND SETTING: A retrospective comparison of a 20% (n = 474/2,372) random sample of public health systematic reviews from the McMaster Health Evidence database (January 2010-October 2015) to establish the proportion of reviews using NS. From those reviews using NS, 30% (n = 75/251) were randomly selected and data were extracted for detailed assessment of: reporting NS methods, management and investigation of heterogeneity, transparency of data presentation, and assessment of robustness of the synthesis. RESULTS: Most reviews used NS (56%, n = 251/446); meta-analysis was the primary method of synthesis for 44%. In the detailed assessment of NS, 95% (n = 71/75) did not describe NS methods; 43% (n = 32) did not provide transparent links between the synthesis data and the synthesis reported in the text; of 14 reviews that identified heterogeneity in direction of effect, only one investigated the heterogeneity; and 36% (n = 27) did not reflect on limitations of the synthesis. CONCLUSION: NS methods are rarely reported in systematic reviews of public health interventions and many NS reviews lack transparency in how the data are presented and the conclusions are reached. This threatens the validity of much of the evidence synthesis used to support public health. Improved guidance on reporting and conduct of NS will contribute to improved utility of NS systematic reviews.

Applying the ROBINS-I tool to natural experiments: an example from public health.

BACKGROUND: A new tool to assess Risk of Bias In Non-randomised Studies of Interventions (ROBINS-I) was published in Autumn 2016. ROBINS-I uses the Cochrane-approved risk of bias (RoB) approach and focusses on internal validity. As such, ROBINS-I represents an important development for those conducting systematic reviews which include non-randomised studies (NRS), including public health researchers. We aimed to establish the applicability of ROBINS-I using a group of NRS which have evaluated non-clinical public health natural experiments. METHODS: Five researchers, all experienced in critical appraisal of non-randomised studies, used ROBINS-I to independently assess risk of bias in five studies which had assessed the health impacts of a domestic energy efficiency intervention. ROBINS-I assessments for each study were entered into a database and checked for consensus across the group. Group discussions were used to identify reasons underpinning lack of consensus for specific questions and bias domains. RESULTS: ROBINS-I helped to systematically articulate sources of bias in NRS. However, the lack of consensus in assessments for all seven bias domains raised questions about ROBINS-I's reliability and applicability for natural experiment studies. The two RoB domains with least consensus were selection (Domain 2) and performance (Domain 4). Underlying the lack of consensus were difficulties in applying an intention to treat or per protocol effect of interest to the studies. This was linked to difficulties in determining whether the intervention status was classified retrospectively at follow-up, i.e. post hoc. The overall risk of bias ranged from moderate to critical; this was most closely linked to the assessment of confounders. CONCLUSION: The ROBINS-I tool is a conceptually rigorous tool which focusses on risk of bias due to the counterfactual. Difficulties in applying ROBINS-I may be due to poor design and reporting of evaluations of natural experiments. While the quality of reporting may improve in the future, improved guidance on applying ROBINS-I is needed to enable existing evidence from natural experiments to be assessed appropriately and consistently. We hope future refinements to ROBINS-I will address some of the issues raised here to allow wider use of the tool.

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health.

BACKGROUND: Review of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods can be incorporated, have received little discussion in the literature. We recently employed systematic review methods in a review of theories about the causal relationship between income and health. METHODS: This article discusses some of the methodological issues we considered in developing the review and offers lessons learnt from our experiences. It examines the stages of a systematic review in relation to how they could be adapted for a review of theory. The issues arising and the approaches taken in the review of theories in income and health are considered, drawing on the approaches of other reviews of theory. RESULTS: Different approaches to searching were required, including electronic and manual searches, and electronic citation tracking to follow the development of theories. Determining inclusion criteria was an iterative process to ensure that inclusion criteria were specific enough to make the review practical and focused, but not so narrow that key literature was excluded. Involving subject specialists was valuable in the literature searches to ensure principal papers were identified and during the inductive approaches used in synthesis of theories to provide detailed understanding of how theories related to another. Reviews of theory are likely to involve iterations and inductive processes throughout, and some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theoretical reviews of this kind. CONCLUSIONS: It may be useful at the outset of a review of theory to consider whether the key aim of the review is to scope out theories relating to a particular issue; to conduct in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations; or to combine both these processes in the review. This can help decide the most appropriate methodological approach to take at particular stages of the review.