Causal Assessment of Income Inequality on Self-Rated Health and All-Cause Mortality: A Systematic Review and Meta-Analysis.

Policy Points Income is thought to impact a broad range of health outcomes. However, whether income inequality (how unequal the distribution of income is in a population) has an additional impact on health is extensively debated. Studies that use multilevel data, which have recently increased in popularity, are necessary to separate the contextual effects of income inequality on health from the effects of individual income on health. Our systematic review found only small associations between income inequality and poor self-rated health and all-cause mortality. The available evidence does not suggest causality, although it remains methodologically flawed and limited, with very few studies using natural experimental approaches or examining income inequality at the national level. CONTEXT: Whether income inequality has a direct effect on health or is only associated because of the effect of individual income has long been debated. We aimed to understand the association between income inequality and self-rated health (SRH) and all-cause mortality (mortality) and assess if these relationships are likely to be causal. METHODS: We searched Medline, ISI Web of Science, Embase, and EconLit (PROSPERO: CRD42021252791) for studies considering income inequality and SRH or mortality using multilevel data and adjusting for individual-level socioeconomic position. We calculated pooled odds ratios (ORs) for poor SRH and relative risk ratios (RRs) for mortality from random-effects meta-analyses. We critically appraised included studies using the Risk of Bias in Nonrandomized Studies - of Interventions tool. We assessed certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework and causality using Bradford Hill (BH) viewpoints. FINDINGS: The primary meta-analyses included 2,916,576 participants in 38 cross-sectional studies assessing SRH and 10,727,470 participants in 14 cohort studies of mortality. Per 0.05-unit increase in the Gini coefficient, a measure of income inequality, the ORs and RRs (95% confidence intervals) for SRH and mortality were 1.06 (1.03-1.08) and 1.02 (1.00-1.04), respectively. A total of 63.2% of SRH and 50.0% of mortality studies were at serious risk of bias (RoB), resulting in very low and low certainty ratings, respectively. For SRH and mortality, we did not identify relevant evidence to assess the specificity or, for SRH only, the experiment BH viewpoints; evidence for strength of association and dose-response gradient was inconclusive because of the high RoB; we found evidence in support of temporality and plausibility. CONCLUSIONS: Increased income inequality is only marginally associated with SRH and mortality, but the current evidence base is too methodologically limited to support a causal relationship. To address the gaps we identified, future research should focus on income inequality measured at the national level and addressing confounding with natural experiment approaches.

Evolution of 'whole institution' approaches to improving health in tertiary education settings: a critical scoping review.

In recent decades, 'whole school' approaches to improving health have gained traction, based on settings-based health promotion understandings which view a setting, its actors and processes as an integrated 'whole' system with multiple intervention opportunities. Much less is known about 'whole institution' approaches to improving health in tertiary education settings. We conducted a scoping review to describe both empirical and non-empirical (e.g. websites) publications relating to 'whole settings', 'complex systems' and 'participatory'/'action' approaches to improving the health of students and staff within tertiary education settings. English-language publications were identified by searching five academic and four grey literature databases and via the reference lists of studies read for eligibility. We identified 101 publications with marked UK overrepresentation. Since the 1970s, publications have increased, spanning a gradual shift in focus from 'aspirational' to 'conceptual' to 'evaluative'. Terminology is geographically siloed (e.g., 'healthy university' (UK), 'healthy campus' (USA)). Publications tend to focus on 'health' generally rather than specific health dimensions (e.g. diet). Policies, arguably crucial for cascading systemic change, were not the most frequently implemented intervention elements. We conclude that, despite the field's evolution, key questions (e.g., insights into who needs to do what, with whom, where and when; or efficacy) remain unanswered.

How do income changes impact on mental health and wellbeing for working-age adults? A systematic review and meta-analysis.

BACKGROUND: Lower incomes are associated with poorer mental health and wellbeing, but the extent to which income has a causal effect is debated. We aimed to synthesise evidence from studies measuring the impact of changes in individual and household income on mental health and wellbeing outcomes in working-age adults (aged 16-64 years). METHODS: For this systematic review and meta-analysis, we searched MEDLINE, Embase, Web of Science, PsycINFO, ASSIA, EconLit, and RePEc on Feb 5, 2020, for randomised controlled trials (RCTs) and quantitative non-randomised studies. We had no date limits for our search. We included English-language studies measuring effects of individual or household income change on any mental health or wellbeing outcome. We used Cochrane risk of bias (RoB) tools. We conducted three-level random-effects meta-analyses, and explored heterogeneity using meta-regression and stratified analyses. Synthesis without meta-analysis was based on effect direction. Critical RoB studies were excluded from primary analyses. Certainty of evidence was assessed using Grading of Recommendations Assessment, Development and Evaluation (GRADE). This study is registered with PROSPERO, CRD42020168379. FINDINGS: Of 16 521 citations screened, 136 were narratively synthesised (12·5% RCTs) and 86 meta-analysed. RoB was high: 30·1% were rated critical and 47·1% serious or high. A binary income increase lifting individuals out of poverty was associated with 0·13 SD improvement in mental health measures (95% CI 0·07 to 0·20; n=42 128; 18 studies), considerably larger than other income increases (0·01 SD improvement, 0·002 to 0·019; n=216 509, 14 studies). For wellbeing, increases out of poverty were associated with 0·38 SD improvement (0·09 to 0·66; n=101 350, 8 studies) versus 0·16 for other income increases (0·07 to 0·25; n=62 619, 11 studies). Income decreases from any source were associated with 0·21 SD worsening of mental health measures (-0·30 to -0·13; n=227 804, 11 studies). Effect sizes were larger in low-income and middle-income settings and in higher RoB studies. Heterogeneity was high (I2=79-87%). GRADE certainty was low or very low. INTERPRETATION: Income changes probably impact mental health, particularly where they move individuals out of poverty, although effect sizes are modest and certainty low. Effects are larger for wellbeing outcomes, and potentially for income losses. To best support population mental health, welfare policies need to reach the most socioeconomically disadvantaged. FUNDING: Wellcome Trust, Medical Research Council, Chief Scientist Office, and European Research Council.

Nature-Based Early Childhood Education and Children's Social, Emotional and Cognitive Development: A Mixed-Methods Systematic Review.

This systematic review synthesised evidence on associations between nature-based early childhood education (ECE) and children's social, emotional, and cognitive development. A search of nine databases was concluded in August 2020. Studies were eligible if: (a) children (2-7 years) attended ECE, (b) ECE integrated nature, and (c) assessed child-level outcomes. Two reviewers independently screened full-text articles and assessed study quality. Synthesis included effect direction, thematic analysis, and results-based convergent synthesis. One thousand three hundred and seventy full-text articles were screened, and 36 (26 quantitative; 9 qualitative; 1 mixed-methods) studies were eligible. Quantitative outcomes were cognitive (n = 11), social and emotional (n = 13), nature connectedness (n = 9), and play (n = 10). Studies included controlled (n = 6)/uncontrolled (n = 6) before-after, and cross-sectional (n = 15) designs. Based on very low certainty of the evidence, there were positive associations between nature-based ECE and self-regulation, social skills, social and emotional development, nature relatedness, awareness of nature, and play interaction. Inconsistent associations were found for attention, attachment, initiative, environmentally responsible behaviour, and play disruption/disconnection. Qualitative studies (n = 10) noted that nature-based ECE afforded opportunities for play, socialising, and creativity. Nature-based ECE may improve some childhood development outcomes, however, high-quality experimental designs describing the dose and quality of nature are needed to explore the hypothesised pathways connecting nature-based ECE to childhood development (Systematic Review Registration: CRD42019152582).

Nature-Based Early Childhood Education and Children's Physical Activity, Sedentary Behavior, Motor Competence, and Other Physical Health Outcomes: A Mixed-Methods Systematic Review.

BACKGROUND: The purpose was to synthesize evidence on the association between nature-based Early Childhood Education (ECE) and children's physical activity (PA) and motor competence (MC). METHODS: A literature search of 9 databases was concluded in August 2020. Studies were eligible if (1) children were aged 2-7 years old and attending ECE, (2) ECE settings integrated nature, and (3) assessed physical outcomes. Two reviewers independently screened full-text articles and assessed study quality. Synthesis was conducted using effect direction (quantitative), thematic analysis (qualitative), and combined using a results-based convergent synthesis. RESULTS: 1370 full-text articles were screened and 39 (31 quantitative and 8 qualitative) studies were eligible; 20 quantitative studies assessed PA and 6 assessed MC. Findings indicated inconsistent associations between nature-based ECE and increased moderate to vigorous PA, and improved speed/agility and object control skills. There were positive associations between nature-based ECE and reduced sedentary time and improved balance. From the qualitative analysis, nature-based ECE affords higher intensity PA and risky play, which could improve some MC domains. The quality of 28/31 studies was weak. CONCLUSIONS: More controlled experimental designs that describe the dose and quality of nature are needed to better inform the effectiveness of nature-based ECE on PA and MC.

Causal assessment in evidence synthesis: A methodological review of reviews.

In fields (such as population health) where randomised trials are often lacking, systematic reviews (SRs) can harness diversity in study design, settings and populations to assess the evidence for a putative causal relationship. SRs may incorporate causal assessment approaches (CAAs), sometimes called 'causal reviews', but there is currently no consensus on how these should be conducted. We conducted a methodological review of self-identifying 'causal reviews' within the field of population health to establish: (1) which CAAs are used; (2) differences in how CAAs are implemented; (3) how methods were modified to incorporate causal assessment in SRs. Three databases were searched and two independent reviewers selected reviews for inclusion. Data were extracted using a standardised form and summarised using tabulation and narratively. Fifty-three reviews incorporated CAAs: 46/53 applied Bradford Hill (BH) viewpoints/criteria, with the remainder taking alternative approaches: Medical Research Council guidance on natural experiments (2/53, 3.8%); realist reviews (2/53, 3.8%); horizontal SRs (1/53, 1.9%); 'sign test' of causal mechanisms (1/53, 1.9%); and a causal cascade model (1/53, 1.9%). Though most SRs incorporated BH, there was variation in application and transparency. There was considerable overlap across the CAAs, with a trade-off between breadth (BH viewpoints considered a greater range of causal characteristics) and depth (many alternative CAAs focused on one viewpoint). Improved transparency in the implementation of CAA in SRs in needed to ensure their validity and allow robust assessments of causality within evidence synthesis.

Assessing the causal relationship between income inequality and mortality and self-rated health: protocol for systematic review and meta-analysis.

BACKGROUND: Income inequality has been linked to health and mortality. While there has been extensive research exploring the relationship, the evidence for whether the relationship is causal remains disputed. We describe the methods for a systematic review that will transparently assess whether a causal relationship exists between income inequality and mortality and self-rated health. METHODS: We will identify relevant studies using search terms relating to income inequality, mortality, and self-rated health (SRH). Four databases will be searched: MEDLINE, ISI Web of Science, EMBASE, and the National Bureau of Economic Research. The inclusion criteria have been developed to identify the study designs best suited to assess causality: multilevel studies that have conditioned upon individual income (or a comparable measure, such as socioeconomic position) and natural experiment studies. Risk of bias assessment of included studies will be conducted using ROBINS-I. Where possible, we will convert all measures of income inequality into Gini coefficients and standardize the effect estimate of income inequality on mortality/SRH. We will conduct random-effects meta-analysis to estimate pooled effect estimates when possible. We will assess causality using modified Bradford Hill viewpoints and assess certainty using GRADE. DISCUSSION: This systematic review protocol lays out the complexity of the relationship between income inequality and individual health, as well as our approach for assessing causality. Understanding whether income inequality impacts the health of individuals within a population has major policy implications. By setting out our methods and approach as transparently as we can, we hope this systematic review can provide clarity to an important topic for public policy and public health, as well as acting as an exemplar for other "causal reviews".

Using candidacy theory to explore unemployed over-50s perceptions of suitability of a welfare to work programme: A longitudinal qualitative study.

Welfare to work interventions seek to move out-of-work individuals from claiming unemployment benefits towards paid work. However, previous research has highlighted that for over-50s, particularly those with chronic health conditions, participation in such activities are less likely to result in a return to work. Using longitudinal semi-structured interviews, we followed 26 over-50s during their experience of a mandated welfare to work intervention (the Work Programme) in the United Kingdom. Focusing on their perception of suitability, we utilise and adapt Candidacy Theory to explore how previous experiences of work, health, and interaction with staff (both in the intervention, and with healthcare practitioners) influence these perceptions. Despite many participants acknowledging the benefit of work, many described a pessimism regarding their own ability to return to work in the future, and therefore their lack of suitability for this intervention. This was particularly felt by those with chronic health conditions, who reflected on difficulties with managing their conditions (e.g., attending appointments, adhering to treatment regimens). By adapting Candidacy Theory, we highlighted the ways that mandatory intervention was navigated by all the participants, and how some discussed attempts to remove themselves from this intervention. We also discuss the role played by decision makers such as employment-support staff and healthcare practitioners in supporting or contesting these feelings. Findings suggest that greater effort is required by policy makers to understand the lived experience of chronic illness in terms of ability to RTW, and the importance of inter-agency work in shaping perceptions of those involved.

Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group.

BACKGROUND AND OBJECTIVE: This article explores the need for conceptual advances and practical guidance in the application of the GRADE approach within public health contexts. METHODS: We convened an expert workshop and conducted a scoping review to identify challenges experienced by GRADE users in public health contexts. We developed this concept article through thematic analysis and an iterative process of consultation and discussion conducted with members electronically and at three GRADE Working Group meetings. RESULTS: Five priority issues can pose challenges for public health guideline developers and systematic reviewers when applying GRADE: (1) incorporating the perspectives of diverse stakeholders; (2) selecting and prioritizing health and "nonhealth" outcomes; (3) interpreting outcomes and identifying a threshold for decision-making; (4) assessing certainty of evidence from diverse sources, including nonrandomized studies; and (5) addressing implications for decision makers, including concerns about conditional recommendations. We illustrate these challenges with examples from public health guidelines and systematic reviews, identifying gaps where conceptual advances may facilitate the consistent application or further development of the methodology and provide solutions. CONCLUSION: The GRADE Public Health Group will respond to these challenges with solutions that are coherent with existing guidance and can be consistently implemented across public health decision-making contexts.

Assessing causality in epidemiology: revisiting Bradford Hill to incorporate developments in causal thinking.

The nine Bradford Hill (BH) viewpoints (sometimes referred to as criteria) are commonly used to assess causality within epidemiology. However, causal thinking has since developed, with three of the most prominent approaches implicitly or explicitly building on the potential outcomes framework: directed acyclic graphs (DAGs), sufficient-component cause models (SCC models, also referred to as 'causal pies') and the grading of recommendations, assessment, development and evaluation (GRADE) methodology. This paper explores how these approaches relate to BH's viewpoints and considers implications for improving causal assessment. We mapped the three approaches above against each BH viewpoint. We found overlap across the approaches and BH viewpoints, underscoring BH viewpoints' enduring importance. Mapping the approaches helped elucidate the theoretical underpinning of each viewpoint and articulate the conditions when the viewpoint would be relevant. Our comparisons identified commonality on four viewpoints: strength of association (including analysis of plausible confounding); temporality; plausibility (encoded by DAGs or SCC models to articulate mediation and interaction, respectively); and experiments (including implications of study design on exchangeability). Consistency may be more usefully operationalised by considering an effect size's transportability to a different population or unexplained inconsistency in effect sizes (statistical heterogeneity). Because specificity rarely occurs, falsification exposures or outcomes (i.e., negative controls) may be more useful. The presence of a dose-response relationship may be less than widely perceived as it can easily arise from confounding. We found limited utility for coherence and analogy. This study highlights a need for greater clarity on BH viewpoints to improve causal assessment.

The effect direction plot revisited: Application of the 2019 Cochrane Handbook guidance on alternative synthesis methods.

Effect direction (evidence to indicate improvement, deterioration, or no change in an outcome) can be used as a standardized metric which enables the synthesis of diverse effect measures in systematic reviews. The effect direction (ED) plot was developed to support the synthesis and visualization of effect direction data. Methods for the ED plot require updating in light of new Cochrane guidance on alternative synthesis methods. To update the ED plot, statistical significance was removed from the algorithm for within-study synthesis and use of a sign test was considered to examine whether patterns of ED across studies could be due to chance alone. The revised methods were applied to an existing Cochrane review of the health impacts of housing improvements. The revised ED plot provides a method of data visualization in synthesis without meta-analysis that incorporates information about study characteristics and study quality, using ED as a common metric, without relying on statistical significance to combine outcomes of single studies. The results of sign tests, when appropriate, suggest caution in over-interpreting apparent patterns in effect direction, especially when the number of included studies is small. The revised ED plot meets the need for alternative methods of synthesis and data visualization when meta-analysis is not possible, enabling a transparent link between the data and conclusions of a systematic review. ED plots may be particularly useful in reviews that incorporate nonrandomized studies, complex systems approaches, and diverse sources of evidence, due to the variety of study designs and outcomes in such reviews.

Regression Discontinuity Designs in Health: A Systematic Review.

BACKGROUND: Regression discontinuity designs are non-randomized study designs that permit strong causal inference with relatively weak assumptions. Interest in these designs is growing but there is limited knowledge of the extent of their application in health. We aimed to conduct a comprehensive systematic review of the use of regression discontinuity designs in health research. METHODS: We included studies that used regression discontinuity designs to investigate the physical or mental health outcomes of any interventions or exposures in any populations. We searched 32 health, social science, and gray literature databases (1 January 1960 to 1 January 2019). We critically appraised studies using eight criteria adapted from the What Works Clearinghouse Standards for regression discontinuity designs. We conducted a narrative synthesis, analyzing the forcing variables and threshold rules used in each study. RESULTS: The literature search retrieved 7658 records, producing 325 studies that met the inclusion criteria. A broad range of health topics was represented. The forcing variables used to implement the design were age, socioeconomic measures, date or time of exposure or implementation, environmental measures such as air quality, geographic location, and clinical measures that act as a threshold for treatment. Twelve percent of the studies fully met the eight quality appraisal criteria. Fifteen percent of studies reported a prespecified primary outcome or study protocol. CONCLUSIONS: This systematic review demonstrates that regression discontinuity designs have been widely applied in health research and could be used more widely still. Shortcomings in study quality and reporting suggest that the potential benefits of this method have not yet been fully realized.

Nature-based early childhood education for child health, wellbeing and development: a mixed-methods systematic review protocol.

BACKGROUND: Several systematic reviews have reviewed the evidence relating to nature on aspects of children and adolescent's health and wellbeing; however, none have looked at the associations or effectiveness of attending nature-based early childhood education (ECE). The main objective is to systematically review and synthesise the evidence to determine if nature-based ECE enhances children's health, wellbeing and development. METHODS: We will search the following electronic databases (from inception onwards): MEDLINE, Scopus, PsycINFO, ERIC, SportDiscus, Australian Education Index, British Education Index, Child Development and Adolescent studies, and Applied Social Sciences Index and Abstracts. Grey literature will be identified searching dissertations and reports (e.g. Open Grey, Dissertations Theses Database [ProQuest], and Google Scholar). All types of studies (quantitative and qualitative) conducted in children (aged 2-7 years old) attending ECE who had not started education at primary or elementary school will be included. The exposure of interest will be nature-based ECE settings that integrate nature into their philosophy and/or curriculum and environment. The outcomes of interest will be all aspects of the child's physical, cognitive, social and emotional health wellbeing and development. Two reviewers will independently screen full-text articles. The study methodological quality (or bias) will be appraised using appropriate tools. If feasible, a meta-analysis will be conducted using a random-effect model for studies similar in exposure and outcome. Where studies cannot be included in a meta-analysis, findings will be summarised based on the effect directions and a thematic analysis will be conducted for qualitative studies. DISCUSSION: This systematic review will capture the state of the current literature on nature-based ECE for child health, wellbeing and development. The results of this study will be of interest to multiple audiences (including researchers and policy makers). Results will be published in a peer-reviewed journal. Gaps for future research will be identified and discussed. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42019152582.

Evidence synthesis for constructing directed acyclic graphs (ESC-DAGs): a novel and systematic method for building directed acyclic graphs.

BACKGROUND: Directed acyclic graphs (DAGs) are popular tools for identifying appropriate adjustment strategies for epidemiological analysis. However, a lack of direction on how to build them is problematic. As a solution, we propose using a combination of evidence synthesis strategies and causal inference principles to integrate the DAG-building exercise within the review stages of research projects. We demonstrate this idea by introducing a novel protocol: 'Evidence Synthesis for Constructing Directed Acyclic Graphs' (ESC-DAGs)'. METHODS: ESC-DAGs operates on empirical studies identified by a literature search, ideally a novel systematic review or review of systematic reviews. It involves three key stages: (i) the conclusions of each study are 'mapped' into a DAG; (ii) the causal structures in these DAGs are systematically assessed using several causal inference principles and are corrected accordingly; (iii) the resulting DAGs are then synthesised into one or more 'integrated DAGs'. This demonstration article didactically applies ESC-DAGs to the literature on parental influences on offspring alcohol use during adolescence. CONCLUSIONS: ESC-DAGs is a practical, systematic and transparent approach for developing DAGs from background knowledge. These DAGs can then direct primary data analysis and DAG-based sensitivity analysis. ESC-DAGs has a modular design to allow researchers who are experienced DAG users to both use and improve upon the approach. It is also accessible to researchers with limited experience of DAGs or evidence synthesis.

Role of age and health in perceptions of returning to work: a qualitative study.

BACKGROUND: People aged over 50 years form a growing proportion of the working age population, but are at increased risk of unemployment compared to other age groups. It is often difficult to return to work after unemployment, particularly for those with health issues. In this paper, we explored the perceptions, attitudes, and experiences of returning to work after a period of unemployment (hereafter RTW) barriers among unemployed adults aged over 50 years. METHOD: In-depth semi-structured interviews were conducted with a diverse sample of 26 unemployed individuals aged 50-64 years who were engaged with the UK Government's Work Programme. Data were thematically analysed. RESULTS: Age alone was not discussed by participants as a barrier to work; rather their discussions of barriers to work focused on the ways in which age influenced other issues in their lives. For participants reporting chronic health conditions, or disabilities, there was a concern about being unfit to return to their previous employment area, and therefore having to "start again" in a new career, with associated concerns about their health status and managing their treatment burden. Some participants also reported experiencing either direct or indirect ageism (including related to their health status or need to access healthcare) when looking for work. Other issues facing older people included wider socio-political changes, such as the increased pension age, were felt to be unfair in many ways and contradicted existing expectations of social roles (such as acting as a carer for other family members). CONCLUSION: Over-50s experienced multiple and interacting issues, at both the individual and societal level, that created RTW barriers. There is a need for employability interventions that focus on supporting the over-50s who have fallen out of the labour market to take a holistic approach, working across healthcare, employability and the local labour market, providing treatment and skills training for both those out of work and for employers, in order to create an intervention that that helps achieve RTW and its associated health benefit.

Effects of Housing First approaches on health and well-being of adults who are homeless or at risk of homelessness: systematic review and meta-analysis of randomised controlled trials.

BACKGROUND: Homelessness is associated with poor health. A policy approach aiming to end homelessness across Europe and North America, the 'Housing First' (HF) model, provides rapid housing, not conditional on abstinence from substance use. We aimed to systematically review the evidence from randomised controlled trials for the effects of HF on health and well-being. METHODS: We searched seven databases for randomised controlled trials of interventions providing rapid access to non-abstinence-contingent, permanent housing. We extracted data on the following outcomes: mental health; self-reported health and quality of life; substance use; non-routine use of healthcare services; housing stability. We assessed risk of bias and calculated standardised effect sizes. RESULTS: We included four studies, all with 'high' risk of bias. The impact of HF on most short-term health outcomes was imprecisely estimated, with varying effect directions. No clear difference in substance use was seen. Intervention groups experienced fewer emergency department visits (incidence rate ratio (IRR)=0.63; 95% CI 0.48 to 0.82), fewer hospitalisations (IRR=0.76; 95% CI 0.70 to 0.83) and less time spent hospitalised (standardised mean difference (SMD)=-0.14; 95% CI -0.41 to 0.14) than control groups. In all studies intervention participants spent more days housed (SMD=1.24; 95% CI 0.86 to 1.62) and were more likely to be housed at 18-24 months (risk ratio=2.46; 95% CI 1.58 to 3.84). CONCLUSION: HF approaches successfully improve housing stability and may improve some aspects of health. Implementation of HF would likely reduce homelessness and non-routine health service use without an increase in problematic substance use. Impacts on long-term health outcomes require further investigation. TRIAL REGISTRATION NUMBER: CRD42017064457.

Lack of transparency in reporting narrative synthesis of quantitative data: a methodological assessment of systematic reviews.

OBJECTIVE: To assess the adequacy of reporting and conduct of narrative synthesis of quantitative data (NS) in reviews evaluating the effectiveness of public health interventions. STUDY DESIGN AND SETTING: A retrospective comparison of a 20% (n = 474/2,372) random sample of public health systematic reviews from the McMaster Health Evidence database (January 2010-October 2015) to establish the proportion of reviews using NS. From those reviews using NS, 30% (n = 75/251) were randomly selected and data were extracted for detailed assessment of: reporting NS methods, management and investigation of heterogeneity, transparency of data presentation, and assessment of robustness of the synthesis. RESULTS: Most reviews used NS (56%, n = 251/446); meta-analysis was the primary method of synthesis for 44%. In the detailed assessment of NS, 95% (n = 71/75) did not describe NS methods; 43% (n = 32) did not provide transparent links between the synthesis data and the synthesis reported in the text; of 14 reviews that identified heterogeneity in direction of effect, only one investigated the heterogeneity; and 36% (n = 27) did not reflect on limitations of the synthesis. CONCLUSION: NS methods are rarely reported in systematic reviews of public health interventions and many NS reviews lack transparency in how the data are presented and the conclusions are reached. This threatens the validity of much of the evidence synthesis used to support public health. Improved guidance on reporting and conduct of NS will contribute to improved utility of NS systematic reviews.