Assessor burden, inter-rater agreement and user experience of the RoB-SPEO tool for assessing risk of bias in studies estimating prevalence of exposure to occupational risk factors: An analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: As part of the development of the World Health Organization (WHO)/International Labour Organization (ILO) Joint Estimates of the Work-related Burden of Disease and Injury, WHO and ILO carried out several systematic reviews to determine the prevalence of exposure to selected occupational risk factors. Risk of bias assessment for individual studies is a critical step of a systematic review. No tool existed for assessing the risk of bias in prevalence studies of exposure to occupational risk factors, so WHO and ILO developed and pilot tested the RoB-SPEO tool for this purpose. Here, we investigate the assessor burden, inter-rater agreement, and user experience of this new instrument, based on the abovementioned WHO/ILO systematic reviews. METHODS: Twenty-seven individual experts applied RoB-SPEO to assess risk of bias. Four systematic reviews provided a total of 283 individual assessments, carried out for 137 studies. For each study, two or more assessors independently assessed risk of bias across the eight RoB-SPEO domains selecting one of RoB-SPEO's six ratings (i.e., "low", "probably low", "probably high", "high", "unclear" or "cannot be determined"). Assessors were asked to report time taken (i.e. indicator of assessor burden) to complete each assessment and describe their user experience. To gauge assessor burden, we calculated the median and inter-quartile range of times taken per individual risk of bias assessment. To assess inter-rater reliability, we calculated a raw measure of inter-rater agreement (Pi) for each RoB-SPEO domain, between Pi = 0.00, indicating no agreement and Pi = 1.00, indicating perfect agreement. As subgroup analyses, Pi was also disaggregated by systematic review, assessor experience with RoB-SPEO (≤10 assessments versus > 10 assessments), and assessment time (tertiles: ≤25 min versus 26-66 min versus ≥ 67 min). To describe user experience, we synthesised the assessors' comments and recommendations. RESULTS: Assessors reported a median of 40 min to complete one assessment (interquartile range 21-120 min). For all domains, raw inter-rater agreement ranged from 0.54 to 0.82. Agreement varied by systematic review and assessor experience with RoB-SPEO between domains, and increased with increasing assessment time. A small number of users recommended further development of instructions for selected RoB-SPEO domains, especially bias in selection of participants into the study (domain 1) and bias due to differences in numerator and denominator (domain 7). DISCUSSION: Overall, our results indicated good agreement across the eight domains of the RoB-SPEO tool. The median assessment time was comparable to that of other risk of bias tools, indicating comparable assessor burden. However, there was considerable variation in time taken to complete assessments. Additional time spent on assessments may improve inter-rater agreement. Further development of the RoB-SPEO tool could focus on refining instructions for selected RoB-SPEO domains and additional testing to assess agreement for different topic areas and with a wider range of assessors from different research backgrounds.

The effect of exposure to long working hours on depression: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates), supported by a large number of individual experts. Evidence from previous reviews suggests that exposure to long working hours may cause depression. In this article, we present a systematic review and meta-analysis of parameters for estimating (if feasible) the number of deaths and disability-adjusted life years from depression that are attributable to exposure to long working hours, for the development of the WHO/ILO Joint Estimates. OBJECTIVES: We aimed to systematically review and meta-analyse estimates of the effect of exposure to long working hours (three categories: 41-48, 49-54 and ≥55 h/week), compared with exposure to standard working hours (35-40 h/week), on depression (three outcomes: prevalence, incidence and mortality). DATA SOURCES: We developed and published a protocol, applying the Navigation Guide as an organizing systematic review framework where feasible. We searched electronic academic databases for potentially relevant records from published and unpublished studies, including the WHO International Clinical Trial Registers Platform, Medline, PubMed, EMBASE, Web of Science, CISDOC and PsycInfo. We also searched grey literature databases, Internet search engines and organizational websites; hand-searched reference lists of previous systematic reviews; and consulted additional experts. STUDY ELIGIBILITY AND CRITERIA: We included working-age (≥15 years) workers in the formal and informal economy in any WHO and/or ILO Member State but excluded children (aged <15 years) and unpaid domestic workers. We included randomized controlled trials, cohort studies, case-control studies and other non-randomized intervention studies with an estimate of the effect of exposure to long working hours (41-48, 49-54 and ≥55 h/week), compared with exposure to standard working hours (35-40 h/week), on depression (prevalence, incidence and/or mortality). STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors independently screened titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. Missing data were requested from principal study authors. We combined odds ratios using random-effects meta-analysis. Two or more review authors assessed the risk of bias, quality of evidence and strength of evidence, using Navigation Guide and GRADE tools and approaches adapted to this project. RESULTS: Twenty-two studies (all cohort studies) met the inclusion criteria, comprising a total of 109,906 participants (51,324 females) in 32 countries (as one study included multiple countries) in three WHO regions (Americas, Europe and Western Pacific). The exposure was measured using self-reports in all studies, and the outcome was assessed with a clinical diagnostic interview (four studies), interview questions about diagnosis and treatment of depression (three studies) or a validated self-administered rating scale (15 studies). The outcome was defined as incident depression in all 22 studies, with first time incident depression in 21 studies and recurrence of depression in one study. We did not identify any study on prevalence of depression or on mortality from depression. For the body of evidence for the outcome incident depression, we had serious concerns for risk of bias due to selection because of incomplete outcome data (most studies assessed depression only twice, at baseline and at a later follow-up measurement, and likely have missed cases of depression that occurred after baseline but were in remission at the time of the follow-up measurement) and due to missing information on life-time prevalence of depression before baseline measurement. Compared with working 35-40 h/week, we are uncertain about the effect on acquiring (or incidence of) depression of working 41-48 h/week (pooled odds ratio (OR) 1.05, 95% confidence interval (CI) 0.86 to 1.29, 8 studies, 49,392 participants, I2 46%, low quality of evidence); 49-54 h/week (OR 1.06, 95% CI 0.93 to 1.21, 8 studies, 49,392 participants, I2 40%, low quality of evidence); and ≥ 55 h/week (OR 1.08, 95% CI 0.94 to 1.24, 17 studies, 91,142 participants, I2 46%, low quality of evidence). Subgroup analyses found no evidence for statistically significant (P < 0.05) differences by WHO region, sex, age group and socioeconomic status. Sensitivity analyses found no statistically significant differences by outcome measurement (clinical diagnostic interview [gold standard] versus other measures) and risk of bias ("high"/"probably high" ratings in any domain versus "low"/"probably low" in all domains). CONCLUSIONS: We judged the existing bodies of evidence from human data as "inadequate evidence for harmfulness" for all three exposure categories, 41-48, 48-54 and ≥55 h/week, for depression prevalence, incidence and mortality; the available evidence is insufficient to assess effects of the exposure. Producing estimates of the burden of depression attributable to exposure to long working appears not evidence-based at this point. Instead, studies examining the association between long working hours and risk of depression are needed that address the limitations of the current evidence.

Global, regional, and national burdens of ischemic heart disease and stroke attributable to exposure to long working hours for 194 countries, 2000-2016: A systematic analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: World Health Organization (WHO) and International Labour Organization (ILO) systematic reviews reported sufficient evidence for higher risks of ischemic heart disease and stroke amongst people working long hours (≥55 hours/week), compared with people working standard hours (35-40 hours/week). This article presents WHO/ILO Joint Estimates of global, regional, and national exposure to long working hours, for 194 countries, and the attributable burdens of ischemic heart disease and stroke, for 183 countries, by sex and age, for 2000, 2010, and 2016. METHODS AND FINDINGS: We calculated population-attributable fractions from estimates of the population exposed to long working hours and relative risks of exposure on the diseases from the systematic reviews. The exposed population was modelled using data from 2324 cross-sectional surveys and 1742 quarterly survey datasets. Attributable disease burdens were estimated by applying the population-attributable fractions to WHO's Global Health Estimates of total disease burdens. RESULTS: In 2016, 488 million people (95% uncertainty range: 472-503 million), or 8.9% (8.6-9.1) of the global population, were exposed to working long hours (≥55 hours/week). An estimated 745,194 deaths (705,786-784,601) and 23.3 million disability-adjusted life years (22.2-24.4) from ischemic heart disease and stroke combined were attributable to this exposure. The population-attributable fractions for deaths were 3.7% (3.4-4.0) for ischemic heart disease and 6.9% for stroke (6.4-7.5); for disability-adjusted life years they were 5.3% (4.9-5.6) for ischemic heart disease and 9.3% (8.7-9.9) for stroke. CONCLUSIONS: WHO and ILO estimate exposure to long working hours (≥55 hours/week) is common and causes large attributable burdens of ischemic heart disease and stroke. Protecting and promoting occupational and workers' safety and health requires interventions to reduce hazardous long working hours.

WHO/ILO work-related burden of disease and injury: Protocol for systematic reviews of exposure to long working hours and of the effect of exposure to long working hours on depression.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing a joint methodology for estimating the national and global work-related burden of disease and injury (WHO/ILO joint methodology), with contributions from a large network of experts. In this paper, we present the protocol for two systematic reviews of parameters for estimating the number of deaths and disability-adjusted life years from depression attributable to exposure to long working hours, to inform the development of the WHO/ILO joint methodology. OBJECTIVES: We aim to systematically review studies on occupational exposure to long working hours (Systematic Review 1) and systematically review and meta-analyse estimates of the effect of long working hours on depression (Systematic Review 2), applying the Navigation Guide systematic review methodology as an organizing framework, conducting both systematic reviews in tandem and in a harmonized way. DATA SOURCES: Separately for Systematic Reviews 1 and 2, we will search electronic academic databases for potentially relevant records from published and unpublished studies, including Medline, EMBASE, Web of Science, CISDOC and PsycINFO. We will also search electronic grey literature databases, Internet search engines and organizational websites; hand search reference list of previous systematic reviews and included study records; and consult additional experts. STUDY ELIGIBILITY AND CRITERIA: We will include working-age (≥15 years) participants in the formal and informal economy in any WHO and/or ILO Member State, but exclude child workers (<15 years) and unpaid domestic workers. For Systematic Review 1, we will include quantitative prevalence studies of relevant levels of occupational exposure to long working hours (i.e. 35-40, 41-48, 49-54 and ≥55 h/week) stratified by country, sex, age and industrial sector or occupation, in the years 2005-2018. For Systematic Review 2, we will include randomized controlled trials, cohort studies, case-control studies and other non-randomized intervention studies with an estimate of the relative effect of relevant level(s) of long working hours on the incidence of or mortality due to depression, compared with the theoretical minimum risk exposure level (i.e. 35-40 h/week). STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors will independently screen titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. At least two review authors will assess risk of bias and the quality of evidence, using the most suited tools currently available. For Systematic Review 2, if feasible, we will combine relative risks using meta-analysis. We will report results using the guidelines for accurate and transparent health estimates reporting (GATHER) for Systematic Review 1 and the preferred reporting items for systematic reviews and meta-analyses guidelines (PRISMA) for Systematic Review 2. PROSPERO REGISTRATION NUMBER: CRD42018085729.