Considering planetary health in health guidelines and health technology assessments: a scoping review protocol.

BACKGROUND: This protocol outlines a scoping review with the objective of identifying and exploring planetary health considerations within existing health guidelines and health technology assessments (HTA). The insights gained from this review will serve as a basis for shaping future Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) guidance on planetary health. METHODS: We will adhere to the JBI methodology for scoping reviews. We will conduct a comprehensive search and screening of results in all languages across various databases including MEDLINE, EMBASE, CINAHL, Global Health, Health Systems Evidence, Greenfile, and Environmental Issues. Additionally, we will supplement this search with resources such as the GIN library, BIGG database, Epistemonikos, GRADE guidelines repository, GRADEpro Guideline Development Tool Database, MAGICapp, NICE website, WHO websites, and a manual exploration of unpublished relevant documents using Google incognito mode. Two independent reviewers will screen and assess the full texts of identified documents according to the eligibility criteria. The following information from each full text will be extracted: document title; first author's name; publication year; language; document type; document as a guideline or HTA; the topic/discipline; document purpose/study objective; developing/sponsoring organization; the country in which the study/guideline/HTA report was conducted; definition of planetary health or related concept provided; types of planetary health experts engaged; study methods; suggested methods to assess planetary health; use of secondary data on planetary health outcomes; description for use of life cycle assessment; description for assessing the quality of life cycle; population/intended audience; interventions; category; applicable planetary health boundaries; consideration of social justice/global equity; phase of intervention in life cycle related to planetary health addressed; the measure of planetary health impact; impact on biodiversity/land use; one health/animal welfare mention; funding; and conflict of interest. Data analysis will involve a combination of descriptive statistics and directed content analysis, with results presented in a narrative format and displayed in tables and graphs. DISCUSSION: The final review results will be submitted to open-access peer-reviewed journals for publication when they become available. The research findings will also be disseminated at relevant planetary health conferences and workshops. SYSTEMATIC REVIEW REGISTRATION: Open Science Framework ( https://osf.io/3jmsa ).

Certainty of Evidence Assessment in Systematic Reviews Published by High-Impact Sports Science Journals: A Meta-epidemiological Study.

BACKGROUND: Assessing certainty of evidence is a key element of any systematic review. The aim of this meta-epidemiology study was to understand the frequency and ways with which certainty of evidence is assessed in contemporary systematic reviews published in high-impact sports science journals. METHODS: We searched PubMed and relevant journal web sites from 1 August 2016 to 11 October 2022 for systematic reviews published in the top-ten highest-impact journals within the 2020 Journal Citation Report for the Sports Sciences category. Pairs of independent reviewers screened items using a priori established criteria. RESULTS: Of 1250 eligible documents, 258 (20.6%) assessed the certainty of evidence, defined as using two or more distinct domains to provide an overall rating of the trustworthiness of findings across studies. Nine methods were cited for assessing certainty, with the most common being the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (61.6%). The proportion of systematic reviews assessing certainty of evidence appeared to increase over the 6-year timeframe analyzed. Across all reviews analyzed, a large majority addressed the domains of risk of bias, imprecision, and inconsistency of the results. Other certainty domains including indirectness/applicability were less commonly assessed. DISCUSSION: Only one in five recent contemporary systematic reviews in the field of exercise and sports science assessed certainty of evidence. Organizational and institutional education on methods for assessing evidence may help further increase uptake of these methods and improve both the quality and clinical impact of systematic reviews in the field.

An Assessment of National Strategic Action Plans for Viral Hepatitis Elimination, 2016-2021.

In 2016, the World Health Organization (WHO) released the Global Health Sector Strategy (GHSS) setting goals for global hepatitis elimination. To inform new or revised viral hepatitis national strategic action plans (NSAPs) for 2022-2030, NSAPs developed during 2016-2021 were assessed for alignment with the WHO GHSS. Country NSAPs were assessed to determine if they included components in the 2016 GHSS. Of 55 country NSAPs, 19 (35%) did not include hepatitis B and C virus elimination goals, only 18 (33%) included targets for needles and syringes for persons who inject drugs, and 21 (38%) had a national budget or financing plan for hepatitis activities. Gaps identified indicate need for technical support in NSAP development.

Improving grading of recommendations assessment, development, and evaluation evidence tables part 4: a three-arm noninferiority randomized trial demonstrates improved understanding of content in summary of findings tables with a new format.

OBJECTIVES: To evaluate alternative formats of summary of findings (SoF) tables for single comparison with multiple outcomes. STUDY DESIGN AND SETTING: We conducted a three-arm randomized controlled noninferiority trial (RCT) in the following systematic review (SR) users: researchers, clinical practice guideline developers, health care providers, policymakers, and knowledge transfer organizations to measure understanding, accessibility, satisfaction, and preference across the current grading of recommendations assessment, development, and evaluation (GRADE) SoF, an alternative GRADE SoF, or an adapted evidence-based practice center (EPC) program SoF table. RESULTS: One Hundred Seventy-Nine participants were randomized, and 129 participants completed the RCT (n = 47 current GRADE, n = 41 alternative GRADE, n = 41 adapted EPC). Understanding the certainty of evidence and treatment effect was comparable across groups. The adapted EPC SoF table was inferior for quantifying risk and RD compared to the alternatives (<35% correct vs. >85% correct). Participants reported increased satisfaction when SoF tables presented number needed to treat (NNT), anticipated absolute effect differences, and narrative syntheses for evidence that could not be meta-analyzed. Participants reported accessibility to information as significantly better in both GRADE SoF tables, when compared with the adapted EPC SoF table. Participants preferred the alternative GRADE SoF table format. CONCLUSION: The alternative GRADE SoF table is a promising format for SR users preferring a comprehensive presentation of SR results for single comparisons.

The effect of occupational exposure to welding fumes on trachea, bronchus and lung cancer: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are the producers of the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). Welding fumes have been classified as carcinogenic to humans (Group 1) by the WHO International Agency for Research on Cancer (IARC) in IARC Monograph 118; this assessment found sufficient evidence from studies in humans that welding fumes are a cause of lung cancer. In this article, we present a systematic review and meta-analysis of parameters for estimating the number of deaths and disability-adjusted life years from trachea, bronchus, and lung cancer attributable to occupational exposure to welding fumes, to inform the development of WHO/ILO Joint Estimates on this burden of disease (if considered feasible). OBJECTIVES: We aimed to systematically review and meta-analyse estimates of the effect of any (or high) occupational exposure to welding fumes, compared with no (or low) occupational exposure to welding fumes, on trachea, bronchus, and lung cancer (three outcomes: prevalence, incidence, and mortality). DATA SOURCES: We developed and published a protocol, applying the Navigation Guide as an organizing systematic review framework where feasible. We searched electronic databases for potentially relevant records from published and unpublished studies, including Medline, EMBASE, Web of Science, CENTRAL and CISDOC. We also searched grey literature databases, Internet search engines, and organizational websites; hand-searched reference lists of previous systematic reviews; and consulted additional experts. STUDY ELIGIBILITY AND CRITERIA: We included working-age (≥15 years) workers in the formal and informal economy in any Member State of WHO and/or ILO but excluded children (<15 years) and unpaid domestic workers. We included randomized controlled trials, cohort studies, case-control studies, and other non-randomized intervention studies with an estimate of the effect of any (or high) occupational exposure to welding fumes, compared with occupational exposure to no (or low) welding fumes, on trachea, bronchus, and lung cancer (prevalence, incidence, and mortality). STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors independently screened titles and abstracts against the eligibility criteria at a first review stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. If studies reported odds ratios, these were converted to risk ratios (RRs). We combined all RRs using random-effects meta-analysis. Two or more review authors assessed the risk of bias, quality of evidence, and strength of evidence, using the Navigation Guide tools and approaches adapted to this project. Subgroup (e.g., by WHO region and sex) and sensitivity analyses (e.g., studies judged to be of "high"/"probably high" risk of bias compared with "low"/"probably low" risk of bias) were conducted. RESULTS: Forty-one records from 40 studies (29 case control studies and 11 cohort studies) met the inclusion criteria, comprising over 1,265,512 participants (≥22,761 females) in 21 countries in three WHO regions (Region of the Americas, European Region, and Western Pacific Region). The exposure and outcome were generally assessed by job title or self-report, and medical or administrative records, respectively. Across included studies, risk of bias was overall generally probably low/low, with risk judged high or probably high for several studies in the domains for misclassification bias and confounding. Our search identified no evidence on the outcome of having trachea, bronchus, and lung cancer (prevalence). Compared with no (or low) occupational exposure to welding fumes, any (or high) occupational exposure to welding fumes increased the risk of acquiring trachea, bronchus, and lung cancer (incidence) by an estimated 48 % (RR 1.48, 95 % confidence interval [CI] 1.29-1.70, 23 studies, 57,931 participants, I2 24 %; moderate quality of evidence). Compared with no (or low) occupational exposure to welding fumes, any (or high) occupational exposure to welding fumes increased the risk dying from trachea, bronchus, and lung cancer (mortality) by an estimated 27 % (RR 1.27, 95 % CI 1.04-1.56, 3 studies, 8,686 participants, I2 0 %; low quality of evidence). Our subgroup analyses found no evidence for difference by WHO region and sex. Sensitivity analyses supported the main analyses. CONCLUSIONS: Overall, for incidence and mortality of trachea, bronchus, and lung cancer, we judged the existing body of evidence for human data as "sufficient evidence of harmfulness" and "limited evidence of harmfulness", respectively. Occupational exposure to welding fumes increased the risk of acquiring and dying from trachea, bronchus, and lung cancer. Producing estimates for the burden of trachea, bronchus, and lung cancer attributable to any (or high) occupational exposure to welding fumes appears evidence-based, and the pooled effect estimates presented in this systematic review could be used as input data for the WHO/ILO Joint Estimates. PROTOCOL IDENTIFIER: https://doi.org/10.1016/j.envint.2020.106089.

Development of an Evidence-Based Risk Assessment Framework.

Assessment of potential human health risks associated with environmental and other agents requires careful evaluation of all available and relevant evidence for the agent of interest, including both data-rich and data-poor agents. With the advent of new approach methodologies in toxicological risk assessment, guidance on integrating evidence from mul-tiple evidence streams is needed to ensure that all available data is given due consideration in both qualitative and quantitative risk assessment. The present report summarizes the discussions among academic, government, and private sector participants from North America and Europe in an international workshop convened to explore the development of an evidence-based risk assessment framework, taking into account all available evidence in an appropriate manner in order to arrive at the best possible characterization of potential human health risks and associated uncertainty. Although consensus among workshop participants was not a specific goal, there was general agreement on the key consider-ations involved in evidence-based risk assessment incorporating 21st century science into human health risk assessment. These considerations have been embodied into an overarching prototype framework for evidence integration that will be explored in more depth in a follow-up meeting.

Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks.

The workshop titled "Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks" was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders' trust for implementation of NAM evidence and AOPs into chemical risk assessment.

Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Quality of Evidence in Studies estimating Prevalence of Exposure to Occupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further. METHODS: Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (Pi) between individual raters and rater teams for the downgrade domains. Pi ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews. RESULTS: We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range Pi: 0.36-1.00) and rater teams (0.20-1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings. CONCLUSIONS: We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area.

Assessor burden, inter-rater agreement and user experience of the RoB-SPEO tool for assessing risk of bias in studies estimating prevalence of exposure to occupational risk factors: An analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: As part of the development of the World Health Organization (WHO)/International Labour Organization (ILO) Joint Estimates of the Work-related Burden of Disease and Injury, WHO and ILO carried out several systematic reviews to determine the prevalence of exposure to selected occupational risk factors. Risk of bias assessment for individual studies is a critical step of a systematic review. No tool existed for assessing the risk of bias in prevalence studies of exposure to occupational risk factors, so WHO and ILO developed and pilot tested the RoB-SPEO tool for this purpose. Here, we investigate the assessor burden, inter-rater agreement, and user experience of this new instrument, based on the abovementioned WHO/ILO systematic reviews. METHODS: Twenty-seven individual experts applied RoB-SPEO to assess risk of bias. Four systematic reviews provided a total of 283 individual assessments, carried out for 137 studies. For each study, two or more assessors independently assessed risk of bias across the eight RoB-SPEO domains selecting one of RoB-SPEO's six ratings (i.e., "low", "probably low", "probably high", "high", "unclear" or "cannot be determined"). Assessors were asked to report time taken (i.e. indicator of assessor burden) to complete each assessment and describe their user experience. To gauge assessor burden, we calculated the median and inter-quartile range of times taken per individual risk of bias assessment. To assess inter-rater reliability, we calculated a raw measure of inter-rater agreement (Pi) for each RoB-SPEO domain, between Pi = 0.00, indicating no agreement and Pi = 1.00, indicating perfect agreement. As subgroup analyses, Pi was also disaggregated by systematic review, assessor experience with RoB-SPEO (≤10 assessments versus > 10 assessments), and assessment time (tertiles: ≤25 min versus 26-66 min versus ≥ 67 min). To describe user experience, we synthesised the assessors' comments and recommendations. RESULTS: Assessors reported a median of 40 min to complete one assessment (interquartile range 21-120 min). For all domains, raw inter-rater agreement ranged from 0.54 to 0.82. Agreement varied by systematic review and assessor experience with RoB-SPEO between domains, and increased with increasing assessment time. A small number of users recommended further development of instructions for selected RoB-SPEO domains, especially bias in selection of participants into the study (domain 1) and bias due to differences in numerator and denominator (domain 7). DISCUSSION: Overall, our results indicated good agreement across the eight domains of the RoB-SPEO tool. The median assessment time was comparable to that of other risk of bias tools, indicating comparable assessor burden. However, there was considerable variation in time taken to complete assessments. Additional time spent on assessments may improve inter-rater agreement. Further development of the RoB-SPEO tool could focus on refining instructions for selected RoB-SPEO domains and additional testing to assess agreement for different topic areas and with a wider range of assessors from different research backgrounds.

An International Needs Assessment Survey of Guideline Developers Demonstrates Variability in Resources and Challenges to Collaboration between Organizations.

BACKGROUND: The development of rigorous, high-quality clinical guidelines increases the need for resources and skilled personnel within guideline-producing organizations. While collaboration between organizations provides a unique opportunity to pool resources and save time and effort, the collaboration presents its own unique challenges. OBJECTIVE: To assess the perceived needs and current challenges of guideline producers worldwide related to guideline development and collaboration efforts. DESIGN: Survey questions were developed by the Guidelines International Network and the US GRADE Network, pilot-tested among attendees of a guideline development workshop, and disseminated electronically using convenience and snowball sampling methods. PARTICIPANTS: A total of 171 respondents representing 30 countries and more than 112 unique organizations were included in this analysis. MAIN MEASURES: The survey included free-response, multiple-choice, and seven-point Likert-scale questions. Questions assessed respondents' perceived value of guidelines, resource availability and needs, guideline development processes, and collaboration efforts of their organization. KEY RESULTS: Time required to develop high-quality systematic reviews and guidelines was the most relevant need (median=7; IQR=5.5-7). In-house resources to conduct literature searches (median=4; IQR=3-6) and the resources to develop rigorous guidelines rapidly (median=4; IQR=2-5) were perceived as the least available resources. Difficulties reconciling differences in guideline methodology (median=6; IQR=4-7) and the time required to establish collaborative agreements (median=6; IQR=5-6) were the most relevant barriers to collaboration between organizations. Results also indicated a general need for improvement in conflict of interest (COI) disclosure policies. CONCLUSION: The survey identified organizational challenges in supporting rigorous guideline development, including the time, resources, and personnel required. Connecting guideline developers to existing databases of high-quality systematic reviews and the use of freely available online platforms may facilitate guideline development. Guideline-producing organizations may also consider allocating resources to hiring or training personnel with expertise in systematic review methodologies or utilizing resources more effectively by establishing collaborations with other organizations.

Systematic review in evidence-based risk assessment.

Systematic reviews provide a structured framework for summarizing the available evidence in a comprehensive, objective, and transparent manner. They inform evidence-based guidelines in medicine, public policy, and more recently, in environmental health and toxicology. Many regulatory agencies have extended and adapted the well-established systematic review methods, initially developed for clinical studies, for their assessment needs. The use of systematic reviews to summarize evidence from existing human, animal, and mechanistic studies can reduce reliance on animal test data in risk assessment and can help avoid unnecessary duplication of animal experiments that have already been conducted. As alternative test methods can be expected to play an increasing role in human health risk assessment in the future, systematic reviews can be particularly helpful in validating these alternatives. The field of evidence-based toxicology has undergone extensive development since its first meeting in 2007 as a result of collaborative efforts among international experts and public health agencies, particularly with respect to the use of mechanistic data and evidence integration. The continued development and wider adoption of systematic review methodology can lead to better 3R implementation. As undertaking a systematic review can be a complex and lengthy process, it is important to understand the main steps involved. Key steps, along with current best practices, are described with references to guidance from organizations with expertise in evidence synthesis. Applications of systematic reviews in clinical, observational, and experimental studies are presented. Finally, software tools available to facilitate and increase the efficiency of completing a systematic review are described.

GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines.

BACKGROUND AND OBJECTIVE: This is the 24th in the ongoing series of articles describing the GRADE approach for assessing the certainty of a body of evidence in systematic reviews and health technology assessments and how to move from evidence to recommendations in guidelines. METHODS: Guideline developers and authors of systematic reviews and other evidence syntheses use randomized controlled studies (RCTs) and non-randomized studies of interventions (NRSI) as sources of evidence for questions about health interventions. RCTs with low risk of bias are the most trustworthy source of evidence for estimating relative effects of interventions because of protection against confounding and other biases. However, in several instances, NRSI can still provide valuable information as complementary, sequential, or replacement evidence for RCTs. RESULTS: In this article we offer guidance on the decision regarding when to search for and include either or both types of studies in systematic reviews to inform health recommendations. CONCLUSION: This work aims to help methodologists in review teams, technology assessors, guideline panelists, and anyone conducting evidence syntheses using GRADE.

Standardisation of Clinical Assessment, Management and Follow-Up of Acute Hospitalised Exacerbation of COPD: A Europe-Wide Consensus.

BACKGROUND: Despite hospitalization for exacerbation being a high-risk event for morbidity and mortality, there is little consensus globally regarding the assessment and management of hospitalised exacerbations of COPD. We aimed to establish a consensus list of symptoms, physiological measures, clinical scores, patient questionnaires and investigations to be obtained at time of hospitalised COPD exacerbation and follow-up. METHODS: A modified Delphi online survey with pre-defined consensus of importance, feasibility and frequency of measures at hospitalisation and follow-up of a COPD exacerbation was undertaken. FINDINGS: A total of 25 COPD experts from 18 countries contributed to all 3 rounds of the survey. Experts agreed that a detailed history and examination were needed. Experts also agreed on which treatments are needed and how soon these should be delivered. Experts recommended that a full blood count, renal function, C-reactive protein and cardiac blood biomarkers (BNP and troponin) should be measured within 4 hours of admission and that the modified Medical Research Council dyspnoea scale (mMRC) and COPD assessment test (CAT) should be performed at time of exacerbation and follow-up. Experts encouraged COPD clinicians to strongly consider discussing palliative care, if indicated, at time of hospitalisation. INTERPRETATION: This Europe-wide consensus document is the first attempt to standardise the assessment and care of patients hospitalised for COPD exacerbations. This should be regarded as the starting point to build knowledge and evidence on patients hospitalised for COPD exacerbations.

The prevalence of occupational exposure to ergonomic risk factors: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates), with contributions from a large network of experts. Evidence from mechanistic and human data suggests that occupational exposure to ergonomic (or physical) risk factors may cause osteoarthritis and other musculoskeletal diseases (excluding rheumatoid arthritis, gout, and back and neck pain). In this paper, we present a systematic review and meta-analysis of the prevalence of occupational exposure to physical ergonomic risk factors for estimating the number of disability-adjusted life years from these diseases that are attributable to exposure to this risk factor, for the development of the WHO/ILO Joint Estimates. OBJECTIVES: We aimed to systematically review and meta-analyse estimates of the prevalence of occupational exposure to ergonomic risk factors for osteoarthritis and other musculoskeletal diseases. DATA SOURCES: We searched electronic bibliographic databases for potentially relevant records from published and unpublished studies, including Ovid Medline, EMBASE, and CISDOC. We also searched electronic grey literature databases, Internet search engines and organizational websites; hand-searched reference list of previous systematic reviews and included study records; and consulted additional experts. STUDY ELIGIBILITY AND CRITERIA: We included working-age (≥15 years) workers in the formal and informal economy in any WHO and/or ILO Member State but excluded children (<15 years) and unpaid domestic workers. The exposure was defined as any occupational exposure to one or more of: force exertion, demanding posture, repetitive movement, hand-arm vibration, kneeling or squatting, lifting, and/or climbing. We included all study types with an estimate of the prevalence of occupational exposure to ergonomic risk factors. STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors independently screened titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. We combined prevalence estimates using random-effect meta-analysis. Two or more review authors assessed the risk of bias and the quality of evidence, using the ROB-SPEO tool and QoE-SPEO approach developed specifically for the WHO/ILO Joint Estimates. RESULTS: Five studies (three cross-sectional studies and two cohort studies) met the inclusion criteria, comprising 150,895 participants (81,613 females) in 36 countries in two WHO regions (Africa, Europe). The exposure was generally assessed with questionnaire data about self-reported exposure. Estimates of the prevalence of occupational exposure to ergonomic risk factors are presented for all five included studies, disaggregated by country, sex, 5-year age group, industrial sector or occupational group where feasible. The pooled prevalence of any occupational exposure to ergonomic risk factors was 0.76 (95% confidence interval 0.69 to 0.84, 3 studies, 148,433 participants, 35 countries in the WHO Europe region, I2 100%, low quality of evidence). Subgroup analyses found no statistically significant differences in exposure by sex but differences by age group, occupation and country. No evidence was found for publication bias. We assessed this body evidence to be of low quality, based on serious concerns for risk of bias due to exposure assessment only being based on self-report and for indirectness due to evidence from two WHO regions only. CONCLUSIONS: Our systematic review and meta-analysis found that occupational exposure to ergonomic risk factors is highly prevalent. The current body of evidence is, however, limited, especially by risk of bias and indirectness. Producing estimates for the burden of disease attributable to occupational exposure to ergonomic risk factors appears evidence-based, and the pooled effect estimates presented in this systematic review may perhaps be used as input data for the WHO/ILO Joint Estimates. Protocol identifier:https://doi.org/10.1016/j.envint.2018.09.053. PROSPERO registration number: CRD42018102631.

The effect of occupational exposure to welding fumes on trachea, bronchus and lung cancer: A protocol for a systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates), with contributions from a large network of experts. Welding fumes have been classified as carcinogenic to humans (Group 1) by the International Agency for Research on Cancer (IARC); this assessment found sufficient evidence from studies in humans that welding fumes are a cause of lung cancer. In this article, we present the protocol for a systematic review of parameters for estimating the number of deaths and disability-adjusted life years from trachea, bronchus and lung cancer attributable to occupational exposure to welding fumes, to inform the development of the WHO/ILO Joint Estimates. OBJECTIVES: We aim to systematically review and meta-analyse estimates of the effect of occupational exposure to welding fumes on trachea, bronchus and lung cancer, applying the Navigation Guide systematic review methodology as an organizing framework. DATA SOURCES: We will search electronic bibliographic databases for potentially relevant records from published and unpublished studies, including Medline, EMBASE, Web of Science, and CISDOC. We will also search electronic grey literature databases, Internet search engines and organizational websites; hand search reference list of previous systematic reviews and included study records; and consult additional experts. STUDY ELIGIBILITY AND CRITERIA: We will include working-age (≥15 years) workers in the formal and informal economy in any Member State of WHO and/or ILO but exclude children (<15 years) and unpaid domestic workers. The eligible risk factor will be occupational exposure to welding fumes, measured directly or indirectly (i.e., through proxy of relevant occupation, work task, job-exposure matrix, expert judgment or self-report). The eligible outcomes will be trachea, bronchus and lung cancer. We will include randomized controlled trials, cohort studies, case-control studies and other non-randomized intervention studies with an estimate of the relative effect of any occupational exposure to welding fumes on the prevalence of, incidence of or mortality from trachea, bronchus and lung cancer, compared with the theoretical minimum risk exposure level of no occupational exposure to welding fumes. STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors will independently screen titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. Two or more review authors will assess risk of bias and the quality of evidence, using the Navigation Guide tool or approach. If feasible, we will combine relative risks using meta-analysis. We will report results using the preferred reporting items for systematic reviews and meta-analyses guidelines (PRISMA).

The effect of exposure to long working hours on ischaemic heart disease: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing Joint Estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates), with contributions from a large network of experts. Evidence from mechanistic data suggests that exposure to long working hours may cause ischaemic heart disease (IHD). In this paper, we present a systematic review and meta-analysis of parameters for estimating the number of deaths and disability-adjusted life years from IHD that are attributable to exposure to long working hours, for the development of the WHO/ILO Joint Estimates. OBJECTIVES: We aimed to systematically review and meta-analyse estimates of the effect of exposure to long working hours (three categories: 41-48, 49-54 and ≥55 h/week), compared with exposure to standard working hours (35-40 h/week), on IHD (three outcomes: prevalence, incidence and mortality). DATA SOURCES: We developed and published a protocol, applying the Navigation Guide as an organizing systematic review framework where feasible. We searched electronic databases for potentially relevant records from published and unpublished studies, including MEDLINE, Scopus, Web of Science, CISDOC, PsycINFO, and WHO ICTRP. We also searched grey literature databases, Internet search engines and organizational websites; hand-searched reference lists of previous systematic reviews; and consulted additional experts. STUDY ELIGIBILITY AND CRITERIA: We included working-age (≥15 years) workers in the formal and informal economy in any WHO and/or ILO Member State but excluded children (aged < 15 years) and unpaid domestic workers. We included randomized controlled trials, cohort studies, case-control studies and other non-randomized intervention studies which contained an estimate of the effect of exposure to long working hours (41-48, 49-54 and ≥55 h/week), compared with exposure to standard working hours (35-40 h/week), on IHD (prevalence, incidence or mortality). STUDY APPRAISAL AND SYNTHESIS METHODS: At least two review authors independently screened titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. Missing data were requested from principal study authors. We combined relative risks using random-effect meta-analysis. Two or more review authors assessed the risk of bias, quality of evidence and strength of evidence, using Navigation Guide and GRADE tools and approaches adapted to this project. RESULTS: Thirty-seven studies (26 prospective cohort studies and 11 case-control studies) met the inclusion criteria, comprising a total of 768,751 participants (310,954 females) in 13 countries in three WHO regions (Americas, Europe and Western Pacific). The exposure was measured using self-reports in all studies, and the outcome was assessed with administrative health records (30 studies) or self-reported physician diagnosis (7 studies). The outcome was defined as incident non-fatal IHD event in 19 studies (8 cohort studies, 11 case-control studies), incident fatal IHD event in two studies (both cohort studies), and incident non-fatal or fatal ("mixed") event in 16 studies (all cohort studies). Because we judged cohort studies to have a relatively lower risk of bias, we prioritized evidence from these studies and treated evidence from case-control studies as supporting evidence. For the bodies of evidence for both outcomes with any eligible studies (i.e. IHD incidence and mortality), we did not have serious concerns for risk of bias (at least for the cohort studies). No eligible study was found on the effect of long working hours on IHD prevalence. Compared with working 35-40 h/week, we are uncertain about the effect on acquiring (or incidence of) IHD of working 41-48 h/week (relative risk (RR) 0.98, 95% confidence interval (CI) 0.91 to 1.07, 20 studies, 312,209 participants, I2 0%, low quality of evidence) and 49-54 h/week (RR 1.05, 95% CI 0.94 to 1.17, 18 studies, 308,405 participants, I2 0%, low quality of evidence). Compared with working 35-40 h/week, working ≥55 h/week may have led to a moderately, clinically meaningful increase in the risk of acquiring IHD, when followed up between one year and 20 years (RR 1.13, 95% CI 1.02 to 1.26, 22 studies, 339,680 participants, I2 5%, moderate quality of evidence). Compared with working 35-40 h/week, we are very uncertain about the effect on dying (mortality) from IHD of working 41-48 h/week (RR 0.99, 95% CI 0.88 to 1.12, 13 studies, 288,278 participants, I2 8%, low quality of evidence) and 49-54 h/week (RR 1.01, 95% CI 0.82 to 1.25, 11 studies, 284,474 participants, I2 13%, low quality of evidence). Compared with working 35-40 h/week, working ≥55 h/week may have led to a moderate, clinically meaningful increase in the risk of dying from IHD when followed up between eight and 30 years (RR 1.17, 95% CI 1.05 to 1.31, 16 studies, 726,803 participants, I2 0%, moderate quality of evidence). Subgroup analyses found no evidence for differences by WHO region and sex, but RRs were higher among persons with lower SES. Sensitivity analyses found no differences by outcome definition (exclusively non-fatal or fatal versus "mixed"), outcome measurement (health records versus self-reports) and risk of bias ("high"/"probably high" ratings in any domain versus "low"/"probably low" in all domains). CONCLUSIONS: We judged the existing bodies of evidence for human evidence as "inadequate evidence for harmfulness" for the exposure categories 41-48 and 49-54 h/week for IHD prevalence, incidence and mortality, and for the exposure category ≥55 h/week for IHD prevalence. Evidence on exposure to working ≥55 h/week was judged as "sufficient evidence of harmfulness" for IHD incidence and mortality. Producing estimates for the burden of IHD attributable to exposure to working ≥55 h/week appears evidence-based, and the pooled effect estimates presented in this systematic review could be used as input data for the WHO/ILO Joint Estimates.

Guideline developers in the United States were inconsistent in applying criteria for appropriate Grading of Recommendations, Assessment, Development and Evaluation use.

OBJECTIVES: We assessed whether guidelines published by organizations based in the United States comply with published criteria for the use of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach. STUDY DESIGN AND SETTING: We performed a cross-sectional study of all clinical practice guidelines that indicated the use of the GRADE approach, were published between 2011 and 2018, and listed in the National Guidelines Clearinghouse. RESULTS: We included 67 guideline documents from 44 of 135 (32.6%) US-based organizations that indicated the use of the GRADE approach. The majority (60/67, 89.6%) of guidelines defined the certainty of evidence consistent with GRADE, but only approximately 1 in 10 (7/67, 10.4%) explicitly reported consideration of all eight criteria to assess the certainty in the evidence for rating down and up. A majority of guidelines (36/67, 53.7%) provided a summary of the evidence, described explicit consideration of all four central domains (36/67, 53.7%), and rated the strength of recommendation consistent with GRADE (36/67, 53.7%). CONCLUSION: Approximately one in three US-based organizations developing evidence-based guidelines report the use of GRADE, but adherence to published criteria is inconsistent. As uptake of the GRADE approach increases in the United States, continued efforts to train guideline methodologists and panel members are important.

Grading of Recommendations Assessment, Development, and Evaluations (GRADE) notes: extremely serious, GRADE's terminology for rating down by three levels.

OBJECTIVES: The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) system for assessing certainty in a body of evidence currently uses two levels, serious and very serious, for downgrading on a single domain. In the context of newer risk of bias instruments, such as Risk of Bias in Non-Randomized Studies I (ROBINS-I), evidence generated by nonrandomized studies may justify rating down by more than two levels on a single domain. Given the importance users of GRADE assign to terminology, our objective was to assess what term GRADE stakeholders would prefer for rating down certainty by three levels. STUDY DESIGN AND SETTING: We conducted a purposefully sampled online survey of GRADE stakeholders to assess possible terms including "critically serious," "extremely serious," "most serious," and "very, very serious" and conducted a descriptive and thematic analysis of responses. We then facilitated a GRADE working group workshop to generate consensus. RESULTS: A total of 225 respondents ranked and rated "extremely serious" highest, closely followed by "critically serious." Respondents felt that "extremely serious" was "more understandable" and "easiest to interpret". GRADE working group members described that the terms "extremely serious" appeared clearer and easier to translate in other languages. CONCLUSION: Based on this stakeholder-driven study, "extremely serious" is the preferred term to rate down certainty of evidence by three levels in the GRADE approach.

Defining ranges for certainty ratings of diagnostic accuracy: a GRADE concept paper.

OBJECTIVE: The objective of the study was to clarify how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept of certainty of evidence applies to certainty ratings of test accuracy. STUDY DESIGN AND SETTING: After initial brainstorming with GRADE Working Group members, we iteratively refined and clarified the approaches for defining ranges when assessing the certainty of evidence for test accuracy within a systematic review, health technology assessment, or guideline. RESULTS: Ranges can be defined both for single test accuracy and for comparative accuracy of multiple tests. For systematic reviews and health technology assessments, approaches for defining ranges include some that do not require value judgments regarding downstream health outcomes. Key challenges arise in the context of a guideline that requires ranges for sensitivity and specificity that are set considering possible effects on all critical outcomes. We illustrate possible approaches and provide an example from a systematic review of a direct comparison between two test strategies. CONCLUSIONS: This GRADE concept paper provides a framework for assessing, presenting, and making decisions based on the certainty of evidence for test accuracy. More empirical research is needed to support future GRADE guidance on how to best operationalize the candidate approaches.

GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence.

OBJECTIVE: To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty rating process. STUDY DESIGN AND SETTING: The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group. RESULTS: We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. CONCLUSIONS: The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.