Systematic Review Methodology for the Fatigue in Emergency Medical Services Project.

BACKGROUND: Guidance for managing fatigue in the Emergency Medical Services (EMS) setting is limited. The Fatigue in EMS Project sought to complete multiple systematic reviews guided by seven explicit research questions, assemble the best available evidence, and rate the quality of that evidence for purposes of producing an Evidence Based Guideline (EBG) for fatigue risk management in EMS operations. METHODS: We completed seven systematic reviews that involved searches of six databases for literature relevant to seven research questions. These questions were developed a priori by an expert panel and framed in the Population, Intervention, Comparison, and Outcome (PICO) format and pre-registered with PROSPERO. Our target population was defined as persons 18 years of age and older classified as EMS personnel or similar shift worker groups. A panel of experts selected outcomes for each PICO question as prescribed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology. We pooled findings, stratified by study design (experimental vs. observational) and presented results of each systematic review in narrative and quantitative form. We used meta-analyses of select outcomes to generate pooled effects. We used the GRADE methodology and the GRADEpro software to designate a quality of evidence rating for each outcome. RESULTS: We present the results for each systematic review in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). More than 38,000 records were screened across seven systematic reviews. The median, minimum, and maximum inter-rater agreements (Kappa) between screeners for our seven systematic reviews were 0.66, 0.49, and 0.88, respectively. The median, minimum, and maximum number of records retained for the seven systematic reviews was 13, 1, and 100, respectively. We present key findings in GRADE Evidence Profile Tables in separate publications for each systematic review. CONCLUSIONS: We describe a protocol for conducting multiple, simultaneous systematic reviews connected to fatigue with the goal of creating an EBG for fatigue risk management in the EMS setting. Our approach may be informative to others challenged with the creation of EBGs that address multiple, inter-related systematic reviews with overlapping outcomes.

Proposed Performance Measures and Strategies for Implementation of the Fatigue Risk Management Guidelines for Emergency Medical Services.

BACKGROUND: Performance measures are a key component of implementation, dissemination, and evaluation of evidence-based guidelines (EBGs). We developed performance measures for Emergency Medical Services (EMS) stakeholders to enable the implementation of guidelines for fatigue risk management in the EMS setting. METHODS: Panelists associated with the Fatigue in EMS Project, which was supported by the National Highway Traffic Safety Administration (NHTSA), used an iterative process to develop a draft set of performance measures linked to 5 recommendations for fatigue risk management in EMS. We used a cross-sectional survey design and the Content Validity Index (CVI) to quantify agreement among panelists on the wording and content of draft measures. An anonymous web-based tool was used to solicit the panelists' perceptions of clarity and relevance of draft measures. Panelists rated the clarity and relevance separately for each draft measure on a 4-point scale. CVI scores ≥0.78 for clarity and relevance were specified a priori to signify agreement and completion of measurement development. RESULTS: Panelists judged 5 performance measures for fatigue risk management as clear and relevant. These measures address use of fatigue and/or sleepiness survey instruments, optimal duration of shifts, access to caffeine as a fatigue countermeasure, use of napping during shift work, and the delivery of education and training on fatigue risk management for EMS personnel. Panelists complemented performance measures with suggestions for implementation by EMS agencies. CONCLUSIONS: Performance measures for fatigue risk management in the EMS setting will facilitate the implementation and evaluation of the EBG for Fatigue in EMS.

Evidence-Based Guidelines for Fatigue Risk Management in Emergency Medical Services.

BACKGROUND: Administrators of Emergency Medical Services (EMS) operations lack guidance on how to mitigate workplace fatigue, which affects greater than half of all EMS personnel. The primary objective of the Fatigue in EMS Project was to create an evidence-based guideline for fatigue risk management tailored to EMS operations. METHODS: Systematic searches were conducted from 1980 to September 2016 and guided by seven research questions framed in the Population, Intervention, Comparison, Outcome (PICO) framework. Teams of investigators applied inclusion criteria, which included limiting the retained literature to EMS personnel or similar shift worker groups. The expert panel reviewed summaries of the evidence based on the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology. The panel evaluated the quality of evidence for each PICO question separately, considered the balance between benefits and harms, considered the values and preferences of the targeted population, and evaluated the resource requirements/needs. The GRADE Evidence-to-Decision (EtD) Framework was used to prepare draft recommendations based on the evidence, and the Content Validity Index (CVI) was used to quantify the panel's agreement on the relevance and clarity of each recommendation. CVI scores for relevance and clarity were measured separately on a 1-4 scale to indicate consensus/agreement among panel members and conclusion of recommendation development. RESULTS: The EtD framework was applied to all 7 PICO questions, and the panel created 5 recommendations. PICO1: The panel recommends using fatigue/sleepiness survey instruments to measure and monitor fatigue in EMS personnel. PICO2: The panel recommends that EMS personnel work shifts shorter than 24 hours in duration. PICO3: The panel recommends that EMS personnel have access to caffeine as a fatigue countermeasure. PICO4: The panel recommends that, EMS personnel have the opportunity to nap while on duty to mitigate fatigue. PICO5: The panel recommends that EMS personnel receive education and training to mitigate fatigue and fatigue-related risks. The panel referenced insufficient evidence as the reason for making no recommendation linked to 2 PICO questions. CONCLUSIONS: Based on a review of the evidence, the panel developed a guideline with 5 recommendations for fatigue risk management in EMS operations.

Does Implementation of Biomathematical Models Mitigate Fatigue and Fatigue-related Risks in Emergency Medical Services Operations? A Systematic Review.

BACKGROUND: Work schedules like those of Emergency Medical Services (EMS) personnel have been associated with increased risk of fatigue-related impairment. Biomathematical modeling is a means of objectively estimating the potential impacts of fatigue on performance, which may be used in the mitigation of fatigue-related safety risks. In the context of EMS operations, our objective was to assess the evidence in the literature regarding the effectiveness of using biomathematical models to help mitigate fatigue and fatigue-related risks. METHODS: A systematic review of the evidence evaluating the use of biomathematical models to manage fatigue in EMS personnel or similar shift workers was performed. Procedures proposed by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology were used to summarize and rate the certainty in the evidence. Potential bias attached to retained studies was documented using the Cochrane Collaboration's Risk of Bias tool for experimental studies. RESULTS: The literature search strategy, which focused on both EMS personnel and non-EMS shift workers, yielded n = 2,777 unique records. One paper, which investigated non-EMS shift workers, met inclusion criteria. As part of a larger effort, managers and dispatchers of a trucking operation were provided with monthly biomathematical model analyses of predicted fatigue in the driver workforce, and educated on how they could reduce predicted fatigue by means of schedule adjustments. The intervention showed a significant reduction in the number and cost of vehicular accidents during the period in which biomathematical modeling was used. The overall GRADE assessment of evidence quality was very low due to risk of bias, indirectness, imprecision, and publication bias. CONCLUSIONS: This systematic review identified no studies that investigated the impact of biomathematical models in EMS operations. Findings from one study of non-EMS shift workers were favorable toward use of biomathematical models as a fatigue mitigation scheduling aid, albeit with very low quality of evidence pertaining to EMS operations. We propose three focus areas of research priorities that, if addressed, could help better elucidate the utility and impact of biomathematical models as a fatigue-mitigation tool in the EMS environment.

Evidence-Based Guidelines for Fatigue Risk Management in EMS: Formulating Research Questions and Selecting Outcomes.

BACKGROUND: Greater than half of Emergency Medical Services (EMS) personnel report work-related fatigue, yet there are no guidelines for the management of fatigue in EMS. A novel process has been established for evidence-based guideline (EBG) development germane to clinical EMS questions. This process has not yet been applied to operational EMS questions like fatigue risk management. The objective of this study was to develop content valid research questions in the Population, Intervention, Comparison, and Outcome (PICO) framework, and select outcomes to guide systematic reviews and development of EBGs for EMS fatigue risk management. METHODS: We adopted the National Prehospital EBG Model Process and Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework for developing, implementing, and evaluating EBGs in the prehospital care setting. In accordance with steps one and two of the Model Process, we searched for existing EBGs, developed a multi-disciplinary expert panel and received external input. Panelists completed an iterative process to formulate research questions. We used the Content Validity Index (CVI) to score relevance and clarity of candidate PICO questions. The panel completed multiple rounds of question editing and used a CVI benchmark of ≥0.78 to indicate acceptable levels of clarity and relevance. Outcomes for each PICO question were rated from 1 = less important to 9 = critical. RESULTS: Panelists formulated 13 candidate PICO questions, of which 6 were eliminated or merged with other questions. Panelists reached consensus on seven PICO questions (n = 1 diagnosis and n = 6 intervention). Final CVI scores of relevance ranged from 0.81 to 1.00. Final CVI scores of clarity ranged from 0.88 to 1.00. The mean number of outcomes rated as critical, important, and less important by PICO question was 0.7 (SD 0.7), 5.4 (SD 1.4), and 3.6 (SD 1.9), respectively. Patient and personnel safety were rated as critical for most PICO questions. PICO questions and outcomes were registered with PROSPERO, an international database of prospectively registered systematic reviews. CONCLUSIONS: We describe formulating and refining research questions and selection of outcomes to guide systematic reviews germane to EMS fatigue risk management. We outline a protocol for applying the Model Process and GRADE framework to create evidence-based guidelines.