A comparative assessment of adverse event classification in the out-of-hospital setting.

OBJECTIVES: We sought to test reliability of two approaches to classify adverse events (AEs) associated with helicopter EMS (HEMS) transport. METHODS: The first approach for AE classification involved flight nurses and paramedics (RN/Medics) and mid-career emergency physicians (MC-EMPs) independently reviewing 50 randomly selected HEMS medical records. The second approach involved RN/Medics and MC-EMPs meeting as a group to openly discuss 20 additional medical records and reach consensus-based AE decision. We compared all AE decisions to a reference criterion based on the decision of three senior emergency physicians (Sr-EMPs). We designed a study to detect an improvement in agreement (reliability) from fair (kappa = 0.2) to moderate (kappa = 0.5). We calculated sensitivity, specificity, percent agreement, and positive and negative predictive values (PPV/NPV). RESULTS: For the independent reviews, the Sr-EMP group identified 26 AEs while individual clinician reviewers identified between 19 and 50 AEs. Agreement on the presence/absence of an AE between Sr-EMPs and three MC-EMPs ranged from κ = 0.20 to κ = 0.25. Agreement between Sr-EMPs and three RN/Medics ranged from κ = 0.11 to κ = 0.19. For the consensus/open-discussion approach, the Sr-EMPs identified 13 AEs, the MC-EMP group identified 18 AEs, and RN/medic group identified 36 AEs. Agreement between Sr-EMPs and MC-EMP group was (κ = 0.30 95%CI -0.12, 0.72), whereas agreement between Sr-EMPs and RN/medic group was (κ = 0.40 95%CI 0.01, 0.79). Agreement between all three groups was fair (κ = 0.33, 95%CI 0.06, 0.66). Percent agreement (58-68%) and NPV (63-76%) was moderately dissimilar between clinicians, while sensitivity (25-80%), specificity (43-97%), and PPV (48-83%) varied. CONCLUSIONS: We identified a higher level of agreement/reliability in AE decisions utilizing a consensus-based approach for review rather than independent reviews.

Measuring adverse events in helicopter emergency medical services: establishing content validity.

INTRODUCTION: We sought to create a valid framework for detecting adverse events (AEs) in the high-risk setting of helicopter emergency medical services (HEMS). METHODS: We assembled a panel of 10 expert clinicians (n = 6 emergency medicine physicians and n = 4 prehospital nurses and flight paramedics) affiliated with a large multistate HEMS organization in the Northeast US. We used a modified Delphi technique to develop a framework for detecting AEs associated with the treatment of critically ill or injured patients. We used a widely applied measure, the content validity index (CVI), to quantify the validity of the framework's content. RESULTS: The expert panel of 10 clinicians reached consensus on a common AE definition and four-step protocol/process for AE detection in HEMS. The consensus-based framework is composed of three main components: (1) a trigger tool, (2) a method for rating proximal cause, and (3) a method for rating AE severity. The CVI findings isolate components of the framework considered content valid. CONCLUSIONS: We demonstrate a standardized process for the development of a content-valid framework for AE detection. The framework is a model for the development of a method for AE identification in other settings, including ground-based EMS.

Identification of adverse events in ground transport emergency medical services.

The purpose of this study was to develop a method to define and rate the severity of adverse events (AEs) in emergency medical services (EMS) safety research. They used a modified Delphi technique to develop a consensus definition of an AE. The consensus definition was as follows: "An adverse event in EMS is a harmful or potentially harmful event occurring during the continuum of EMS care that is potentially preventable and thus independent of the progression of the patient's condition." Physicians reviewed 250 charts from 3 EMS agencies for AEs. The authors examined physician agreement using κ, Fleiss's κ, and corresponding 95% confidence intervals (CIs). Overall physician agreement on presence of an AE per chart was fair (κ = 0.24; 95% CI = 0.19, 0.29). These findings should serve as a basis for refining and implementing an AE evaluation instrument.

Association between poor sleep, fatigue, and safety outcomes in emergency medical services providers.

OBJECTIVE: To determine the association between poor sleep quality, fatigue, and self-reported safety outcomes among emergency medical services (EMS) workers. METHODS: We used convenience sampling of EMS agencies and a cross-sectional survey design. We administered the 19-item Pittsburgh Sleep Quality Index (PSQI), 11-item Chalder Fatigue Questionnaire (CFQ), and 44-item EMS Safety Inventory (EMS-SI) to measure sleep quality, fatigue, and safety outcomes, respectively. We used a consensus process to develop the EMS-SI, which was designed to capture three composite measurements of EMS worker injury, medical errors and adverse events (AEs), and safety-compromising behaviors. We used hierarchical logistic regression to test the association between poor sleep quality, fatigue, and three composite measures of EMS worker safety outcomes. RESULTS: We received 547 surveys from 30 EMS agencies (a 35.6% mean agency response rate). The mean PSQI score exceeded the benchmark for poor sleep (6.9, 95% confidence interval [CI] 6.6, 7.2). More than half of the respondents were classified as fatigued (55%, 95% CI 50.7, 59.3). Eighteen percent of the respondents reported an injury (17.8%, 95% CI 13.5, 22.1), 41% reported a medical error or AE (41.1%, 95% CI 36.8, 45.4), and 90% reported a safety-compromising behavior (89.6%, 95% CI 87, 92). After controlling for confounding, we identified 1.9 greater odds of injury (95% CI 1.1, 3.3), 2.2 greater odds of medical error or AE (95% CI 1.4, 3.3), and 3.6 greater odds of safety-compromising behavior (95% CI 1.5, 8.3) among fatigued respondents versus nonfatigued respondents. CONCLUSIONS: In this sample of EMS workers, poor sleep quality and fatigue are common. We provide preliminary evidence of an association between sleep quality, fatigue, and safety outcomes.

Medical necessity in emergency medical services transports.

The purpose of this study was to generate national estimates of the prevalence of medically unnecessary emergency medical services (EMS) transports to emergency departments (EDs) over time and to identify characteristics that may be associated with medically unnecessary transports. A previously published algorithm was applied to operationalize medical necessity based on ED diagnosis to 10 years of data from the National Hospital Ambulatory Medical Care Survey. The trend over time was reported using descriptive statistics weighted to produce national estimates. Nationally, the proportion of EMS transports that were medically unnecessary increased from 13% to 17% over the 10-year study period. Individual demographic characteristics, including insurance status, were not predictive of inappropriate utilization. EMS transports for medically unnecessary complaints increased from 1997 to 2007. Our findings from a nationally representative sample highlight the opportunity for alternative patient delivery strategies for select patients seeking EMS services.

Variation in emergency medical services workplace safety culture.

INTRODUCTION: Workplace attitude, beliefs, and culture may impact the safety of patient care. This study characterized perceptions of safety culture in a nationwide sample of emergency medical services (EMS) agencies. METHODS: We conducted a cross-sectional survey involving 61 advanced life support EMS agencies in North America. We administered a modified version of the Safety Attitudes Questionnaire (SAQ), a survey instrument measuring dimensions of workplace safety culture (Safety Climate, Teamwork Climate, Perceptions of Management, Job Satisfaction, Working Conditions, and Stress Recognition). We included full-time and part-time paramedics and emergency medical technicians. We determined the variation in safety culture scores across EMS agencies. Using hierarchical linear models, we determined associations between safety culture scores and individual and EMS agency characteristics. RESULTS: We received 1,715 completed surveys from 61 EMS agencies (mean agency response rate 47%; 95% confidence interval [CI] 10%, 83%). There was wide variation in safety culture scores across EMS agencies [mean (minimum, maximum)]: Safety Climate 74.5 (min 49.9, max 89.7), Teamwork Climate 71.2 (min 45.1, max 90.1), Perceptions of Management 67.2 (min 31.1, max 92.2), Job Satisfaction 75.4 (min 47.5, max 93.8), Working Conditions 66.9 (min 36.6, max 91.4), and Stress Recognition 55.1 (min 31.3, max 70.6). Air medical EMS agencies tended to score higher across all safety culture domains. Lower safety culture scores were associated with increased annual patient contacts. Safety Climate domain scores were not associated with other individual or EMS agency characteristics. CONCLUSION: In this sample, workplace safety culture varies between EMS agencies.

Utilization of emergency medical transports and hospital admissions among persons with behavioral health conditions.

OBJECTIVE: Emergency medical services transport and emergency department misuse among persons with behavioral health conditions is a concern. Administrative data were used to examine medical transports and hospital admissions among persons with behavioral health conditions. METHODS: Data on 70,126 medical transports to emergency departments in three southeastern counties were analyzed. RESULTS: Compared with general medical transports, fewer behavioral health transports resulted in a hospital admission. Among behavioral health transports, persons with schizophrenia were 2.62 times more likely than those with substance use disorders to be admitted, and persons with mood disorders were 4.36 times more likely than those with substance use disorders to be admitted. Also, among behavioral health transports, rural transports were less likely than more urban transports to result in a hospital admission. CONCLUSIONS: More training of emergency medical services personnel and more behavioral health crisis resources, especially targeting rural areas and substance use disorders, are needed.