Pilot study of a new comprehensive radiology report categorization (RADCAT) system in the emergency department.

PURPOSE: The purpose of this study was to describe a new, broadly applicable radiology report categorization (RADCAT) system that was developed collaboratively between radiologists and emergency department (ED) physicians, and to establish its usability and performance by interobserver variation. METHODS: In collaboration with our ED colleagues, we developed the RADCAT system for all imaging studies performed in our level-1 trauma center, including five categories that span the spectrum of normal through emergent life-threatening findings. During a pilot phase, four radiologists used the system real-time to categorize a minimum of 400 reports in the ED. From this pool of categorized studies, 58 reports were then selected semi-randomly, de-identified, stripped of their original categorization, and recategorized based on the narrative radiology report by 12 individual reviewers (6 radiologists, and 6 ED physicians). Interobserver variation between all reviewers, radiologists only, and ED physicians only was calculated using Cohen's Kappa statistic and Kendall's coefficient of concordance. RESULTS: Altogether, agreement among radiologists and ED physicians was substantial (κ = 0.73, p < 0.0001) and agreement for each category was substantial (all κ > 0.60, p < 0.0001). The lowest agreement was observed with RADCAT-3 (κ > 0.61, p < 0.0001) and the highest agreement with RADCAT-1 (κ > 0.85, p < 0.0001). A high trend in agreement was observed for radiologists and ED physicians and their combination (all W > 0.90, p < 0.0001). CONCLUSIONS: Our RADCAT system is understandable between radiologists and ED physicians for categorizing a wide range of imaging studies, and warrants further assessment and validation. Based upon these pilot results, we plan to adopt this RADCAT scheme and further assess its performance.

The RADCAT-3 system for closing the loop on important non-urgent radiology findings: a multidisciplinary system-wide approach.

The goal of this project was to create a system that was easy for radiologists to use and that could reliably identify, communicate, and track communication of important but non-urgent radiology findings to providers and patients. Prior to 2012, our workflow for communicating important non-urgent diagnostic imaging results was cumbersome, rarely used by our radiologists, and resulted in delays in report turnaround time. In 2012, we developed a new system to communicate important non-urgent findings (the RADiology CATegorization 3 (RADCAT-3) system) that was easy for radiologists to use and documented communication of results in the electronic medical record. To evaluate the performance of the new system, we reviewed our radiology reports before (June 2011-June 2012) and after (June 2012-June 2014) the implementation of the new system to compare utilization by the radiologists and success in communicating these findings. During the 12 months prior to implementation, 250 radiology reports (0.06 % of all reports) entered our workflow for communicating important non-urgent findings. One-hundred percent were successfully communicated. During the 24 months after implementation, 13,158 radiology reports (1.4 % of all reports) entered our new RADCAT-3 workflow (3995 (0.8 % of all reports) during year 1 and 9163 (1.9 % of all reports) during year 2). 99.7 % of those reports were successfully communicated. We created a reliable system to ensure communication of important but non-urgent findings with providers and/or patients and to document that communication in the electronic medical record. The rapid adoption of the new system by radiologists suggests that they found it easy to use and had confidence in its integrity. This system has the potential to improve patient care by improving the likelihood of appropriate follow-up for important non-urgent findings that could become life threatening.

In situ medical simulation investigation of emergency department procedural sedation with randomized trial of experimental bedside clinical process guidance intervention.

INTRODUCTION: Patient safety during emergency department procedural sedation (EDPS) can be difficult to study. Investigators sought to delineate and experimentally assess EDPS performance and safety practices of senior-level emergency medicine residents through in situ simulation. METHODS: Study sessions used 2 pilot-tested EDPS scenarios with critical action checklists, institutional forms, embedded probes, and situational awareness questionnaires. An experimental informatics system was separately developed for bedside EDPS process guidance. Postgraduate year 3 and 4 subjects completed both scenarios in randomized order; only experimental subjects were provided with the experimental system during second scenarios. RESULTS: Twenty-four residents were recruited into a control group (n = 12; 6.2 ± 7.4 live EDPS experience) and experimental group (n = 12; 11.3 ± 8.2 live EDPS experience [P = 0.10]). Critical actions for EDPS medication selection, induction, and adverse event recognition with resuscitation were correctly performed by most subjects. Presedation evaluations, sedation rescue preparation, equipment checks, time-outs, and documentation were frequently missed. Time-outs and postsedation assessments increased during second scenarios in the experimental group. Emergency department procedural sedation safety probe detection did not change across scenarios in either group. Situational awareness scores were 51% ± 7% for control group and 58% ± 12% for experimental group. Subjects using the experimental system completed more time-outs and scored higher Simulation EDPS Safety Composite Scores, although without comprehensive improvements in EDPS practice or safety. CONCLUSIONS: Study simulations delineated EDPS and assessed safety behaviors in senior emergency medicine residents, who exhibited the requisite medical knowledge base and procedural skill set but lacked some nontechnical skills that pertain to emergency department microsystem functions and patient safety. The experimental system exhibited limited impact only on in-simulation time-out compliance.

Use of in situ simulation and human factors engineering to assess and improve emergency department clinical systems for timely telemetry-based detection of life-threatening arrhythmias.

BACKGROUND AND OBJECTIVES: Medical simulation and human factors engineering (HFE) may help investigate and improve clinical telemetry systems. Investigators sought to (1) determine the baseline performance characteristics of an Emergency Department (ED) telemetry system implementation at detecting simulated arrhythmias and (2) improve system performance through HFE-based intervention. METHODS: The prospective study was conducted in a regional referral ED over three 2-week periods from 2010 to 2012. Subjects were clinical providers working at the time of unannounced simulation sessions. Three-minute episodes of sinus bradycardia (SB) and of ventricular tachycardia (VT) were simulated. An experimental HFE-based multi-element intervention was developed to (1) improve system accessibility, (2) increase system relevance and utility for ED clinical practice and (3) establish organisational processes for system maintenance and user base cultivation. The primary outcome variable was overall simulated arrhythmia detection. Pre-intervention system characterisation, post-intervention end-user feedback and real-world correlates of system performance were secondary outcome measures. RESULTS: Baseline HFE assessment revealed limited accessibility, suboptimal usability, poor utility and general neglect of the telemetry system; one simulated VT episode (5%) was detected during 20 pre-intervention sessions. Systems testing during intervention implementation recorded detection of 4 out of 10 arrhythmia simulations (p=0.03). Twenty post-intervention sessions revealed more VT detections (8 of 10) than SB detections (3 of 10) for a 55% overall simulated arrhythmia detection rate (p=0.001). CONCLUSIONS: Experimental investigations helped reveal and mitigate weaknesses in an ED clinical telemetry system implementation. In situ simulation and HFE methodologies can facilitate the assessment and abatement of patient safety hazards in healthcare environments.