System Configuration Evaluation for a Province-Wide Clinical Information System Using the eSafety Checklist.

BACKGROUND: According to Digital Health Canada 2013 eSafety Guidelines, an estimated one-third of patient safety incidents following implementation of clinical information systems (CISs) are technology-related. An eSafety checklist was previously developed to improve CIS safety by providing a comprehensive listing of system-agnostic, evidence-based configuration recommendations. OBJECTIVES: We sought to use the checklist to support safe initial configuration of a provincial system-wide CIS (Alberta, Canada), referred to as Connect Care. METHODS: The checklist was applied to 13 Connect Care modules in three successive phases. First, the checklist was adapted to an abbreviated high-priority version. Second, demonstrations of each module were recorded. Finally, independent evaluation of each recording was conducted by two eSafety evaluators using the abbreviated eSafety checklist. RESULTS: All modules achieved greater than 72% compliance, with an average of 84%. Overall, 273 opportunities for improvement were identified, with four major areas or themes emerging: (1) inconsistent date and time, (2) unclear patient identification, (3) ineffective alert system, and (4) insufficient decision support. These opportunities were forwarded to the appropriate build teams for review and implementation. CONCLUSION: This work is the first to utilize the eSafety checklist in a real-world CIS, which will become one of the largest in Canada. The checklist has shown clinical applicability in identifying gaps in CIS configuration and should be considered for use in future and pre-existing CISs.

Influenza-like illness-related emergency department visits: Christmas and New Year holiday peaks and relationships with laboratory-confirmed respiratory virus detections, Edmonton, Alberta, 2004-2014.

BACKGROUND: Emergency department (ED) visit volumes can be especially high during the Christmas-New Year holidays, a period occurring during the influenza season in Canada. METHODS: Using daily data, we examined the relationship between ED visits for the chief complaint "cough" (for Edmonton, Alberta residents) and laboratory detections for influenza A and respiratory syncytial virus (RSV) (for Edmonton and surrounding areas), lagged 0-5 days ahead, for non-pandemic years (2004-2008 and 2010-2014) using multivariable linear regression adjusting for temporal variables. We defined these cough-related visits as influenza-like illness (ILI)-related ED visits and, for 2004-2014, compared Christmas-New Year holiday (December 24-January 3) and non-holiday volumes during the influenza season (October-April). RESULTS: Adjusting for temporal variables, ILI-related ED visits were significantly associated with laboratory detections for influenza A and RSV. During non-pandemic years, the highest peak in ILI-related visit volumes always occurred during the holidays. The median number of holiday ILI-related visits/day (42.5) was almost twice the non-holiday median (24) and was even higher in 2012-2013 (80) and 2013-2014 (86). Holiday ILI-related ED visit volumes/100 000 population ranged from 56.0 (2010-2011) to 117.4 (2012-2013). In contrast, lower visit volumes occurred during the holidays of pandemic-affected years (2008-2010). CONCLUSIONS: During non-pandemic years, ILI-related ED visit volumes were associated with variations in detections for influenza A and RSV and always peaked during the Christmas-New Year holidays. This predictability should be used to prepare for, and possibly prevent, this increase in healthcare use; however, interventions beyond disease prevention strategies are likely needed.

Pilot study comparing sepsis management with and without electronic clinical practice guidelines in an academic emergency department.

BACKGROUND: Sepsis is a potentially life-threatening condition that requires urgent management in an Emergency Department (ED). Evidence-based guidelines for managing sepsis have been developed; however, their integration into routine practice is often incomplete. Care maps may help clinicians meet guideline targets more often. OBJECTIVES: To determine if electronic clinical practice guidelines (eCPGs) improve management of patients with severe sepsis and septic shock (SS/SS). METHODS: The impact of an eCPG on the management of patients presenting with SS/SS over a 3-year period at a tertiary care ED was evaluated using retrospective case-control design and chart review methods. Cases and controls, matched by age and sex, were chosen from an electronic database using physician sepsis diagnoses. Data were compared using McNemar tests or paired t-tests, as appropriate. RESULTS: Overall, 51 cases and controls were evaluated; the average age was 62 years, and 60% were male. eCPG patients were more likely to have a central venous pressure and central venous oxygen saturation measured; however, lactate measurement, blood cultures, and other investigations were similarly ordered (all p > 0.05). The administration of antibiotics within 3 h (63% vs. 41%; p = 0.03) and vasopressors (45% vs. 20%; p = 0.02) was more common in the eCPG group; however, use of corticosteroids and other interventions did not differ between the groups. Overall, survival was high and similar between groups. CONCLUSION: A sepsis eCPG experienced variable use; however, physicians using the eCPG achieved more quality-of-care targets for SS/SS. Strategies to increase the utilization of eCPGs in Emergency Medicine seem warranted.

Febrile neutropenia in EDs: the role of an electronic clinical practice guideline.

PURPOSE: Evidence-based clinical practice guidelines (CPGs) for managing febrile neutropenia (FN) are widely available; however, the integration of guidelines into routine practice is often incomplete. This study evaluated the uptake and clinical impact of implementing an electronic CPG on the management and outcomes of patients presenting with FN at 4 urban emergency departments (ED). METHODS: A retrospective chart review over a 3-year period at 4 hospitals in Edmonton, Alberta, was performed. Potentially eligible patient visits were identified by searching the Ambulatory Care Classification System database using International Classification of Diseases, 10th Edition, codes and ED physician diagnoses of FN. ED patients with fever (>38°C at home or in ED) and neutropenia (white blood cell count of <1000 cells/mm(3) or a neutrophil count of <500 cells/mm(3)) who received an ED diagnosis of FN were included. RESULTS: From 371 potential cases, 201 unique cases of FN were included. Overall, the electronic CPG was used in 76 (37.8%) of 201 patient visits; however, there were significant differences in CPG utilization between hospitals. Clinical practice guideline usage was greatest at the University of Alberta Hospital (57%). This finding correlated with a decrease in time from triage to first antibiotic by 1 hour compared to the 3 control hospitals (3.9 vs 4.9 hours, P = .022). CONCLUSIONS: The electronic CPG is a useful clinical tool that can improve patient management in the ED, and strategies to increase its utilization in this and other regions should be pursued.

A multi-function public health surveillance system and the lessons learned in its development: the Alberta Real Time Syndromic Surveillance Net.

OBJECTIVE: We describe a centralized automated multi-function detection and reporting system for public health surveillance--the Alberta Real Time Syndromic Surveillance Net (ARTSSN). This improves upon traditional paper-based systems which are often fragmented, limited by incomplete data collection and inadequate analytical capacity, and incapable of providing timely information for public health action. METHODS: ARTSSN concurrently analyzes multiple electronic data sources in real time to describe results in tables, charts and maps. Detected anomalies are immediately disseminated via alerts to decision-makers for action. RESULTS: ARTSSN provides richly integrated information on a variety of health conditions for early detection of and prompt action on abnormal events such as clusters, outbreaks and trends. Examples of such health conditions include chronic and communicable disease, injury and environment-mediated adverse incidents. DISCUSSION: Key advantages of ARTSSN over traditional paper-based methods are its timeliness, comprehensiveness and automation. Public health surveillance of communicable disease, injury, environmental hazard exposure and chronic disease now occurs in a single system in real time year round. Examples are given to demonstrate the public health value of this system, particularly during Pandemic (H1N1) 2009.

How usability of a web-based clinical decision support system has the potential to contribute to adverse medical events.

INTRODUCTION: Clinical decision support systems (CDSS) have the potential to reduce adverse medical events, but improper design can introduce new forms of error. CDSS pertaining to community acquired pneumonia and neutropenic fever were studied to determine whether usability of the graphical user interface might contribute to potential adverse medical events. METHODS: Automated screen capture of 4 CDSS being used by volunteer emergency physicians was analyzed using structured methods. RESULTS: 422 events were recorded over 56 sessions. In total, 169 negative comments, 55 positive comments, 130 neutral comments, 21 application events, 34 problems, 6 slips, and 5 mistakes were identified. Three mistakes could have had life-threatening consequences. CONCLUSION: Evaluation of CDSS will be of utmost importance in the future with increasing use of electronic health records. Usability engineering principles can identify interface problems that may lead to potential medical adverse events, and should be incorporated early in the software design phase.

Development of a flexible healthcare form builder using Ruby on Rails.

A collaboratively maintained, centralized web-based clinical decision support system (CDSS) repository could lead to improved quality and standardization of care and decrease duplication of resources. Ruby on Rails (RoR) is an open source web application framework that enables rapid iterative development of database backed applications. We have created a prototype form building application in RoR that has the potential to create and maintain such a CDSS repository. Further study and refinement are required.

The effect of training on nurse agreement using an electronic triage system.

OBJECTIVES: Emergency department (ED) triage prioritizes patients based on urgency of care, and the Canadian Triage and Acuity Scale (CTAS) is the national standard. We describe the inter-rater agreement and manual overrides of nurses using a CTAS-compliant web-based triage tool (eTRIAGE) for 2 different intensities of staff training. METHODS: This prospective study was conducted in an urban tertiary care ED. In phase 1, eTRIAGE was deployed after a 3-hour training course for 24 triage nurses who were asked to share this knowledge during regular triage shifts with colleagues who had not received training (n = 77). In phase 2, a targeted group of 8 triage nurses underwent further training with eTRIAGE. In each phase, patients were assessed first by the duty triage nurse and then by a blinded independent study nurse, both using eTRIAGE. Inter-rater agreement was calculated using kappa (weighted kappa) statistics. RESULTS: In phase 1, 569 patients were enrolled with 513 (90.2%) complete records; 577 patients were enrolled in phase 2 with 555 (96.2%) complete records. Inter-rater agreement during phase 1 was moderate (weighted kappa = 0.55; 95% confidence interval [CI] 0.49-0.62); agreement improved in phase 2 (weighted kappa = 0.65; 95% CI 0.60-0.70). Manual overrides of eTRIAGE scores were infrequent (approximately 10%) during both periods. CONCLUSIONS: Agreement between study nurses and duty triage nurses, both using eTRIAGE, was moderate to good, with a trend toward improvement with additional training. Triage overrides were infrequent. Continued attempts to refine the triage process and training appear warranted.

Predictive validity of a computerized emergency triage tool.

BACKGROUND: Emergency department (ED) triage prioritizes patients on the basis of the urgency of need for care. eTRIAGE is a Web-based triage decision support tool that is based on the Canadian Triage and Acuity Scale (CTAS), a five level triage system (CTAS 1 = resuscitation, CTAS 5 = nonurgent). OBJECTIVES: To examine the validity of eTRIAGE on the basis of resource utilization and cost as measures of acuity. METHODS: Scores on the CTAS, specialist consultations, computed-tomography use, ED length of stay, ED disposition, and estimated ED and hospital costs (if the patient was subsequently admitted to hospital) were collected for each patient over a six month period. These data were queried from a database that captures all regional ED visits. Correlations between CTAS score and each outcome were measured by using logistic regression models (categorical variables), univariate analysis of variance (continuous variables), and the Kruskal-Wallis analysis of variance (costs). A multivariate regression model that used cost as the outcome was used to identify interaction between the variables presented. RESULTS: Over the six month study, 29,524 patients were triaged by using eTRIAGE. When compared with CTAS level 3, the odds ratios for consultation, CT scan, and admission were significantly higher in CTAS 1 and 2 and were significantly lower in CTAS 4 and 5 (p < 0.001). When compared with CTAS levels 2-5 combined, the odds ratio for death in CTAS 1 was 664.18 (p < 0.001). The length of stay also demonstrated significant correlation with CTAS score (p < 0.001). Costs to the ED and hospital also correlated significantly with increasing acuity (median costs for CTAS levels in Canadian dollars: CTAS 1 = 2,690 dollars, CTAS 2 = 433 dollars, CTAS 3 = 288 dollars, CTAS 4 = 164 dollars, CTAS 5 = 139 dollars, and p < 0.001). Significant interactions between the data collected were found in a multivariate regression model, although CTAS score remained highly associated with costs. CONCLUSIONS: Acuity measured by eTRIAGE demonstrates excellent predictive validity for resource utilization and ED and hospital costs. Future research should focus on specific presenting complaints and targeted resources to more accurately assess eTRIAGE validity.

Reliability of computerized emergency triage.

OBJECTIVES: Emergency department (ED) triage prioritizes patients based on urgency of care. This study compared agreement between two blinded, independent users of a Web-based triage tool (eTRIAGE) and examined the effects of ED crowding on triage reliability. METHODS: Consecutive patients presenting to a large, urban, tertiary care ED were assessed by the duty triage nurse and an independent study nurse, both using eTRIAGE. Triage score distribution and agreement are reported. The study nurse collected data on ED activity, and agreement during different levels of ED crowding is reported. Two methods of interrater agreement were used: the linear-weighted kappa and quadratic-weighted kappa. RESULTS: A total of 575 patients were assessed over nine weeks, and complete data were available for 569 patients (99.0%). Agreement between the two nurses was moderate if using linear kappa (weighted kappa = 0.52; 95% confidence interval = 0.46 to 0.57) and good if using quadratic kappa (weighted kappa = 0.66; 95% confidence interval = 0.60 to 0.71). ED overcrowding data were available for 353 patients (62.0%). Agreement did not significantly differ with respect to periods of ambulance diversion, number of admitted inpatients occupying stretchers, number of patients in the waiting room, number of patients registered in two hours, or nurse perception of busyness. CONCLUSIONS: This study demonstrated different agreement depending on the method used to calculate interrater reliability. Using the standard methods, it found good agreement between two independent users of a computerized triage tool. The level of agreement was not affected by various measures of ED crowding.

Emergency triage: comparing a novel computer triage program with standard triage.

BACKGROUND: Emergency department (ED) triage prioritizes patients based on urgency of care; however, little previous testing of triage tools in a live ED environment has been performed. OBJECTIVES: To determine the agreement between a computer decision tool and memory-based triage. METHODS: Consecutive patients presenting to a large, urban, tertiary care ED were assessed in the usual fashion and by a blinded study nurse using a computerized decision support tool. Triage score distribution and agreement between the two triage methods were reported. A random subset of patients was selected and reviewed by a blinded expert panel as a consensus standard. RESULTS: Over five weeks, 722 ED patients were assessed; complete data were available from 693 (96%) score pairs. Agreement between the two methods was poor (kappa = 0.202; 95% confidence interval [95% CI] = 0.150 to 0.254); however, agreement improved when using weighted kappa (0.360; 95% CI = 0.305 to 0.415) or "within one" level kappa (0.732; 95% CI = 0.644 to 0.821). When compared with the expert panel, the nurse triage scores showed lower agreement (0.263; 95% CI = 0.133 to 0.394) than the tool (kappa = 0.426; 95% CI = 0.289 to 0.564). There was a significant down-triaging of patients when patients were triaged without the computerized tool. Admission rates also differed between the triage systems. CONCLUSIONS: There was significant discrepancy by nurses using memory-based triage when compared with a computer tool. Triage decision support tools can mitigate this drift, which has administrative implications for EDs.

Supporting clinical practice at the bedside using wireless technology.

OBJECTIVES: Despite studies that show improvements in both standards of care and outcomes with the judicious application of clinical practice guidelines (CPGs), their clinical utilization remains low. This randomized controlled trial examined the use of a wirelessly networked mobile computer (MC) by physicians at the bedside with access to an emergency department information system, decision support tools (DSTs), and other software options. METHODS: Each of ten volunteer emergency physicians was randomized using a matched-pair design to work five shifts in standard fashion (desktop computer [DC] access) and five shifts with a wirelessly networked MC. Work pattern issues and electronic CPG/DST use were compared using end-of-shift satisfaction questionnaires and review of a CPG/DST database. Repeated-measures analysis of variance was used to examine between-shift differences. RESULTS: A total of 100 eight-hour shifts were evaluated; 99% compliance with postshift questionnaires was achieved. Using a seven-point Likert scale (MC values first), MCs were rated as being as fast (5.04 vs. 4.54; p=0.13) and convenient (5.08 vs. 4.14; p=0.07) as DCs. Overall, physicians rated MCs to be less efficient (3.18 vs. 4.30; p=0.02) but encouraged more frequent use of DSTs (4.10 vs. 3.47; p=0.03) without impacting doctor-patient communication (2.78 vs. 2.96; p=0.51). During the study period, physician use of an intranet Web application (eCPG) was more frequent during shifts assigned to the MC when compared with the DC (eCPG uses/shift, 3.6 vs. 2.0; p=0.033). CONCLUSIONS: The MC technology permitted physicians to access information at the bedside and increased the use of CPG/DST tools. According to physicians, patients appeared to accept their use of information technology to assist in decision making. Development of improved computer technology may address the major limitation of MC portability.