Using Time Series Analysis to Predict Cardiac Arrest in a PICU.

OBJECTIVES: To build and test cardiac arrest prediction models in a PICU, using time series analysis as input, and to measure changes in prediction accuracy attributable to different classes of time series data. DESIGN: Retrospective cohort study. SETTING: Thirty-one bed academic PICU that provides care for medical and general surgical (not congenital heart surgery) patients. SUBJECTS: Patients experiencing a cardiac arrest in the PICU and requiring external cardiac massage for at least 2 minutes. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: One hundred three cases of cardiac arrest and 109 control cases were used to prepare a baseline dataset that consisted of 1,025 variables in four data classes: multivariate, raw time series, clinical calculations, and time series trend analysis. We trained 20 arrest prediction models using a matrix of five feature sets (combinations of data classes) with four modeling algorithms: linear regression, decision tree, neural network, and support vector machine. The reference model (multivariate data with regression algorithm) had an accuracy of 78% and 87% area under the receiver operating characteristic curve. The best model (multivariate + trend analysis data with support vector machine algorithm) had an accuracy of 94% and 98% area under the receiver operating characteristic curve. CONCLUSIONS: Cardiac arrest predictions based on a traditional model built with multivariate data and a regression algorithm misclassified cases 3.7 times more frequently than predictions that included time series trend analysis and built with a support vector machine algorithm. Although the final model lacks the specificity necessary for clinical application, we have demonstrated how information from time series data can be used to increase the accuracy of clinical prediction models.

Evaluation of a problem-specific SBAR tool to improve after-hours nurse-physician phone communication: a randomized trial.

BACKGROUND: After-hours telephone communications are common in patient management. Patterns of communication of key information during after-hours phone calls were evaluated, and the utility of problem-specific Situation, Background, Assessment, Recommendation (SBAR) forms in improving this communication was assessed. METHODS: In a randomized trial using a simulated on-call setting, 20 nurses called physicians regarding six cases adapted from inpatient records and based on the six most common reasons for after-hours nurse-physician communication. Three of the cases were handled without the SBAR forms (control cases), and three cases were handled with the forms (SBAR cases). Two cue types of communication were evaluated: situation cues, which conveyed the patient's situation (for example, a patient is confused), and background cues, which conveyed problem-specific data indicated on the SBAR forms (for example, the patient has a low sodium level). RESULTS: Ninety-two phone calls were analyzed (43 SBAR/49 controls). Most of the nurses reported the situation cues (SBAR 88%, control 84%, p = .60) but not the background cues. There was a trend toward fewer background cues communicated in the SBAR cases (14% versus 31%, p = .08). In 14% of the cases, on average, nurses omitted information or reported wrong information regarding the background cue. Physicians asked questions that resulted in the communication of the cues in a minority of the cases when the background cues were not originally provided by the nurses (SBAR 6%, control 16%, p = .39). CONCLUSIONS: In after-hours phone communication between physicians and nurses, significant information was often not communicated and physicians did not elicit the necessary information. Simply providing an SBAR-based form did not ensure complete communication of key information.

Time series analysis as input for clinical predictive modeling: modeling cardiac arrest in a pediatric ICU.

BACKGROUND: Thousands of children experience cardiac arrest events every year in pediatric intensive care units. Most of these children die. Cardiac arrest prediction tools are used as part of medical emergency team evaluations to identify patients in standard hospital beds that are at high risk for cardiac arrest. There are no models to predict cardiac arrest in pediatric intensive care units though, where the risk of an arrest is 10 times higher than for standard hospital beds. Current tools are based on a multivariable approach that does not characterize deterioration, which often precedes cardiac arrests. Characterizing deterioration requires a time series approach. The purpose of this study is to propose a method that will allow for time series data to be used in clinical prediction models. Successful implementation of these methods has the potential to bring arrest prediction to the pediatric intensive care environment, possibly allowing for interventions that can save lives and prevent disabilities. METHODS: We reviewed prediction models from nonclinical domains that employ time series data, and identified the steps that are necessary for building predictive models using time series clinical data. We illustrate the method by applying it to the specific case of building a predictive model for cardiac arrest in a pediatric intensive care unit. RESULTS: Time course analysis studies from genomic analysis provided a modeling template that was compatible with the steps required to develop a model from clinical time series data. The steps include: 1) selecting candidate variables; 2) specifying measurement parameters; 3) defining data format; 4) defining time window duration and resolution; 5) calculating latent variables for candidate variables not directly measured; 6) calculating time series features as latent variables; 7) creating data subsets to measure model performance effects attributable to various classes of candidate variables; 8) reducing the number of candidate features; 9) training models for various data subsets; and 10) measuring model performance characteristics in unseen data to estimate their external validity. CONCLUSIONS: We have proposed a ten step process that results in data sets that contain time series features and are suitable for predictive modeling by a number of methods. We illustrated the process through an example of cardiac arrest prediction in a pediatric intensive care setting.

Informaticians: how they may benefit your healthcare organization.

The heightened focus on health information technology deployment, specifically the electronic health record, has magnified the need for the knowledge and skills of informaticians. The clinical informatician is frequently confused with system analyst and project management roles. In this article, the authors explore the knowledge and skills of informaticians and how to determine the skill combinations most suited to an organization and compare the roles of informaticians, project managers, and information technology professionals.

Self-assessed emergency readiness and training needs of nurses in rural Texas.

OBJECTIVE: Nurses, particularly public health nurses, play a key role in emergency preparedness and response in rural areas. To prepare rural jurisdictions for unforeseen disastrous events it is imperative to assess the public health emergency readiness and training needs of nurses. The objective of this study was to assess the self-reported terrorism preparedness and training needs of a nurse workforce. DESIGN AND SAMPLE: Cross-sectional prevalence of practicing nurses in regions of North Texas. 3,508 rural nurses practicing in North Texas participated in the study. MEASUREMENTS: Data were collected through a mailed survey; analyses included multinominal logistic regression and descriptive statistics. RESULTS: A total of 941 (27%) nurses completed the survey. The majority of respondents reported limited bioterrorism-related training. Fewer than 10% were confident in their ability to diagnose or treat bioterrorism-related conditions. Although only 30% expressed a willingness to collaborate with state and local authorities during a bioterrorism event, more than 69% indicated interest in future training opportunities. Preferred training modalities included small group workshops with instructor-led training, and Internet-based training. CONCLUSIONS: Licensing agencies, professional organizations, and community constituencies may need to play a stronger role in improving the bioterrorism-related emergency preparedness of rural nurses.

Legibility of a volumetric infusion pump in a shock trauma ICU.

BACKGROUND: Reports submitted to the Federal Food and Drug Administration (FDA) indicate that legibility of infusion pump interfaces contributes to medication errors (for example, patients have been seriously injured when nurses overinfused them after reading the number 7 as a 1). Health care has experienced an influx of medical devices with small-screen interfaces, generically described as small-screen devices. Legibility is widely acknowledged as a necessary aspect of safe operation of medical devices. Contextual analysis was used to observe conditions affecting the legibility of the screen interface of a dual-channel infusion volumetric infusion pump in a shock trauma intensive care unit (STICU). METHODS: Observations were made of registered nurses and physicians using the infusion pumps during a four-hour period. RESULTS: Results from the observations indicated that there was reduced legibility of the infusion pump screen interface because of an inherently subdued light situation, reduced screen contrast, and the small font size of the lettering. DISCUSSION: It was clear from the beginning of the observations that the screen of the infusion pump had limited legibility. In some instances, nurses attached handmade tape labels to the infusion pump to enhance and supplement the small screen. The pump was often positioned facing away from the nurses, who then had to reposition it to view the screen, contributing to interruptions in work flow and creating a potential safety hazard. A variety of strategies are recommended to ensure legibility of infusion pump interfaces and of medical-device labels. CONCLUSIONS: A more complete environmental approach is needed to determine the legibility and usefulness of microdisplay and small-screen devices in health care.

When your words count: a discriminative model to predict approval of referrals.

OBJECTIVE: To develop and test a statistical model which correctly predicts the approval of outpatient referrals when reviewed by a specialty service based on nine discriminating variables. DESIGN: Retrospective cross-sectional study. SETTING: Large public county hospital system in a southern US city. PARTICIPANTS: Written documents and associated data from 500 random adult referrals made by primary care providers to various specialty services during the course of one month. MAIN OUTCOME MEASURES: The resulting correct prediction rates obtained by the model. RESULTS: The model correctly predicted 78.6% of approved referrals using all nine discriminating variables, 75.3% of approved referrals using all variables in a stepwise manner and 74.7% of approved referrals using only the referral total word count as a single discriminating variable. CONCLUSIONS: Three iterations of the model correctly predicted at least 75% of the approved referrals in the validation set. A correct prediction of whether or not a referral will be approved can be made in three out of four cases.

Health informatics: moving from a discipline to a science.

This paper examines the historical definitions of Health (Biomedical) Informatics. It is clear that a majority of the definitions refer to Health Informatics as a discipline. Rather it can be argued that the maturation of Health Informatics is beginning to culminate in a distinct science. This progress need to be reflected in academic programs as well as our conferences and publications.

Developing a taxonomy and an ontology of nurses' patient clinical summaries.

Nurses prepare a summary of patient information that they consult and update throughout the shift. This document is believed to be integral to cognition, working memory, and decision-making. While serving as a key support to nursing practice, this summary also represents risks to patient safety. Characterized as a PCCAT, or Personally Created Cognitive Artifact, studies of this document in the context of nursing practice have not been reported. The absence of reported research, the importance of the document to nurse cognition and practice, and related safety risks prompted the research that this paper discusses. A taxonomy was developed through the analysis and coding of 151 PCCATs. Further analysis and mapping provided an ontology of the PCCAT. Content differences were noted between nursing units and among nurses. This may reflect differences in unit-based culture and/or differences in the patient complexity. The interaction between culture and perceived complexity of practice is one of the great difficulties in generating automated information systems for clinical practice settings. This paper is part of a larger research protocol that explores meta-level knowledge structures and revision to the understanding of the granularity of nursing knowledge. Development of a taxonomy and ontology of the nurse PCCAT, an important component of the larger research protocol, is described in this paper.

Second Life and health information.

Second Life (SL) is emerging as a repository of health information. The purpose of this study was to determine the number of places and groups that offer specific health information in SL. A retrospective analysis of SL places and groups was conducted for cancer. Findings indicate that more cancer information is available from groups than specific places in SL. All current and future cancer information will need to be evaluated for reliability and efficacy.

Interruptions in a level one trauma center: a case study.

BACKGROUND: The emergency department has been characterized as interrupt-driven. Government agencies and patient safety organizations recognize that interruptions contribute to medical errors. The purpose of this study was to observe, record, and contextualize activities and interruptions experienced by physicians and Registered Nurses (RNs) working in a Level One Trauma Center. DESIGN: A case study that relied on an ethnographic study design using the shadowing method. SUBJECTS: A convenience sample of physicians and RNs, each with at least 6 months of experience in the Emergency Department (ED), were asked to participate. In these kinds of detailed qualitative investigations, it is quite common to have a small sample size. Ethical approval: Approval was obtained from institutional ethic committees prior to initiating the study. Community consent was obtained from the ED staff through in-service education. SETTING: All observations were made in the trauma section of the ED of a tertiary teaching hospital. The hospital is situated in a major medical center in the Gulf Coast region of the United States of America (USA). FINDINGS: Five attending ED physicians were observed for a total of 29h, 31min. Eight RNs were shadowed for a total of 40 h, 9min. Interruptions and activities were categorized using the Hybrid Method to Categorize Interruptions and Activities (HyMCIA). Registered Nurses received slightly more interruptions per hour than physicians. People, pagers, and telephones were identified as mediums through which interruptions were delivered. The physical environment was found to contribute to interruptions in workflow because of physical design and when supplies were not available. Physicians and RNs usually returned to the original, interrupted activity more often than leaving the activity unfinished. CONCLUSION: This research provides an enhanced understanding of interruptions in workflow in the ED, the identification of work constraints, and the need to develop interventions to manage interruptions. It is crucial that interruptions be delivered in such a way that there is minimal negative impact on performance. The significance and importance of the interruption must always be weighed against the negative impact that it has on smooth, efficient workflow.

Reasons for after-hours calls by hospital floor nurses to on-call physicians.

BACKGROUND: Communication failure is a common root cause of preventable medical errors affecting hospitalized patients. A study was conducted to determine the reasons for calls made by nurses working on the general medical wards to on-call physicians from 6:00 P.M. to 7:00 A.M. METHODS: A retrospective review was performed of a random sample of 500 inpatients admitted to general medical wards at an urban public teaching hospital in Houston between January 1, 2000, and February 28, 2003. RESULTS: In 139 (47%) of 293 medical records there were 304 documented calls. The majority of calls (81%) took place between 6:00 P.M. and 2:00 A.M., with peak call volume between midnight and 2:00 P.M. Patients with one or more calls had an average of 2.2 calls during their stay. Ten categories accounted for 65% of all the nurse calls. In 44% of calls, physicians responded by ordering a medication. DISCUSSION: Communication between floor nurses and on-call physicians might be improved by several interventions. Because 10 reasons accounted for 65% of after-hours calls, protocols could allow nurses to resolve some acute problems without physician involvement. For example, appropriate standing orders (e.g., PR.N. medications) may prevent some calls. In addition, sign-out procedures can be tailored to address common problems that are likely to require future telephone communication. With efforts to change error-prone systems, it seems prudent to focus on after-hours coverage.

Attitudes toward medical device use errors and the prevention of adverse events.

BACKGROUND: The design of a device's user interface often contributes to the chance of a user making an error in using the device. However, there is evidence that most such errors that occur in practice are attributed solely to the user and that the primary method of error prevention is to retrain the user. Yet this attitude may decrease the quality of error reports and the use of more effective error prevention strategies. A qualitative study was conducted to assess health care employees' attitudes toward device use errors and the prevention of adverse events. METHODS: Twenty-six health care employees from three hospital systems, including 11 device users and 15 nonusers who had participated in infusion pump purchasing decisions were given a scenario describing a device use error involving an infusion pump. Several open-ended questions assessed what they felt led to the event and how they would prevent the event from reccurring. RESULTS: The top three reported types of factors leading to the adverse event, in decreasing order of frequency, were the user, pump design problems, and lack of training. The top three prevention strategies reported by the participants were retraining the user, redesigning the device, and telling the user to be careful. DISCUSSION: These results suggest that health care employees still put too much emphasis on the traditional view of blaming and retraining the user.

Towards a hybrid method to categorize interruptions and activities in healthcare.

OBJECTIVE: Interruptions are known to have a negative impact on activity performance. Understanding how an interruption contributes to human error is limited because there is not a standard method for analyzing and classifying interruptions. Qualitative data are typically analyzed by either a deductive or an inductive method. Both methods have limitations. In this paper, a hybrid method was developed that integrates deductive and inductive methods for the categorization of activities and interruptions recorded during an ethnographic study of physicians and registered nurses in a Level One Trauma Center. Understanding the effects of interruptions is important for designing and evaluating informatics tools in particular as well as improving healthcare quality and patient safety in general. METHOD: The hybrid method was developed using a deductive a priori classification framework with the provision of adding new categories discovered inductively in the data. The inductive process utilized line-by-line coding and constant comparison as stated in Grounded Theory. RESULTS: The categories of activities and interruptions were organized into a three-tiered hierarchy of activity. Validity and reliability of the categories were tested by categorizing a medical error case external to the study. No new categories of interruptions were identified during analysis of the medical error case. CONCLUSIONS: Findings from this study provide evidence that the hybrid model of categorization is more complete than either a deductive or an inductive method alone. The hybrid method developed in this study provides the methodical support for understanding, analyzing, and managing interruptions and workflow.

Clinical communication ontology for medical errors.

Clinical communication failures caused 60% of sentinel events reported by the Joint Commission on Accreditation of Healthcare Organizations. The difficulties of communication have been the primary cause of errors leading to patients' death. For analyzing medical error events, uncovering the patterns of clinical communication, this paper reports the design and development of clinical communication ontology. The ontology contains eight axes and was validated using ten medical error cases, where communication was the main factor. The coding process demonstrates that the ontology can be used as a guideline for future medical error reporting system, through which the root cause of medical error due to communication will be revealed in a clear pattern. This ontology contributes to the generation of proper interventions and effective strategies for reducing medical errors.

Initiators of interruption in workflow: the role of MDs and RNs.

The healthcare environment has been characterized as interrupt-driven with medical doctors (MDs) and registered nurses (RNs) receiving many interruptions during a shift. Previous research studies have focused on the recipient because of the negative impact on task performance. It is equally important to understand the initiator of an interruption to help design strategies to lessen the number of interruptions and the possible negatives consequences. The purpose of this instrumental study was to examine MDs and RNs as initiators of interruptions. Results of this study indicate that MDs and RNs initiate interruptions most often through face-to-face situations and use of the telephone. Strategies to successfully manage interruptions must consider both the role of initiator as well as the recipient in an interruption event.

What if nurses get what they have always sought: totally personalized care? Trends affecting nursing informatics.

We may be moving towards an era where nurses may get what they say they have always wanted--totally personalized care. This paper examines four major trends, i.e. the increasing importance of the genome, the informed patient, the rise of the patient-payer, and the 'instant gratification generation'. Each individually, and in combination, will have potential impacts on the future of health and nursing. The implications of these issues, and others are explored, as well as nursing's readiness to deal with the implications of truly and totally personalized care.

Operating manual-based usability evaluation of medical devices: an effective patient safety screening method.

BACKGROUND: Use of medical devices often directly contributes to medical errors. Because it is difficult or impossible to change the design of existing devices, the best opportunity for improving medical device safety is during the purchasing process. However, most hospital personnel are not familiar with the usability evaluation methods designed to identify aspects of a user interface that do not support intuitive and safe use. A review of medical device operating manuals is proposed as a more practical method of usability evaluation. METHOD: Operating manuals for five volumetric infusion pumps from three manufacturers were selected for this study (January-April 2003). Each manual's safety message content was evaluated to determine whether the message indicated a device design characteristic that violated known usability principles (heuristics) or indicated a violation of an affordance of the device. RESULTS: "Minimize memory load," with 65 violations, was the heuristic violated most frequently across pumps. Variations between pumps, including the frequency and severity of violations for each, were noted. DISCUSSION: Results suggest that manual review can provide a proxy for heuristic evaluation of the actual medical device. This method, intended to be a component of prepurchasing evaluation, can complement more formal usability evaluation methods and be used to select a subset of devices for more extensive and formal testing.

The significance of cognitive modeling in building healthcare interfaces.

BACKGROUND: Although there are many reasons that widespread adoption of healthcare information systems has not transpired, one reason is a failure to take into account the cognitive needs of the users. AIM: To understand the cognitive needs of nurses and physicians and determine how these needs should influence the design of healthcare interfaces. DESIGN OF STUDY: A qualitative and quantitative study that compares how nurses and physicians comprehend patient information. SETTING: Twenty-four registered nurses and twenty-four physicians working in the specialties of gastrointestinal or internal medicine. METHODS: Each clinician reviewed two mock electronic medical records and summarized the cases using a think-aloud protocol. All verbalizations were coded for medical and conceptual information. RESULTS: The nurses included a larger mean proportion (p<0.001) of recalls than did the physicians. As compared to the nurses, the physicians included a statistically significant (p<0.001) larger mean proportion of inferences, conditional statements, and interventions. The nurses concentrated on functional problems, whereas the physicians focused on diagnosis, treatment, and management. CONCLUSION: The main cognitive differences between the physicians and the nurses are explained through the differences in their practice models. Therefore, healthcare IT must develop separate interfaces for each discipline to address their unique needs.

Assessing informatics students' satisfaction with a web-based courseware system.

This study assessed health informatics student satisfaction with two subsequent versions of Prometheus, a web-based courseware system. Prometheus versions 4 and 5 were assessed to gauge the effect of modifications to improve the usability of the system. The Questionnaire for User Interaction Satisfaction (QUIS, version 7.0) was administered at the end of fall semester 2001 (in which Prometheus version 4 was used) and again at the end of Spring 2002 (in which Prometheus version 5 was used). QUIS contains measures of user satisfaction of the overall system and 11 specific dimensions, including screen, terminology and system information, learning, system capabilities, manuals and online help, multimedia, and teleconferencing. In general, students had favorable judgments of Prometheus, and their satisfaction level remained relatively stable across the two versions. However, usability enhancements incorporated into version 5 produced no significant differences in student satisfaction ratings. The results of this study provide a benchmark for comparing the relative usability of alternative courseware systems and demonstrate the utility of user satisfaction surveys for assessing and improving courseware systems. With increases in the need for compliance education and the education of clinicians with 'just in time knowledge', courseware systems will become an integral part of the clinical information systems, increasing the importance of usability studies such as this one.