Evaluation of a Measurement System to Assess ICU Team Performance.

OBJECTIVE: Measuring teamwork is essential in critical care, but limited observational measurement systems exist for this environment. The objective of this study was to evaluate the reliability and validity of a behavioral marker system for measuring teamwork in ICUs. DESIGN: Instances of teamwork were observed by two raters for three tasks: multidisciplinary rounds, nurse-to-nurse handoffs, and retrospective videos of medical students and instructors performing simulated codes. Intraclass correlation coefficients were calculated to assess interrater reliability. Generalizability theory was applied to estimate systematic sources of variance for the three observed team tasks that were associated with instances of teamwork, rater effects, competency effects, and task effects. SETTING: A 15-bed surgical ICU at a large academic hospital. SUBJECTS: One hundred thirty-eight instances of teamwork were observed. Specifically, we observed 88 multidisciplinary rounds, 25 nurse-to-nurse handoffs, and 25 simulated code exercises. INTERVENTIONS: No intervention was conducted for this study. MEASUREMENTS AND MAIN RESULTS: Rater reliability for each overall task ranged from good to excellent correlation (intraclass correlation coefficient, 0.64-0.81), although there were seven cases where reliability was fair and one case where it was poor for specific competencies. Findings from generalizability studies provided evidence that the marker system dependably distinguished among teamwork competencies, providing evidence of construct validity. CONCLUSIONS: Teamwork in critical care is complex, thereby complicating the judgment of behaviors. The marker system exhibited great potential for differentiating competencies, but findings also revealed that more context specific guidance may be needed to improve rater reliability.

Eight critical factors in creating and implementing a successful simulation program.

BACKGROUND: Recognizing the need to minimize human error and adverse events, clinicians, researchers, administrators, and educators have strived to enhance clinicians' knowledge, skills, and attitudes through training. Given the risks inherent in learning new skills or advancing underdeveloped skills on actual patients, simulation-based training (SBT) has become an invaluable tool across the medical education spectrum. The large simulation, training, and learning literature was used to provide a synthesized yet innovative and "memorable" heuristic of the important facets of simulation program creation and implementation, as represented by eight critical "S" factors-science, staff, supplies, space, support, systems, success, and sustainability. These critical factors advance earlier work that primarily focused on the science of SBT success, to also include more practical, perhaps even seemingly obvious but significantly challenging components of SBT, such as resources, space, and supplies. SYSTEMS: One of the eight critical factors-systems-refers to the need to match fidelity requirements to training needs and ensure that technological infrastructure is in place. The type of learning objectives that the training is intended to address should determine these requirements. For example, some simulators emphasize physical fidelity to enable clinicians to practice technical and nontechnical skills in a safe environment that mirrors real-world conditions. Such simulators are most appropriate when trainees are learning how to use specific equipment or conduct specific procedures. CONCLUSION: The eight factors-science, staff, supplies, space, support, systems, success, and sustainability-represent a synthesis of the most critical elements necessary for successful simulation programs. The order of the factors does not represent a deliberate prioritization or sequence, and the factors' relative importance may change as the program evolves.

Potential impact of data source and interoperability messaging on health information technology (HIT) users: a study series from the United States Department of Veterans Affairs.

BACKGROUND: The promise of Health Information Exchange (HIE) systems rests in their potential to provide clinicians and administrative staff rapid access to relevant patient data to support judgement and decision-making. However, HIE systems can have usability and technical issues, as well as fail to support user workflow. OBJECTIVE: Share the findings from a series of studies that address HIE system deficiencies for an Electronic Health Record (EHR) viewer which accesses multiple data sources. METHODS: A variety of methods were used, in a series of studies, to gain a better understanding of issues and their mitigation through use of promising EHR viewer features. RESULTS: The study series results are presented by the themes that underscore the importance for users to distinguish between data that are available but missing due to connection or system errors, data that are omitted entirely because they are not available and data that are excluded due to filtered search criteria. CONCLUSIONS: The principal findings from this study series led to improvement recommendations for the EHR viewer, as well as citing areas that are ripe for further investigation and analysis.

Human factors-based risk analysis to improve the safety of doffing enhanced personal protective equipment.

OBJECTIVE: To systematically assess enhanced personal protective equipment (PPE) doffing safety risks. DESIGN: We employed a 3-part approach to this study: (1) hierarchical task analysis (HTA) of the PPE doffing process; (2) human factors-informed failure modes and effects analysis (FMEA); and (3) focus group sessions with a convenience sample of infection prevention (IP) subject matter experts. SETTING: A large academic US hospital with a regional Special Pathogens Treatment Center and enhanced PPE doffing protocol experience.ParticipantsEight IP experts. METHODS: The HTA was conducted jointly by 2 human-factors experts based on the Centers for Disease Control and Prevention PPE guidelines. The findings were used as a guide in 7 focus group sessions with IP experts to assess PPE doffing safety risks. For each HTA task step, IP experts identified failure mode(s), assigned priority risk scores, identified contributing factors and potential consequences, and identified potential risk mitigation strategies. Data were recorded in a tabular format during the sessions. RESULTS: Of 103 identified failure modes, the highest priority scores were associated with team members moving between clean and contaminated areas, glove removal, apron removal, and self-inspection while preparing to doff. Contributing factors related to the individual (eg, technical/ teamwork competency), task (eg, undetected PPE contamination), tools/technology (eg, PPE design characteristics), environment (eg, inadequate space), and organizational aspects (eg, training) were identified. Participants identified 86 types of risk mitigation strategies targeting the failure modes. CONCLUSIONS: Despite detailed guidelines, our study revealed 103 enhanced PPE doffing failure modes. Analysis of the failure modes suggests potential mitigation strategies to decrease self-contamination risk during enhanced PPE doffing.

Teamwork in healthcare: Key discoveries enabling safer, high-quality care.

Few industries match the scale of health care. In the United States alone, an estimated 85% of the population has at least 1 health care encounter annually and at least one quarter of these people experience 4 to 9 encounters annually. A single visit requires collaboration among a multidisciplinary group of clinicians, administrative staff, patients, and their loved ones. Multiple visits often occur across different clinicians working in different organizations. Ineffective care coordination and the underlying suboptimal teamwork processes are a public health issue. Health care delivery systems exemplify complex organizations operating under high stakes in dynamic policy and regulatory environments. The coordination and delivery of safe, high-quality care demands reliable teamwork and collaboration within, as well as across, organizational, disciplinary, technical, and cultural boundaries. In this review, we synthesize the evidence examining teams and teamwork in health care delivery settings in order to characterize the current state of the science and to highlight gaps in which studies can further illuminate our evidence-based understanding of teamwork and collaboration. Specifically, we highlight evidence concerning (a) the relationship between teamwork and multilevel outcomes, (b) effective teamwork behaviors, (c) competencies (i.e., knowledge, skills, and attitudes) underlying effective teamwork in the health professions, (d) teamwork interventions, (e) team performance measurement strategies, and (f) the critical role context plays in shaping teamwork and collaboration in practice. We also distill potential avenues for future research and highlight opportunities to understand the translation, dissemination, and implementation of evidence-based teamwork principles into practice. (PsycINFO Database Record

Sensor-based measurement of critical care nursing workload: Unobtrusive measures of nursing activity complement traditional task and patient level indicators of workload to predict perceived exertion.

OBJECTIVE: To establish the validity of sensor-based measures of work processes for predicting perceived mental and physical exertion of critical care nurses. MATERIALS AND METHODS: Repeated measures mixed-methods study in a surgical intensive care unit. Wearable and environmental sensors captured work process data. Nurses rated their mental (ME) and physical exertion (PE) for each four-hour block, and recorded patient and staffing-level workload factors. Shift was the grouping variable in multilevel modeling where sensor-based measures were used to predict nursing perceptions of exertion. RESULTS: There were 356 work hours from 89 four-hour shift segments across 35 bedside nursing shifts. In final models, sensor-based data accounted for 73% of between-shift, and 5% of within-shift variance in ME; and 55% of between-shift, and 55% of within-shift variance in PE. Significant predictors of ME were patient room noise (ß = 0.30, p < .01), the interaction between time spent and activity levels outside main work areas (ß = 2.24, p < .01), and the interaction between the number of patients on an insulin drip and the burstiness of speaking (ß = 0.19, p < .05). Significant predictors of PE were environmental service area noise (ß = 0.18, p < .05), and interactions between: entropy and burstiness of physical transitions (ß = 0.22, p < .01), time speaking outside main work areas and time at nursing stations (ß = 0.37, p < .001), service area noise and time walking in patient rooms (ß = -0.19, p < .05), and average patient load and nursing station speaking volume (ß = 0.30, p < .05). DISCUSSION: Analysis yielded highly predictive models of critical care nursing workload that generated insights into workflow and work design. Future work should focus on tighter connections to psychometric test development methods and expansion to a broader variety of settings and professional roles. CONCLUSIONS: Sensor-based measures are predictive of perceived exertion, and are viable complements to traditional task demand measures of workload.

An integrative framework for sensor-based measurement of teamwork in healthcare.

There is a strong link between teamwork and patient safety. Emerging evidence supports the efficacy of teamwork improvement interventions. However, the availability of reliable, valid, and practical measurement tools and strategies is commonly cited as a barrier to long-term sustainment and spread of these teamwork interventions. This article describes the potential value of sensor-based technology as a methodology to measure and evaluate teamwork in healthcare. The article summarizes the teamwork literature within healthcare, including team improvement interventions and measurement. Current applications of sensor-based measurement of teamwork are reviewed to assess the feasibility of employing this approach in healthcare. The article concludes with a discussion highlighting current application needs and gaps and relevant analytical techniques to overcome the challenges to implementation. Compelling studies exist documenting the feasibility of capturing a broad array of team input, process, and output variables with sensor-based methods. Implications of this research are summarized in a framework for development of multi-method team performance measurement systems. Sensor-based measurement within healthcare can unobtrusively capture information related to social networks, conversational patterns, physical activity, and an array of other meaningful information without having to directly observe or periodically survey clinicians. However, trust and privacy concerns present challenges that need to be overcome through engagement of end users in healthcare. Initial evidence exists to support the feasibility of sensor-based measurement to drive feedback and learning across individual, team, unit, and organizational levels. Future research is needed to refine methods, technologies, theory, and analytical strategies.

Towards expanding the acute care team: Learning how to involve families in care processes.

INTRODUCTION: Effective teamwork is known to be important to improving health care outcomes. Current research often highlights teamwork among health care professionals without consideration of approaches to including family as part of the health care team. In this study, the authors assess family and provider openness to expanding the care team to include family participation and introduce the Family Involvement Menu as a tool to facilitate family engagement. METHOD: They collected 37 family surveys and 37 clinician surveys to understand the perception, comfort level, experience, and interest of family and clinicians in including family in the care of the patient. The majority of family reported being interested and comfortable in participating in care (95% and 92%, respectively). RESULTS: The majority of clinicians considered family already to be part of the health care team (92%) though only 16% reported routinely inviting families to participate in direct patient care all the time. Multiple direct patient care activities were identified as promising opportunities for family engagement. Barriers to family engagement reported included the family being scared (19%), uncomfortable (19%), or unwilling (14%) or nurses not having enough time (14%) to involve families. DISCUSSION: Engaging family has the potential to increase nursing availability for other tasks, enhance relationship building, and is an opportunity to introduce early education for family, better preparing them for transition of care and discharge. The Family Involvement Menu supports family engagement and can be a strategy to include family members as part of the health care team.

Validation of a teamwork perceptions measure to increase patient safety.

BACKGROUND: TeamSTEPPS (Team Strategies and Tools to Enhance Performance and Patient Safety) is a team-training intervention which shows promise in aiding the mitigation of medical errors. This article examines the construct validity of the TeamSTEPPS Teamwork Perceptions Questionnaire (T-TPQ), a self-report survey that examines multiple dimensions of perceptions of teamwork within healthcare settings. METHOD: Using survey-based methods, 1700 multidisciplinary healthcare professionals and support staff were measured on their perceptions of teamwork. Confirmatory factor analysis was conducted to examine the relationship between the five TeamSTEPPS dimensions: Leadership, Mutual Support, Situation Monitoring, Communication, and Team Structure. RESULTS: The analysis indicated that the T-TPQ measure is more reliable than previously thought (Cronbach's α=0.978). Further, our final tested model showed a good fit with the data (x(2) (df) 3601.27 (546), p<0.0001, Tucker-Lewis Index (TLI)=0.942, Comparative fit index (CFI)=0.947, root mean square error of approximation (RMSEA)=0.057), indicating that the measure appears to have construct validity. Further, all dimensions correlated with one another, but were shown to be independent constructs. CONCLUSIONS: The T-TPQ is a construct-valid instrument for measuring perceptions of teamwork. This has beneficial implications for patient safety and future research that studies medical teamwork.

A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application?

OBJECTIVE: Behavioural marker systems are advocated as a method for providing accurate assessments, directing feedback and determining the impact of teamwork improvement initiatives. The present article reports on the state of quality surrounding their use in healthcare and discusses the implications of these findings for future research, development and application. In doing so, this article provides a practical resource where marker systems can be selected and evaluated based on their strengths and limitations. METHODS: Four research questions framed this review: what are the attributes of behavioural marker systems? What evidence of reliability and validity exists? What skills and expertise are required for their use? How have they been applied to investigate the relationship between teamwork and other constructs? RESULTS: Behavioural markers systems are generally designed for specific work domains or tasks. They often cover similar content with inconsistent terminology, which complicates the comparison of research findings across clinical domains. Although several approaches were used to establish the reliability and validity of marker systems, the marker system literature, as a whole, requires more robust reliability and validity evidence. The impact of rater training on rater proficiency was mixed, but evidence suggests that improvements can be made over time. CONCLUSIONS: A consensus of definitions for teamwork constructs must be reached to ensure that the meaning behind behavioural measurement is understood across disciplines, work domains and task types. Future development efforts should focus on the cost effectiveness and feasibility of measurement tools including time spent training raters. Further, standards for the testing and reporting of psychometric evidence must be established. Last, a library of tools should be generated around whether the instrument measures general or domain-specific behaviours.

A systematic review of teamwork in the intensive care unit: what do we know about teamwork, team tasks, and improvement strategies?

PURPOSE: Teamwork is essential for ensuring the quality and safety of health care delivery in the intensive care unit (ICU). This article addresses what we know about teamwork, team tasks, and team improvement strategies in the ICU to identify the strengths and limitations of the existing knowledge base to guide future research. METHODS: A keyword search of the PubMed database was conducted in February 2013. Keyword combinations focused on 3 areas: (1) teamwork, (2) the ICU, and (3) training/quality improvement interventions. All studies that investigated teamwork, team tasks, or team interventions within the ICU (ie, intradepartment) were selected for inclusion. RESULTS: Teamwork has been investigated across an array of research contexts and task types. The terminology used to describe team factors varied considerably across studies. The most common team tasks involved strategy and goal formulation. Team training and structured protocols were the most widely implemented quality improvement strategies. CONCLUSIONS: Team research is burgeoning in the ICU, yet low-hanging fruit remains that can further advance the science of teams in the ICU if addressed. Constructs must be defined, and theoretical frameworks should be referenced. The functional characteristics of tasks should also be reported to help determine the extent to which study results might generalize to other contexts of work.