Observational teamwork assessment for surgery: construct validation with expert versus novice raters.

OBJECTIVE: To test the construct validity of the Observational Teamwork Assessment for Surgery (OTAS) tool. SUMMARY BACKGROUND DATA: Poor teamwork in surgical teams has been implicated in adverse events to patients. The OTAS is a tool that assesses teamwork in real time for the entire surgical team. Existing empirical research on OTAS has yet to explore how expert versus novice tool users use the tool to assess teamwork in the operating room. METHODS: Data were collected in 12 elective procedures by an expert/expert (N = 6) and an expert/novice (N = 6) pair of raters. Five teamwork behaviors (communication, coordination, leadership, monitoring, and cooperation) were scored via observation pre, intra, and postoperatively by blind raters. RESULTS: Significant and sizeable correlations were obtained in 12 of 15 behaviors in the expert/expert pair, but only in 3 of 15 behaviors in the expert/novice pair. Significant differences in mean scores were obtained in 3 of 15 behaviors in the expert/expert pair, but in 11 of 15 behaviors in the expert/novice pair. Total OTAS scores exhibited strong correlations and no significant differences in ratings in the expert/expert pair. In the expert/novice pair no correlations were obtained and there were significant differences in mean scores. The overall size of inconsistency in the scoring was 2% for expert/expert versus 15% for expert/novice. CONCLUSIONS: OTAS exhibits adequate construct validity as assessed by consistency in the scoring by expert versus novices-ie, expert raters produce significantly more consistent scoring than novice raters. Further validation should assess the learning curve for novices in OTAS. Relationships between OTAS, measures of technical skill, and behavioral responses to surgical crises should also be quantified.

Distracting communications in the operating theatre.

RATIONALE AND AIMS: Research suggests that there are problems of communication effectiveness in surgery. Here we describe the content, initiators and recipients of communications that intrude or interfere with individual surgical cases. We also consider the level at which the surgical team and its team members are distracted by these case-irrelevant communications (CICs). METHODS: Two psychologist observers sampled 48 general surgery procedures and they recorded the initiator and the recipient of CIC events, their content and the level of observable distraction that they caused. RESULTS: Irrelevant comments and queries (i.e. 'small-talk') accounted for half of the observed CICs. From the remaining CICs that we observed, most were related to the organization and administration of the case-list, to operating theatre provisions and to teaching junior staff. Surgeons initiated a third of the observed CICs, while receiving two thirds of them. External staff visiting the operating theatre initiated the most distracting communications. The CICs addressed to surgeons introduced significantly less distraction to the operating theatre than those addressed to anaesthetists and nurses. CONCLUSIONS: Some of the observed CICs contributed to the administration of the operating theatre case-list. Nonetheless, this communication can interfere with highly sensitive work. More effectively co-ordinated communication could reduce this interference. More research should assess the communication effectiveness and the impact of CICs on task performance in the operating theatre.

Teamwork in the operating theatre: cohesion or confusion?

RATIONALE: The aim of the research that we report here was to empirically assess the cohesiveness of the multidisciplinary operating theatre (OT) team. METHOD: We used concepts from the team performance and team mental models literature to assess OT professionals' perceptions of their teamwork, the structure of their teams and their respective roles within them and their teams' performance. RESULTS: Team structure: OT professionals would welcome a change from the current structure of the team, although there was no agreement on what that structure is. Nurses perceived the team as unitary, surgeons and anaesthetists perceived it as comprising multiple subteams. Team roles: OT professionals tended to overrate their own understanding of their colleagues' role in the OT relative to the role understanding that the colleagues attributed to them. This tendency was especially marked for the surgeons. Team communication and team performance: OT professionals agreed on the relative importance of the various communicating pairs in the OT. Moreover, they were satisfied with the quality of communication among them, except for the communication between the surgeon and the anaesthetist, which received lower ratings. Finally, the quality of the teamwork in the OT was deemed acceptable, although there is room for improvement. CONCLUSIONS: The OT environment need not be as cohesive as previously assumed--a finding that carries implications for the effectiveness of team training interventions. Further research is needed in order to fully comprehend the dynamics of the OT as a working environment and, most importantly, their relation to patient safety.

Observational teamwork assessment for surgery (OTAS): refinement and application in urological surgery.

BACKGROUND: Teamwork in surgical teams is at the forefront of good practice guidelines and empirical research as an important aspect of safe surgery. We have developed a comprehensive assessment for teamwork in surgery-the Observational Teamwork Assessment for Surgery (OTAS)-and we have tested it for general surgical procedures. The aim of the research reported here was to extend the assessment to urology procedures. METHODS: After refining the original assessment, we used it to observe 50 urology procedures. The OTAS comprises a procedural task checklist that assesses patient, equipment/provisions, and communication tasks as well as ratings on five team behavior constructs (communication, cooperation, coordination, leadership, and monitoring). Teamwork was assessed separately in the surgical, anesthesia, and nursing subteams in the operating theater. We also assessed the reliability of the behavioral scoring. RESULTS: Regarding task completion, a number of communication and equipment/provisions tasks were not routinely performed during the operations we observed. Regarding teamwork-related behaviors, adequate reliability was obtained in the scoring of behaviors. Anesthetists and nurses obtained their lowest scores on communication. Surgeons' scores revealed a more complex pattern. In addition to low scores on communication, surgeons' teamwork behaviors appeared to deteriorate as the procedures were finishing. CONCLUSIONS: Our findings suggest that OTAS is applicable to various branches of surgery. Separate assessment of the subteams in the operating theater provides useful information that can be used to build targeted teamwork training aiming to improve surgical patients' safety and outcomes.

Observational assessment of surgical teamwork: a feasibility study.

BACKGROUND: Teamwork is fundamental to effective surgery, yet there are currently no measures of teamwork to guide training, evaluate team interventions or assess the impact of teamwork on outcomes. We report the first steps in the development of an observational assessment of teamwork and preliminary findings. METHOD: We observed 50 operations in general surgery from a single operating theater using a measure of teamwork specifically developed for use in the operating theater. The OTAS (Observational Teamwork Assessment for Surgery) comprises a procedural task checklist centered on the patient, equipment and communications tasks and ratings on team behavior constructs, namely: communication, co-operation, co-ordination, shared-leadership and monitoring. RESULTS: Ratings of overall team performance were reasonably high, though variable, but there was evidence that clinically significant steps were being missed which at the very least eroded safety margins. There was, for instance, a frequent failure to check both surgical and anesthetic equipment and a failure to confirm the procedure verbally, patient notes were missing in about one-eighth of the cases and delays or changes occurred in over two-thirds of the cases. CONCLUSIONS: This study takes an initial step towards developing measures of team performance in surgery that are defined in relation to tasks and behaviors of the team. The observational method of assessment is feasible and can provide a wealth of potentially valuable research data. However, for these measures to be used for formal assessment, more research is needed to make them robust and standardized.

The complexity of measuring interprofessional teamwork in the operating theatre.

Surgery depends on interprofessional teamwork, which is becoming increasingly specialized. If surgery is to become a highly reliable system, it must adapt and professionals must learn from, and share, tested models of interprofessional teamwork. Trainers also need valid measures of teamwork to assess individual and team performance. However, measurement and assessment of interprofessional teamwork is lacking and interprofessional team training is scarce in the surgical domain. This paper addresses the complexity of measuring interprofessional teamwork in the operating theatre. It focuses mainly on the design and properties of observational assessment tools. The report and analysis serves to inform the researcher or clinician of the issues to consider when designing or choosing from alternative measures of team performance for training or assessment.