Teamwork matters: team situation awareness to build high-performing healthcare teams, a narrative review.

Healthcare today is the prerogative of teams rather than of individuals. In acute care domains such as anaesthesia, intensive care, and emergency medicine, the work is complex and fast-paced, and the team members are diverse and interdependent. Three decades of research into the behaviours of high-performing teams provides us with clear guidance on team training, demonstrating positive effects on patient safety and staff wellbeing. Here we consider team performance through the lens of situation awareness. Maintaining situation awareness is an absolute requirement for safe and effective patient management. Situation awareness is a dynamic process of perceiving cues in the environment, understanding what they mean, and predicting how the situation may evolve. In the context of acute clinical care, situation awareness can be improved if the whole team actively contributes to monitoring the environment, processing information, and planning next steps. In this narrative review, we explore the concept of situation awareness at the level of the team, the conditions required to maintain team situation awareness, and the relationship between team situation awareness, shared mental models, and team performance. Our ultimate goal is to help clinicians create the conditions required for high-functioning teams, and ultimately improve the safety of clinical care.

Anaesthesia and patient safety in the socio-technical operating theatre: a narrative review spanning a century.

We review the development of technology in anaesthesia over the course of the past century, from the invention of the Boyle apparatus to the modern anaesthetic workstation with artificial intelligence assistance. We define the operating theatre as a socio-technical system, being necessarily comprised of human and technological parts, the ongoing development of which has led to a reduction in mortality during anaesthesia by an order of four magnitudes over a century. The remarkable technological advances in anaesthesia have been accompanied by important paradigm shifts in the approach to patient safety, and we describe the inter-relationship between technology and the human work environment in the development of such paradigm shifts, including the systems approach and organisational resilience. A better understanding of emerging technological advances and their effects on patient safety will allow anaesthesia to continue to be a leader in both patient safety and in the design of equipment and workspaces.

Desire paths for workplace assessment in postgraduate anaesthesia training: analysing informal processes to inform assessment redesign.

BACKGROUND: In postgraduate specialist training, workplace assessments are expected to provide the information required for decisions on trainee progression. Research suggests that meeting this expectation can be difficult in practice, which has led to the development of informal processes, or 'shadow systems' of assessment. Rather than rejecting these informal approaches to workplace assessment, we propose borrowing from sociology the concept of 'desire paths' to legitimise and strengthen these well-trodden approaches. We asked what information about trainees is currently used or desired by those charged with making decisions on trainee progression, and how is it obtained? METHODS: We undertook a qualitative study with thematic analysis of semi-structured interviews of supervisors of training across Australia and New Zealand. RESULTS: From 21 interviews, we identified four interrelated themes, the first being the local context of training sites. The other three themes represent dilemmas in the desire for authentic and representative information about the trainee: 1) how the process of gathering and documenting information can filter, transform, or limit the original message; 2) deciding when possible trainee deviation from performance norms warrants a closer look; and 3) how transparent vs covert information gathering affects the information supervisors will provide, and how control over assessment is distributed between trainee and supervisor. CONCLUSION: From these themes, we propose a set of design principles for future workplace assessment. Understanding the reasons desire paths exist can inform future assessment redesign, and may address the current disjunct between the formal workplace assessment system and what happens in practice.

Trust, power and learning in workplace-based assessment: The trainee perspective.

For trainees to participate meaningfully in workplace-based assessment (WBA), they must have trust in their assessor. However, the trainee's dependent position complicates such trust. Understanding how power and trust influence WBAs may help us make them more effective learning opportunities. We conducted semi-structured interviews with 17 postgraduate anaesthesia trainees across Australia and New Zealand. Sensitised by notions of power, we used constructivist grounded theory methodology to examine trainees' experiences with trusting their supervisors in WBAs. In our trainee accounts, we found that supervisors held significant power to mediate access to learning opportunities and influence trainee progress in training. All episodes where supervisors could observe trainees, from simply working together to formal WBAs, were seen to generate assessment information with potential consequences. In response, trainees actively acquiesced to a deferential role, which helped them access desirable expertise and minimise the risk of reputational harm. Trainees granted trust based on how they anticipated a supervisor would use the power inherent in their role. Trainees learned to ration exposure of their authentic practice to supervisors in proportion to their trust in them. Trainees were more trusting and open to learning when supervisors used their power for the trainee's benefit and avoided WBAs with supervisors they perceived as less trustworthy. If assessment for learning is to flourish, then the trainee-supervisor power dynamic must evolve. Enhancing supervisor behaviour through reflection and professional development to better reward trainee trust would invite more trainee participation in assessment for learning. Modifying the assessment system design to nudge the power balance towards the trainee may also help. Modifications could include designated formative and summative assessments or empowering trainees to select which assessments count towards progress decisions. Attending to power and trust in WBA may stimulate progress towards the previously aspirational goal of assessment for learning in the workplace.

Communication and team function affect patient outcomes in anaesthesia: getting the message across.

A study in this edition of the Journal has added to data showing that failures in communication in the operating room contribute to patient harm. These data support the view that multidisciplinary teamwork and communication training should be part of the continuous professional development of all members of the perioperative team. Achieving change will require efforts to win the hearts and minds of all concerned, but these data also support an expectation that engagement in initiatives and techniques to enhance communication and teamwork should not be optional.

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review.

BACKGROUND: Specialist training bodies continue to devise innovative methods of gathering information on trainee workplace performance to meet the requirements of competency-based medical education. We reviewed recent innovations in workplace-based assessment (WBA) tools to identify strengths, weaknesses, and trade-offs inherent in their design and use. METHODS: In this scoping review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we systematically searched databases between 2009 and 2019 for WBA tools with novel characteristics not typically seen in traditional WBAs. These included innovations in rating scales, ways of collecting information, technological innovations, ways of triggering WBAs, and approaches to compiling and using information. RESULTS: We identified 30 innovative WBA tools whose characteristics could be categorised into seven dimensions: frequency of assessment, granularity (unit of performance assessed), coverage of the curriculum, rating method, initiation of the WBA, information use, and incentives. These dimensions had multiple interdependencies and trade-offs, often balancing generating assessment data with available resources. Philosophical stance on assessment also influenced WBA choice, for example prioritising trainee-centred learning (i.e. initiation of WBA and transparency of assessment data), perceptions of assessment and feedback as burdensome or beneficial, and holistic vs reductionist views on assessment of performance. CONCLUSIONS: Our synthesis of the literature on innovative WBAs provides a framework for categorising tool characteristics across seven dimensions, systematically teasing apart the considerations in design and use of workplace assessments. It also draws attention to the trade-offs inherent in tool design and selection, and enables a more deliberate consideration of the tool characteristics most appropriate to the local context.

Systematic review and narrative synthesis of competency-based medical education in anaesthesia.

BACKGROUND: Competency-based medical education (CBME) addresses the accountability of postgraduate training programmes to graduate specialists capable of independent practice. METHODS: We undertook a systematic review and narrative synthesis of the published CBME literature in anaesthesia training programmes to identify current practices and areas requiring further exploration. RESULTS: We grouped the 23 studies that met our inclusion criteria into the following categories: demonstrating outcomes of CBME, developing a consensus on an achievable CBME curriculum, CBME curriculum framework, design and implementation of workplace-based assessment (WBA) tools, trainee self-assessment, perceptions of trainees and supervisors on WBA tools, and technological solutions for assessment and feedback. Included studies reported variable success in reaching consensus in competency outcome frameworks for sequenced progression and limited research on approaches to curriculum delivery, whilst the majority of studies focused on workplace assessment. Studies supported the use of entrustment scales, where assessors make a judgement on the extent to which the trainee can manage a case independently. While evidence supported the reliability of WBA tools, and predicted the numbers needed for high-stakes decisions, areas of concern related to factors influencing the value WBA tools in promoting trainee learning, and variable perceptions of their value in making decisions on progression. CONCLUSIONS: Evidence on outcomes of CBME was limited to acquisition of specific competencies during training. The large number of unanswered questions and the dearth of studies across the core components of CBME suggest that we need a collaborative approach to create the evidence required to implement CBME wisely and cost effectively, to have positive impacts on patients, trainees, and healthcare systems.

Challenging gender stereotypes and advancing inclusive leadership in the operating theatre.

Modern healthcare is delivered by interprofessional teams, and good leadership of these teams is integral to safe patient care. Good leadership in the operating theatre has traditionally been considered as authoritative, confident and directive, and stereotypically associated with men. We argue that this may not be the best model for team-based patient care and promote the concept of inclusive leadership as a valid alternative. Inclusive leadership encourages all team members to contribute to decision-making, thus engendering more team cohesion, information sharing and speaking up, and ultimately enhancing team effectiveness. However, the relational behaviours associated with inclusive leadership are stereotypically associated with women and may not in fact be recognised as leadership. In this article we provide evidence on the advantages of inclusive leadership over authoritative leadership and explore gender stereotypes and obstacles that limit the recognition of inclusive leadership. We propose that operating teams rise above gender stereotypes of leadership. Inclusive leadership can elicit maximum performance of every team member, thus realising the full potential of interprofessional healthcare teams to provide the best care for patients.

Shadow systems in assessment: how supervisors make progress decisions in practice.

Medical educators are tasked with decisions on trainee progression and credentialing for independent clinical practice, which requires robust evidence from workplace-based assessment. It is unclear how the current promotion of workplace-based assessment as a pedagogical approach to promote learning has impacted this use of assessments for decision-making; meeting both these purposes may present unforeseen challenges. In this study we explored how supervisors make decisions on trainee progress in practice. We conducted semi-structured interviews with 19 supervisors of postgraduate anesthesia training across Australia and New Zealand and undertook thematic analysis of the transcripts. Supervisors looked beyond the formal assessment portfolio when making performance decisions. They instead used assessment 'shadow systems' based on their own observation and confidential judgements from trusted colleagues. Supervisors' decision making involved expert judgement of the perceived salient aspects of performance and the standard to be attained while making allowances for the opportunities and constraints of the local learning environment. Supervisors found making progress decisions an emotional burden. When faced with difficult decisions, they found ways to share the responsibility and balance the potential consequences for the trainee with the need to protect their patients. Viewed through the lens of community of practice theory, the development of assessment 'shadow systems' indicates a lack of alignment between local workplace assessment practices and the prescribed programmatic assessment approach to high-stakes progress decisions. Avenues for improvement include cooperative development of formal assessment processes to better meet local needs or incorporating the information in 'shadow systems' into formal assessment processes.

The reliability of a portfolio of workplace-based assessments in anesthesia training.

PURPOSE: Competency-based anesthesia training programs require robust assessment of trainee performance and commonly combine different types of workplace-based assessment (WBA) covering multiple facets of practice. This study measured the reliability of WBAs in a large existing database and explored how they could be combined to optimize reliability for assessment decisions. METHODS: We used generalizability theory to measure the composite reliability of four different types of WBAs used by the Australian and New Zealand College of Anaesthetists: mini-Clinical Evaluation Exercise (mini-CEX), direct observation of procedural skills (DOPS), case-based discussion (CbD), and multi-source feedback (MSF). We then modified the number and weighting of WBA combinations to optimize reliability with fewer assessments. RESULTS: We analyzed 67,405 assessments from 1,837 trainees and 4,145 assessors. We assumed acceptable reliability for interim (intermediate stakes) and final (high stakes) decisions of 0.7 and 0.8, respectively. Depending on the combination of WBA types, 12 assessments allowed the 0.7 threshold to be reached where one assessment of any type has the same weighting, while 20 were required for reliability to reach 0.8. If the weighting of the assessments is optimized, acceptable reliability for interim and final decisions is possible with nine (e.g., two DOPS, three CbD, two mini-CEX, two MSF) and 15 (e.g., two DOPS, eight CbD, three mini-CEX, two MSF) assessments respectively. CONCLUSIONS: Reliability is an important factor to consider when designing assessments, and measuring composite reliability can allow the selection of a WBA portfolio with adequate reliability to provide evidence for defensible decisions on trainee progression.

RéSUMé: OBJECTIF: Les programmes de formation en anesthésie basés sur les compétences nécessitent de solides évaluations des performances des stagiaires et combinent habituellement des évaluations sur le lieu de travail (ÉLT) couvrant de nombreux aspects de la pratique. Cette étude a mesuré la fiabilité des ÉLT dans une grande base de données existante et a exploré comment elles pourraient être combinées pour accroître leur fiabilité pour des décisions sur les évaluations. MéTHODES: Nous avons utilisé la théorie de la généralisation pour mesurer un critère composite de fiabilité de quatre types d'ÉLT utilisés par les collèges d'anesthésiologistes d'Australie et de Nouvelle-Zélande : un exercice de mini-évaluation clinique (mini-CEX), l'observation directe des habiletés procédurales (DOPS), une discussion de cas (CbD) et une rétroaction de multiples sources (MSF). Nous avons alors modifié le nombre et la pondération des combinaisons d'ÉLT pour optimiser la fiabilité avec moins d'évaluations. RéSULTATS: Nous avons analysé 67 405 évaluations de 1 837 stagiaires et 4 145 assesseurs. Nous avons supposé une fiabilité acceptable pour les décisions intérimaires (enjeux intermédiaires) et définitives (enjeux élevés) à, respectivement, 0,7 et 0,8. Selon la combinaison des types d'ÉLT, 12 évaluations ont permis d'atteindre le seuil de 0,7 lorsqu'une évaluation de chaque type a le même poids, alors qu'il en a fallu 20 pour que la fiabilité atteigne 0,8. Si la pondération des évaluations est optimisée, la fiabilité acceptable pour les décisions intérimaires et finales est possible avec, respectivement, neuf évaluations (p. ex., deux DOPS, trois CbD, deux mini-CEX, deux MSF) et quinze évaluations (p. ex. deux DOPS, huit CbD, trois mini-CEX, deux MSF). CONCLUSIONS: La fiabilité est un facteur important dont il faut tenir compte quand on conçoit les évaluations et la mesure d'une fiabilité composite permet la sélection d'un éventail d'ÉLT avec une fiabilité adéquate pour l'obtention de données probantes et la défense de décisions sur les progrès des stagiaires.

Retesting the Hypothesis of a Clinical Randomized Controlled Trial in a Simulation Environment to Validate Anesthesia Simulation in Error Research (the VASER Study).

BACKGROUND: Simulation has been used to investigate clinical questions in anesthesia, surgery, and related disciplines, but there are few data demonstrating that results apply to clinical settings. We asked "would results of a simulation-based study justify the same principal conclusions as those of a larger clinical study?" METHODS: We compared results from a randomized controlled trial in a simulated environment involving 80 cases at three centers with those from a randomized controlled trial in a clinical environment involving 1,075 cases. In both studies, we compared conventional methods of anesthetic management with the use of a multimodal system (SAFERsleep; Safer Sleep LLC, Nashville, Tennessee) designed to reduce drug administration errors. Forty anesthesiologists each managed two simulated scenarios randomized to conventional methods or the new system. We compared the rate of error in drug administration or recording for the new system versus conventional methods in this simulated randomized controlled trial with that in the clinical randomized controlled trial (primary endpoint). Six experts were asked to indicate a clinically relevant effect size. RESULTS: In this simulated randomized controlled trial, mean (95% CI) rates of error per 100 administrations for the new system versus conventional groups were 6.0 (3.8 to 8.3) versus 11.6 (9.3 to 13.8; P = 0.001) compared with 9.1 (6.9 to 11.4) versus 11.6 (9.3 to 13.9) in the clinical randomized controlled trial (P = 0.045). A 10 to 30% change was considered clinically relevant. The mean (95% CI) difference in effect size was 27.0% (-7.6 to 61.6%). CONCLUSIONS: The results of our simulated randomized controlled trial justified the same primary conclusion as those of our larger clinical randomized controlled trial, but not a finding of equivalence in effect size.

Discovery and characterization of [(cyclopentyl)ethyl]benzoic acid inhibitors of microsomal prostaglandin E synthase-1.

We describe a novel class of acidic mPGES-1 inhibitors with nanomolar enzymatic and human whole blood (HWB) potency. Rational design in conjunction with structure-based design led initially to the identification of anthranilic acid 5, an mPGES-1 inhibitor with micromolar HWB potency. Structural modifications of 5 improved HWB potency by over 1000×, reduced CYP2C9 single point inhibition, and improved rat clearance, which led to the selection of [(cyclopentyl)ethyl]benzoic acid compound 16 for clinical studies. Compound 16 showed an IC80 of 24nM for inhibition of PGE2 formation in vitro in LPS-stimulated HWB. A single oral dose resulted in plasma concentrations of 16 that exceeded its HWB IC80 in both rat (5mg/kg) and dog (3mg/kg) for over twelve hours.

Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training.

INTRODUCTION: Workplace-based assessment is integral to programmatic assessment in a competency-based curriculum. In 2013, one such assessment, a mini-Clinical Evaluation Exercise (mini-CEX) with a novel "entrustability scale", became compulsory for over 1,200 Australia and New Zealand College of Anaesthetists (ANZCA) trainees. We explored trainees' and supervisors' understanding of the mini-CEX, their experience with the assessment, and their perceptions of its influence on learning and supervision. METHODS: We conducted semi-structured telephone interviews with anesthesia supervisors and trainees and performed an inductive thematic analysis of the verbatim transcripts. RESULTS: Eighteen supervisors and 17 trainees participated (n = 35). Interrelated themes concerned the perceived purpose of the mini-CEX, its value in trainee learning and supervision, and the process of performing the assessment. While few participants saw the mini-CEX primarily as an administrative burden, most focused on its potential for facilitating trainee improvement and reported positive impacts on the quantity and quality of feedback, trainee learning, and supervision. Finding time to schedule assessments and deliver timely feedback proved to be difficult in busy clinical workplaces. Views on case selection were divided and driven by contrasting goals - i.e., receiving useful feedback on challenging cases or receiving a high score by choosing lenient assessors or easy cases. Whether individual mini-CEXs were summative or formative was subject to intense debate, while the intended summative use of multiple mini-CEXs in programmatic assessment was poorly understood. CONCLUSION: Greater clarity of purpose and consistency of time commitment are necessary to embed the mini-CEX in the culture of the workplace, to realize the full potential for trainee learning, and to reach decisions on trainee progression.

Assessing the similarity of mental models of operating room team members and implications for patient safety: a prospective, replicated study.

BACKGROUND: Patient safety depends on effective teamwork. The similarity of team members' mental models - or their shared understanding-regarding clinical tasks is likely to influence the effectiveness of teamwork. Mental models have not been measured in the complex, high-acuity environment of the operating room (OR), where professionals of different backgrounds must work together to achieve the best surgical outcome for each patient. Therefore, we aimed to explore the similarity of mental models of task sequence and of responsibility for task within multidisciplinary OR teams. METHODS: We developed a computer-based card sorting tool (Momento) to capture the information on mental models in 20 six-person surgical teams, each comprised of three subteams (anaesthesia, surgery, and nursing) for two simulated laparotomies. Team members sorted 20 cards depicting key tasks according to when in the procedure each task should be performed, and which subteam was primarily responsible for each task. Within each OR team and subteam, we conducted pairwise comparisons of scores to arrive at mean similarity scores for each task. RESULTS: Mean similarity score for task sequence was 87 % (range 57-97 %). Mean score for responsibility for task was 70 % (range = 38-100 %), but for half of the tasks was only 51 % (range = 38-69 %). Participants believed their own subteam was primarily responsible for approximately half the tasks in each procedure. CONCLUSIONS: We found differences in the mental models of some OR team members about responsibility for and order of certain tasks in an emergency laparotomy. Momento is a tool that could help elucidate and better align the mental models of OR team members about surgical procedures and thereby improve teamwork and outcomes for patients.

Trainee anaesthetist self-assessment using an entrustment scale in workplace-based assessment.

The role of self-assessment in workplace-based assessment remains contested. However, anaesthesia trainees need to learn to judge the quality of their own work. Entrustment scales have facilitated a shared understanding of performance standards among supervisors by aligning assessment ratings with everyday clinical supervisory decisions. We hypothesised that if the entrustment scale similarly helped trainees in their self-assessment, there would be substantial agreement between supervisor and trainee ratings. We collected separate mini-clinical evaluation exercises forms from 113 anaesthesia trainee-supervisor pairs from three hospitals in Australia and New Zealand. We calculated the agreement between trainee and supervisor ratings using Pearson and intraclass correlation coefficients. We also tested for associations with demographic variables and examined narrative comments for factors influencing rating. We found ratings agreed in 32% of cases, with 66% of trainee ratings within one point of the supervisor rating on a nine-point scale. The correlation between trainee and supervisor ratings was 0.71, and the degree of agreement measured by the intraclass correlation coefficient was 0.67. With higher supervisor ratings, trainee ratings better correlated with supervisor ratings. We found no strong association with demographic variables. Possible explanations of divergent ratings included one party being unaware of a vital aspect of the performance and different interpretations of the prospective nature of the scale. The substantial concordance between trainee and supervisor ratings supports the contention that the entrustment scale helped produce a shared understanding of the desired performance standard. Discussion between trainees and supervisors on the reasoning underlying their respective judgements would provide further opportunities to enhance this shared understanding.

Simulation in clinical teaching and learning.

Simulation-based education (SBE) is a rapidly developing method of supplementing and enhancing the clinical education of medical students. Clinical situations are simulated for teaching and learning purposes, creating opportunities for deliberate practice of new skills without involving real patients. Simulation takes many forms, from simple skills training models to computerised full-body mannequins, so that the needs of learners at each stage of their education can be targeted. Emerging evidence supports the value of simulation as an educational technique; to be effective it needs to be integrated into the curriculum in a way that promotes transfer of the skills learnt to clinical practice. Currently, SBE initiatives in Australia are fragmented and depend on local enthusiasts; Health Workforce Australia is driving initiatives to develop a more coordinated national approach to optimise the benefits of simulation.

Hospital accreditation processes in Saudi Arabia: a thematic analysis of hospital staff experiences.

BACKGROUND: Hospital accreditation by an international organisation can play an important role in health quality and safety. However, little is known about how managers and front-line employees experience and perceive the effects of accreditation. Their views could inform quality improvement processes and procedures. OBJECTIVE: To explore perceptions of employees at the managerial level on the Joint Commission International (JCI) accreditation process and its impact on quality of patient care in Saudi Arabian JCI-accredited hospitals. METHODS: We undertook a qualitative study using semi-structured interviews to explore the perspectives of senior staff from three accredited public hospitals in Saudi Arabia. Interviews were transcribed prior to thematic analysis. RESULTS: Twenty managers participated in the interviews. The following inter-related themes emerged concerning the JCI accreditation process and its impact on quality of patient care: drivers for the change; the plan for the change; the process of the change; maintaining changes post-accreditation and patients' issues. Participants were positive in their accounts of: drivers for the change; planning for the change needed to achieve accreditation and managing patients' issues. However, participants reported less favourably on: the process of the change; and maintaining changes post-accreditation. CONCLUSION: The planning stage was perceived as the easiest component of JCI accreditation. Implementing and maintaining changes post-accreditation that demonstrably promote patient safety and quality of care was perceived as more difficult. When planning for accreditation, institutions need to incorporate strategies to ensure that improvements to care continue beyond the accreditation period.