Competencies for proficiency in basic point-of-care ultrasound in anesthesiology: national expert recommendations using Delphi methodology.

PURPOSE: Point-of-care ultrasound (POCUS) allows for rapid bedside assessment and guidance of patient care. Recently, POCUS was included as a mandatory component of Canadian anesthesiology training; however, there is no national consensus regarding the competencies to guide curriculum development. We therefore aimed to define national residency competencies for basic perioperative POCUS proficiency. METHODS: We adopted a Delphi process to delineate relevant POCUS competencies whereby we circulated an online survey to academic anesthesiologists identified as POCUS leads/experts (n = 25) at all 17 Canadian anesthesiology residency programs. After reviewing a list of competencies derived from the Royal College of Physicians and Surgeons of Canada's National Curriculum, we asked participants to accept, refine, delete, or add competencies. Three rounds were completed between 2022 and 2023. We discarded items with < 50% agreement, revised those with 50-79% agreement based upon feedback provided, and maintained unrevised those items with ≥ 80% agreement. RESULTS: We initially identified and circulated (Round 1) 74 competencies across 19 clinical domains (e.g., basics of ultrasound [equipment, nomenclature, clinical governance, physics]; cardiac [left ventricle, right ventricle, valve assessment, pericardial effusion, intravascular volume status] and lung ultrasound anatomy, image acquisition, and image interpretation; and clinical applications [monitoring and serial assessments, persistent hypotension, respiratory distress, cardiac arrest]). After three Delphi rounds (and 100% response rate maintained), panellists ultimately agreed upon 75 competencies. CONCLUSION: Through national expert consensus, this study identified POCUS competencies suitable for curriculum development and assessment in perioperative anesthesiology. Next steps include designing and piloting a POCUS curriculum and assessment tool(s) based upon these nationally defined competencies.

RéSUMé: OBJECTIF: L'échographie ciblée (POCUS) permet une évaluation rapide au chevet des patient·es et l'orientation des soins aux patient·es. Récemment, l'échographie ciblée a été incluse en tant que composante obligatoire de la formation en anesthésiologie au Canada; cependant, il n'y a pas de consensus national sur les compétences qui guideront l'élaboration des programmes d'études. Nous avons donc cherché à définir les compétences à inclure dans les programmes de résidence nationaux pour acquérir des compétences de base en échographie ciblée périopératoire. MéTHODE: Nous avons adopté un processus Delphi pour délimiter les compétences pertinentes en échographie ciblée, processus dans le cadre duquel nous avons fait circuler un sondage en ligne auprès d'anesthésiologistes universitaires identifié·es comme des responsables/expert·es en échographie ciblée (n = 25) dans les 17 programmes canadiens de résidence en anesthésiologie. Après avoir examiné une liste de compétences tirées du programme d'études national du Collège royal des médecins et chirurgiens du Canada, nous avons demandé aux participant·es d'accepter, de peaufiner, de supprimer ou d'ajouter des compétences. Trois rondes ont été complétées entre 2022 et 2023. Nous avons écarté les éléments ayant < 50 % d'accord, révisé ceux avec 50 à 79 % d'accord en fonction des commentaires fournis, et maintenu sans révision les éléments obtenant ≥ 80 % d'accord. RéSULTATS: Nous avons d'abord identifié et diffusé (ronde 1) 74 compétences dans 19 domaines cliniques (p. ex., les bases de l'échographie [équipement, nomenclature, gouvernance clinique, physique]; anatomie échographique cardiaque [ventricule gauche, ventricule droit, évaluation valvulaire, épanchement péricardique, état du volume intravasculaire] et pulmonaire [acquisition et interprétation d'images]; et applications cliniques [surveillance et évaluations en série, hypotension persistante, détresse respiratoire, arrêt cardiaque]). Après trois rondes Delphi (et un taux de réponse de 100 % maintenu), les panélistes se sont finalement mis·es d'accord sur 75 compétences. CONCLUSION: Grâce à un consensus d'expert·es au pays, cette étude a permis d'identifier les compétences en échographie ciblée adaptées à l'élaboration et à l'évaluation de programmes d'études en anesthésiologie périopératoire. Les prochaines étapes comprennent la conception et la mise à l'essai d'un programme d'études et d'outils d'évaluation en échographie ciblée basés sur ces compétences définies à l'échelle nationale.

Cognitive load theory in action: e-learning modules improve performance in simulation-based education. A pilot study.

PURPOSE: Learners participating in simulation-based education may experience cognitive overload with potential detrimental effects to learning and performance. Multiple strategies have been proposed to mitigate this detrimental response. However, these strategies have not fully considered the potential benefits of using online platforms, such as accessibility, cost-effectiveness, efficiency, and scalability. Addressing this gap in the literature, preparatory online modules were developed by applying concepts from cognitive load theory and simulation-based education. This study assessed whether using preparatory online modules to deliver weekly pre-briefing content could impact cognitive load and performance. The participants were first-year postgraduate medical trainees participating in a simulation-based resuscitation curriculum. METHODS: Fifty-three trainees were allocated to receive preparatory online modules (online modules group, n = 27) or not (control group, n = 26) during the course component of a simulation-based resuscitation curriculum. Then, these trainees participated in a simulation-based objective structured clinical examination (OSCE). Sources of cognitive load (intrinsic, extraneous, and germane) were measured using a modified cognitive load questionnaire. Performance was assessed using the Ottawa Surgical Competency Operating Room Evaluation. Data were analyzed with descriptive statistics, and principal component analysis. RESULTS: During the course component, the online modules group was found to have higher intrinsic and germane cognitive load, and lower extraneous cognitive load compared to the control group. During the OSCE, the online modules group scored significantly higher in performance scores (p = 0.0077, d = 0.39, 95% confidence interval = 0.10;0.68) compared to the control group. Principal component analysis supported the results obtained with the modified cognitive load questionnaire. CONCLUSION: Trainees using preparatory online modules during the course component of a simulation-based resuscitation curriculum experienced cognitive load changes consistent with cognitive optimization. This may have contributed to their superior performance in the subsequent OSCE. Future research should explore the long-term impacts of online preparatory training and consider potential barriers to implementation in diverse healthcare environments.

RéSUMé: OBJECTIF: Les personnes apprenantes qui participent à un enseignement basé sur la simulation peuvent éprouver une surcharge cognitive pouvant avoir des effets néfastes sur l'apprentissage et le rendement. Plusieurs stratégies ont été proposées pour atténuer cette réaction préjudiciable. Toutefois, ces stratégies n'ont pas pleinement tenu compte des avantages potentiels de l'utilisation de plateformes en ligne, comme l'accessibilité, la rentabilité, l'efficience et l'évolutivité. Pour combler cette lacune dans la littérature, des modules préparatoires en ligne ont été développés en appliquant les concepts de la théorie de la charge cognitive et de l'éducation basée sur la simulation. Cette étude a évalué si l'utilisation de modules en ligne préparatoires pour fournir un contenu de pré-briefing hebdomadaire pourrait avoir un impact sur la charge cognitive et la performance. Les participants étaient des étudiants en médecine de troisième cycle de première année participant à un programme de réanimation par simulation. MéTHODES: Cinquante-trois stagiaires ont reçu des modules préparatoires en ligne (groupe de modules en ligne, n = 27) ou non (groupe témoin, n = 26) au cours de la composante de cours d'un programme de réanimation par simulation. Ces stagiaires ont ensuite participé à un examen clinique objectif structuré basé sur la simulation (OSCE). Les sources de charge cognitive (intrinsèque, étrangère et pertinente) ont été mesurées à l'aide d'un questionnaire sur la charge cognitive modifiée. Le rendement a été évalué à l'aide de l'évaluation de la salle d'opération des compétences en chirurgie d'Ottawa. Les données ont été analysées à l'aide de statistiques descriptives et d'une analyse en composantes principales. RéSULTATS: Au cours de la composante de cours, le groupe des modules en ligne s'est avéré avoir une charge cognitive intrinsèque plus élevée et une charge cognitive étrangère plus faible par rapport au groupe témoin. Au cours de l'OSCE, le groupe des modules en ligne a obtenu des scores de performance significativement plus élevés (p = 0,0077, d = 0,39, intervalle de confiance à 95 % = 0,10;0,68) que le groupe témoin. L'analyse en composantes principales a appuyé les résultats obtenus avec le questionnaire sur la charge cognitive modifiée. CONCLUSION: Les stagiaires utilisant des modules préparatoires en ligne pendant la composante de cours d'un programme de réanimation basé sur la simulation ont subi des changements de charge cognitive compatibles avec l'optimisation cognitive. Cela peut avoir contribué à leur performance supérieure dans l'OSCE ultérieure. Les recherches futures devraient explorer les impacts à long terme de la formation préparatoire en ligne et examiner les obstacles potentiels à la mise en œuvre dans divers environnements de soins de santé.

The Assessment Burden in Competency-Based Medical Education: How Programs Are Adapting.

Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing.

Multi-source feedback following simulated resuscitation scenarios: a qualitative study.

Background: The direct observation and assessment of learners' resuscitation skills by an attending physician is challenging due to the unpredictable and time-sensitive nature of these events. Multisource feedback (MSF) may address this challenge and improve the quality of assessments provided to learners. We aimed to describe the similarities and differences in the assessment rationale of attending physicians, registered nurses, and resident peers in the context of a simulation-based resuscitation curriculum. Methods: We conducted a qualitative content analysis of narrative MSF of medical residents in their first postgraduate year of training who were participating in a simulation-based resuscitation course at two Canadian institutions. Assessments included an entrustment score and narrative comments from attending physicians, registered nurses, and resident peers in addition to self-assessment. Narrative comments were transcribed and analyzed thematically using a constant comparative method. Results: All 87 residents (100%) participating in the 2017-2018 course provided consent. A total of 223 assessments were included in our analysis. Four themes emerged from the narrative data: 1) Communication, 2) Leadership, 3) Demeanor, and 4) Medical Expert. Relative to other assessor groups, feedback from nurses focused on patient-centred care and communication while attending physicians focused on the medical expert theme. Peer feedback was the most positive. Self-assessments included comments within each of the four themes. Conclusions: In the context of a simulation-based resuscitation curriculum, MSF provided learners with different perspectives in their narrative assessment rationale and may offer a more holistic assessment of resuscitation skills within a competency-based medical education (CBME) program of assessment.

Contexte: Le contexte imprévisible et contraignant au niveau du temps lors de l'observation directe et de la rétroaction associée sur les compétences en réanimation des apprenants constituent un défi pour un médecin superviseur. La rétroaction multisources (RMS) peut être un moyen de relever ce défi et d'améliorer la qualité des rétroactions fournies aux apprenants. Nous visons à décrire les similitudes et les différences quant à la démarche évaluative auprès de médecins traitants, d'infirmières cliniciennes et de pairs résidents dans le cadre d'un cours de réanimation offert par simulation. Méthodes: Nous avons réalisé une analyse de contenu à partir des rétroactions narratives offertes aux résidents en première année de formation postdoctorale dans deux universités canadiennes dans le cadre d'un cours de réanimation offert par simulation. En plus de l'auto-évaluation, la rétroaction comportait un score de confiance et des commentaires narratifs de la part de médecins superviseurs, d'infirmières cliniciennes et des pairs. Les commentaires ont été transcrits et analysés par thèmes en appliquant la méthode générale de comparaison constante. Résultats: Un consentement pour participer à l'étude a été obtenu auprès des 87 résidents (100 %) qui ont suivi le cours en 2017-2018. Nous avons analysé un total de 223 rétroactions. Quatre thèmes ont émergé à partir des données narratives soit : 1) la communication, 2) le leadership, 3) le comportement, et 4) l'expertise médicale. Alors que les infirmières ont ciblé leurs commentaires sur les soins centrés sur le patient et la communication, les médecins superviseurs ont les ont ciblés sur l'expertise médicale. Les commentaires des pairs étaient les plus positifs. Les auto-évaluations comportaient des commentaires sur chacun des quatre thèmes. Conclusions: Dans le contexte d'un cours de réanimation offert par simulation, la RMS a permis aux apprenants d'obtenir des évaluations narratives selon différentes perspectives. Permettant ainsi une approche plus holistique de rétroaction sur les habiletés en réanimation dans le cadre d'un programme d'évaluation axé sur les compétences .

A scoping review of nontechnical skill assessment tools to evaluate trauma team performance.

BACKGROUND: The objective of this scoping review was to identify assessment tools of trauma team performance (outside of technical skills) and assess the validity and reliability of each tool in assessing trauma team performance. METHODS: We searched Embase, Cochrane Library, Web of Science, Ovid Medline, and Cumulative Index to Nursing and Allied Health Literature from inception to June 1, 2021. English studies that evaluated trauma team performance using nontechnical skill assessment tools in a simulation or real-world setting were included. Studies were assessed by two independent reviewers for meeting inclusion/exclusion criteria. Data regarding team assessment tools were extracted and synthesized into behavior domains. Each tool was then assessed for validity and reliability. RESULTS: The literature search returned 4,215 articles with 29 meeting inclusion criteria. Our search identified 12 trauma team performance assessment tools. Most studies were conducted in the United States (n = 20 [69%]). Twenty studies (69%) assessed trauma team performance in a simulation setting; Team Emergency Assessment Measure (TEAM) and Trauma Nontechnical Skills Scale (T-NOTECHS) were the only tools to be applied in a simulation and real-world setting. Most studies assessed trauma team performance using video review technology (n = 17 [59%]). Five overarching themes were designed to encompass behavioral domains captured across the 12 tools: (1) Leadership, (2) Communication, (3) Teamwork, (4) Assessment, and (5) Situation Awareness. The reliability and validity of T-NOTECHS were investigated by the greatest number of studies (n = 13); however, TEAM had the most robust evidence of reliability and validity. CONCLUSION: We identified 12 trauma team performance tools that assessed nontechnical skills to varying degrees. Trauma Nontechnical Skills Scale and TEAM tools had the most evidence to support their reliability and validity. Considering the limited research in the impact of trauma team performance on patient outcomes, future studies could use video review technology in authentic trauma cases to further study this important relationship. LEVEL OF EVIDENCE: Systematic reviews and meta-analyses, level IV.

Cognitive load and processes during chest radiograph interpretation in the emergency department across the spectrum of expertise.

BACKGROUND: In the emergency department (ED), chest radiographs (CXRs) provide essential information for clinical diagnostic reasoning. Errors in interpretation by emergency physicians can lead to negative patient outcomes. To aid in teaching this important skill, an understanding of cognitive processes and cognitive load (CL) in CXR interpretation in emergency medicine (EM) personnel is warranted. METHODS: This study adopted a concurrent mixed-methods research design. Participant groups included medical students (M), junior (J) and senior (S) EM residents, and attending emergency physicians (P) in the ED at an academic hospital. To elucidate cognitive processes, a real-time cognitive task analysis during CXR interpretation was performed. Interviews were audio recorded, transcribed verbatim, and analyzed thematically. The interview was followed by a questionnaire, where participants rated their CL, stress, and confidence level. RESULTS: Levels of CL (M vs. S and M vs. P, p = 0.002; J vs. S, p = 0.004; J vs. P, p = 0.005) and stress (J vs. P, p = 0.002) decreased, while confidence levels increased (M vs. S, p = 0.006; J vs. S, p ≤ 0.001; J vs. P, p = 0.003) as experience level increased. Qualitative analysis of interviews revealed four themes: checking behavior, information reduction, pattern recognition versus systematic viewing, and recognizing scope of practice. Experts commonly utilized checking behavior (e.g., comparison to prior radiographs) and deprioritized task irrelevant data. Experts used a general overview technique as their initial approach as opposed to a systematic viewing approach, and they more readily recognized an EM physicians' scope of practice in this task. CONCLUSION: This study characterized differences in cognition that led to increased CL, stress, and lower level of confidence in EM learners during CXR interpretation and provided insight into expertise development in this important skill.

Shaken and stirred: emotional state, cognitive load, and performance of junior residents in simulated resuscitation.

BACKGROUND: Patient resuscitation can be overwhelming for junior postgraduate medical residents due to its inherent complexity and high-stakes environment. Emotional states of unpleasant hyperarousal burden cognitive resources, contributing to cognitive overload and performance decline. Our objective is to characterize the associations between pre-scenario emotional state and junior residents' cognitive load and performance in a simulated-resuscitation, to provide evidence for informed curricular development. METHODS: PGY-1 residents self-rated their emotional state before four simulated-resuscitation scenarios, and their cognitive load after. Faculty assessed performance with entrustment scores. Factor analysis identified the principal components of emotional state data. Linear regression models examined the relationship between pre-scenario emotional components, cognitive load, and performance scores. RESULTS: 47/47 medical and surgical residents (100%) participated and completed Emotional State (99.5%) and Cognitive Load (98.9%) surveys. Positive invigoration and negative tranquility were the principal components. Pre-scenario tranquility was negatively associated with cognitive load (b= -0.23, p < 0.0001), and cognitive load was negatively associated with performance scores (b= -0.27, p < 0.0001). Pre-scenario invigoration was negatively associated with cognitive load (b=-0.18, p = 0.0001), and positively associated with performance scores (b= 0.08, p = 0.0193). CONCLUSION: Amongst junior residents participating in simulated resuscitation scenarios, pre-scenario agitation (negative tranquility) is associated with increased cognitive load, which itself is associated with lower performance scores. These findings suggest residency programs should consider developing curriculum aimed at modulating residents' emotional agitation and reducing residents' cognitive burden to improve resuscitation performance.

CONTEXTE: La réanimation de patients peut être éprouvante pour les stagiaires postdoctoraux juniors en raison de la complexité qui y est inhérente et de la gravité de l'enjeu. Les états émotionnels désagréables d'hyperexcitation épuisent les ressources cognitives, contribuant ainsi à la surcharge cognitive et à la baisse de la performance. Notre objectif était de mettre en évidence le rapport entre l'état émotionnel des résidents juniors avant une simulation de réanimation d'un côté et leur charge cognitive et leur performance lors de celle-ci de l'autre, pour produire des données probantes pouvant servir à la conception éclairée de programmes d'enseignement. MÉTHODES: Des résidents de première année ont autoévalué leur état émotionnel avant chacun des quatre scénarios de réanimation simulée, ainsi que leur charge cognitive après les simulations. Des membres du corps professoral ont évalué leur performance par l'attribution de scores de confiance. Les principales composantes des données sur l'état émotionnel ont été déterminées par le biais d'une analyse factorielle. On s'est servi de modèles de régression linéaire pour établir la relation entre les composantes émotionnelles avant la simulation, la charge cognitive et les scores de performance. RÉSULTATS: Les 47 résidents en médecine et en chirurgie qui ont participé à l'étude (100 %) ont rempli les questionnaires sur l'état émotionnel (99,5 %) et la charge cognitive (98,9 %). La stimulation positive et la tranquillité négative sont les principales composantes dégagées. La tranquillité avant la simulation était négativement corrélée avec la charge cognitive (b= -0,23, p<0,0001), et la charge cognitive était négativement liée aux scores de performance (b= -0,27, p<0,0001). La stimulation avant la simulation était négativement corrélée avec la charge cognitive (b=-0.18, p=0.0001), et positivement corrélée avec les scores de performance (b= 0.08, p=0.0193). CONCLUSION: Chez les résidents juniors qui ont participé à des scénarios de réanimation simulée, l'agitation précédant cette dernière (tranquillité négative) était liée à une charge cognitive accrue, qui elle-même a donné lieu à des scores de performance plus faibles. Ces résultats montrent la pertinence de concevoir des programmes qui visent à réduire l'agitation émotionnelle et la charge cognitive des résidents afin d'améliorer leurs performances en réanimation.

An exploratory investigation of the measurement of cognitive load on shift: Application of cognitive load theory in emergency medicine.

BACKGROUND: Emergency physicians often experience a high cognitive load (CL) due to the inherent nature of working in acute care settings. CL has traditionally been measured in educational studies but has not been well studied in the clinical environment. METHODS: Emergency medicine attending physicians and residents working in an academic urgent care center completed psychometric questionnaires while on shift to measure overall CL, intrinsic cognitive load (ICL), extraneous cognitive load (ECL), and acute stress. Data regarding the patient load, patient acuity, and the number of patients in the waiting room were also collected. Correlational analysis and simple linear regression were used to evaluate predictors of CL on shift. RESULTS: Forty-two questionnaires were completed (26 by attending physicians, 16 by residents). Attending physicians carried a significantly higher patient load compared to residents (p < 0.001). No differences in mean overall CL, ICL, ECL, and acute stress were observed between attending physicians and residents. Bivariate analysis demonstrated associations between ICL, ECL, acute stress, and overall CL in attending physicians. In residents, acute stress was the only variable associated with overall CL and the number of high-acuity patients was associated with ICL. CONCLUSIONS: Factors influencing reported CL during clinical work are different between attending emergency physicians and residents. Further study to appreciate the impact of these differences is required and may help educators elucidate strategies to better manage CL, thereby improving clinical performance and potentially improving patient care.

The future of simulation-based medical education: Adaptive simulation utilizing a deep multitask neural network.

BACKGROUND: In resuscitation medicine, effectively managing cognitive load in high-stakes environments has important implications for education and expertise development. There exists the potential to tailor educational experiences to an individual's cognitive processes via real-time physiologic measurement of cognitive load in simulation environments. OBJECTIVE: The goal of this research was to test a novel simulation platform that utilized artificial intelligence to deliver a medical simulation that was adaptable to a participant's measured cognitive load. METHODS: The research was conducted in 2019. Two board-certified emergency physicians and two medical students participated in a 10-minute pilot trial of a novel simulation platform. The system utilized artificial intelligence algorithms to measure cognitive load in real time via electrocardiography and galvanic skin response. In turn, modulation of simulation difficulty, determined by a participant's cognitive load, was facilitated through symptom severity changes of an augmented reality (AR) patient. A postsimulation survey assessed the participants' experience. RESULTS: Participants completed a simulation that successfully measured cognitive load in real time through physiological signals. The simulation difficulty was adapted to the participant's cognitive load, which was reflected in changes in the AR patient's symptoms. Participants found the novel adaptive simulation platform to be valuable in supporting their learning. CONCLUSION: Our research team created a simulation platform that adapts to a participant's cognitive load in real-time. The ability to customize a medical simulation to a participant's cognitive state has potential implications for the development of expertise in resuscitation medicine.

Exploring the use of rating scales with entrustment anchors in workplace-based assessment.

PURPOSE: Competency-based medical education (CBME) has prompted widespread implementation of workplace-based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. METHOD: A convenience sample of 50 semi-structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line-by-line coding to identify themes. RESULTS: Interviews captured interactions between 70% (26/37) of full-time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were 'in the room' or not. CONCLUSIONS: By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of 'in-the-moment' assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.

The medical pause: Importance, processes and training.

Research has shown that taking 'timeouts' in medical practice improves performance and patient safety. However, the benefits of taking timeouts, or pausing, are not sufficiently acknowledged in workplaces and training programmes. To promote this acknowledgement, we suggest a systematic conceptualisation of the medical pause, focusing on its importance, processes and implementation in training programmes. By employing insights from educational and cognitive psychology, we first identified pausing as an important skill to interrupt negative momentum and bolster learning. Subsequently, we categorised constituent cognitive processes for pausing skills into two phases: the decision-making phase (determining when and how to take pauses) and the executive phase (applying relaxation or reflection during pauses). We present a model that describes how relaxation and reflection during pauses can optimise cognitive load in performance. Several strategies to implement pause training in medical curricula are proposed: intertwining pause training with training of primary skills, providing second-order scaffolding through shared control and employing auxiliary tools such as computer-based simulations with a pause function.

Differences in Gaze Fixation Location and Duration Between Resident and Fellowship Sonographers Interpreting a Focused Assessment With Sonography in Trauma.

OBJECTIVES: We quantified the gaze fixation duration of resident and fellowship sonographers interpreting a prerecorded focused assessment with sonography in trauma (FAST). We hypothesized that all sonographers would fixate on each relevant anatomic relationship but that the duration of fixation would differ. METHODS: We conducted a cross-sectional study collecting and analyzing the gaze fixations of a convenience sample of current resident and fellowship sonographers. All sonographers viewed a standardized FAST video, and their gaze fixations were recorded using a Tobii X3-120 eye-tracking bar. Gaze fixations over nine anatomic regions of interest (ROIs) were identified. These were assessed for normality and analyzed using the Wilcoxon rank sum test at an alpha of 0.05 and Bonferroni correction p value of <0.0034. The chi-square test and Pearson's correlation were performed to assess statistical association. RESULTS: The gaze fixation recordings of 24 resident and eight fellowship sonographers were suitable for analysis. Fourteen of the 24 resident sonographers viewed all ROIs in the FAST, whereas all eight fellowship sonographers viewed each of the nine relevant ROIs. Five ROIs were identified over which at least one resident sonographer did not have a gaze fixation. No statistically significant difference was identified between groups. Resident sonographers gaze fixated over the left upper quadrant (LUQ) splenorenal interface for a median (interquartile range) of 10.64 (9.73-11.60) seconds. The fellowship group viewed the same ROI for 8.43 (6.64-8.95) seconds (p < 0.003). All participants viewed this ROI. No other ROIs had a statistical difference. CONCLUSION: Five ROIs were identified that were not visually interrogated by all resident sonographers. Only 14 of 24 resident sonographers visually interrogated every area in the FAST, whereas all fellowship sonographers interrogated every ROI. A statistically significant difference was found in gaze fixation duration between resident and fellowship sonographers in one ROI. Further study is required for gaze fixation assessment to become a tool for the interpretation component of point-of-care ultrasound.

Simulation in the Continuing Professional Development of Academic Emergency Physicians: A Canadian National Survey.

INTRODUCTION: Simulation is becoming a popular educational modality for physician continuing professional development (CPD). This study sought to characterize how simulation-based CPD (SBCPD) is being used in Canada and what academic emergency physicians (AEPs) desire in an SBCPD program. METHODS: Two national surveys were conducted from March to June 2018. First, the SBCPD Needs Assessment Survey was administered online to all full-time AEPs across 9 Canadian academic emergency medicine (EM) sites. Second, the SBCPD Status Survey was administered by telephone to the department representatives (DRs)-simulation directors or equivalent-at 20 Canadian academic EM sites. RESULTS: Response rates for the SBCPD Needs Assessment and the SBCPD Status Survey were 40% (252/635) and 100% (20/20) respectively. Sixty percent of Canadian academic EM sites reported using SBCPD, although only 30% reported dedicated funding support. Academic emergency physician responses demonstrated a median annual SBCPD of 3 hours. Reported incentivization for SBCPD participation varied with AEPs reporting less incentivization than DRs. Academic emergency physicians identified time commitments outside of shift, lack of opportunities, and lack of departmental funding as their top barriers to participation, whereas DRs thought AEPs fear of peer judgment and inexperience with simulation were substantial barriers. Content areas of interest for SBCPD were as follows: rare procedures, pediatric resuscitation, and neonatal resuscitation. Lastly, interprofessional involvement in SBCPD was valued by both DRs and AEPs. CONCLUSIONS: Simulation-based CPD programs are becoming common in Canadian academic EM sites. Our findings will guide program coordinators in addressing barriers to participation, selecting content, and determining the frequency of SBCPD events.

From Theory to Practice: The Application of Cognitive Load Theory to the Practice of Medicine.

Cognitive load theory has become a leading model in educational psychology and has started to gain traction in the medical education community over the last decade. The theory is rooted in our current understanding of human cognitive architecture in which an individual's limited working memory and unlimited long-term memory interact during the process of learning. Though initially described as primarily a theory of learning, parallels between cognitive load theory and broader aspects of medical education as well as clinical practice are now becoming clear. These parallels are particularly relevant and evident in complex clinical environments, like resuscitation medicine. The authors have built on these connections to develop a recontextualized version of cognitive load theory that applies to complex professional domains and in which the connections between the theory and clinical practice are made explicit, with resuscitation medicine as a case study. Implications of the new model for medical education are also presented along with suggested applications.

The Development of Visual Expertise in ECG Interpretation: An Eye-Tracking Augmented Re Situ Interview Approach.

Phenomenon: Visual expertise in medicine involves a complex interplay between expert visual behavior patterns and higher-level cognitive processes. Previous studies of visual expertise in medicine have centered around traditionally visually intensive disciplines such as radiology and pathology. However, there is limited study of visual expertise in electrocardiogram (ECG) interpretation, a common clinical task that is associated with high error rates. This qualitatively driven multi-methods study aimed to describe differences in cognitive approaches to ECG interpretation between medical students, emergency medicine (EM) residents, and EM attending physicians. Approach: Ten medical students, 10 EM residents, and 10 EM attending physicians were recruited from one tertiary academic center to participate in this study. Participants interpreted 10 ECGs with a screen-based eye-tracking device, then underwent a subjective re situ interview augmented by playback of the participants' own gaze scan-paths via eye-tracking. Interviews were transcribed verbatim and an emergent thematic analysis was performed across participant groups. Diagnostic speed, accuracy, and heat maps of fixation distribution were collected to supplement the qualitative findings. Findings: Qualitative analysis demonstrated differences among the cohorts in three major themes: dual-process reasoning, ability to prioritize, and clinical implications. These qualitative findings were aligned with differences in visual behavior demonstrated by heat maps of fixation distribution across each ECG. More experienced participants completed ECG interpretation significantly faster and more accurately than less experienced participants. Insights: The cognitive processes related to ECG interpretation differed between novices and more experienced providers in EM. Understanding the differences in cognitive approaches to ECG interpretation between these groups may help inform best practices in teaching this ubiquitous diagnostic skill.

A case for feedback and monitoring assessment in competency-based medical education.

PURPOSE: Within competency-based medical education, self-regulated learning (SRL) requires residents to leverage self-assessment and faculty feedback. We sought to investigate the potential for competency-based assessments to foster SRL by quantifying the relationship between faculty feedback and entrustment ratings as well as the congruence between faculty assessment and resident self-assessment. MATERIALS AND METHODS: We collected comments in (a) an emergency medicine objective structured clinical examination group (objective structured clinical examinations [OSCE] and emergency medicine OSCE group [EMOG]) and (b) a first-year resident multidisciplinary resuscitation "Nightmares" course assessment group (NCAG) and OSCE group (NOG). We assessed comments across five domains including Initial Assessment (IA), Diagnostic Action (DA), Therapeutic Action (TA), Communication (COM), and entrustment. Analyses included structured qualitative coding and (non)parametric and descriptive analyses. RESULTS: In the EMOG, faculty's positive comments in the entrustment domain corresponded to lower entrustment score Mean Ranks (MRs) for IA (<11.1), DA (<11.2), and entrustment (<11.6). In NOG, faculty's negative comments resulted in lower entrustment score MRs for TA (<11.8 and <10) and DA (<12.4), and positive comments resulted in higher entrustment score MRs for IA (>15.4) and COM (>17.6). In the NCAG, faculty's positive IA comments were negatively correlated with entrustment scores (ρ = -.27, P = .04). Across programs, faculty and residents made similar domain-specific comments 13% of the time. CONCLUSIONS: Minimal and inconsistent associations were found between narrative and numerical feedback. Performance monitoring accuracy and feedback should be included in assessment validation.

Toward Dynamically Adaptive Simulation: Multimodal Classification of User Expertise Using Wearable Devices.

Simulation-based training has been proven to be a highly effective pedagogical strategy. However, misalignment between the participant's level of expertise and the difficulty of the simulation has been shown to have significant negative impact on learning outcomes. To ensure that learning outcomes are achieved, we propose a novel framework for adaptive simulation with the goal of identifying the level of expertise of the learner, and dynamically modulating the simulation complexity to match the learner's capability. To facilitate the development of this framework, we investigate the classification of expertise using biological signals monitored through wearable sensors. Trauma simulations were developed in which electrocardiogram (ECG) and galvanic skin response (GSR) signals of both novice and expert trauma responders were collected. These signals were then utilized to classify the responders' expertise, successive to feature extraction and selection, using a number of machine learning methods. The results show the feasibility of utilizing these bio-signals for multimodal expertise classification to be used in adaptive simulation applications.

Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring.

BACKGROUND: Simulation is increasingly being used in postgraduate medical education as an opportunity for competency assessment. However, there is limited direct evidence that supports performance in the simulation lab as a surrogate of workplace-based clinical performance for non-procedural tasks such as resuscitation in the emergency department (ED). We sought to directly compare entrustment scoring of resident performance in the simulation environment to clinical performance in the ED. METHODS: The resuscitation assessment tool (RAT) was derived from the previously implemented and studied Queen's simulation assessment tool (QSAT) via a modified expert review process. The RAT uses an anchored global assessment scale to generate an entrustment score and narrative comments. Emergency medicine (EM) residents were assessed using the RAT on cases in simulation-based examinations and in the ED during resuscitation cases from July 2016 to June 2017. Resident mean entrustment scores were compared using Pearson's correlation coefficient to determine the relationship between entrustment in simulation cases and in the ED. Inductive thematic analysis of written commentary was conducted to compare workplace-based with simulation-based feedback. RESULTS: There was a moderate, positive correlation found between mean entrustment scores in the simulated and workplace-based settings, which was statistically significant (r = 0.630, n = 17, p < 0.01). Further, qualitative analysis demonstrated overall management and leadership themes were more common narratives in the workplace, while more specific task-based feedback predominated in the simulation-based assessment. Both workplace-based and simulation-based narratives frequently commented on communication skills. CONCLUSIONS: In this single-center study with a limited sample size, assessment of residents using entrustment scoring in simulation settings was demonstrated to have a moderate positive correlation with assessment of resuscitation competence in the workplace. This study suggests that resuscitation performance in simulation settings may be an indicator of competence in the clinical setting. However, multiple factors contribute to this complicated and imperfect relationship. It is imperative to consider narrative comments in supporting the rationale for numerical entrustment scores in both settings and to include both simulation and workplace-based assessment in high-stakes decisions of progression.

Starting to Think Like an Expert: An Analysis of Resident Cognitive Processes During Simulation-Based Resuscitation Examinations.

STUDY OBJECTIVE: Simulation is commonly used to teach crisis resource management skills and assess them in emergency medicine residents. However, our understanding of the cognitive processes underlying crisis resource management skills is limited because these processes are difficult to assess and describe. The objective of this study is to uncover and characterize the cognitive processes underlying crisis resource management skills and to describe how these processes vary between residents according to performance in a simulation-based examination. METHODS: Twenty-two of 24 eligible emergency medicine trainees from 1 tertiary academic center completed 1 or 2 resuscitation-based examinations in the simulation laboratory. Resident performance was assessed by a blinded expert using an entrustment-based scoring tool. Participants wore eye-tracking glasses that generated first-person video that was used to augment subsequent interviews led by an emergency medicine faculty member. Interviews were audio recorded and then transcribed. An emergent thematic analysis was completed with a codebook that was developed by 4 research assistants, with subsequent analyses conducted by the lead research assistant with input from emergency medicine faculty. Themes from high- and low-performing residents were subsequently qualitatively compared. RESULTS: Higher-performing residents were better able to anticipate, selectively attend to relevant information, and manage cognitive demands, and took a concurrent (as opposed to linear) approach to managing the simulated patient. CONCLUSION: The results provide new insights into residents' cognitive processes while managing simulated patients in an examination environment and how these processes vary with performance. More work is needed to determine how best to apply these findings to improve crisis resource management education.