How Emergency Medicine Residents Have Conversations About Life-Sustaining Treatments in Critical Illness: A Qualitative Study Using Inductive Thematic Analysis.

STUDY OBJECTIVE: The inherent pressures of high-acuity, critical illness in the emergency department create a unique environment whereby acute goals-of-care discussions must be had with patients or substitute decision makers to rapidly decide between divergent treatment paths. Among university-affiliated hospitals, resident physicians are often conducting these highly consequential discussions. This study aimed to use qualitative methods to explore how emergency medicine residents make recommendations regarding life-sustaining treatments during acute goals-of-care discussions in critical illness. METHODS: Using qualitative methods, semistructured interviews were conducted with a purposive sample of emergency medicine residents in Canada from August to December 2021. Inductive thematic analysis of the interview transcripts was conducted using line-by-line coding, and key themes were identified through comparative analysis. Data collection continued until thematic saturation was reached. RESULTS: Seventeen emergency medicine residents from 9 Canadian universities were interviewed. Two factors guided residents' treatment recommendations (a duty to provide a recommendation and the balance between disease prognosis and patient values). Three factors influenced residents' comfort when making recommendations (time constraints, uncertainty, and moral distress). CONCLUSION: While conducting acute goals-of-care discussions with critically ill patients or their substitute decision makers in the emergency department, residents felt a sense of duty to provide a recommendation informed by an intersection between the patient's disease prognosis and the patient's values. Their comfort in making these recommendations was limited by time constraints, uncertainty, and moral distress. These factors are important for informing future educational strategies.

Assessing the utility of a novel entrustment-supervision assessment tool.

BACKGROUND: Work-based assessments (WBAs) are increasingly used to inform decisions about trainee progression. Unfortunately, WBAs often fail to discriminate between trainees of differing abilities and have poor reliability. Entrustment-supervision scales may improve WBA performance, but there is a paucity of literature directly comparing them to traditional WBA tools. METHODS: The Ottawa Emergency Department Shift Observation Tool (O-EDShOT) is a previously published WBA tool employing an entrustment-supervision scale with strong validity evidence. This pre-/post-implementation study compares the performance of the O-EDShOT with that of a traditional WBA tool using norm-based anchors. All assessments completed in 12-month periods before and after implementing the O-EDShOT were collected, and generalisability analysis was conducted with year of training, trainees within year and forms within trainee as nested factors. Secondary analysis included assessor as a factor. RESULTS: A total of 3908 and 3679 assessments were completed by 99 and 116 assessors, for 152 and 138 trainees in the pre- and post-implementation phases respectively. The O-EDShOT generated a wider range of awarded scores than the traditional WBA, and mean scores increased more with increasing level of training (0.32 vs. 0.14 points per year, p = 0.01). A significantly greater proportion of overall score variability was attributable to trainees using the O-EDShOT (59%) compared with the traditional tool (21%, p < 0.001). Assessors contributed less to overall score variability for the O-EDShOT than for the traditional WBA (16% vs. 37%). Moreover, the O-EDShOT required fewer completed assessments than the traditional tool (27 vs. 51) for a reliability of 0.8. CONCLUSION: The O-EDShOT outperformed a traditional norm-referenced WBA in discriminating between trainees and required fewer assessments to generate a reliable estimate of trainee performance. More broadly, this study adds to the body of literature suggesting that entrustment-supervision scales generate more useful and reliable assessments in a variety of clinical settings.

Stress and conflict from tacit culture forges professional identity in newly graduated independent physicians.

To transition successfully into independent practice, newly graduated independent physicians (new "attendings") undergo a process of professional identity formation (PIF) as a clinician within a new community of practice (CoP). PIF is crafted by socialization within a CoP including transfer of tacit knowledge. While certain tacit knowledge is critical for professional identity, we understand little how it shapes PIF. We set out to describe the tacit knowledge acquired by new attendings within a CoP and how it contributes to PIF. Informed by constructivist grounded theory, we interviewed 23 new attendings about the tacit knowledge they acquired in early practice. Data collection and analysis occurred iteratively. We identified themes using constant comparative analysis and generated a theory that underwent member checking and feedback. Implicit standards from group culture imparted high expectations on new attendings and led to internal stress. New attendings also encountered a tacit code of conduct as behavioral elements of group culture. These elements created external conflict between new attendings and group members such as departmental colleagues, consulting physicians, and other health professionals. Depending on the support they received, new attendings responded to the stress and conflict in three ways: they doubted, adjusted, or avoided. These strategies molded their professional identity, and moved them towards or away from the CoP as they navigated their transition and PIF. We describe a novel theory of how tacit group culture shaped new attending physicians' professional identity in a new community of practice. Internal stress and external conflict occurred due to high expectations and tacit culture elements. New attendings' doubt, adjust, or avoid responses, shaped by support they received, in turn crafted their professional identity. Education leaders should prepare graduating trainees to navigate aspects of transition to independent practice successfully.

A systematic review of sufentanil for the management of adults with acute pain in the emergency department and pre-hospital setting.

BACKGROUND: Pain is commonly encountered in the Emergency Department (ED) and pre-hospital setting and often requires opioid analgesia. We sought to synthesize the available evidence on the effectiveness of sufentanil for acute pain relief for adult patients in the pre-hospital or ED setting. METHODS: This systematic review was conducted in accordance with PRISMA guidelines. Medline, Embase, Cochrane CENTRAL, and CINAHL were searched from inception to February 1, 2022. The grey literature was also searched. We included randomized controlled trials of adult patients with acute pain who were treated with sufentanil. Two reviewers independently completed screening, full text review, and data extraction. Primary outcome was reduction in pain. Secondary outcomes included adverse events, need for rescue analgesia, and patient and provider satisfaction. Risk of bias was assessed using the Cochrane Risk of Bias 2 tool. A meta-analysis was not performed due to heterogeneity. RESULTS: Of 1120 unique citations, four studies (3 ED and 1 pre-hospital) met full inclusion criteria (n = 467 participants). The overall quality of the included studies was high. Intranasal (IN) sufentanil was superior to placebo for pain relief at 30 min (difference 20.8%, 95% CI 4.0-36.2%, p = 0.01). Both IN (two studies) and IV sufentanil (one study) were comparable to IV morphine. Mild adverse events were common and there was a higher propensity for minor sedation in patients receiving sufentanil. There were no serious adverse events requiring advanced interventions. CONCLUSION: Sufentanil was comparable to IV morphine and was superior to placebo for rapid relief of acute pain in the ED setting. The safety profile of sufentanil is similar to IV morphine in this setting, with minimal concern for serious adverse events. The intranasal formulation may provide an alternative, rapid, non-parenteral route that could benefit our unique emergency department and pre-hospital patient population. Due to the overall small sample size of this review, larger studies are required to confirm safety.

Barriers and facilitators to nursing delirium screening in older emergency patients: a qualitative study using the theoretical domains framework.

BACKGROUND: delirium is common in older emergency department (ED) patients, but vastly under-recognised, in part due to lack of standardised screening processes. Understanding local context and barriers to delirium screening are integral for successful implementation of a delirium screening protocol. OBJECTIVES: we sought to identify barriers and facilitators to delirium screening by nurses in older ED patients. METHODS: we conducted 15 semi-structured, face-to-face interviews based on the Theoretical Domains Framework with bedside nurses, nurse educators and managers at two academic EDs in 2017. Two research assistants independently coded transcripts. Relevant domains and themes were identified. RESULTS: a total of 717 utterances were coded into 14 domains. Three dominant themes emerged: (i) lack of clinical prioritisation because of competing demands, lack of time and heavy workload; (ii) discordance between perceived capabilities and knowledge and (iii) hospital culture. CONCLUSION: this qualitative study explored nursing barriers and facilitators to delirium screening in older ED patients. We found that delirium was recognised as an important clinical problem; however, it was not clinically prioritised; there was a false self-perception of knowledge and ability to recognise delirium and hospital culture was a strong influencer of behaviour. Successful adoption of a delirium screening protocol will only be realised if these issues are addressed.

Implementation of competence committees during the transition to CBME in Canada: A national fidelity-focused evaluation.

PURPOSE: This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME). METHODS: A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design. RESULTS: Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions. CONCLUSIONS: Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.

Ready, set, go! Evaluating readiness to implement competency-based medical education.

PURPOSE: Organizational readiness is critical for successful implementation of an innovation. We evaluated program readiness to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME), among Canadian postgraduate training programs. METHODS: A survey of program directors was distributed 1 month prior to CBD implementation in 2019. Questions were informed by the R = MC2 framework of organizational readiness and addressed: program motivation, general capacity for change, and innovation-specific capacity. An overall readiness score was calculated. An ANOVA was conducted to compare overall readiness between disciplines. RESULTS: Survey response rate was 42% (n = 79). The mean overall readiness score was 74% (30-98%). There was no difference in scores between disciplines. The majority of respondents agreed that successful implementation of CBD was a priority (74%), and that their leadership (94%) and faculty and residents (87%) were supportive of change. Fewer perceived that CBD was a move in the right direction (58%) and that implementation was a manageable change (53%). Curriculum mapping, competence committees and programmatic assessment activities were completed by >90% of programs, while <50% had engaged off-service disciplines. CONCLUSION: Our study highlights important areas where programs excelled in their preparation for CBD, as well as common challenges that serve as targets for future intervention to improve program readiness for CBD implementation.

Does the public feel prepared to be substitute decision-makers? Gaps in preparedness and support for a high school curriculum: a national survey.

BACKGROUND: Substitute decision-makers (SDMs) make decisions on behalf of patients who do not have capacity, in line with previously expressed wishes, values and beliefs. However, miscommunications and poor awareness of previous wishes often lead to inappropriate care. Increasing public preparedness to communicate on behalf of loved ones may improve care in patients requiring an SDM. METHODS: We conducted an online survey in January 2019 with a representative sample of the Canadian population. The primary outcome was self-reported preparedness to be an SDM. The secondary outcome was support for a high school curriculum on the role of SDMs. The effect of socio-demographics, known enablers and barriers to acting as an SDM, and attitudes towards a high school curriculum were assessed using multivariate analysis. RESULTS: Of 1,000 participants, 53.1% felt prepared to be an SDM, and 75.4% stated they understood their loved one's values. However, only 55.6% reported having had a meaningful conversation with their loved one about values and wishes, and only 61.7% reported understanding the SDM role. Engagement in advance care planning for oneself was low (23.1%). Age, experience, training and comfort with communication were associated with preparedness in our multivariate analysis. A high school curriculum was supported by 61.1% of respondents, with 28.3% neutral and 10.6% against it. INTERPRETATION: There is a gap between perceived and actual preparedness to be an SDM. Many report understanding their loved one's values yet have not asked them about wishes in illness or end of life. The majority of respondents support high school education to improve preparedness.

Exploring the use of rating scales with entrustment anchors in workplace-based assessment.

PURPOSE: Competency-based medical education (CBME) has prompted widespread implementation of workplace-based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. METHOD: A convenience sample of 50 semi-structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line-by-line coding to identify themes. RESULTS: Interviews captured interactions between 70% (26/37) of full-time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were 'in the room' or not. CONCLUSIONS: By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of 'in-the-moment' assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.

Twelve tips for learners to succeed in a CBME program.

The international movement to competency-based medical education (CBME) marks a major transition in medical education that requires a shift in educators' and learners' approach to clinical experiences, the way assessment data are collected and integrated, and in learners' mindsets. Learners entering a CBME curriculum must actively drive their learning experiences and education goals. For some, this expectation may be a significant change from their previous approach to learning in medicine. This paper highlights 12 tips to help learners succeed within a CBME model.

Diagnosing conflict: Conflicting data, interpersonal conflict, and conflicts of interest in clinical competency committees.

Clinical competency committees (CCCs) are increasingly used within health professions education as their decisions are thought to be more defensible and fairer than those generated by previous training promotion processes. However, as with most group-based processes, it is inevitable that conflict will arise. In this paper the authors explore three ways conflict may arise within a CCC: (1) conflicting data submissions that are presented to the committee, (2) conflicts between members of the committee, and (3) conflicts of interest between a specific committee member and a trainee. The authors describe each of these conflict situations, dissect out the underlying problems, and explore possible solutions based on the current literature.

Entrustment Checkpoint: Clinical Supervisors' Perceptions of the Emergency Department Oral Case Presentation.

Phenomenon: The oral case presentation represents a unique method of communication and forms the foundation for trainee-supervisor interactions in the clinical setting. Recently, entrustment has been highlighted as an essential element of trainee-supervisor interactions. Despite the growing body of knowledge concerning entrustment in medical education, how supervisors conceptualize the oral case presentation as a contributor to entrustment decision making during clinical supervision remains unknown. Given their widespread use, oral case presentations may represent a potential tool for future frameworks of workplace-based assessment. This study sought to explore what factors influence supervisors' expectations of oral case presentation content and how the oral case presentation may contribute to entrustment decision making. Approach: Using qualitative methodology, semistructured interviews were conducted from a purposive sample of attending emergency medicine physicians at an academic medical center from 2015 to 2016. Thematic analysis of the semistructured interview transcripts was conducted by 2 investigators using line-by-line coding and constant comparative analysis. Key themes were identified through consensus. Theoretical sampling occurred until thematic saturation was reached. Findings: Twenty-one attending physicians were interviewed. Four factors were found to influence supervisor expectations pertaining to oral case presentation content (trainee level, trainee familiarity, clinical context, and clinical task). Further, the oral case presentation was found to serve as a means of indirect observation and an entrustment check point informing future decisions relating to trainee supervision. Insights: The oral case presentation represents a core activity within the trainee-supervisor relationship in which entrustment plays a central role. Given the numerous factors influencing oral case presentation content, we caution supervisors against relying solely on the oral case presentation as an entrustment check point, as this may lead to inaccurate judgments of trainee competence. We recommend that the oral case presentation be used in conjunction with other means of direct and indirect observation to assist with entrustment decisions relating to trainee supervision.

Barriers and enablers that influence guideline-based care of geriatric fall patients presenting to the emergency department.

BACKGROUND: Geriatric patients commonly present to the ED after a fall. Recent evidence suggests that ED physicians are poorly adherent to published ED-specific geriatric fall guidelines. This study applied a theoretical domains framework (TDF) approach to systematically investigate barriers and enablers in the provision of guideline-based care to ED geriatric fall patients. METHODS: From June to September 2017, semistructured interviews of staff ED physicians practising in Ontario, Canada, were conducted and analysed. An interview guide based on the TDF was used to capture 14 domains influencing provision of guideline-based care. Relevant domains were identified based on frequencies of beliefs, existence of conflicting beliefs and evidence of strong beliefs that would influence provision of guideline-based care. RESULTS: Eleven interviews were conducted with practising ED physicians. Thirty belief statements were identified across 13 relevant TDF domains (all except Optimism). Prominent themes included lack of knowledge, paucity of evidence, heterogeneous self-perceived skills, perceived increased time and workload, importance of allied health support, inconsistently available allied health workers, lack of positive reinforcement, emotions negatively impacting these clinical encounters and support for memory aids. Overall, ED physicians were supportive of guideline implementation, and believe it will lead to better outcomes for geriatric fall patients. CONCLUSION: This study identified important barriers and enablers to provision of guideline-based care in geriatric ED fall patients. Based on these findings, future implementation of guidelines nationally and internationally should focus on improving knowledge and training on guidelines, improving positive reinforcement for guideline-appropriate management, greater allied health support and further research to support guidelines.

Supervisor-trainee continuity and the quality of work-based assessments.

CONTEXT: Work-based assessments (WBAs) represent an increasingly important means of reporting expert judgements of trainee competence in clinical practice. However, the quality of WBAs completed by clinical supervisors is of concern. The episodic and fragmented interaction that often occurs between supervisors and trainees has been proposed as a barrier to the completion of high-quality WBAs. OBJECTIVES: The primary purpose of this study was to determine the effect of supervisor-trainee continuity on the quality of assessments documented on daily encounter cards (DECs), a common form of WBA. The relationship between trainee performance and DEC quality was also examined. METHODS: Daily encounter cards representing three differing degrees of supervisor-trainee continuity (low, intermediate, high) were scored by two raters using the Completed Clinical Evaluation Report Rating (CCERR), a previously published nine-item quantitative measure of DEC quality. An analysis of variance (anova) was performed to compare mean CCERR scores among the three groups. Linear regression analysis was conducted to examine the relationship between resident performance and DEC quality. RESULTS: Differences in mean CCERR scores were observed between the three continuity groups (p = 0.02); however, the magnitude of the absolute differences was small (partial eta-squared = 0.03) and not educationally meaningful. Linear regression analysis demonstrated a significant inverse relationship between resident performance and CCERR score (p < 0.001, r2  = 0.18). This inverse relationship was observed in both groups representing on-service residents (p = 0.001, r2  = 0.25; p = 0.04, r2  = 0.19), but not in the Off-service group (p = 0.62, r2  = 0.05). CONCLUSIONS: Supervisor-trainee continuity did not have an educationally meaningful influence on the quality of assessments documented on DECs. However, resident performance was found to affect assessor behaviours in the On-service group, whereas DEC quality remained poor regardless of performance in the Off-service group. The findings suggest that greater attention should be given to determining ways of improving the quality of assessments reported for off-service residents, as well as for those residents demonstrating appropriate clinical competence progression.

Decreased patient discharges on weekends: part 2-what do the ward nurses tell us?

BACKGROUND: Hospital and emergency department (ED) crowding is exacerbated on Mondays because fewer in-patients are discharged during the weekend. We evaluated the experiences and attitudes of in-patient ward nurses to better understand the challenges they face when considering the weekend discharge of their patients. METHODS: We conducted a qualitative study of in-patient ward nurses, using the theoretical domains framework (TDF), at two campuses of a major academic health sciences centre. The interview guides consisted of, first, a series of questions to explore the typical processes involved for safe patient discharges and, second, exploration of the influence of the 14 TDF domains. All interviews were audio-recorded, transcribed verbatim, and anonymized and then imported into NVivo qualitative software for data management and analysis. Analysis was conducted in three stages (coding, generation of specific beliefs, identification of relevant and nonrelevant domains). RESULTS: The 28 interviewed nurses represented a variety of medical, surgical and other wards, and reported being acutely aware of the pressures to discharge patients on weekends (knowledge). They believed that increasing weekend discharges would improve hospital flow and aid in decanting the ED (beliefs about consequences). However, they also acknowledged that the weekend discharge pressures might result in patients being discharged prematurely and bouncing back to the hospital (beliefs about consequences). Overall, the nurses reported that as a hospital culture, discharging patients was not much of a priority (goals; environmental context and resources). CONCLUSION: We know there are much fewer discharges on weekends, and this is associated with significant hospital and ED crowding on Mondays. This study has illuminated the many challenges faced by in-patient ward nurses when considering the discharge of admitted patients on weekends. In order to decrease ED and hospital crowding related to decreased weekend discharges, hospitals will need to effect a culture change amongst all staff.

ABSTRAIT: CONTEXTE: Le surpeuplement des hôpitaux et des services d'urgence (SU) est exacerbé le lundi parce que moins de patients hospitalisés sont libérés pendant la fin de semaine. Nous avons évalué les expériences et les attitudes des infirmières en salle afin de mieux comprendre les défis auxquels elles font face lorsqu'elles envisagent le congé de fin de semaine de leurs patients. MéTHODES: Nous avons mené une étude qualitative sur les infirmières en salle, en utilisant le cadre des domaines théoriques (TDF), sur deux campus d'un grand centre universitaire des sciences de la santé. Les guides d'entrevue ont consisté, d'une part, en une série de questions visant à explorer les processus typiques de sortie sécuritaire des patients et, d'autre part, en une exploration de l'influence des 14 domaines du TDF. Toutes les entrevues ont été enregistrées, transcrites mot à mot et rendues anonymes, puis importées dans le logiciel qualitatif NVivo pour la gestion et l'analyse des données. L'analyse a été menée en trois étapes (codage, génération de croyances spécifiques, identification de domaines pertinents et non pertinents). RéSULTATS: Les 28 infirmières interrogées représentaient une variété de services médicaux, chirurgicaux et autres, et ont déclaré être très conscientes des pressions exercées sur les patients pour qu'ils quittent la clinique les fins de semaine (connaissance). Ils croyaient que l'augmentation des congés de fin de semaine améliorerait le flux hospitalier et aiderait à décanter le DE (croyances sur les conséquences). Cependant, ils ont également reconnu que les pressions de sortie du week-end pourraient entraîner le renvoi prématuré des patients à l'hôpital (croyances sur les conséquences). Dans l'ensemble, les infirmières ont indiqué qu'en tant que culture hospitalière, le congé des patients n'était pas une grande priorité (objectifs ; contexte environnemental et ressources). CONCLUSION: Nous savons qu'il y a beaucoup moins de congés la fin de semaine, et cela est associé à un grand nombre d'hôpitaux et de services d'urgence le lundi. Cette étude a mis en lumière les nombreux défis auxquels font face les infirmières en salle lorsqu'elles envisagent de quitter les patients admis les fins de semaine. Afin de réduire le surpeuplement des urgences et des hôpitaux liés à la diminution des congés de fin de semaine, les hôpitaux devront effectuer un changement de culture parmi tout le personnel.

The Mean Delta Method: Quantifying Assessor Stringency and Leniency and Identifying Outliers in Workplace-Based Assessments.

ABSTRACT: Assessor stringency and leniency (ASL)-an assessor's tendency to award low or high scores-has a significant effect on workplace-based assessments. Outliers on this spectrum have a disproportionate effect. However, no method has been published for quantifying ASL or identifying outlier stringent or lenient assessors using workplace-based assessment data. The authors propose the mean delta method, which compares the scores that an assessor awards to trainees with those trainees' mean scores. This novel, simple method can be used to quantify ASL and identify outlier assessors without requiring specialized statistical knowledge or software. As a worked example, the mean delta method was applied to a set of end-of-shift assessments completed in a large Canadian academic emergency department from July 1, 2017, to May 31, 2018, and used to examine the net effect of ASL on learners' assessment scores. A total of 3,908 assessments were completed by 99 assessors for 151 trainees, with a median (interquartile range) of 37 (12-39) completed assessments per trainee. Using cutoff values of 1.5 and 2 standard deviations, a total of 11 and 3 outlier assessors were identified, respectively. Moreover, ASL changed overall scores by more than the mean difference between years of training for nearly 1 in 4 learners. The mean delta method was able to quantify ASL and identify outlier lenient and stringent assessors. It was also used to quantify the net effect of ASL on individual trainees. This method could be used to further study outlier assessors, to identify assessors who may benefit most from targeted coaching and feedback, and to measure changes in assessors' tendencies over time or with specific intervention.

Deidentifying Narrative Assessments to Facilitate Data Sharing in Medical Education.

PROBLEM: Narrative assessments are commonly incorporated into competency-based medical education programs. However, efforts to share competency-based medical education assessment data among programs to support the evaluation and improvement of assessment systems have been limited in part because of security concerns. Deidentifying assessment data mitigates these concerns, but deidentifying narrative assessments is time-consuming, resource intensive, and error prone. The authors developed and tested a tool to automate the deidentification of narrative assessments and facilitate their review. APPROACH: The authors met throughout 2021 and 2022 to iteratively design, test, and refine the deidentification algorithm and data review interface. Preliminary testing of the prototype deidentification algorithm was performed using narrative assessments from the University of Saskatchewan emergency medicine program. The algorithm's accuracy was assessed by the authors using the review interface designed for this purpose. Formal testing included 2 rounds of deidentification and review by members of the authorship team. Both the algorithm and data review interface were refined during the testing process. OUTCOMES: Authors from 3 institutions, including 3 emergency medicine programs, an anesthesia program, and a surgical program, participated in formal testing. In the final round of review, 99.4% of the narrative assessments were fully deidentified (names, nicknames, and pronouns removed). The results were comparable for each institution and specialty. The data review interface was improved with feedback obtained after each round of review and found to be intuitive. NEXT STEPS: This innovation has demonstrated viability evidence of an algorithmic approach to the deidentification of assessment narratives while reinforcing that a small number of errors are likely to persist. Future steps include the refinement of both the algorithm to improve its accuracy and the data review interface to support additional data set formats.

Comparison of peer-assisted learning with expert-led learning in medical school ultrasound education: a systematic review and meta-analysis.

INTRODUCTION: Teaching point-of-care ultrasonography (PoCUS) to medical students is resource intensive. Peer-assisted learning, where the teacher can be a medical student, may be a feasible alternative to expert-led learning. The objective of this systematic review and meta-analysis was to compare the PoCUS performance assessments of medical students receiving peer-assisted vs expert-led learning. METHODS: This study was submitted to PROSPERO (CRD42023383915) and reported with PRISMA guidelines. MEDLINE, Embase, ERIC, Education Source, Scopus, and Web of Science were searched from inception to November 2022. Inclusion criteria were studies comparing peer-assisted vs expert-led PoCUS teaching for undergraduate medical students. The primary outcome was performance assessment of PoCUS skills. Two reviewers independently screened citations and extracted data. The Cochrane risk-of-bias tool for randomized trials was used to assess study quality. Studies were included in the meta-analysis if mean performance assessment scores with standard deviations and sample sizes were available. A random-effects meta-analysis was conducted to estimate the accuracy score of practical knowledge test for each group. A meta-regression evaluated difference in mean scores. RESULTS: The search yielded 2890 citations; 1417 unique citations remained after removing duplicates. Nine randomized-controlled studies conducted in Germany, USA, and Israel, with 593 participants, were included in the meta-analysis. The included studies assessed teaching of abdominal, cardiac, thoracic, musculoskeletal, and ocular PoCUS skills. Most studies had some risk-of-bias concerns. The estimate accuracy score after weighting is 0.56 (95% CI [0.47, 0.65]) for peer-assisted learning and 0.59 (95% CI [0.49, 0.69]) for expert-led learning. The regression coefficient estimate is 0.0281 (95% CI [- 0.1121, 0.1683]); P value is 0.69. CONCLUSION: This meta-analysis found that peer-assisted learning was a reasonable alternative to expert-led learning for teaching PoCUS skills to medical students.

RéSUMé: INTRODUCTION: L'enseignement de l'échographie au point d'intervention (PoCUS) aux étudiants en médecine nécessite des ressources importantes. L'apprentissage assisté par les pairs, où l'enseignant peut être un étudiant en médecine, peut être une alternative possible à l'apprentissage dirigé par des experts. L'objectif de cette revue systématique et de cette méta-analyse était de comparer les évaluations de performance PoCUS d'étudiants en médecine bénéficiant d'un apprentissage assisté par des pairs par rapport à un apprentissage dirigé par des experts. MéTHODES: Cette étude a été soumise à PROSPERO (CRD42023383915) et rapportée selon les directives PRISMA. MEDLINE, Embase, ERIC, Education Source, Scopus et Web of Science ont été recherchés depuis leur création jusqu'en novembre 2022. Les critères d'inclusion étaient les études comparant l'enseignement du PoCUS assisté par des pairs à celui dirigé par des experts pour les étudiants en médecine de premier cycle. Le principal résultat était l'évaluation du rendement des compétences PoCUS. Deux évaluateurs ont indépendamment examiné les citations et extrait les données. L'outil Cochrane d'évaluation du risque de biais pour les essais randomisés a été utilisé pour évaluer la qualité des études. Les études ont été incluses dans la méta-analyse si les scores moyens d'évaluation des performances avec les écarts types et la taille des échantillons étaient disponibles. Une méta-analyse à effets aléatoires a été réalisée pour estimer le score de précision du test de connaissances pratiques pour chaque groupe. Une méta-régression a évalué la différence dans les scores moyens. RéSULTATS: La recherche a donné lieu à 2890 citations ; 1417 citations uniques ont été conservées après suppression des doublons. Neuf études contrôlées randomisées menées en Allemagne, aux États-Unis et en Israël, avec 593 participants, ont été incluses dans la méta-analyse. Les études incluses ont évalué l'enseignement des compétences PoCUS abdominales, cardiaques, thoraciques, musculo-squelettiques et oculaires. La plupart des études présentaient des risques de biais. Le score de précision estimé après pondération est de 0,56 (IC à 95 % : [0,47, 0,65]) pour l'apprentissage assisté par les pairs et de 0,59 (IC à 95 % : [0,49, 0,69]) pour l'apprentissage dirigé par des experts. L'estimation du coefficient de régression est de 0,0281 (IC à 95 % : [-0,1121, 0,1683]) ; la valeur P est de 0,69. CONCLUSION: Cette méta-analyse a montré que l'apprentissage assisté par les pairs était une alternative raisonnable à l'apprentissage dirigé par des experts pour enseigner les compétences PoCUS aux étudiants en médecine.

Implementing Competence Committees on a National Scale: Design and Lessons Learned.

Competence committees (CCs) are a recent innovation to improve assessment decision-making in health professions education. CCs enable a group of trained, dedicated educators to review a portfolio of observations about a learner's progress toward competence and make systematic assessment decisions. CCs are aligned with competency based medical education (CBME) and programmatic assessment. While there is an emerging literature on CCs, little has been published on their system-wide implementation. National-scale implementation of CCs is complex, owing to the culture change that underlies this shift in assessment paradigm and the logistics and skills needed to enable it. We present the Royal College of Physicians and Surgeons of Canada's experience implementing a national CC model, the challenges the Royal College faced, and some strategies to address them. With large scale CC implementation, managing the tension between standardization and flexibility is a fundamental issue that needs to be anticipated and addressed, with careful consideration of individual program needs, resources, and engagement of invested groups. If implementation is to take place in a wide variety of contexts, an approach that uses multiple engagement and communication strategies to allow for local adaptations is needed. Large-scale implementation of CCs, like any transformative initiative, does not occur at a single point but is an evolutionary process requiring both upfront resources and ongoing support. As such, it is important to consider embedding a plan for program evaluation at the outset. We hope these shared lessons will be of value to other educators who are considering a large-scale CBME CC implementation.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.