Evaluating ExpandED: Evaluating the effectiveness of a serious game expansion pack in teaching health professional students about interprofessional care.

Background: The emergency department (ED) is a challenging fast-paced environment with high-acuity, undifferentiated patients who often require extensive interdisciplinary care. This paper introduces ExpandED, an expansion pack to the serious board game GridlockED, designed to enhance players' understanding of interprofessional collaboration in the ED and the diverse scope of practice of different ED professionals including physicians, residents, registered nurses, registered practical nurses, social workers, occupational therapists, and physiotherapists. This investigation evaluates the effectiveness of ExpandED as a teaching tool for medical and allied health professions students about interprofessional collaboration in the ED. Methods: A program evaluation harnessing a playtest framework was employed. Participants completed pre- and postgame surveys including quantitative measures (e.g., Likert scales) and qualitative free-text feedback that focused on participant familiarity with ED functioning, valuation of interprofessional collaboration before and after playing, and feedback on game usability and effectiveness. Results: Recruitment was open to students in all health care and allied health professional programs at the institution. Forty-five participants were recruited from medical doctor, nursing, physiotherapy, and speech language pathology programs. ExpandED enhances participants' understanding of ED workflow (p < 0.001) and provides an enjoyable playing experience. However, participants' valuation of interdisciplinary teamwork did not change significantly before and after game play (p = 0.17). Participants expressed satisfaction with the game's accuracy in simulating the ED environment and appreciated the opportunity to collaborate with peers from different disciplines. Challenges reported included some tension among players, potential biases, and limitations of fidelity to a real-life ED. Conclusions: While this study has limitations regarding participant sampling and duration of gameplay sessions, it highlights the potential of ExpandED for teaching interprofessional collaboration in the ED. These findings will guide further development to optimize the expansion pack's effectiveness and its implementation into health care curricula.

Measuring Burnout and Professional Fulfillment Among Emergency Medicine Residency Program Leaders in the United States: A Cross-Sectional Survey Study.

Objective: Emergency medicine (EM) physicians face high burnout rates, even in academic settings. Research on burnout among EM residency program leaders is limited, despite their role in shaping the training environment and influencing resident well-being. This study aims to measure burnout and professional fulfillment among EM residency program leaders and identify contributing factors. Methods: A cross-sectional survey using the adapted Stanford Professional Fulfillment Index (SPFI) was conducted in 2023 to assess burnout and professional fulfillment among EM residency program leaders at US programs. The survey, tailored to EM leaders, was distributed to all current EM Program Directors (PDs) and Assistant or Associate Program Directors (APDs) from accredited US programs. Descriptive statistics and odds ratios were used to compare burnout and professional fulfillment across various groups. Results: A total of 112 of 281 PDs (39.9% response rate) and 130 of 577 APDs (22.5% response rate) participated. Professional fulfillment was reported by 59.8% of PDs and 58.5% of APDs. Burnout was experienced by 42.0% of PDs and 26.9% of APDs. Higher professional fulfillment correlated with alignment with expectations, positive work environments, and perceived appreciation, while burnout was strongly associated with negative impacts on personal health and relationships. About 27.7% of PDs and 23.8% of APDs expressed intentions to leave their current positions within 18 months. Conclusion: A significant portion of US EM residency program leaders experience burnout and low professional fulfillment. Addressing well-being in this population has important implications for education and mentorship provided to future physicians in the field.

Bias in Observed Assessments in Medical Education: A Scoping Review.

PURPOSE: Observed assessments are integral to medical education but may be biased against structurally marginalized communities. Current understanding of assessment bias is limited because studies have focused on single specialties, levels of training, or social identity characteristics (SIDCs). This scoping review maps studies investigating bias in observed assessments in medical education arising from trainees' observable SIDCs at different medical training levels, with consideration of medical specialties, assessment environments, and assessment tools. METHOD: MEDLINE, Embase, ERIC, PsycINFO, Scopus, Web of Science Core Collection, and Cochrane Library were searched for articles published between January 1, 2008, and March 15, 2023, on assessment bias related to 6 observable SIDCs: gender (binary), gender nonconformance, race and ethnicity, religious expression, visible disability, and age. Two authors reviewed the articles, with conflicts resolved by consensus or a third reviewer. Results were interpreted through group review and informed by consultation with experts and stakeholders. RESULTS: Sixty-six of 2,920 articles (2.3%) were included. These studies most frequently investigated graduate medical education (44 [66.7%]), used quantitative methods (52 [78.8%]), and explored gender bias (63 [95.5%]). No studies investigated gender nonconformance, religious expression, or visible disability. One evaluated intersectionality, with SIDCs described inconsistently. General surgery (16 [24.2%]) and internal medicine (12 [18.2%]) were the most studied specialties. Simulated environments (37 [56.0%]) were studied more frequently than clinical environments (29 [43.9%]). Bias favoring men was found more in assessments of intraoperative autonomy (5 of 9 [55.6%]), whereas clinical examination bias often favored women (15 of 19 [78.9%]). When race and ethnicity bias was identified, it consistently favored White students. CONCLUSIONS: This review mapped studies of gender, race, and ethnicity bias in the medical education assessment literature, finding limited studies on other SIDCs and intersectionality. These findings will guide future research by highlighting the importance of consistent terminology, unexplored SIDCs, and intersectionality.

Shifting paradigms: A collective and structural strategy for addressing healthcare inequity.

Healthcare inequity is a persistent systemic problem, yet many solutions have historically focused on "debiasing" individuals. Individualistic strategies fit within a competency-based medical education and assessment paradigm, whereby professional values of social accountability, patient safety, and healthcare equity are linked to an individual clinician's competence. Unfortunately, efforts to realise the conceptual linkages between medical education curricula and goals to improve healthcare equity fail to address the institutional values, policies, and practices that enable structural racism. In this article, we explore alternative approaches that target collective and structural causes of health inequity. We first describe the structural basis of healthcare inequity by identifying the ways in which institutional culture, power and privilege erode patient-centred care and contribute to epistemic injustice. We then outline some reasons that stereotypes, which are a culturally supported foundation for discrimination, bias and racism in healthcare, cannot be modified effectively through individualistic strategies or education curricula. Finally, we propose a model that centres shared values for leadership by individuals and institutions with consistency in goal setting, knowledge translation, and talent development. Figure 1 summarises the key recommendations. We have provided cases to supplement this work and facilitate discussion about the model's application to practice.

2023 Society for Academic Emergency Medicine Consensus Conference on Precision Emergency Medicine: Development of a policy-relevant, patient-centered research agenda.

OBJECTIVES: Precision medicine is data-driven health care tailored to individual patients based on their unique attributes, including biologic profiles, disease expressions, local environments, and socioeconomic conditions. Emergency medicine (EM) has been peripheral to the precision medicine discourse, lacking both a unified definition of precision medicine and a clear research agenda. We convened a national consensus conference to build a shared mental model and develop a research agenda for precision EM. METHODS: We held a conference to (1) define precision EM, (2) develop an evidence-based research agenda, and (3) identify educational gaps for current and future EM clinicians. Nine preconference workgroups (biomedical ethics, data science, health professions education, health care delivery and access, informatics, omics, population health, sex and gender, and technology and digital tools), comprising 84 individuals, garnered expert opinion, reviewed relevant literature, engaged with patients, and developed key research questions. During the conference, each workgroup shared how they defined precision EM within their domain, presented relevant conceptual frameworks, and engaged a broad set of stakeholders to refine precision EM research questions using a multistage consensus-building process. RESULTS: A total of 217 individuals participated in this initiative, of whom 115 were conference-day attendees. Consensus-building activities yielded a definition of precision EM and key research questions that comprised a new 10-year precision EM research agenda. The consensus process revealed three themes: (1) preeminence of data, (2) interconnectedness of research questions across domains, and (3) promises and pitfalls of advances in health technology and data science/artificial intelligence. The Health Professions Education Workgroup identified educational gaps in precision EM and discussed a training roadmap for the specialty. CONCLUSIONS: A research agenda for precision EM, developed with extensive stakeholder input, recognizes the potential and challenges of precision EM. Comprehensive clinician training in this field is essential to advance EM in this domain.

From Slow Shifts to Fast Flips: Unraveling problem-based learning group function dynamics.

PURPOSE: Problem-Based Learning (PBL) relies on self-directed learning in small groups in the presence of a tutor. While the effectiveness of PBL is often attributed to the dynamics of group function, change in group function over time and factors influencing group function development are less understood. This study aims to explore the development of PBL group function over time to better understand the factors that give rise to high-functioning groups. METHOD: We examined time-function graphs of group function and conducted semi-structured focus group discussions in 2023 with medical students enrolled in a PBL curriculum. Students reflected on their experiences in four different PBL groups, creating time-function graphs to characterize development of group function over 8-12-week periods. We analyzed graphs and transcripts in a staged approach using qualitative description and direct content analysis, sensitized by two frameworks: Tuckman's Stages of Group Development and the Dimensions of PBL Group Function. RESULTS: Three archetypes of PBL group function development were identified: Slow Shifters, Fast Flippers, and Coasters. (1) Slow Shifters were characterized by a complex and extended pattern of growth consistent with Tuckman's model, typically occurring amongst inexperienced groups, or groups faced with a novel task. (2) Fast Flippers were characterized by abrupt state changes in group function arising from internal or external disruptions. (3) Coasters were characterized by plateaus, where maintenance of group function was a frequently cited challenge. Abrupt changes and plateaus occurred more among mature groups and groups with significant PBL experience. CONCLUSIONS: PBL group function varies over time in 3 different patterns. Classic Tuckman's stages are apparent among inexperienced groups, or groups facing novel tasks, whereas experienced groups often face abrupt change or plateaus. PBL educators and students should consider the need for novelty and disruption in more experienced groups to incite growth.

Precision medicine within health professions education: Defining a research agenda for emergency medicine using a foresight and strategy technique (FaST) review.

Background: Precision medicine, sometimes referred to as personalized medicine, is rapidly changing the possibilities for how people will engage health care in the near future. As technology to support precision medicine exponentially develops, there is an urgent need to proactively improve our understanding of precision medicine and pose important research questions (RQs) related to its inclusion in the education and training of future emergency physicians. Methods: A seven-step process was employed to develop a research agenda exploring the intersection of precision and emergency medicine education/training. A literature search of articles about precision medicine was conducted first, which informed the creation of future four scenarios in which trainees and practicing physicians regularly discuss and incorporate precision medicine tools into their discussions and work. Based on these futurist narratives, potential education RQs were generated by an expert panel. A total of 59 initial questions were subsequently categorized and refined to a priority list through a nominal group voting method. The top/priority questions were presented at the 2023 SAEM Consensus Conference on Precision Medicine, Austin, Texas, for further input. Results: Eight high-value education RQs were developed, reflecting a holistic view of the challenges and opportunities for precision medicine education in the knowledge, skills, and attitudes relevant to emergency medicine. These questions contend with topics such as most effective pedagogical methods; intended resulting outcomes and behaviors; the generational differences between practicing emergency physicians, educators, and future trainees; and the desires and expectations of patients. Conclusions: Emergency medicine and emergency physicians must be prepared to understand precision medicine and incorporate this information into their "toolbox" of thinking, problem solving, and communication with patients and colleagues. This research agenda on how best to educate future emergency physicians in the use of personalized data to provide optimal health care is the focus of this article.

ReThinking clinical reasoning: A paradigm shift.

Numerous studies have demonstrated that our healthcare systems and medical education programs are fundamentally flawed. In North America and Europe, most systems were built upon values and structures that have historically benefitted middle and upper class males of European descent in the global north. As a result, there continue to be systemic biases that are pervasive throughout our healthcare systems and medical education programs. This has led to inequities in health outcomes and clinical reasoning practices which marginalize several communities. These biases are perpetuated as we continue to lead medical education research and practice with traditional values and views of evidence. To address these issues, we proposed a 'flipped' conference in which three interdisciplinary writing teams, comprised of both junior and senior academics, clinicians, and researchers, were invited to rethink the foundations of clinical reasoning. In the months leading up to the conference, each writing team explored a specific topic related to clinical reasoning and racial equity. The papers, presented during the virtual conference are now available in this issue of the Journal for the Evaluation of Clinical Practice. In addition, 6 more publications were added to this special topic to showcase new evidence and theory that builds on the recommendations in the three core papers.

A Longitudinal Survey on Canadian Emergency Physician Burnout.

STUDY OBJECTIVE: Since Canada eased pandemic restrictions, emergency departments have experienced record levels of patient attendance, wait times, bed blocking, and crowding. The aim of this study was to report Canadian emergency physician burnout rates compared with the same physicians in 2020 and to describe how emergency medicine work has affected emergency physician well-being. METHODS: This longitudinal study on Canadian emergency physician wellness enrolled participants in April 2020. In September 2022, participants were invited to a follow-up survey consisting of the Maslach Burnout Inventory and an optional free-text explanation of their experience. The primary outcomes were emotional exhaustion and depersonalization levels, which were compared with the Maslach Burnout Inventory survey conducted at the end of 2020. A thematic analysis identified common stressors, challenges, emotions, and responses among participants. RESULTS: The response rate to the 2022 survey was 381 (62%) of 615 between September 28 and October 28, 2022, representing all provinces or territories in Canada (except Yukon). The median participant age was 42 years. In total, 49% were men, and 93% were staff physicians with a median of 12 years of work experience. 59% of respondents reported high emotional exhaustion, and 64% reported high depersonalization. Burnout levels in 2022 were significantly higher compared with 2020. Prevalent themes included a broken health care system, a lack of societal support, and systemic workplace challenges leading to physician distress and loss of physicians from the emergency workforce. CONCLUSION: We found very high burnout levels in emergency physician respondents that have increased since 2020.

Perceptions of gender equity in emergency medicine in Canada.

INTRODUCTION: Women-identifying emergency physicians face gender-based discrimination throughout their careers. The purpose of this study was to explore emergency physician's perceptions and experiences of gender equity in emergency medicine. METHODS: We conducted a secondary analysis of data from a previously conducted survey of Canadian emergency physicians on barriers to gender equity in emergency medicine. Survey responses were analyzed using logistic regression to determine the impact that gender, practice setting, years since graduation, race, equity-seeking status, and parental status had on agreement about gender equity in emergency medicine and five of the problem statements. RESULTS: A total of 710 participants completed the survey. Most identified as women (58.8%), white (77.4%), graduated between 2010 and 2019 (40%), had CCFP (Emergency Medicine) designation (47.9%), an urban practice (84.4%), were parents (62.4%) and did not identify as equity-seeking (79.9%). Women-identifying physicians were less likely to perceive gender equity in emergency medicine, OR 0.52, CI [0.38, 0.73]. Women-identifying physicians were more likely to agree with statements about microaggressions, OR 4.39, CI [2.66, 7.23]; barriers to leadership, OR 3.51, CI [2.25, 5.50]; gender wage gap, OR 13.46, CI [8.27, 21.91]; lack of support for parental leave, OR 2.85, CI [1.82, 4.44]; and education on allyship, OR 2.23 CI [1.44, 3.45] than men-identifying physicians. CONCLUSION: In this study, women-identifying physicians were less likely to perceive that there was gender equity in emergency medicine than men-identifying physicians. Women-identifying physicians agreed that there are greater barriers for career advancement including fewer opportunities for leadership, a gender wage gap, a lack of parental leave policies to support a return to work and a lack of education for men to become allies. Men-identifying physicians were less aware of these inequities. Health systems must work to improve gender equity in emergency medicine and this will require education and allyship from men-identifying physicians.

RéSUMé: INTRODUCTION: Les femmes médecins urgentistes sont confrontées à une discrimination fondée sur le sexe tout au long de leur carrière. L'objectif de cette étude était d'explorer les perceptions et les expériences des médecins urgentistes en matière d'équité entre les sexes en médecine d'urgence. MéTHODES: Nous avons procédé à une analyse secondaire des données d'une enquête menée précédemment auprès des médecins urgentistes canadiens sur les obstacles à l'équité entre les sexes en médecine d'urgence. Les réponses au sondage ont été analysées à l'aide d'une régression logistique pour déterminer l'incidence que le sexe, le milieu de pratique, les années écoulées depuis l'obtention du diplôme, la race, le statut de demandeur d'équité et le statut parental avaient sur l'accord sur l'équité entre les sexes en médecine d'urgence et cinq des énoncés de problème. RéSULTATS: Au total, 710 participants ont répondu à l'enquête. La plupart d'entre eux sont des femmes (58.8 %), de race blanche (77.4 %), ont obtenu leur diplôme entre 2010 et 2019 (40 %), ont le titre de CCMF (médecine d'urgence) (47.9 %), exercent en milieu urbain (84.4 %), sont parents (62.4 %) et ne se déclarent pas en quête d'équité (79.9 %). Les médecins s'identifiant à des femmes étaient moins susceptibles de percevoir l'équité entre les sexes en médecine d'urgence, OR 0.52, IC [0.38,0.73]. Les médecins s'identifiant comme femmes étaient plus susceptibles d'être d'accord avec les déclarations sur les microagressions, OR 4.39, IC [2.66, 7.23] ; obstacles au leadership, OR 3.51, IC [2.25, 5.50] ; écart salarial entre les hommes et les femmes, OR 13.46, IC [8.27, 21.91] ; le manque de soutien pour le congé parental, OR 2.85, IC [1.82, 4.44]; et l'éducation sur l'alliance, OR 2.23 IC [1.44, 3.45] que les médecins s'identifiant comme hommes. CONCLUSION: Dans cette étude, les médecins s'identifiant à des femmes étaient moins susceptibles de percevoir qu'il y avait une équité entre les sexes en médecine d'urgence que les médecins s'identifiant à des hommes. Les femmes médecins s'accordent à dire qu'il existe davantage d'obstacles à l'avancement professionnel, notamment moins d'opportunités de leadership, un écart salarial entre les hommes et les femmes, un manque de politiques de congé parental pour favoriser le retour au travail et un manque d'éducation des hommes pour qu'ils deviennent des alliés. Les médecins s'identifiant à des hommes étaient moins conscients de ces inégalités. Les systèmes de santé doivent s'efforcer d'améliorer l'équité entre les sexes dans la médecine d'urgence, ce qui nécessitera une formation et un allié de la part des médecins qui s'identifient aux hommes.

Using Natural Language Processing to Evaluate the Quality of Supervisor Narrative Comments in Competency-Based Medical Education.

PURPOSE: Learner development and promotion rely heavily on narrative assessment comments, but narrative assessment quality is rarely evaluated in medical education. Educators have developed tools such as the Quality of Assessment for Learning (QuAL) tool to evaluate the quality of narrative assessment comments; however, scoring the comments generated in medical education assessment programs is time intensive. The authors developed a natural language processing (NLP) model for applying the QuAL score to narrative supervisor comments. METHOD: Samples of 2,500 Entrustable Professional Activities assessments were randomly extracted and deidentified from the McMaster (1,250 comments) and Saskatchewan (1,250 comments) emergency medicine (EM) residency training programs during the 2019-2020 academic year. Comments were rated using the QuAL score by 25 EM faculty members and 25 EM residents. The results were used to develop and test an NLP model to predict the overall QuAL score and QuAL subscores. RESULTS: All 50 raters completed the rating exercise. Approximately 50% of the comments had perfect agreement on the QuAL score, with the remaining resolved by the study authors. Creating a meaningful suggestion for improvement was the key differentiator between high- and moderate-quality feedback. The overall QuAL model predicted the exact human-rated score or 1 point above or below it in 87% of instances. Overall model performance was excellent, especially regarding the subtasks on suggestions for improvement and the link between resident performance and improvement suggestions, which achieved 85% and 82% balanced accuracies, respectively. CONCLUSIONS: This model could save considerable time for programs that want to rate the quality of supervisor comments, with the potential to automatically score a large volume of comments. This model could be used to provide faculty with real-time feedback or as a tool to quantify and track the quality of assessment comments at faculty, rotation, program, or institution levels.

The Promise and Perils of Artificial Intelligence in Health Professions Education Practice and Scholarship.

ABSTRACT: Artificial intelligence (AI) methods, especially machine learning and natural language processing, are increasingly affecting health professions education (HPE), including the medical school application and selection processes, assessment, and scholarship production. The rise of large language models over the past 18 months, such as ChatGPT, has raised questions about how best to incorporate these methods into HPE. The lack of training in AI among most HPE faculty and scholars poses an important challenge in facilitating such discussions. In this commentary, the authors provide a primer on the AI methods most often used in the practice and scholarship of HPE, discuss the most pressing challenges and opportunities these tools afford, and underscore that these methods should be understood as part of the larger set of statistical tools available.Despite their ability to process huge amounts of data and their high performance completing some tasks, AI methods are only as good as the data on which they are trained. Of particular importance is that these models can perpetuate the biases that are present in those training datasets, and they can be applied in a biased manner by human users. A minimum set of expectations for the application of AI methods in HPE practice and scholarship is discussed in this commentary, including the interpretability of the models developed and the transparency needed into the use and characteristics of such methods.The rise of AI methods is affecting multiple aspects of HPE including raising questions about how best to incorporate these models into HPE practice and scholarship. In this commentary, we provide a primer on the AI methods most often used in HPE and discuss the most pressing challenges and opportunities these tools afford.

Five hats of effective leaders: teacher, mentor, coach, supervisor and sponsor.

BACKGROUND/AIM: Teaching, mentoring, coaching, supervising and sponsoring are often conflated in the literature. In this reflection, we clarify the distinctions, the benefits and the drawbacks of each approach. We describe a conceptual model for effective leadership conversations where leaders dynamically and deliberately 'wear the hats' of teacher, mentor, coach, supervisor and/or sponsor during a single conversation. METHODS: As three experienced physician leaders and educators, we collaborated to write this reflection on how leaders may deliberately alter their approach during dynamic conversations with colleagues. Each of us brings our own perspective and lens. RESULTS: We articulate how each of the 'five hats' of teacher, mentor, coach, supervisor and sponsor may help or hinder effectiveness. We discuss how a leader may 'switch' hats to engage, support and develop colleagues across an ever-expanding range of contexts and settings. We demonstrate how a leader might 'wear the five hats' during conversations about career advancement and burn-out. CONCLUSION: Effective leaders teach, mentor, coach, supervise and sponsor during conversations with colleagues. These leaders employ a deliberate, dynamic and adaptive approach to better serve the needs of their colleagues at the moment.

Considering the Secondary Use of Clinical and Educational Data to Facilitate the Development of Artificial Intelligence Models.

ABSTRACT: Medical training programs and health care systems collect ever-increasing amounts of educational and clinical data. These data are collected with the primary purpose of supporting either trainee learning or patient care. Well-established principles guide the secondary use of these data for program evaluation and quality improvement initiatives. More recently, however, these clinical and educational data are also increasingly being used to train artificial intelligence (AI) models. The implications of this relatively unique secondary use of data have not been well explored. These models can support the development of sophisticated AI products that can be commercialized. While these products have the potential to support and improve the educational system, there are challenges related to validity, patient and learner consent, and biased or discriminatory outputs. The authors consider the implications of developing AI models and products using educational and clinical data from learners, discuss the uses of these products within medical education, and outline considerations that should guide the appropriate use of data for this purpose. These issues are further explored by examining how they have been navigated in an educational collaborative.

GridlockED as an Intervention for Nurses (GAAIN) Study.

BACKGROUND: GridlockED (The Game Crafter, LLC) is a serious game that was developed to teach challenges that face nursing and medical professionals in the emergency department (ED). However, few studies have explored nurses' perceptions of the utility, fidelity, acceptability, and applicability of the serious game modality. This study examined how ED nurses view GridlockED as a continuing education platform. METHOD: This single-center observational study explored how nurses engage with and respond to Grid-lockED. The convenience sample included participants recruited from a local continuing nursing education day. Participants completed a presurvey, engaged in a full game play session with the GridlockED game for approximately 45 minutes, and immediately completed a post-game play survey. RESULTS: Of the 48 participants (11 male, 37 female; 44 of 48 were RNs), most (91%) agreed that the workflow reflected in the game was equivalent to the flow in a typical ED. Almost all (96%) found the cases in the game reflective of real ED patients, and most (92%) found the game a useful educational tool to prepare new nurses to transition into the ED environment. CONCLUSION: The GridlockED game shows potential as a serious game to support nursing education, particularly for new ED nurse orientation and transition to ED practice. [J Contin Educ Nurs. 2024;55(5):231-238.].

Deidentifying Narrative Assessments to Facilitate Data Sharing in Medical Education.

PROBLEM: Narrative assessments are commonly incorporated into competency-based medical education programs. However, efforts to share competency-based medical education assessment data among programs to support the evaluation and improvement of assessment systems have been limited in part because of security concerns. Deidentifying assessment data mitigates these concerns, but deidentifying narrative assessments is time-consuming, resource intensive, and error prone. The authors developed and tested a tool to automate the deidentification of narrative assessments and facilitate their review. APPROACH: The authors met throughout 2021 and 2022 to iteratively design, test, and refine the deidentification algorithm and data review interface. Preliminary testing of the prototype deidentification algorithm was performed using narrative assessments from the University of Saskatchewan emergency medicine program. The algorithm's accuracy was assessed by the authors using the review interface designed for this purpose. Formal testing included 2 rounds of deidentification and review by members of the authorship team. Both the algorithm and data review interface were refined during the testing process. OUTCOMES: Authors from 3 institutions, including 3 emergency medicine programs, an anesthesia program, and a surgical program, participated in formal testing. In the final round of review, 99.4% of the narrative assessments were fully deidentified (names, nicknames, and pronouns removed). The results were comparable for each institution and specialty. The data review interface was improved with feedback obtained after each round of review and found to be intuitive. NEXT STEPS: This innovation has demonstrated viability evidence of an algorithmic approach to the deidentification of assessment narratives while reinforcing that a small number of errors are likely to persist. Future steps include the refinement of both the algorithm to improve its accuracy and the data review interface to support additional data set formats.

Systems-Based Practice in Graduate Medical Education: Evolving Toward an Ideal Future State.

ABSTRACT: Systems-based practice (SBP) was first introduced as a core competency in graduate medical education (GME) in 2002 by the Accreditation Council for Graduate Medical Education as part of the Outcomes Project. While inclusion of SBP content in GME has become increasingly common, there have also been well-documented stumbling blocks, including perceptions that SBP has eroded the amount of curricular time available for more medically focused competencies, is not relevant for some practice contexts, and is not introduced early enough in training. As a result, SBP learning experiences often feel disconnected from medical trainees' practical reality. In this commentary, the authors provide guidance regarding potential changes that may facilitate the evolution of SBP toward an ideal future state where graduates bring a systems science mindset to all aspects of their work. Specific suggestions include the following: (1) expanding the SBP toolbox to reflect current-day health system needs, (2) evolve the teaching methodology, (3) broadening the scope of relevant SBP content areas, and (4) emphasizing SBP as an integrated responsibility for all health care team members. Levers to enact this transformation exist and must be used to influence change at the learner, faculty, program, and clinical learning environment levels.Physicians operate within an increasingly complex health care system that highlights the intersection of health care with complex social, environmental, and relational contexts. Consequently, the role of SBP in both physician work responsibilities and educational requirements continues to expand. To meet this growing demand, GME must adapt how it supports and trains the next generation of systems thinkers, ensuring they understand how levers in the health care system directly affect health outcomes for their patients, and integrate SBP into the foundation of GME curricula in an inclusive, holistic, and unrestrained way.

Brain versus bot: Distinguishing letters of recommendation authored by humans compared with artificial intelligence.

Objectives: Letters of recommendation (LORs) are essential within academic medicine, affecting a number of important decisions regarding advancement, yet these letters take significant amounts of time and labor to prepare. The use of generative artificial intelligence (AI) tools, such as ChatGPT, are gaining popularity for a variety of academic writing tasks and offer an innovative solution to relieve the burden of letter writing. It is yet to be determined if ChatGPT could aid in crafting LORs, particularly in high-stakes contexts like faculty promotion. To determine the feasibility of this process and whether there is a significant difference between AI and human-authored letters, we conducted a study aimed at determining whether academic physicians can distinguish between the two. Methods: A quasi-experimental study was conducted using a single-blind design. Academic physicians with experience in reviewing LORs were presented with LORs for promotion to associate professor, written by either humans or AI. Participants reviewed LORs and identified the authorship. Statistical analysis was performed to determine accuracy in distinguishing between human and AI-authored LORs. Additionally, the perceived quality and persuasiveness of the LORs were compared based on suspected and actual authorship. Results: A total of 32 participants completed letter review. The mean accuracy of distinguishing between human- versus AI-authored LORs was 59.4%. The reviewer's certainty and time spent deliberating did not significantly impact accuracy. LORs suspected to be human-authored were rated more favorably in terms of quality and persuasiveness. A difference in gender-biased language was observed in our letters: human-authored letters contained significantly more female-associated words, while the majority of AI-authored letters tended to use more male-associated words. Conclusions: Participants were unable to reliably differentiate between human- and AI-authored LORs for promotion. AI may be able to generate LORs and relieve the burden of letter writing for academicians. New strategies, policies, and guidelines are needed to balance the benefits of AI while preserving integrity and fairness in academic promotion decisions.

Educator's blueprint: A how-to guide for creating analog serious games for learning in medical education.

Serious games are an emerging tool for teaching and learning within medical education. These games can be used to facilitate learning or to demonstrate complex concepts in short bursts of interactive learning. This educator's blueprint will provide 10 strategies for creating a serious game, focusing on card and board games. These strategies include creating a project charter; determining the nature of the game; establishing game mechanics; selecting the best medium; prototyping and playtesting; reviewing sensitivity to equity, diversity, and inclusion; reviewing and refining content; funding game development, manufacture, and distribution; marketing and publicizing the game; and future-proofing the game. This blueprint hopes to help aspiring serious game designers and educators to conceptualize the steps for successfully creating a new serious game for medical education.