ReThinking clinical reasoning: A paradigm shift.

Numerous studies have demonstrated that our healthcare systems and medical education programs are fundamentally flawed. In North America and Europe, most systems were built upon values and structures that have historically benefitted middle and upper class males of European descent in the global north. As a result, there continue to be systemic biases that are pervasive throughout our healthcare systems and medical education programs. This has led to inequities in health outcomes and clinical reasoning practices which marginalize several communities. These biases are perpetuated as we continue to lead medical education research and practice with traditional values and views of evidence. To address these issues, we proposed a 'flipped' conference in which three interdisciplinary writing teams, comprised of both junior and senior academics, clinicians, and researchers, were invited to rethink the foundations of clinical reasoning. In the months leading up to the conference, each writing team explored a specific topic related to clinical reasoning and racial equity. The papers, presented during the virtual conference are now available in this issue of the Journal for the Evaluation of Clinical Practice. In addition, 6 more publications were added to this special topic to showcase new evidence and theory that builds on the recommendations in the three core papers.

Perceptions of gender equity in emergency medicine in Canada.

INTRODUCTION: Women-identifying emergency physicians face gender-based discrimination throughout their careers. The purpose of this study was to explore emergency physician's perceptions and experiences of gender equity in emergency medicine. METHODS: We conducted a secondary analysis of data from a previously conducted survey of Canadian emergency physicians on barriers to gender equity in emergency medicine. Survey responses were analyzed using logistic regression to determine the impact that gender, practice setting, years since graduation, race, equity-seeking status, and parental status had on agreement about gender equity in emergency medicine and five of the problem statements. RESULTS: A total of 710 participants completed the survey. Most identified as women (58.8%), white (77.4%), graduated between 2010 and 2019 (40%), had CCFP (Emergency Medicine) designation (47.9%), an urban practice (84.4%), were parents (62.4%) and did not identify as equity-seeking (79.9%). Women-identifying physicians were less likely to perceive gender equity in emergency medicine, OR 0.52, CI [0.38, 0.73]. Women-identifying physicians were more likely to agree with statements about microaggressions, OR 4.39, CI [2.66, 7.23]; barriers to leadership, OR 3.51, CI [2.25, 5.50]; gender wage gap, OR 13.46, CI [8.27, 21.91]; lack of support for parental leave, OR 2.85, CI [1.82, 4.44]; and education on allyship, OR 2.23 CI [1.44, 3.45] than men-identifying physicians. CONCLUSION: In this study, women-identifying physicians were less likely to perceive that there was gender equity in emergency medicine than men-identifying physicians. Women-identifying physicians agreed that there are greater barriers for career advancement including fewer opportunities for leadership, a gender wage gap, a lack of parental leave policies to support a return to work and a lack of education for men to become allies. Men-identifying physicians were less aware of these inequities. Health systems must work to improve gender equity in emergency medicine and this will require education and allyship from men-identifying physicians.

RéSUMé: INTRODUCTION: Les femmes médecins urgentistes sont confrontées à une discrimination fondée sur le sexe tout au long de leur carrière. L'objectif de cette étude était d'explorer les perceptions et les expériences des médecins urgentistes en matière d'équité entre les sexes en médecine d'urgence. MéTHODES: Nous avons procédé à une analyse secondaire des données d'une enquête menée précédemment auprès des médecins urgentistes canadiens sur les obstacles à l'équité entre les sexes en médecine d'urgence. Les réponses au sondage ont été analysées à l'aide d'une régression logistique pour déterminer l'incidence que le sexe, le milieu de pratique, les années écoulées depuis l'obtention du diplôme, la race, le statut de demandeur d'équité et le statut parental avaient sur l'accord sur l'équité entre les sexes en médecine d'urgence et cinq des énoncés de problème. RéSULTATS: Au total, 710 participants ont répondu à l'enquête. La plupart d'entre eux sont des femmes (58.8 %), de race blanche (77.4 %), ont obtenu leur diplôme entre 2010 et 2019 (40 %), ont le titre de CCMF (médecine d'urgence) (47.9 %), exercent en milieu urbain (84.4 %), sont parents (62.4 %) et ne se déclarent pas en quête d'équité (79.9 %). Les médecins s'identifiant à des femmes étaient moins susceptibles de percevoir l'équité entre les sexes en médecine d'urgence, OR 0.52, IC [0.38,0.73]. Les médecins s'identifiant comme femmes étaient plus susceptibles d'être d'accord avec les déclarations sur les microagressions, OR 4.39, IC [2.66, 7.23] ; obstacles au leadership, OR 3.51, IC [2.25, 5.50] ; écart salarial entre les hommes et les femmes, OR 13.46, IC [8.27, 21.91] ; le manque de soutien pour le congé parental, OR 2.85, IC [1.82, 4.44]; et l'éducation sur l'alliance, OR 2.23 IC [1.44, 3.45] que les médecins s'identifiant comme hommes. CONCLUSION: Dans cette étude, les médecins s'identifiant à des femmes étaient moins susceptibles de percevoir qu'il y avait une équité entre les sexes en médecine d'urgence que les médecins s'identifiant à des hommes. Les femmes médecins s'accordent à dire qu'il existe davantage d'obstacles à l'avancement professionnel, notamment moins d'opportunités de leadership, un écart salarial entre les hommes et les femmes, un manque de politiques de congé parental pour favoriser le retour au travail et un manque d'éducation des hommes pour qu'ils deviennent des alliés. Les médecins s'identifiant à des hommes étaient moins conscients de ces inégalités. Les systèmes de santé doivent s'efforcer d'améliorer l'équité entre les sexes dans la médecine d'urgence, ce qui nécessitera une formation et un allié de la part des médecins qui s'identifient aux hommes.

The Promise and Perils of Artificial Intelligence in Health Professions Education Practice and Scholarship.

ABSTRACT: Artificial intelligence (AI) methods, especially machine learning and natural language processing, are increasingly affecting health professions education (HPE), including the medical school application and selection processes, assessment, and scholarship production. The rise of large language models over the past 18 months, such as ChatGPT, has raised questions about how best to incorporate these methods into HPE. The lack of training in AI among most HPE faculty and scholars poses an important challenge in facilitating such discussions. In this commentary, the authors provide a primer on the AI methods most often used in the practice and scholarship of HPE, discuss the most pressing challenges and opportunities these tools afford, and underscore that these methods should be understood as part of the larger set of statistical tools available.Despite their ability to process huge amounts of data and their high performance completing some tasks, AI methods are only as good as the data on which they are trained. Of particular importance is that these models can perpetuate the biases that are present in those training datasets, and they can be applied in a biased manner by human users. A minimum set of expectations for the application of AI methods in HPE practice and scholarship is discussed in this commentary, including the interpretability of the models developed and the transparency needed into the use and characteristics of such methods.The rise of AI methods is affecting multiple aspects of HPE including raising questions about how best to incorporate these models into HPE practice and scholarship. In this commentary, we provide a primer on the AI methods most often used in HPE and discuss the most pressing challenges and opportunities these tools afford.

Tuition Costs of Master's of Health Professions Education Programs: A Cross-Sectional Analysis.

PURPOSE: The increasing professionalization of medical education during the past 2 decades has ushered in an era in which formal degrees, particularly master's of health professions education (MHPE), have become important for career advancement in medical education. Although tuition costs can pose a substantial barrier for many seeking advanced degrees in health professions education, data on tuition associated with these programs are lacking. This study examines the accessibility of pertinent cost-related information available to prospective students and the variability of costs among programs worldwide. METHOD: The authors conducted an Internet-based, cross-sectional study, augmented with emails and direct contact with educators, to extract tuition-related data for MHPE programs between March 29, 2022, and September 20, 2022. Costs were converted to an annual total within each jurisdiction's currency and converted to U.S. dollars on August 18, 2022. RESULTS: Of the 121 programs included in the final cost analysis, only 56 had publicly available cost information. Excluding programs free to local students, the mean (SD) total tuition cost was $19,169 ($16,649), and the median (interquartile range) cost was $13,784 ($9,401- $22,650) (n = 109). North America had the highest mean (SD) tuition for local students ($26,751 [$22,538]), followed by Australia and New Zealand ($19,778 [$10,514]) and Europe ($14,872 [$7,731]), whereas Africa had the lowest ($2,598 [$1,650]). The region with the highest mean (SD) tuition for international students was North America ($38,217 [$19,500]), followed by Australia and New Zealand ($36,891 [$10,397]) and Europe ($22,677 [$10,010]), whereas Africa had the lowest ($3,237 [$1,189]). CONCLUSIONS: There is substantial variability in the geographic distribution of MHPE programs and marked differences in tuition. Incomplete program websites and limited responsiveness from many programs contributed to a lack of transparency regarding potential financial implications. Greater efforts are necessary to ensure equitable access to health professions education.

His opportunity, her burden: A narrative critical review of why women decline academic opportunities.

OBJECTIVES: This paper stems from a desire to deepen our own understanding of why women might 'say no' when allies and sponsors offer or create opportunities for advancement, leadership or recognition. The resulting disparity between representation by men and women in leadership positions, invited keynote speakers and publication counts in academic medicine is a stubborn and wicked problem that requires a synthesis of knowledge across multidisciplinary literature. Acknowledging the complexity of this topic, we selected a narrative critical review methodology to explore reasons why one man's opportunity might be a woman's burden in academic medicine. METHODS: We engaged with an iterative process of identifying, reviewing and interpreting literature from Psychology (cognitive, industrial and educational), Sociology, Health Professions Education and Business, placing no restrictions on context or year of publication. Knowledge synthesis and interpretation were guided by our combined expertise, lived experience, consultations with experts outside the author team and these guiding questions: (1) Why might women have less time for career advancement opportunities? (2) Why do women have less time for research and leadership? (3) How are these disparities maintained? RESULTS: Turning down an opportunity may be a symptom of a much larger issue. The power of social expectations, culture and gender stereotypes remains a resistant force against calls for action. Consequently, women disproportionately take on other tasks that are not as well recognised. This disparity is maintained through social consequences for breaking with firmly entrenched stereotypes. CONCLUSIONS: Popular strategies like 'lean into opportunities', 'fake it till you make it' and 'overcome your imposter syndrome' suggest that women are standing in their own way. Critically, these axioms ignore powerful systemic barriers that shape these choices and opportunities. We offer strategies that allies, sponsors and peers can implement to offset the power of stereotypes.

The Quality of Assessment for Learning score for evaluating written feedback in anesthesiology postgraduate medical education: a generalizability and decision study.

Background: Competency based residency programs depend on high quality feedback from the assessment of entrustable professional activities (EPA). The Quality of Assessment for Learning (QuAL) score is a tool developed to rate the quality of narrative comments in workplace-based assessments; it has validity evidence for scoring the quality of narrative feedback provided to emergency medicine residents, but it is unknown whether the QuAL score is reliable in the assessment of narrative feedback in other postgraduate programs. Methods: Fifty sets of EPA narratives from a single academic year at our competency based medical education post-graduate anesthesia program were selected by stratified sampling within defined parameters [e.g. resident gender and stage of training, assessor gender, Competency By Design training level, and word count (≥17 or <17 words)]. Two competency committee members and two medical students rated the quality of narrative feedback using a utility score and QuAL score. We used Kendall's tau-b co-efficient to compare the perceived utility of the written feedback to the quality assessed with the QuAL score. The authors used generalizability and decision studies to estimate the reliability and generalizability coefficients. Results: Both the faculty's utility scores and QuAL scores (r = 0.646, p < 0.001) and the trainees' utility scores and QuAL scores (r = 0.667, p < 0.001) were moderately correlated. Results from the generalizability studies showed that utility scores were reliable with two raters for both faculty (Epsilon=0.87, Phi=0.86) and trainees (Epsilon=0.88, Phi=0.88). Conclusions: The QuAL score is correlated with faculty- and trainee-rated utility of anesthesia EPA feedback. Both faculty and trainees can reliability apply the QuAL score to anesthesia EPA narrative feedback. This tool has the potential to be used for faculty development and program evaluation in Competency Based Medical Education. Other programs could consider replicating our study in their specialty.

Contexte: La qualité de la rétroaction à la suite de l'évaluation d'activités professionnelles confiables (APC) est d'une importance capitale dans les programmes de résidence fondés sur les compétences. Le score QuAL (Quality of Assessment for Learning) est un outil développé pour évaluer la qualité de la rétroaction narrative dans les évaluations en milieu de travail. Sa validité a été démontrée dans le cas des commentaires narratifs fournis aux résidents en médecine d'urgence, mais sa fiabilité n'a pas été évaluée dans d'autres programmes de formation postdoctorale. Méthodes: Cinquante ensembles de commentaires portant sur des APC d'une seule année universitaire dans notre programme postdoctoral en anesthésiologie ­ un programme fondé sur les compétences ­ ont été sélectionnés par échantillonnage stratifié selon des paramètres préétablis [par exemple, le sexe du résident et son niveau de formation, le sexe de l'évaluateur, le niveau de formation en Compétence par conception, et le nombre de mots (≥17 ou <17 mots)]. Deux membres du comité de compétence et deux étudiants en médecine ont évalué la qualité de la rétroaction narrative à l'aide d'un score d'utilité et d'un score QuAL. Nous avons utilisé le coefficient tau-b de Kendall pour comparer l'utilité perçue de la rétroaction écrite et sa qualité évaluée à l'aide du score QuAL. Les auteurs ont utilisé des études de généralisabilité et de décision pour estimer les coefficients de fiabilité et de généralisabilité. Résultats: Les scores d'utilité et les scores QuAL des enseignants (r = 0,646, p < 0,001) et ceux des étudiants (r = 0,667, p < 0,001) étaient modérément corrélés. Les résultats des études de généralisabilité ont montré qu'avec deux évaluateurs les scores d'utilité étaient fiables tant pour les enseignants (Epsilon=0,87, Phi=0,86) que pour les étudiants (Epsilon=0,88, Phi=0,88). Conclusions: Le score QuAL est en corrélation avec l'utilité de la rétroaction sur les APC en anesthésiologie évaluée par les enseignants et les étudiants. Les uns et les autres peuvent appliquer de manière fiable le score QuAL aux commentaires narratifs sur les APC en anesthésiologie. Cet outil pourrait être utilisé pour le perfectionnement professoral et l'évaluation des programmes dans le cadre d'une formation médicale fondée sur les compétences. D'autres programmes pourraient envisager de reproduire notre étude dans leur spécialité.

Validity evidence for the Quality of Assessment for Learning score: a quality metric for supervisor comments in Competency Based Medical Education.

Background: Competency based medical education (CBME) relies on supervisor narrative comments contained within entrustable professional activities (EPA) for programmatic assessment, but the quality of these supervisor comments is unassessed. There is validity evidence supporting the QuAL (Quality of Assessment for Learning) score for rating the usefulness of short narrative comments in direct observation. Objective: We sought to establish validity evidence for the QuAL score to rate the quality of supervisor narrative comments contained within an EPA by surveying the key end-users of EPA narrative comments: residents, academic advisors, and competence committee members. Methods: In 2020, the authors randomly selected 52 de-identified narrative comments from two emergency medicine EPA databases using purposeful sampling. Six collaborators (two residents, two academic advisors, and two competence committee members) were recruited from each of four EM Residency Programs (Saskatchewan, McMaster, Ottawa, and Calgary) to rate these comments with a utility score and the QuAL score. Correlation between utility and QuAL score were calculated using Pearson's correlation coefficient. Sources of variance and reliability were calculated using a generalizability study. Results: All collaborators (n = 24) completed the full study. The QuAL score had a high positive correlation with the utility score amongst the residents (r = 0.80) and academic advisors (r = 0.75) and a moderately high correlation amongst competence committee members (r = 0.68). The generalizability study found that the major source of variance was the comment indicating the tool performs well across raters. Conclusion: The QuAL score may serve as an outcome measure for program evaluation of supervisors, and as a resource for faculty development.

Contexte: Dans la formation médicale fondée sur les compétences (FMFC), l'évaluation programmatique s'appuie sur les commentaires narratifs des superviseurs en lien avec les activités professionnelles confiables (EPA). En revanche, la qualité de ces commentaires n'est pas évaluée. Il existe des preuves de la validité du score QuAL (qualité de l'évaluation pour l'apprentissage, Quality of Assessment for Learning en anglais) pour l'évaluation de l'utilité des commentaires de rétroaction courts lors de la supervision par observation directe. Objectif: Nous avons tenté de démontrer la validité du score QuAL aux fins de l'évaluation de la qualité des commentaires narratifs des superviseurs pour une APC en interrogeant les principaux utilisateurs finaux des rétroactions : les résidents, les conseillers pédagogiques et les membres du comité de compétence. Méthodes: En 2020, les auteurs ont sélectionné au hasard 52 commentaires narratifs anonymisés dans deux bases de données d'APC en médecine d'urgence au moyen d'un échantillonnage intentionnel. Six collaborateurs (deux résidents, deux conseillers pédagogiques et deux membres de comités de compétence) ont été recrutés dans chacun des quatre programmes de résidence en médecine d'urgence (Saskatchewan, McMaster, Ottawa et Calgary) pour évaluer ces commentaires à l'aide d'un score d'utilité et du score QuAL. La corrélation entre l'utilité et le score QuAL a été calculée à l'aide du coefficient de corrélation de Pearson. Les sources de variance et la fiabilité ont été calculées à l'aide d'une étude de généralisabilité. Résultats: Tous les collaborateurs (n=24) ont réalisé l'étude complète. Le score QuAL présentait une corrélation positive élevée avec le score d'utilité parmi les résidents (r=0,80) et les conseillers pédagogiques (r=0,75) et une corrélation modérément élevée parmi les membres du comité de compétence (r=0,68). L'étude de généralisation a révélé que la principale source de variance était le commentaire, ce qui indique que l'outil a fonctionné avec une efficacité égale pour tous les évaluateurs. Conclusion: Le score QuAL peut servir de mesure des résultats pour l'évaluation des superviseurs par les programmes, et de ressource pour le perfectionnement du corps professoral.

Cost of free open-access medical education (FOAM): An economic analysis of the top 20 FOAM sites.

Background: Free open-access medical education (FOAM) is a growing resource within the field of medicine, in particular, emergency medicine. Yet despite FOAM's contribution to advancing medical education, the precise value of FOAM has never been calculated. As a result, content creators have not been acknowledged, either financially or academically, for their deliverables. The aim of this paper was to meet this challenge by determining a value for the top 20 FOAM sites as determined by web traffic in emergency medicine through two approaches. The first approach was to value the websites through a market-based method, where the value of the website was extrapolated from the number of blog posts published. The second approach was through a traffic analysis for each website. Methods: The top 20 FOAM websites in emergency medicine were identified and the monetary value of each resource was calculated through two methods, the first by extrapolating the number of blog posts published by each resource and the second through traffic analysis conducted by a third-party industry specialist based on the number of unique visitors and page visits. Results: The median page views per month was 194,850 and the median number of unique visitors was 138,350. Based on the content valuation method, the median value of content produced in a year was $2337.06 per website. Through the traffic valuation method, the median overall value of a website was $22,815. Conclusions: Although two different approaches were used to value FOAM, both came to the same conclusion that there is substantial economic value being produced. This value should not go unrewarded and content creators should be acknowledged either academically or financially for their contributions.

The Future of Emergency Medicine (EM) Sim Cases: A Modified Massive Online Needs Assessment.

Objective Emergency Medicine (EM) Sim Cases was initially developed in 2015 as a free open-access simulation resource. To ensure the future of EM Sim Cases remains relevant and up to date, we performed a needs assessment to better define our audience and facilitate long-term goals. Methods We delivered a survey using a modified massive-online-needs-assessment methodology through an iterative process with simulation experts from the EM Simulation Educators Research Collaborative. We distributed the survey via email and Twitter and analyzed the data using descriptive statistics and thematic analysis. Results We obtained 106 responses. EM Sim Cases is commonly used by physicians primarily as an educational resource for postgraduate level trainees. Perceived needs included resuscitation, pediatrics, trauma, and toxicology content. Prompted needs included non-simulation-case educational resources, increased case database, and improved website organization. Conclusions Data collected from our needs assessment has defined our audience allowing us to design our long-term goals and strategies.

Harnessing Natural Language Processing to Support Decisions Around Workplace-Based Assessment: Machine Learning Study of Competency-Based Medical Education.

BACKGROUND: Residents receive a numeric performance rating (eg, 1-7 scoring scale) along with a narrative (ie, qualitative) feedback based on their performance in each workplace-based assessment (WBA). Aggregated qualitative data from WBA can be overwhelming to process and fairly adjudicate as part of a global decision about learner competence. Current approaches with qualitative data require a human rater to maintain attention and appropriately weigh various data inputs within the constraints of working memory before rendering a global judgment of performance. OBJECTIVE: This study explores natural language processing (NLP) and machine learning (ML) applications for identifying trainees at risk using a large WBA narrative comment data set associated with numerical ratings. METHODS: NLP was performed retrospectively on a complete data set of narrative comments (ie, text-based feedback to residents based on their performance on a task) derived from WBAs completed by faculty members from multiple hospitals associated with a single, large, residency program at McMaster University, Canada. Narrative comments were vectorized to quantitative ratings using the bag-of-n-grams technique with 3 input types: unigram, bigrams, and trigrams. Supervised ML models using linear regression were trained with the quantitative ratings, performed binary classification, and output a prediction of whether a resident fell into the category of at risk or not at risk. Sensitivity, specificity, and accuracy metrics are reported. RESULTS: The database comprised 7199 unique direct observation assessments, containing both narrative comments and a rating between 3 and 7 in imbalanced distribution (scores 3-5: 726 ratings; and scores 6-7: 4871 ratings). A total of 141 unique raters from 5 different hospitals and 45 unique residents participated over the course of 5 academic years. When comparing the 3 different input types for diagnosing if a trainee would be rated low (ie, 1-5) or high (ie, 6 or 7), our accuracy for trigrams was 87%, bigrams 86%, and unigrams 82%. We also found that all 3 input types had better prediction accuracy when using a bimodal cut (eg, lower or higher) compared with predicting performance along the full 7-point rating scale (50%-52%). CONCLUSIONS: The ML models can accurately identify underperforming residents via narrative comments provided for WBAs. The words generated in WBAs can be a worthy data set to augment human decisions for educators tasked with processing large volumes of narrative assessments.

The Impact of Electronic Data to Capture Qualitative Comments in a Competency-Based Assessment System.

Introduction Digitalizing workplace-based assessments (WBA) holds the potential for facilitating feedback and performance review, wherein we can easily record, store, and analyze data in real time. When digitizing assessment systems, however, it is unclear what is gained and lost in the message as a result of the change in medium. This study evaluates the quality of comments generated in paper vs. electronic media and the influence of an assessor's seniority. Methods Using a realist evaluation framework, a retrospective database review was conducted with paper-based and electronic medium comments. A sample of assessments was examined to determine any influence of the medium on the word count and the Quality of Assessment for Learning (QuAL) score. A correlation analysis evaluated the relationship between word count and QuAL score. Separate univariate analyses of variance (ANOVAs) were used to examine the influence of the assessor's seniority and medium on word count, QuAL score, and WBA scores. Results The analysis included a total of 1,825 records. The average word count for the electronic comments (M=16) was significantly higher than the paper version (M=12; p=0.01). Longer comments positively correlated with QuAL score (r=0.2). Paper-based comments received lower QuAL scores (0.41) compared to electronic (0.51; p<0.01). Years in practice was negatively correlated with QuAL score (r=-0.08; p<0.001) as was word count (r=-0.2; p<0.001). Conclusion Digitization of WBAs increased the length of comments and did not appear to jeopardize the quality of WBAs; these results indicate higher-quality assessment data. True digital transformation may be possible by harnessing trainee data repositories and repurposing them to analyze for faculty-relevant metrics.

Beyond the CLAIM: A comprehensive needs assessment strategy for creating an Advanced Medical Education Research Training Program (ARMED-MedEd).

BACKGROUND: The health professions education (HPE) landscape has shifted substantively with increasing professionalization of research and scholarship. Clinician educators seeking to become competitive in this domain often pursue fellowships and master's degrees in HPE, but there are few resources for the continuing professional development (CPD) of those who seek to continue developing their scholarly practice within HPE. Acknowledging the multiple players in this landscape, the authors sought to design a new "beyond beginners" HPE research program using a novel needs assessment planning process. METHODS: The authors developed and conducted a new three-phase, five-step process that sets forth a programmatic approach to conducting a needs assessment for a CPD course in HPE research. The five steps of the CLAIM method are: Competitive analysis, Literature review with thematic analysis, Ask stakeholders, Internal review by experts, and Mapping of a curriculum. These steps are organized into three phases (Discovery, Convergence, and Synthesis). RESULTS: Over a 12-month period, the authors completed a comprehensive needs assessment. The CLAIM process revealed that longitudinal digital connection, diverse and in depth exposure to HPE research methods, skills around scholarly publishing, and leadership and management of research would be beneficial to our design. CONCLUSIONS: The CLAIM method provided scaffolding to help the authors create a robust curriculum that adopts a scholarly approach for developing a HPE research course. This needs assessment methodology may be useful in other CPD contexts.

Priority strategies to improve gender equity in Canadian emergency medicine: proceedings from the CAEP 2021 Academic Symposium on leadership.

OBJECTIVES: Gender inequities are deeply rooted in our society and have significant negative consequences. Female physicians experience numerous gender-related inequities (e.g., microaggressions, harassment, violence). These inequities have far-reaching consequences on health, well-being and career longevity and may result in the devaluing of various strengths that female emergency physicians bring to the table. This, in turn, has an impact on patient healthcare experience and outcomes. During the 2021 Canadian Association of Emergency Physicians (CAEP) Academic Symposium, a national collaborative sought to understand gender inequities in emergency medicine in Canada. METHODS: We used a multistep stakeholder-engagement-based approach (harnessing both quantitative and qualitative methods) to identify and prioritize problems with gender equity in emergency medicine in Canada. Based on expert consultation and literature review, we developed recommendations to effect change for the higher priority problems. We then conducted a nationwide consultation with the Canadian emergency medicine community via online engagement and the CAEP Academic Symposium to ensure that these priority problems and solutions were appropriate for the Canadian context. CONCLUSION: Via the above process, 15 recommendations were developed to address five unique problem areas. There is a dearth of research in this important area and we hope this preliminary work will serve as a starting point to fuel further research. To facilitate these scholarly endeavors, we have appended additional documents identifying other key problems with gender equity in emergency medicine in Canada as well as proposed next steps for future research.

RéSUMé: OBJECTIFS: Les inégalités entre les sexes sont profondément ancrées dans notre société et ont des conséquences négatives importantes. Les femmes médecins subissent de nombreuses inégalités liées au genre (par exemple, microagressions, harcèlement, violence). Ces inégalités ont des conséquences considérables sur la santé, le bien-être et la longévité de la carrière et peuvent entraîner la dévalorisation des différents atouts que les femmes médecins urgentistes apportent à la table. Ceci, à son tour, a un impact sur l'expérience et les résultats des soins de santé des patients. Au cours du Symposium académique 2021 de l'Association canadienne des médecins d'urgence (ACMU), une collaboration nationale a cherché à comprendre les inégalités entre les sexes en médecine d'urgence au Canada. MéTHODES: Nous avons utilisé une approche en plusieurs étapes basée sur l'engagement des parties prenantes (en utilisant des méthodes quantitatives et qualitatives) pour identifier et classer par ordre de priorité les problèmes d'équité entre les sexes en médecine d'urgence au Canada. À partir d'une consultation d'experts et d'une revue de la littérature, nous avons élaboré des recommandations visant à apporter des changements aux problèmes les plus prioritaires. Nous avons ensuite mené une consultation nationale auprès de la communauté canadienne de médecine d'urgence par le biais d'un engagement en ligne et du symposium universitaire de l'ACMU afin de nous assurer que ces problèmes prioritaires et ces solutions étaient adaptés au contexte canadien. CONCLUSION: Grâce au processus ci-dessus, 15 recommandations ont été élaborées pour traiter 5 domaines problématiques uniques. Il existe un manque de recherche dans ce domaine important et nous espérons que ce travail préliminaire servira de point de départ pour alimenter d'autres recherches. Pour faciliter ces efforts de recherche, nous avons annexé d'autres documents identifiant d'autres problèmes clés en matière d'équité entre les sexes en médecine d'urgence au Canada, ainsi que des propositions d'étapes pour de futures recherches.

Redesigning continuing professional development: Harnessing design thinking to go from needs assessment to mandate.

BACKGROUND: The world of medicine is constantly changing, and with it the continuing professional development (CPD) needs of physicians. As the CPD landscape is shifting away from unidirectional delivery of knowledge through live large group learning (conferences) and is placing increased emphasis on new approaches for skills training not taught a decade ago, a new approach is needed. APPROACH: Using design thinking techniques, we hosted a full-day retreat for emergency medicine stakeholders in Hamilton and the surrounding region. Prior to the retreat we collected medico-legal data on emergency physicians in our region and performed a needs assessment survey. At the retreat, we had participants brainstorm ideas for CPD, generate archetypes for end-users, then generate solutions to the problems they had identified. These proposals were presented to the larger group for feedback and refinement. EVALUATION: The Design Thinking Retreat generated five main pillars for action by our CPD team. 1) Simulation/procedural learning (staff simulation, procedural skills day, in situ simulation); 2) Asynchronous learning (website and podcast); 3) Synchronous learning (small group sessions for staff); 4) Community connectivity (online platform for collaboration and communication); and 5) Coaching & mentorship (focused coaching for specific practice improvement, improved onboarding for new staff). REFLECTION: These ideas have vastly increased engagement in CPD. Stakeholder consultation via design thinking may be a key approach for educators to use.

Workplace-based Assessment Data in Emergency Medicine: A Scoping Review of the Literature.

OBJECTIVE: In the era of competency-based medical education (CBME), the collection of more and more trainee data is being mandated by accrediting bodies such as the Accreditation Council for Graduate Medical Education and the Royal College of Physicians and Surgeons of Canada. However, few efforts have been made to synthesize the literature around the current issues surrounding workplace-based assessment (WBA) data. This scoping review seeks to synthesize the landscape of literature on the topic of data collection and utilization for trainees' WBAs in emergency medicine (EM). METHODS: The authors conducted a scoping review in the style of Arksey and O'Malley, seeking to synthesize and map literature on collecting, aggregating, and reporting WBA data. The authors extracted, mapped, and synthesized literature that describes, supports, and substantiates effective data collection and utilization in the context of the CBME movement within EM. RESULTS: Our literature search retrieved 189 potentially relevant references (after removing duplicates) that were screened to 29 abstracts and papers relevant to collecting, aggregating, and reporting WBAs. Our analysis shows that there is an increasing temporal trend toward contributions in these topics, with the majority of the papers (16/29) being published in the past 3 years alone. CONCLUSION: There is increasing interest in the areas around data collection and utilization in the age of CBME. The field, however, is only beginning to emerge, leaving more work that can and should be done in this area.

Becoming a deliberately developmental organization: Using competency based assessment data for organizational development.

Medical education is situated within health care and educational organizations that frequently lag in their use of data to learn, develop, and improve performance. How might we leverage competency-based medical education (CBME) assessment data at the individual, program, and system levels, with the goal of redefining CBME from an initiative that supports the development of physicians to one that also fosters the development of the faculty, administrators, and programs within our organizations? In this paper we review the Deliberately Developmental Organization (DDO) framework proposed by Robert Kegan and Lisa Lahey, a theoretical framework that explains how organizations can foster the development of their people. We then describe the DDO's conceptual alignment with CBME and outline how CBME assessment data could be used to spur the transformation of health care and educational organizations into digitally integrated DDOs. A DDO-oriented use of CBME assessment data will require intentional investment into both the digitalization of assessment data and the development of the people within our organizations. By reframing CBME in this light, we hope that educational and health care leaders will see their investments in CBME as an opportunity to spur the evolution of a developmental culture.

Good practices in harnessing social media for scholarly discourse, knowledge translation, and education.

INTRODUCTION: There still remains a gap between those who conduct science and those who engage in educating others about health sciences through various forms of social media. Few empirical studies have sought to define useful practices for engaging in social media for academic use in the health professions. Given the increasing importance of these platforms, we sought to define good practices and potential pitfalls with help of those respected for their work in this new field. METHODS: We conducted a qualitative study, guided by constructivist grounded theory principles, of 17 emerging experts in the field of academic social media. We engaged in a snowball sampling technique and conducted a series of semi-structured interviews. The analytic team consisted of a diverse group of researchers with a range of experience in social media. RESULTS: Understanding the strengths of various platforms was deemed to be of critical importance across all the participants. Key to building online engagement were the following: 1) Culture-building strategies; 2) Tailoring the message; 3) Responsiveness; and 4) Heeding rules of online engagement. Several points of caution were noted within our participants' interviews. These were grouped into caveat emptor and the need for critical appraisal, and common pitfalls when broadcasting one's self. DISCUSSION: Our participants were able to share a number of key practices that are central to developing and sharing educational content via social media. The findings from the study may guide future practitioners seeking to enter the space. These good practices support professionals for effective engagement and knowledge translation without being harmed.

Evaluation of a National Competency-Based Assessment System in Emergency Medicine: A CanDREAM Study.

BACKGROUND: In 2018, Canadian postgraduate emergency medicine (EM) programs began implementing a competency-based medical education (CBME) assessment program. Studies evaluating these programs have focused on broad outcomes using data from national bodies and lack data to support program-specific improvement. OBJECTIVE: We evaluated the implementation of a CBME assessment program within and across programs to identify successes and opportunities for improvement at the local and national levels. METHODS: Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of entrustable professional activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages were compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations. RESULTS: Data from 15 of 17 (88%) programs containing 9842 EPA observations from 68 of 77 (88%) EM residents in the 2018 cohort were analyzed. Average numbers of EPAs observed per resident in each program varied from 92.5 to 229.6, correlating with the number of blocks spent on EM and pediatric EM (r = 0.83, P < .001). Relative to the specialty committee's guidelines, residents were promoted later than expected (eg, one-third of residents had a 2-month delay to promotion from the first to second stage) and with fewer EPA observations than suggested. CONCLUSIONS: There was demonstrable variation in EPA-based assessment numbers and promotion timelines between programs and with national guidelines.