Analyzing interprofessional teamwork in the operating room: An exploratory observational study using conventional and alternative approaches.

Intraoperative teamwork is vital for patient safety. Conventional tools for studying intraoperative teamwork typically rely on behaviorally anchored rating scales applied at the individual or team level, while others capture narrative information across several units of analysis. This prospective observational study characterizes teamwork using two conventional tools (Operating Theatre Team Non-Technical Skills Assessment Tool [NOTECHS]; Team Emergency Assessment Measure [TEAM]), and one alternative approach (modified-Systems Engineering Initiative for Patient Safety [SEIPS] model). We aimed to explore the advantages and disadvantages of each for providing feedback to improve teamwork practice. Fifty consecutive surgical cases at a Canadian academic hospital were recorded with the OR Black Box®, analyzed by trained raters, and summarized descriptively. Teamwork performance was consistently high within and across cases rated with NOTECHS and TEAMS. For cases analyzed with the modified-SEIPS tool, both optimal and suboptimal teamwork behaviors were identified, and team resilience was frequently observed. NOTECHS and TEAM provided summative assessments and overall pattern descriptions, while SEIPS facilitated a deeper understanding of teamwork processes. As healthcare organizations continue to prioritize teamwork improvement, SEIPS may provide valuable insights regarding teamwork behavior and the broader context influencing performance. This may ultimately enhance the development and effectiveness of multi-level teamwork interventions.

The McMaster Advanced Communication Competencies Model for Psychiatry (MACC Model).

OBJECTIVE: Communication is a core competency for all physicians in training. In Canada, the importance of communication during residency is recognized through the CanMEDS framework. Although literature exists around teaching communication skills to residents, research in psychiatry residents is lacking. The purpose of this study was to explore how faculty members conceptualize the development of communication skills in psychiatry residents and develop a model reflecting this. METHODS: The authors used a constructivist grounded theory approach. Purposive sampling was used to select 14 faculty educators who regularly supervise psychiatry residents in a single university-based residency training program. Semi-structured interviews were conducted to explore educators' perceptions of how communication skills develop during residency. Constant comparative analysis occurred concurrently with data collection until thematic theoretical sufficiency was reached and relationships between themes determined. RESULTS: Five themes underlie the McMaster Advanced Communication Competencies model, describing the progressive development of communication abilities in residents. Three themes identify foundational sets of abilities including: refining common foundational relational abilities, developing foundational specific psychiatric communication abilities, and learning to reflect upon and manage internal reactions. These foundational pillars then allow residents to develop a personalized art of flexible psychiatric interviewing, and skillfully partner with patients in co-creating care plans. CONCLUSIONS: This research describes a preliminary communication competency model for teaching and assessing psychiatry residents. It defines the core communication abilities required for residents to progress to independent practice. Future research could explore and test the model's validity and transferability.

Breaking bad news to cancer patients in times of COVID-19.

Breaking bad news is a mandatory provision in the professional life of nearly every physician. One of its most frequent occasions is the diagnosis of malignancy. Responding to the recipients' emotions is a critical issue in the delivery of unsettling information, and has an impact on the patient's trust in the treating physician, adjustment to illness and ultimately treatment. Since the World Health Organization (WHO) declared COVID-19 a pandemic on March 11, 2020, several measures of social distancing and isolation have been introduced to our clinical setting. In the wake of these restrictions, it is important to reexamine existing communication guidelines to determine their applicability to face-to-face counseling in the context of social distancing, as well as to new communication technologies, such as telemedicine. We address these issues and discuss strategies to convey bad news the most empathetic and comprehensible way possible.

Validity of entrustment scales within anesthesiology residency training.

INTRODUCTION: Competency-based medical education requires robust assessment in authentic clinical environments. Using work-based assessments, entrustment scales have emerged as a means of describing a trainee's ability to perform competently. Nevertheless, psychometric properties of entrustment-based assessment are relatively unknown, particularly in anesthesiology. This study assessed the generalizability and extrapolation evidence for entrustment scales within a program of assessment during anesthesiology training. METHODS: Entrustment scores were collected during the first seven blocks of training for three resident cohorts. Entrustment scores were assessed during daily evaluations using a Clinical Case Assessment Tool (CCAT) within the preoperative, intraoperative, and postoperative setting. The reliability of the entrustment scale was estimated using generalizability theory. Spearman's correlations measured the relationship between median entrustment scores and percentiles scores on the Anesthesia Knowledge Test (AKT)-1 and AKT-6, mean Objective Structured Clinical Examination (OSCE) scores, and rankings of performance by the Clinical Competence Committee (CCC). RESULTS: Analyses were derived from 2,309 CCATs from 35 residents. The reliability or generalizability (G) coefficient of the entrustment scale was 0.73 (95% confidence interval [CI], 0.70 to 0.76), and the internal consistency was 0.86 (95% CI, 0.84 to 0.88). Intraoperative entrustment scores significantly correlated with the AKT-6 (rho = 0.51, P = 0.01), mean OSCE (rho = 0.45, P = 0.04), and CCC performance rankings (rho = 0.52, P = 0.006). CONCLUSION: As part of an assessment program, entrustment scales used early during anesthesiology training showed evidence of validity. Intraoperative entrustment scores had good reliability and showed acceptable internal consistency. Interpreting entrustment scores in this setting may constitute a valuable adjunct complementing traditional summative evaluations.

RéSUMé: INTRODUCTION: La formation médicale fondée sur les compétences nécessite une évaluation rigoureuse dans des environnements cliniques authentiques. Se fondant sur des évaluations basées sur le travail, les échelles de confiance sont apparues comme une méthode pour décrire la capacité d'un résident à performer de façon compétente. Toutefois, les propriétés psychométriques de l'évaluation basée sur la confiance sont relativement peu connues, particulièrement en anesthésiologie. Cette étude a évalué les données de généralisabilité et d'extrapolation des échelles de confiance dans le cadre d'un programme d'évaluation pendant la formation en anesthésiologie. MéTHODE: Les notes sur les échelles de confiance ont été colligées pendant les sept premiers blocs de formation de trois cohortes de résidents. Les notes sur les échelles de confiance ont été évaluées pendant les évaluations quotidiennes à l'aide d'un Outil d'évaluation des compétences cliniques (CCAT ­ Clinical Case Assessment Tool) dans les cadres préopératoire, peropératoire et postopératoire. La fiabilité de l'échelle de confiance a été estimée à l'aide de la théorie de la généralisabilité. Les corrélations de Spearman ont mesuré la relation entre des notes moyennes sur l'échelle de confiance et les scores de percentile aux examens de connaissances en anesthésiologie AKT-1 et AKT-6, les scores moyens à l'Examen clinique objectif structuré (ECOS), et les classements de performance par le Comité de compétences cliniques (CCC). RéSULTATS: Les analyses ont été dérivées à partir de 2309 résultats CCAT de 35 résidents. Le coefficient de fiabilité ou de généralisabilité (G) de l'échelle de confiance était de 0,73 (intervalle de confiance [IC] 95 %, 0,70 à 0,76), et la cohérence interne était de 0,86 (IC 95 %, 0,84 à 0,88). Les scores de confiance peropératoires étaient corrélés de manière significative aux scores sur l'AKT-6 (rho = 0,51, P = 0,01), aux scores moyens à l'ECOS (rho = 0,45, P = 0,04) et aux classements de performance du CCC (rho = 0,52, P = 0,006). CONCLUSION: Dans le cadre d'un programme d'évaluation, la validité des échelles de confiance utilisées en début de la formation en anesthésiologie a été éprouvée. Les scores de confiance peropératoires ont démontré une bonne fiabilité et une cohérence interne acceptable. L'interprétation des scores de confiance dans ce cadre pourrait constituer un ajout précieux qui complèterait les évaluations sommatives.

Assessing the process and outcome of the development of practice guidelines and recommendations: PANELVIEW instrument development.

BACKGROUND: Guideline recommendations may be affected by flaws in the process, inappropriate panel member selection or conduct, conflicts of interest and other factors. To our knowledge, no validated tool exists to evaluate guideline development from the perspective of those directly involved in the process. Our objective was to develop and validate a universal tool, the PANELVIEW instrument, to assess guideline processes, methods and outcomes from the perspective of the participating guideline panellists and group members. METHODS: We performed a systematic literature search and surveys of guideline groups (identified through contacting international organizations and convenience sampling of working panels) to inform item generation. Subsequent groups of guideline methodologists and panellists reviewed items for face validity and missing items. We used surveys, interviews and expert review for item reduction and phrasing. For reliability assessment and feedback, we tested the PANELVIEW tool in 8 international guideline groups. RESULTS: We surveyed 62 members from 13 guideline panels, contacted 19 organizations and reviewed 20 source documents to generate items. Fifty-three additional key informants provided feedback about phrasing of the items and response options. We reduced the number of items from 95 to 34 across domains that included administration, training, conflict of interest, group dynamics, chairing, evidence synthesis, formulating recommendations and publication. The tool takes about 10 minutes to complete and showed acceptable measurement properties. INTERPRETATION: The PANELVIEW instrument fills a gap by enabling guideline organizations to involve clinicians, patients and other participants in evaluating their guideline processes. The tool can inform quality improvement of existing or new guideline programs, focusing on insight into and transparency of the guideline development process, methods and outcomes.

The disconnect between knowing and doing in health professions education and practice.

Safe and effective healthcare requires that new knowledge or skills, once learned, are incorporated into professional practice. However, this process is not always straightforward. Learning takes place in complex contexts, requiring practitioners to overcome various motivational, systemic, emotional, and social barriers to the application of knowledge. This paper explores the mechanisms through which individuals translate knowledge into action to provide insight into why disconnects between knowledge and action can arise. As a critical review, the aim was to draw on key literature from multiple fields to analyse and synthesize existing schools of thought and lay a strong conceptual foundation on which knowledge to action gaps might be considered. We iteratively consulted clinicians and experts in various fields to guide literature searches focused on theoretical perspectives that could inform educational and research efforts around knowledge-to-action gaps. Key theoretical perspectives on motivation address when and how individuals decide to take action. Literatures from cognitive science address how clinicians and learners self-regulate to (sometimes) overcome barriers to action. Sociocultural theories examine the ways in which action might be prevented by social norms that conflict with what the individual knows and believes, potentially also giving rise to counter-normative action. No single perspective will entirely explain how health professionals and learners implement knowledge in practice. As a result, the authors offer multiple lenses through which to view the problem, and then propose how each of these lenses might better guide educational and research efforts to untangle this challenging but important issue.

Logging in: a comparative analysis of electronic health records versus anesthesia resident-driven logbooks.

PURPOSE: Resident logbooks (RLBs) documenting clinical case exposure are widespread in medical education despite evidence of poor accuracy. Electronic health records (e.g., anesthesia information management systems [AIMS]) may provide advantages for auditing longitudinal case exposure. We evaluated the agreement between AIMS and RLBs for tracking case exposure during anesthesiology residency. METHODS: We performed a historical cohort study with anesthesiology residents (2011-2018, all of whom used a RLB contemporaneously with AIMS) working in a multisite academic health sciences network. The primary outcome was total case-load logging; secondary outcomes were volumes for seven surgical specialties (general, gynecology, neuro, orthopedic, thoracic, urology, and vascular surgery). Correlation of case numbers tracked by AIMS vs RLB was assessed using Pearson correlation; agreement was determined using Bland-Altman plots and intraclass correlation coefficients (ICC). RESULTS: Data from 27 anesthesiology residents were collected. Overall, mean (standard deviation) case numbers were generally greater with AIMS vs RLB [649 (103) vs 583 (191); P = 0.049). Total case volumes between systems had moderate correlation (r = 0.50) and agreement (intraclass correlation coefficient [ICC], 0.42; 95% CI, 0.34 to 0.59). Bland-Altman plots showed variable agreement between AIMS and RLB data [mean (SD) bias = 66 (166) cases]. For general, gynecology, neuro, orthopedic, thoracic, urology, and vascular surgery, there was a range of poor to moderate agreement (ICC, 0.23-0.57) between AIMS and RLB. CONCLUSION: For anesthesiology resident case-logging, the number of cases logged in an AIMS was higher with lower variance compared with RLBs. Anesthesia information management systems vs RLB data showed low-moderate correlation and agreement. Given the additional time and resources required for RLBs, AIMS may be a superior method for tracking cases where available.

RéSUMé: OBJECTIF: Les logbooks des résidents documentant leur exposition à des cas cliniques sont répandus dans la formation médicale et ce, malgré des données probantes déplorant leur manque de précision. Les dossiers médicaux informatisés (par ex., les systèmes de gestion de l'information en anesthésie [SGIA]) pourraient présenter des avantages pour le contrôle de l'exposition longitudinale des résidents aux cas. Notre étude a évalué la concordance entre les SGIA et les logbooks pour l'exposition aux cas pendant la résidence en anesthésiologie. MéTHODE: Nous avons réalisé une étude de cohorte historique auprès de résidents en anesthésiologie (2011­2018, tous les résidents ayant utilisé un logbook simultanément aux SGIA) travaillant dans un réseau universitaire de sciences de la santé multisite. Le critère d'évaluation principal était l'enregistrement du nombre de cas total; les critères d'évaluation secondaires comprenaient les volumes pour sept spécialités chirurgicales (soit la chirurgie générale, gynécologique, orthopédique, thoracique, urologique, vasculaire et la neurochirurgie). La corrélation entre le nombre de cas enregistrés dans les SGIA vs les logbooks a été évaluée à l'aide d'une corrélation de Pearson; l'agrément a été déterminé à l'aide d'un graphique de Bland­Altman et de coefficients de corrélation intraclasse (CCI). RéSULTATS: Les données de 27 résidents en anesthésiologie ont été colligées. Globalement, les nombres de cas moyens (écart type) étaient en général plus élevés dans les SGIA que dans les logbooks [649 (103) vs 583 (191); P = 0,049). Les volumes de cas totaux entre les systèmes présentaient une corrélation (r = 0,50) et un agrément (coefficient de corrélation intraclasse [CCI], 0,42; IC 95 %, 0,34 à 0,59) modérés. Les graphiques de Bland­Altman ont démontré un agrément variable entre les données des SGIA et celles des logbooks [biais moyen (ÉT) = 66 (166) cas]. Pour les chirurgies générales, gynécologiques, orthopédiques, thoraciques, urologique, vasculaires et neurochirurgies, l'agrément allait de faible à modéré (CCI, 0,23-0,57) entre les SGIA et les logbooks. CONCLUSION: En ce qui a trait à l'enregistrement des cas des résidents en anesthésiologie, le nombre de cas enregistrés dans un SGIA était plus élevé et présentait une variance moindre que dans les logbooks. Les données des systèmes de gestion de l'information en anesthésie vs des logbooks ont affiché une corrélation et un agrément faible à modéré. Étant donné le temps et les ressources supplémentaires nécessaires pour compléter les logbooks, les SGIA pourraient constituer une méthode supérieure pour le suivi des cas, lorsqu'un tel système est disponible.

Use of the KT-MCC strategy to improve the quality of decision making for multidisciplinary cancer conferences: a pilot study.

BACKGROUND: Multidisciplinary Cancer Conferences (MCCs) are increasingly used to guide treatment decisions for patients with cancer, though numerous barriers to optimal MCC decision-making quality have been identified. We aimed to improve the quality of MCC decision making through the use of an implementation bundle titled the KT-MCC Strategy. The Strategy included use of discussion tools (standard case intake tool and a synoptic discussion tool), workshops, MCC team and chair training, and audit and feedback. Implementation strategies were selected using a theoretically-rooted and integrated KT approach, meaning members of the target population (MCC participants) assisted with the design and implementation of the intervention and strategies. We evaluated implementation quality of the KT-MCC Strategy and initial signals of impact on decision making quality. METHODS: This was a before-and-after study design among 4 MCC teams. Baseline data (before-phase) were collected for a period of 2 months to assess the quality of MCC decision making. Study teams selected the intervention strategies they wished to engage with. Post-intervention data (after-phase) were collected for 4 months. Implementation quality outcomes included reach, adherence/fidelity and adaptation. We also evaluated feasibility of data management. Decision making quality was evaluated on a per-case and per-round level using the MTB-MODe and MDT-OARS tools, respectively. RESULTS: There were a total of 149 cases and 23 MCCs observed in the before phase and 260 cases and 35 MCCs observed in the after phase. Teams implemented 3/5 strategies; adherence to selected strategies varied by MCC team. The per-round quality of MCCs improved by 11% (41.0 to 47.3, p = < 0.0001). The quality of per-case decision-making did not improve significantly (32.3 to 32.6, p = 0.781). CONCLUSION: While per round MCC decision making quality improved significantly, per-case decision-making quality did not. We posit that the limited improvements on decision making quality may be attributed to implementation quality gaps, including a lack of uptake of and adherence to theoretically-identified implementation strategies. Our findings highlight the importance of evaluating implementation quality and processes, iterative testing, and engagement of key gatekeepers in the implementation process.

Use of the theoretical domains framework and behaviour change wheel to develop a novel intervention to improve the quality of multidisciplinary cancer conference decision-making.

BACKGROUND: Multidisciplinary Cancer Conferences (MCCs) are prospective meetings involving cancer specialists to discuss treatment plans for patients with cancer. Despite reported gaps in MCC quality, there have been few efforts to improve its functioning. The purpose of this study was to use theoretically-rooted knowledge translation (KT) theories and frameworks to inform the development of a strategy to improve MCC decision-making quality. METHODS: A multi-phased approach was used to design an intervention titled the KT-MCC Strategy. First, key informant interviews framed using the Theoretical Domains Framework (TDF) were conducted with MCC participants to identify barriers and facilitators to optimal MCC decision-making. Second, identified TDF domains were mapped to corresponding strategies using the COM-B Behavior Change Wheel to develop the KT-MCC Strategy. Finally, focus groups with MCC participants were held to confirm acceptability of the proposed KT-MCC Strategy. RESULTS: Data saturation was reached at n = 21 interviews. Twenty-seven barrier themes and 13 facilitator themes were ascribed to 11 and 10 TDF domains, respectively. Differences in reported barriers by physician specialty were observed. The resulting KT-MCC Strategy included workshops, chair training, team training, standardized intake forms and a synoptic discussion checklist, and, audit and feedback. Focus groups (n = 3, participants 18) confirmed the acceptability of the identified interventions. CONCLUSION: Myriad factors were found to influence MCC decision making. We present a novel application of the TDF and COM-B to the context of MCCs. We comprehensively describe the barriers and facilitators that impact MCC decision making and propose strategies that may positively impact the quality of MCC decision making.

Knowledge about the Developmental Origins of Health and Disease is independently associated with variation in diet quality during pregnancy.

Environmental factors affecting development through embryogenesis, pregnancy, and infancy impact health through all subsequent stages of life. Known as the Developmental Origins of Health and Disease (DOHaD) hypothesis, this concept is widely accepted among health and social scientists. However, it is unclear whether DOHaD-based ideas are reaching the general public and/or influencing behaviour. This study thus investigated whether and under what circumstances pregnant people in Canada are familiar with DOHaD, and if DOHaD familiarity relates to eating behaviour. Survey responses from pregnant people from Hamilton, Canada, were used to assess respondents' knowledge of DOHaD (hereafter, DOHaDKNOWLEDGE ) compared with their knowledge of more general pregnancy health recommendations (Pregnancy GuidelineKNOWLEDGE ). The survey also characterized respondents' pregnancy diet quality and sociodemographic profiles. We fit two multiple, linear, mixed regression models to the data, one with DOHaDKNOWLEDGE score as the dependent variable and the other with diet quality score as the dependent. In both models, responses were clustered by respondents' neighbourhoods. Complete, internally consistent responses were available for 330 study-eligible respondents. Relative to Pregnancy GuidelineKNOWLEDGE , respondents had lower, more variable DOHaDKNOWLEDGE scores. Additionally, higher DOHaDKNOWLEDGE was associated with higher socio-economic position, older age, and lower parity, independent of Pregnancy GuidelineKNOWLEDGE . Diet quality during pregnancy was positively associated with DOHaDKNOWLEDGE , adjusting for sociodemographic factors. A subset of relatively high socio-economic position respondents was familiar with DOHaD. Greater familiarity with DOHaD was associated with better pregnancy diet quality, hinting that translating DOHaD knowledge to pregnant people may motivate improved pregnancy nutrition and thus later-life health for developing babies.

Patient Narratives as a Teaching Tool: A Pilot Study of First-Year Medical Students and Patient Educators Affected by Intellectual/Developmental Disabilities.

PROBLEM: People with intellectual and developmental disabilities (IDD) face complex biopsychosocial challenges and are medically underserved. This is in part due to insufficient resources and supports but can also be attributed to a lack of adequate physician training in addressing the unique needs of this population. INTERVENTION: This study aimed to introduce 1st-year medical students to the IDD population using a blended educational experience that included video narratives of and direct interactions with people affected by IDD. The goal of this intervention was to promote person-centered attitudes and communication among early medical trainees. CONTEXT: The study recruited 27 first-year medical students and randomly assigned each to 1 of 2 groups. The control group received an introductory video lecture about IDD healthcare, followed by a quiz. The narrative group received the same lecture, followed by reflective discussion of videos featuring people living with IDD sharing their perspectives and stories. All students then participated in 4 simulated clinical encounters with patient educators (PEs) who have lived experiences of IDD. Focus groups were conducted with students following the simulated encounters to explore their experiences and perceptions of this blended learning activity. Moreover, secondary quantitative data were collected to assess students' performance in the clinical encounters, along with self-reports of comfort, confidence, and competence of interacting with people with IDD (pre- and postparticipation). OUTCOME: All students thought that the blended educational experience was valuable and enjoyable, commenting on the importance of adaptable language and engagement of people with IDD, as well as the merits of reflecting on patient narratives. Students also discussed feelings of discomfort stemming from a lack of knowledge and previous exposure to IDD and how this discomfort might motivate them to learn more and develop their skills further. In addition, descriptive analyses revealed that students in the narrative group showed greater self-rated measures of comfort, confidence, and competence compared to control; they also had higher mean performance scores across all PE interview stations. LESSONS LEARNED: PEs add a powerful real-life dimension to communication skills teaching and have been shown to be a valuable educational modality. Moreover, exposure to and reflection on video-based patient narratives are useful ways of teaching medical students about patients' lived experiences and promoting person-centered communication, both within and beyond IDD.

Developing Mentoring Competency: Does a One Session Training Workshop Have Impact?

OBJECTIVE: Mentorship remains vital to the career development, research productivity, and professional advancement of healthcare professionals in all disciplines of academic medicine. Recent studies describe mentor training initiatives aimed at increasing mentoring competency through multisession training curricula. Although the published results of these programs are promising, they require the following: (1) substantial financial resources from the institution, and (2) continuous participation and time commitment from faculty, which may reduce participation and effectiveness. METHOD: A single, half-day of evidence-based mentor training would represent a more cost-effective and accessible option for educating mentors. The present study investigates the impact of a half-day interactive mentor training workshop on mentoring competency in faculty, staff, and trainees of the Department of Psychiatry and Behavioural Neurosciences at McMaster University. RESULTS: Overall, participants' self-reported mentoring competency mean scores were significantly higher post-workshop compared to pre-workshop ratings [mean = 4.48 vs. 5.02 pre- and post-workshop, respectively; F(1, 31) = 18.386, P < 0.001, η p2 = 0.37]. Survey respondents gave positive feedback and reported greater understanding of mentorship and specific mentoring changes they planned to apply after attending the workshop. CONCLUSION: Academic and healthcare institutions may use this framework to guide the development of a half-day mentoring workshop into their education programs.

The benefits of testing for learning on later performance.

Testing has been shown to enhance retention of learned information beyond simple studying, a phenomena known as test-enhanced learning (TEL). Research has shown that TEL effects are greater for tests that require the production of responses [e.g., short-answer questions (SAQs)] relative to tests that require the recognition of correct answers [e.g., multiple-choice questions (MCQs)]. High stakes licensure examinations have recently differentiated MCQs that require the application of clinical knowledge (context-rich MCQs) from MCQs that rely on the recognition of "facts" (context-free MCQs). The present study investigated the influence of different types of educational activities (including studying, SAQs, context-rich MCQs and context-free MCQs) on later performance on a mock licensure examination. Fourth-year medical students (n = 224) from four Quebec universities completed four educational activities: one reading-based activity and three quiz-based activities (SAQs, context-rich MCQs, and context-free MCQs). We assessed the influence of the type of educational activity on students' subsequent performance in a mock licensure examination, which consisted of two types of context-rich MCQs: (1) verbatim replications of previous items and (2) items that tested the same learning objective but were new. Mean accuracy scores on the mock licensure exam were higher when intervening educational activities contained either context-rich MCQs (Mean z-score = 0.40) or SAQs (M = 0.39) compared to context-free MCQs (M = -0.38) or study only items (M = -0.42; all p < 0.001). Higher mean scores were only present for verbatim items (p < 0.001). The benefit of testing was observed when intervening educational activities required either the generation of a response (SAQs) or the application of knowledge (context-rich MCQs); however, this effect was only observed for verbatim test items. These data provide evidence that context-rich MCQs and SAQs enhance learning through testing compared to context-free MCQs or studying alone. The extent to which these findings generalize beyond verbatim questions remains to be seen.

Predictable chaos: a review of the effects of emotions on attention, memory and decision making.

Healthcare practice and education are highly emotional endeavors. While this is recognized by educators and researchers seeking to develop interventions aimed at improving wellness in health professionals and at providing them with skills to deal with emotional interpersonal situations, the field of health professions education has largely ignored the role that emotions play on cognitive processes. The purpose of this review is to provide an introduction to the broader field of emotions, with the goal of better understanding the integral relationship between emotions and cognitive processes. Individuals, at any given time, are in an emotional state. This emotional state influences how they perceive the world around them, what they recall from it, as well as the decisions they make. Rather than treating emotions as undesirable forces that wreak havoc on the rational being, the field of health professions education could be enriched by a greater understanding of how these emotions can shape cognitive processes in increasingly predictable ways.

Evaluating the Construct Validity of Competencies: A Retrospective Analysis.

Background: A competency-based framework focuses on alignment between professional standards and assessment design. This alignment implies improved measurement validity, yet it has not been established that competence in one context predicts performance in another context. High-stakes competence assessments offer insights into the relationship between assessment design and competencies. Methods/Analyses: The internationally educated nurses competency assessment program (IENCAP) was developed at Touchstone Institute in collaboration with the College of Nurses of Ontario (CNO) and includes a 12-station OSCE. Each station evaluated the same 10 competencies. We submitted competency scores to a multi-trait multi-method matrix analysis to evaluate the convergent and discriminant validity of competencies. Results/Observations: All correlations were significant and positive; however, we did not find evidence of convergent or discriminant validity. Correlations were higher between different competencies evaluated within the same station (mean correlation = 0.60) compared to identical competencies evaluated across different stations (mean correlation = 0.19). Discussion: The results do not provide evidence of construct validity for competencies. While competency-based approaches emphasize various generalized knowledge, skills, and attitudes, these findings indicate that the clinical context is a major determinant of performance. Conclusion: The context-dependent nature of competencies requires multiple assessments in varied contexts. Performance on a single competency cannot be determined in a single occasion. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-023-01794-z.

Is COVID-19 perceived as a threat to equal career opportunities amongst Swiss medical students? A cross-sectional survey study from Bern and Geneva.

Introduction: Students frequently rely on part-time jobs to earn a living wage. We sought to evaluate the sociodemographic status of Swiss medical students and their perception regarding equal career opportunities in view of impaired part-time job opportunities under the COVID-19 pandemic. Methods: We conducted an anonymous online survey among Swiss medical students from Bern and Geneva over a period of 4 months between December 2020 and April 2021. We evaluated sociodemographic data, current living situation, part-time job occupation as well as other sources of income to fund living expenses, and, by means of a five-point Likert scale (1=strongly disagree and 5=strongly agree), whether COVID-19 was perceived as impeding equal career opportunities. Results: Of 968 participants, corresponding to around 13.8% of all medical students in Switzerland, 81.3% had part-time jobs. Amongst the employed, 54.8% worked to afford living expenses and 28.9% reported a negative financial impact due to reduced part-time jobs under the pandemic. The loss of part-time jobs was perceived to make medical studies a privilege for students with higher socioeconomic status (4.11±1.0), whose opportunity to study is independent of a regular income. A governmental backup plan was considered crucial to support affected students (4.22±0.91). Discussion: COVID-19 and its sequelae are perceived as a threat for Swiss medical students and lead to a disadvantage for those with lower socioeconomic status. Nationwide measures should be established to foster equal career opportunities.

Exploring gender influences in the quality of workplace-based assessments.

INTRODUCTION: Workplace-based assessments are an important tool for trainee feedback and as a means of reporting expert judgments of trainee competence in the workplace. However, the literature has demonstrated that gender bias can exist within these assessments. We aimed to determine whether gender differences in the quality of workplace-based assessment data exist in our residency training program. METHODS: This study was conducted at the University of Ottawa in the Department of Emergency Medicine. Four end-of-shift workplace-based assessments completed by men faculty and four completed by women faculty were randomly selected for each resident during the 2018-2019 academic year. Two blinded raters scored each workplace-based assessment using the Completed Clinical Evaluation Report Rating (CCERR), a published nine-item quantitative measure of workplace-based assessment quality. A 2 × 2 mixed measures analysis of variance (ANOVA) of resident gender and faculty gender was conducted, with mean CCERR score as the dependent variable. The ANOVA was repeated with mean workplace-based assessment rating as the dependent variable. RESULTS: A total of 363 workplace-based assessments were analyzed for 46 residents. There were no significant effects of faculty or resident gender on the quality of workplace-based assessments (p = 0.30). There was no difference in mean workplace-based assessment ratings between women and men residents (p = 0.92), and no interaction between resident and faculty gender (p = 0.62). Mean CCERR score was 25.8, SD = 4.2, indicating average quality assessments. CONCLUSIONS: We did not find faculty or resident gender differences in the quality of workplace-based assessments completed in our training program. While the literature has previously demonstrated gender bias in trainee assessments, our results are not surprising as assessment culture varies by institution and program. Our study cautions against generalizing gender bias across contexts, and offers an approach that educators can use to evaluate whether gender bias in the quality of trainee assessments exists within their program.

RéSUMé: INTRODUCTION: Les évaluations sur le lieu de travail constituent un outil important pour le retour d'information des stagiaires et comme moyen de rapporter les jugements d'experts sur les compétences des stagiaires sur le lieu de travail. Cependant, la littérature a démontré que des préjugés sexistes peuvent exister dans ces évaluations. Nous avons cherché à déterminer s'il existe des différences entre les sexes dans la qualité des données d'évaluation sur le lieu de travail dans notre programme de formation en résidence. MéTHODES: Cette étude a été menée à l'Université d'Ottawa dans le département de médecine d'urgence. Quatre évaluations en fin de poste de travail complétées par des professeurs hommes et 4 complétées par des professeurs femmes ont été sélectionnées au hasard pour chaque résident au cours de l'année universitaire 2018-2019. Deux évaluateurs en aveugle ont noté chaque évaluation sur le lieu de travail à l'aide du Completed Clinical Evaluation Report Rating (CCERR), une mesure quantitative publiée en neuf points de la qualité de l'évaluation sur le lieu de travail. Une analyse de variance (ANOVA) à mesures mixtes 2 × 2 du sexe des résidents et du sexe des enseignants a été réalisée, avec le score CCERR moyen comme variable dépendante. L'ANOVA a été répétée en prenant comme variable dépendante la note moyenne de l'évaluation sur le lieu de travail. RéSULTATS: Au total, 363 évaluations sur le lieu de travail ont été analysées pour 46 résidents. Il n'y avait aucun effet significatif du sexe du corps professoral ou du résident sur la qualité des évaluations en milieu de travail (p = 0,30). Il n'y avait pas de différence dans les évaluations moyennes sur le lieu de travail entre les femmes et les hommes résidents (p = 0,92), et pas d'interaction entre le sexe du résident et celui de la faculté (p = 0,62). Le score moyen du CCERR était de 25,8, SD = 4,2, ce qui indique des évaluations de qualité moyenne. CONCLUSIONS: Nous n'avons pas constaté de différences entre les sexes au sein du corps professoral ou des résidents en ce qui concerne la qualité des évaluations en milieu de travail effectuées dans le cadre de notre programme de formation. Bien que la littérature ait déjà démontré l'existence de préjugés sexistes dans les évaluations des stagiaires, nos résultats ne sont pas surprenants car la culture de l'évaluation varie selon les établissements et les programmes. Notre étude met en garde contre la généralisation des préjugés sexistes dans tous les contextes et propose une approche que les éducateurs peuvent utiliser pour évaluer s'il existe des préjugés sexistes dans la qualité des évaluations des stagiaires au sein de leur programme.

Assessment Practices in Continuing Professional Development Activities in Health Professions: A Scoping Review.

CONTEXT: In continuing professional development (CPD), educators face the need to develop and implement innovative assessment strategies to adhere to accreditation standards and support lifelong learning. However, little is known about the development and validation of these assessment practices. We aimed to document the breadth and depth of what is known about the development and implementation of assessment practices within CPD activities. METHODS: We conducted a scoping review using the framework proposed by Arksey and O'Malley (2005) and updated in 2020. We examined five databases and identified 1733 abstracts. Two team members screened titles and abstracts for inclusion/exclusion. After data extraction, we conducted a descriptive analysis of quantitative data and a thematic analysis of qualitative data. RESULTS: A total of 130 studies were retained for the full review. Most reported assessments are written assessments (n = 100), such as multiple-choice items (n = 79). In 99 studies, authors developed an assessment for research purpose rather than for the CPD activity itself. The assessment validation process was detailed in 105 articles. In most cases, the authors examined the content with experts (n = 57) or pilot-tested the assessment (n = 50). We identified three themes: 1-satisfaction with assessment choices; 2-difficulties experienced during the administration of the assessment; and 3-complexity of the validation process. CONCLUSION: Building on the adage "assessment drives learning," it is imperative that the CPD practices contribute to the intended learning and limit the unintended negative consequences of assessment. Our results suggest that validation processes must be considered and adapted within CPD contexts.