Understanding Emotions Impacted by New Assessment Mandates Implemented in Medical Education: A Survey of Residents and Faculty Across Multiple Specialties.

Background Previous research findings show that the overall perception of residents regarding the new entrustable professional activity (EPA) assessment mandates is primarily negative. Hence, this study aims to explore the link between EPA assessment experiences and resident and faculty emotions and expectancy of successfully completing residency training. Methods A standardized questionnaire (Medical Emotions Scale (MES)), which measures 20 unique emotions on a 5-point Likert scale, was used to explore the emotions of residents and faculty members regarding EPA assessments and residents' expectancy of success. Data analysis included descriptive statistics and analysis of variance (ANOVA). Results Ninety-one (N=91) participants (46 faculty members and 45 residents) completed the survey. The results revealed that residents have more negative emotions toward EPA assessments compared to faculty. Additionally, resident and faculty emotions regarding EPA assessments vary across specialty and gender. Conclusions These findings will be crucial in providing the Royal College of Physicians and Surgeons of Canada and medical education programs with concrete evidence and guidance in understanding the perspectives and emotions of residents and faculty towards EPA assessments and residents' beliefs about successfully completing their medical training.

Performance of generative pre-trained transformers (GPTs) in Certification Examination of the College of Family Physicians of Canada.

INTRODUCTION: The application of large language models such as generative pre-trained transformers (GPTs) has been promising in medical education, and its performance has been tested for different medical exams. This study aims to assess the performance of GPTs in responding to a set of sample questions of short-answer management problems (SAMPs) from the certification exam of the College of Family Physicians of Canada (CFPC). METHOD: Between August 8th and 25th, 2023, we used GPT-3.5 and GPT-4 in five rounds to answer a sample of 77 SAMPs questions from the CFPC website. Two independent certified family physician reviewers scored AI-generated responses twice: first, according to the CFPC answer key (ie, CFPC score), and second, based on their knowledge and other references (ie, Reviews' score). An ordinal logistic generalised estimating equations (GEE) model was applied to analyse repeated measures across the five rounds. RESULT: According to the CFPC answer key, 607 (73.6%) lines of answers by GPT-3.5 and 691 (81%) by GPT-4 were deemed accurate. Reviewer's scoring suggested that about 84% of the lines of answers provided by GPT-3.5 and 93% of GPT-4 were correct. The GEE analysis confirmed that over five rounds, the likelihood of achieving a higher CFPC Score Percentage for GPT-4 was 2.31 times more than GPT-3.5 (OR: 2.31; 95% CI: 1.53 to 3.47; p<0.001). Similarly, the Reviewers' Score percentage for responses provided by GPT-4 over 5 rounds were 2.23 times more likely to exceed those of GPT-3.5 (OR: 2.23; 95% CI: 1.22 to 4.06; p=0.009). Running the GPTs after a one week interval, regeneration of the prompt or using or not using the prompt did not significantly change the CFPC score percentage. CONCLUSION: In our study, we used GPT-3.5 and GPT-4 to answer complex, open-ended sample questions of the CFPC exam and showed that more than 70% of the answers were accurate, and GPT-4 outperformed GPT-3.5 in responding to the questions. Large language models such as GPTs seem promising for assisting candidates of the CFPC exam by providing potential answers. However, their use for family medicine education and exam preparation needs further studies.

Clinical teaching self-efficacy positively predicts professional fulfillment and negatively predicts burnout amongst Thai physicians: a cross-sectional survey.

BACKGROUND: Clinician teachers (physicians who teach in clinical settings) experience considerable psychological challenges in providing both educational training and patient care. This study aimed to determine the prevalence of physician burnout and professional fulfillment, and to identify internal and external factors associated with mental health outcomes among Thai clinician teachers working in non-university teaching hospitals. METHOD: A one-time online questionnaire was completed by physicians at 37 governmental, non-university teaching hospitals in Thailand, with 227 respondents being assessed in the main analyses. Four outcomes were evaluated including burnout, professional fulfillment, quality of life, and intentions to quit. RESULTS: The observed prevalence of professional fulfillment was 20%, and burnout was 30.7%. Hierarchical regression analysis showed a significant internal, psychological predictor (clinical teaching self-efficacy) and external, structural predictors (multiple roles at work, teaching support), controlling for the background variables of gender, years of teaching experience, family roles, and active chronic disease, with clinical teaching self-efficacy positively predicting professional fulfillment (b = 0.29, p ≤.001) and negatively predicting burnout (b = - 0.21, p =.003). CONCLUSIONS: Results highlight the importance of faculty development initiatives to enhance clinical teaching self-efficacy and promote mental health among Thai physicians.

Findings Favor Haptics Feedback in Virtual Simulation Surgical Education: An Updated Systematic and Scoping Review.

BACKGROUND: Virtual simulations (VSs) enhance clinical competencies and skills. However, a previous systematic review of 9 RCT studies highlighted a paucity of literature on the effects of haptic feedback in surgical VSs. An updated systematic and scoping review was conducted to encompass more studies and a broader range of study methodologies. METHODS: A systematic literature search was conducted on July 31, 2023, in MEDLINE, Embase, and Cochrane. English language studies comparing haptic vs non-haptic conditions and using VSs were included. Studies were evaluated and reported using PRISMA-ScR guidelines. RESULTS: Out of 2782 initial studies, 51 were included in the review. Most studies used RCT (21) or crossover (23) methodologies with medical residents, students, and attending physicians. Most used post-intervention metrics, while some used pre- and post-intervention metrics. Overall, 34 performance results from studies favored haptics, 3 favored non-haptics, and the rest showed mixed or equal results. CONCLUSION: This updated review highlights the diverse application of haptic technology in surgical VSs. Haptics generally enhances performance, complements traditional teaching methods, and offers personalized learning with adequate simulator validation. However, a sparsity of orienting to the simulator, pre-/post-study designs, and small sample sizes poses concerns with the validity of the results. We underscore the urgent need for standardized protocols, large-scale studies, and nuanced understanding of haptic feedback integration. We also accentuate the significance of simulator validation, personalized learning potential, and the need for researcher, educator, and manufacturer collaboration. This review is a guidepost for navigating the complexities and advancements in haptic-enhanced surgical VSs.

Improving Pediatric Trauma Education by Teaching Non-technical Skills: A Randomized Controlled Trial.

BACKGROUND: Pediatric trauma is a significant cause of child mortality, and the absence of non-technical skills (NTS) among health providers is linked with errors in patients' care. In this study, we evaluate the effectiveness of a structured debriefing protocol in enhancing NTS during pediatric trauma simulation. METHODS: A total of 45 medical students were successfully recruited from two medical schools, one in Brazil and one in Canada. Medical students were assigned to a control (N = 20) or intervention group (N = 25) in a randomized control trial. Following simulated scenarios, participants in the intervention group underwent NTS debriefing, while the control received standard debriefing based on the Advanced Trauma Life Support (ATLS) protocol. Students' confidence, NTS level, and performance were measured through self-assessment surveys, the Non-Technical Skills for Surgeons (NOTSS) score, and adherence to the trauma protocol, respectively. Baseline characteristics and outcomes were compared using t-tests, Mann-Whitney, Wilcoxon signed-rank Kruskal-Wallis, ANOVA, and a repeated-measures ANCOVA. A significance level was set at p < 0.05. RESULTS: The workshop increased students' confidence in leading trauma resuscitation regardless of their assignment to condition. While controlling for covariates, students in the intervention group significantly improved their overall NOTSS compared to those in the control and in all categories: situational awareness, decision-making, communication and teamwork, and leadership. The intervention teams also demonstrated a significant increase in completing trauma protocol steps. CONCLUSION: Implementing structured debriefing focusing on NTS enhanced these skills and improved adherence to protocol among medical students managing pediatric trauma-simulated scenarios. These findings support integrating NTS training in pediatric trauma education. LEVEL OF EVIDENCE: I.