Internal Medicine Acting Internship Trends in Rotation Structure and Student Responsibilities: Results from a 2023 National Survey.

BACKGROUND: The acting internship (AI) in internal medicine plays a key role in the transition from medical school to residency. While there have been recent changes in medical education including a pass/fail USMLE Step 1 and increasing use of competency-based assessment, there has not been a large survey of the state of the AI in many years. OBJECTIVE: To assess the current landscape of the internal medicine AI and identify areas in need of standardization. DESIGN: This was a voluntary online survey of medical schools in the United States (U.S.). PARTICIPANTS: Course directors of the AI rotation at U.S. medical schools. MAIN MEASURES: Number of AI rotations required for graduation, length of AI rotation, types of services allowed for AI, clinical responsibilities of students, curricular components. KEY RESULTS: Response rate was 50.7% (71/140 LCME accredited schools). All responding institutions require at least one AI for graduation, with nearly all schools integrating students into resident teaching teams, and almost half also allowing AI students to work on hospitalist services. Students carry 3-4 patients per day on average with a maximum of 5-6 in most institutions. Students are responsible for most aspects of patient care including notes, orders, interprofessional communication, and transitions of care. Night call or night float responsibilities are infrequently required. The structured curriculum published by AAIM is used by only 41% of schools. CONCLUSIONS: The internal medicine AI continues to be a staple in the medical school experience, but there is variation in the structure, curriculum, and expectations on the rotation. Opportunities exist to improve standardization of the AI experience and expectations to better prepare medical students for the transition from medical school to residency.

Learning to Lead: An Exploration of Leadership Development in Obstetrics and Gynaecology.

Despite the demonstrated value of leadership in medicine, there is limited research on how physicians develop leadership skills. We administered a national survey (225 respondents) and conducted eight interviews of residents, fellows, and staff physicians in obstetrics and gynaecology to explore leadership skill development. Most (87%) positively rated the effectiveness of their leadership skills; however, the majority (98%) stated they would benefit from further training. Interview themes indicated that leadership skills can be taught, leadership training and roles should be intentional, training should be longitudinal, and barriers to training can be overcome with changes to the current model.

Utilizing natural language processing to analyze student narrative reflections for medical curriculum improvement.

MOTIVATION: Medical curricula improvement is an ongoing process to keep material relevant and improve the student's learning experience to better prepare them for patient care. Many programs utilize end-of-year evaluations, but these frequently have low response rates and lack actionable feedback. We hypothesized that student reflections written during a fourth year Sub-Internship could be used retrospectively to mine additional information as feedback for future curriculum adjustments. However, reflections contain a large amount of narrative content that would require a cumbersome and essentially infeasible manual review process for busy medical education faculty. METHODS: We developed a Natural Language Processing (NLP) pipeline to automatically identify common themes and topics present in the set of reflective writings that could be used to improve the curriculum. The dataset contains required responses to a faculty issued question submitted between August 2016 and July 2018 about challenges experienced during the medical students fourth year Sub-Internship. RESULTS: Eleven distinct topics were identified, with several being subsequently addressed in future iterations of the curriculum. CONCLUSION: Utilizing NLP on reflective writings was able to identify areas of curriculum improvement, and the NLP results provided a quick and easy way to explore the main themes and challenges expressed by students.

The Ottawa resident observation form for nurses (O-RON): evaluation of an assessment tool's psychometric properties in different specialties.

BACKGROUND: Workplace-based assessment (WBA) used in post-graduate medical education relies on physician supervisors' feedback. However, in a training environment where supervisors are unavailable to assess certain aspects of a resident's performance, nurses are well-positioned to do so. The Ottawa Resident Observation Form for Nurses (O-RON) was developed to capture nurses' assessment of trainee performance and results have demonstrated strong evidence for validity in Orthopedic Surgery. However, different clinical settings may impact a tool's performance. This project studied the use of the O-RON in three different specialties at the University of Ottawa. METHODS: O-RON forms were distributed on Internal Medicine, General Surgery, and Obstetrical wards at the University of Ottawa over nine months. Validity evidence related to quantitative data was collected. Exit interviews with nurse managers were performed and content was thematically analyzed. RESULTS: 179 O-RONs were completed on 30 residents. With four forms per resident, the ORON's reliability was 0.82. Global judgement response and frequency of concerns was correlated (r = 0.627, P < 0.001). CONCLUSIONS: Consistent with the original study, the findings demonstrated strong evidence for validity. However, the number of forms collected was less than expected. Exit interviews identified factors impacting form completion, which included clinical workloads and interprofessional dynamics.

Consensus Statement No. 434: Simulation in Obstetrics and Gynaecology.

OBJECTIVE: To provide a comprehensive and current overview of the evidence for the value of simulation for education, team training, patient safety, and quality improvement in obstetrics and gynaecology, to familiarize readers with principles to consider in developing a simulation program, and to provide tools and references for simulation advocates. TARGET POPULATION: Providers working to improve health care for Canadian women and their families; patients and their families. OUTCOMES: Simulation has been validated in the literature as contributing to positive outcomes in achieving learning objectives, maintaining individual and team competence, and enhancing patient safety. Simulation is a well-developed modality with established principles to maximize its utility and create a safe environment for simulation participants. Simulation is most effective when it involves interprofessional collaboration, institutional support, and regular repetition. BENEFITS, HARMS, AND COSTS: This modality improves teamwork skills, patient outcomes, and health care spending. Upholding prescribed principles of psychological safety when implementing a simulation program minimizes harm to participants. However, simulation can be an expensive tool requiring human resources, equipment, and time. EVIDENCE: Articles published between 2003 and 2022 were retrieved through searches of Medline and PubMed using the keywords "simulation" and "simulator." The search was limited to articles published in English and French. The articles were reviewed for their quality, relevance, and value by the SOGC Simulation Working Group. Expert opinion from relevant seminal books was also considered. VALIDATION METHODS: The authors rated the quality of evidence and strength of recommendations using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. See online Appendix A (Tables A1 for definitions and A2 for interpretations of strong and conditional [weak] recommendations). INTENDED AUDIENCE: All health care professionals working to improve Canadian women's health, and relevant stakeholders, including granting agencies, physician/nursing/midwifery colleges, accreditation bodies, academic centres, hospitals, and training programs.

Clinical predictors and outcomes of patients with left ventricular thrombus following ST-segment elevation myocardial infarction.

We aimed to characterize the independent predictors of LVT following STEMI and the association with outcomes. The clinical predictors of left ventricular thrombus (LVT) formation after ST-segment elevation myocardial infarction (STEMI) are not well-defined in the contemporary era. We performed a retrospective analysis of STEMI patients at Duke from 2000 to 2011 who had a transthoracic echocardiogram within 90 days post-STEMI and compared patients with and without LVT (LVT+ vs. LVT-). Univariate Cox proportional hazards regression models of baseline characteristics were examined and significant variables were used in a multivariable model to assess adjusted relationships with LVT. A multivariable Cox PH survival model with covariate adjustments was used for assessment of LVT and long-term mortality. Of all eligible patients, 1734 patients met inclusion criteria and 4.3 % (N = 74) had a LVT. LVT+ patients tended to have a history of heart failure (HF) and higher initial troponin compared to LVT- patients. After adjustment, higher heart rate, non-white race, HF severity, and presence of left anterior descending artery (LAD) disease were independent predictors of LVT. There was a trend toward an association between LVT and increased all-cause mortality (HR 1.36; 95 % CI 0.84-2.21, P = 0.22), however this was not statistically significant. LVT was seen in over 4 % of this contemporary post-STEMI population. Several baseline characteristics were independently associated with LVT: Heart rate, HF severity, LAD disease, and non-white race. Prospective studies are warranted to determine whether anticoagulation in patients at increased risk for LVT improves outcomes.

Using Mixed Reality to Teach Medical Students Lumbar Punctures.

BACKGROUND: Gaining familiarity with bedside procedures during medical school is essential for preparing medical students for intern year. However, this is often not a core component of undergraduate medical education, and students often feel unprepared to perform bedside procedures upon graduation from medical school. Extended reality technologies have rapidly developed in the past few years, become more accessible, and show great promise in education, most recently in healthcare. We aim to show the feasibility and usability of a mixed-reality lumbar puncture simulated experience. METHODS: We created the mixed-reality simulation by filming a procedure conducted by an expert on a lumbar puncture simulator from the first-person view. We performed a study to assess the feasibility and usability of mixed-reality lumbar puncture training compared with traditional faculty-led high-fidelity mannikin-based simulation training. Fourth-year medical students were randomly selected to participate in the mixed-reality lumbar puncture training. Students watched the virtual first-person teaching via a virtual reality headset while mimicking hand movements on a lumbar puncture mannikin. Faculty taught students in the traditional high-fidelity group in small groups before practicing on the high-fidelity manikin. Students in both groups then performed the procedure entirely with performance assessed via a standardized checklist. Number of attempts to achieve a predetermined minimum passing score were recorded, and students were also surveyed regarding their experience with both simulations. RESULTS: Fifty-nine percent of students felt comfortable using mixed reality as a training modality with less than 6% having any prior extended reality experience. More than 55% felt the mixed-reality experience was as effective as the traditional high-fidelity simulation training. Many students (72%) desired more mixed-reality opportunities. In terms of performance, 58.8% of students in the mixed-reality group vs 65.5% in the traditional group completed the checklist in 1 attempt (P = .79). CONCLUSIONS: We demonstrate that mixed reality is an acceptable and effective modality to train health care professionals in lumbar punctures with opportunities to further enhance the immersive learning experience.

Using Virtual Reality to Teach Medical Students Cross-Coverage Skills.

BACKGROUND: Recognizing and attempting management of patients with urgent or emergent conditions is one of the Association of American Medical Colleges Core Entrustable Professional Activities (#10) and a skill desired of new interns. However, given the acuity of these patient conditions, medical students often struggle to gain experience in these situations. Virtual reality could help fill this void while lowering costs and resources compared with high-fidelity simulation. METHODS: We converted a high-fidelity chest pain simulation case to virtual reality format utilizing short video clips filmed with a 360-degree camera and superimposed menus of options at decision points. This virtual reality simulation was offered to fourth-year medical students during their transition to residency course in the spring of 2023. Students were offered a post-survey on the simulation. RESULTS: There were 47 fourth-year students that completed the virtual reality simulation; 41 completed the post-survey (response rate 87.2%). Over 90% of the students agreed or strongly agreed with the following statements: the virtual reality simulation was a valuable part of the transition to residency course, the virtual reality case was similar to what they will face as an intern, and they would like to have more virtual reality simulations earlier in the fourth year; 85.4% agreed or strongly agreed that the virtual reality simulation helped prepare them for the first few days of intern year. CONCLUSIONS: We demonstrated that virtual reality is an acceptable, cost-effective, and feasible modality to teach medical students how to recognize and attempt management of urgent clinical situations (Core Entrustable Professional Activity 10).

Competency-based assessment of procedural skills in resident training.

BACKGROUND: No standardized summative tools exist to assess competency in bedside procedures or provide residents and programs with summative feedback. OBJECTIVE: To provide competency-based procedure training and feedback to residents, we created a procedure competency committee (PCC). Here, we describe the PCC process, its impact on procedure training, and examine residents' attainment of competency in bedside procedures. DESIGN, SETTING, AND PARTICIPANTS: The PCC consisted of hospitalists and met twice annually to review resident procedure portfolios for three academic years 2019-2022 at a university-based internal medicine residency program. Residents were designated to one of the five competency levels; being able to participate, perform under supervision with assistance, perform under direct supervision (DS) without assistance, perform independently with indirect supervision (IS), and perform independently with IS and supervise others. We analyzed the probability of advancing competency levels with each additional procedure using multinomial logistic regression models. RESULTS: Of the 97 residents, 48 (49.5%) were women and 60 (62%) subsequently matched in procedure-oriented fields. More residents achieved IS level for paracentesis than for lumbar puncture (LP) or central venous catheterization (CVC) (62 vs. 25 and 37, respectively; p < .001). Each incremental procedure performed was associated with a higher chance of being advanced to IS group from the DS group; 1.31 times for paracentesis (95% confidence interval [CI] = 1.07, 1.60; p < .008), 1.83 times for LP (95% CI = 1.35, 2.47; p = .0001), and two times for CVC (95% CI = 1.32, 3.05; p = .001). CONCLUSION: The PCC provided competency-based assessment of a resident's procedural skills and may be used to assess the impact of curriculum changes.

Ten frequently asked questions about latent transition analysis.

Latent transition analysis (LTA), also referred to as latent Markov modeling, is an extension of latent class/profile analysis (LCA/LPA) used to model the interrelations of multiple latent class variables. LTA methods have become increasingly accessible and in-turn are being utilized in applied research. The current article provides an introduction to LTA by answering 10 questions commonly asked by applied researchers. Topics discussed include: (1) an overview of LTA; (2) a comparison of LTA to other longitudinal models; (3) software used to run LTA; (4) sample size suggestions; (5) modeling steps in LTA; (6) measurement invariance; (7) the inclusion of auxiliary variables; (8) interpreting results of an LTA; (9) the nature of data (e.g., longitudinal, cross-sectional); and (10) extensions of LTA. An applied example of LTA is included to help understand how to build an LTA and interpret results. Finally, the article suggests future areas of research for LTA. This article provides an overview of LTA, highlighting key decisions researchers need to make to navigate and implement an LTA analysis from start to finish. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Question of Scale? Generalizability of the Ottawa and Chen Scales to Render Entrustment Decisions for the Core EPAs in the Workplace.

PURPOSE: Assessments of the Core Entrustable Professional Activities (Core EPAs) are based on observations of supervisors throughout a medical student's progression toward entrustment. The purpose of this study was to compare generalizability of scores from 2 entrustment scales: the Ottawa Surgical Competency Operating Room Evaluation (Ottawa) scale and an undergraduate medical education supervisory scale proposed by Chen and colleagues (Chen). A secondary aim was to determine the impact of frequent assessors on generalizability of the data. METHOD: For academic year 2019-2020, the Virginia Commonwealth University School of Medicine modified a previously described workplace-based assessment (WBA) system developed to provide feedback for the Core EPAs across clerkships. The WBA scored students' performance using both Ottawa and Chen scales. Generalizability (G) and decision (D) studies were performed using an unbalanced random-effects model to determine the reliability of each scale. Secondary G- and D-studies explored whether faculty who rated more than 5 students demonstrated better reliability. The Phi-coefficient was used to estimate reliability; a cutoff of at least 0.70 was used to conduct D-studies. RESULTS: Using the Ottawa scale, variability attributable to the student ranged from 0.8% to 6.5%. For the Chen scale, student variability ranged from 1.8% to 7.1%. This indicates the majority of variation was due to the rater (42.8%-61.3%) and other unexplained factors. Between 28 and 127 assessments were required to obtain a Phi-coefficient of 0.70. For 2 EPAs, using faculty who frequently assessed the EPA improved generalizability, requiring only 5 and 13 assessments for the Chen scale. CONCLUSIONS: Both scales performed poorly in terms of learner-attributed variance, with some improvement in 2 EPAs when considering only frequent assessors using the Chen scale. Based on these findings in conjunction with prior evidence, the authors provide a root cause analysis highlighting challenges with WBAs for Core EPAs.

Evaluating the Association of a Core EPA-Oriented Patient Handover Curriculum on Medical Students' Self-reported Frequency of Observation and Skill Acquisition.

Introduction: "Giving or receiving a patient handover to transition patient care responsibility" is one of the thirteen Core Entrustable Professional Activities (Core EPAs) for Entering Residency. However, implementing a patient handover curriculum in undergraduate medical education (UME) remains challenging. Educational leaders in the multi-institutional Core EPA8 pilot workgroup developed a longitudinal patient handover UME curriculum that was implemented at two pilot institutions. Materials and Methods: We utilized multi-school graduation questionnaire data to assess the association of our patient handover curriculum on self-reported frequency of observation/feedback and skill acquisition by comparing data from the shared curriculum schools to data from other Core EPA pilot schools (three schools with school-specific curriculum; five without a dedicated handover curriculum). Questionnaire data from 1,278 graduating medical students of the class of 2020 from all ten Core EPA pilot schools were analyzed. Results: Graduates from the two medical schools that implemented the shared patient handover curriculum reported significantly greater frequency of handover observation/feedback compared to graduates at the other schools (school-specific curriculum (p < .05) and those without a handover curriculum (p < .05)). Graduates from the two shared approach schools also more strongly agreed that they possessed the skill to perform handovers compared to graduates from the other eight pilot schools that did not implement this curriculum. Conclusion: The findings of this study suggest that the implementation of a multi-institutional Core EPA-based curricular model for teaching and assessing patient handovers was successful and could be implemented at other UME institutions.

Facing hierarchy: a qualitative study of residents' experiences in an obstetrical simulation scenario.

BACKGROUND: Residents in surgical specialties face a steep hierarchy when managing medical crises. Hierarchy can negatively impact patient safety when team members are reluctant to speak up. Yet, simulation has scarcely been previously utilized to qualitatively explore the way residents in surgical specialities navigate this challenge. The study aimed to explore the experiences of residents in one surgical specialty, obstetrics and gynecology (Ob/Gyn), when challenging hierarchy, with the goal of informing future interventions to optimize resident learning and patient safety. METHODS: Eight 3rd- and 4th-year Ob/Gyn residents participated in a simulation scenario in which their supervising physician made an erroneous medical decision that jeopardized the wellbeing of the labouring mother and her foetus. Residents participated in 30-45 min semi-structured interviews that explored their approach to managing this scenario. Transcribed interviews were analysed using qualitative thematic inquiry by three research team members, finalizing the identified themes once consensus was reached. RESULTS: Study results show that the simulated scenario did create an experience of hierarchy that challenged residents. In response, residents adopted three distinct communication strategies while confronting hierarchy: (1) messaging - a mere reporting of existing clinical information; (2) interpretive - a deliberate construction of clinical facts aimed at swaying supervising physician's clinical decision; and (3) advocative - a readiness to confront the staff physician's clinical decision. Furthermore, residents utilized coping mechanisms to mitigate challenges related to confronting hierarchy, namely deflecting responsibility, diminishing urgency, and drafting allies. Both these communication strategies and coping mechanisms shaped their practice when challenging hierarchy to preserve patient safety. CONCLUSIONS: Understanding the complex processes in which residents engage when confronting hierarchy can serve to inform the development and study of curricular innovations. Informed by these processes, we must move beyond solely teaching residents to speak up and consider a broader curriculum that targets not only residents but also faculty physicians and the learning environment within the organization.

Core EPAs in the Acting Internship: Early Outcomes from an Interdepartmental Experience.

INTRODUCTION: The Core Entrustable Professional Activities for Entering Residency (Core EPAs) are clinical activities all interns should be able to perform on the first day of residency with indirect supervision. The acting (sub) internship (AI) rotation provides medical students the opportunity to be assessed on advanced Core EPAs. MATERIALS AND METHODS: All fourth-year AI students were taught Core EPA skills and performed these clinical skills under direct supervision. Formative feedback and direct observation data were provided via required workplace-based assessments (WBAs). Supervising physicians rated learner performance using the Ottawa Clinic Assessment Tool (OCAT). WBA and pre-post student self-assessment data were analyzed to assess student performance and gauge curriculum efficacy. RESULTS: In the 2017-2018 academic year, 167 students completed two AI rotations at our institution. By their last WBA, 91.2% of students achieved a target OCAT supervisory scale rating for both patient handoffs and calling consults. Paired sample t tests of the student pre-post surveys showed statistically significant improvement in self-efficacy on key clinical functions of the EPAs. DISCUSSION: This study demonstrates that the AI rotation can be structured to include a Core EPA curriculum that can assess student performance utilizing WBAs of directly observed clinical skills. CONCLUSIONS: Our clinical outcomes data demonstrates that the majority of fourth-year medical students are capable of performing advanced Core EPAs at a level acceptable for intern year by the conclusion of their AI rotations. WBA data collected can also aid in ad hoc and longitudinal summative Core EPA entrustment decisions. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40670-021-01208-y.

"When in Doubt, Ask the Patient": A Quantitative, Patient-Oriented Approach to Formative Assessment of CanMEDS Roles.

Introduction: Since the introduction of competency-based frameworks into postgraduate medical curricula, educators have struggled to implement robust assessment tools that document the progression of necessary skills. The global movement towards competency-based medical education demands validated assessment tools. Our objective was to provide validity evidence for the Ottawa CanMEDS Competency Assessment Tool (OCCAT), designed to assess clinical performance in the communicator, professional, and health advocate CanMEDS roles. Methods: We developed the OCCAT, a 29-item questionnaire informed by specialty-specific Entrustable Professional Activities and consultation with stakeholders, including patients. Our sample included nine neonatal-perinatal medicine and maternal fetal medicine fellows rotating through antenatal high-risk clinics at the Ottawa Hospital. Following 70 unique encounters, the OCCAT was completed by patients and learners. Generalizability theory was used to determine overall reliability of scores. Differences in self and patient ratings were assessed using analyses of variance. Results: Generalizability analysis demonstrated that both questionnaires produced reliable scores (G-coefficient > 0.9). Self-scores were significantly lower than patient scores across all competencies, F(1, 6) = 13.9, p = .007. Variability analysis demonstrated that trainee scores varied across all competencies, suggesting both groups were able to recognize competencies as distinct and discriminate favorable behaviors belonging to each. Discussion: Our findings lend support to the movement to integrate self-assessment and patient feedback in formal evaluations for the purpose of enriched learner experiences and improved patient outcomes. We anticipate that the OCCAT will facilitate bridging to competency-based medical education.

Flipping Out! Utilizing an Online Micro-lecture for Asynchronous Learning Within the Acting Internship.

BACKGROUND: The acting (Sub)internship (AI) provides fourth-year medical students the opportunity to gain essential clinical experience, making it challenging to develop an effective curriculum without detracting from clinical time. ACTIVITY: This flipped classroom, asynchronous learning curriculum utilized a short online video, called a micro-lecture, to teach one to two key concepts, associated with online case-based questions. RESULTS AND DISCUSSION: Over one academic year, 96% (64/67) of internal medicine AI students at our institution completed the online questions. The majority of students selected the single best response for both questions and preferred this online micro-lecture format over traditional methods.