Outcomes of the Internal Medicine Supplemental Application: Preliminary Data on the 2022-2023 Match.

INTRODUCTION: Preference signaling (program signals and geographic preference divisions) was introduced as a component of the supplemental application for internal medicine applicants applying to programs within the United States (USA) during the 2021-22 cycle. These signals were intended to address application inflation by allowing applicants to express interest in and increase their likelihood of receiving interviews from their top programs. There is little published data, however, to describe the impact of preference signaling on the likelihood of receiving interviews from a program. This study thus sought to analyze, in a small subset of US applicants, whether preference signals were associated with a higher likelihood of obtaining a residency interview. METHODS: A survey was distributed in March 2023 to US MD seniors from the four allopathic medical schools in North Carolina who applied to categorical internal medicine residency programs during the 2022-23 application cycle. The survey was developed by the research team to provide respondents with the opportunity to report data from the electronic residency application service (ERAS) application and provide data on interviews received, actions taken throughout the application season, and outcomes of the National Residency Match Program (NRMP) using a combination of free response and multiple choice questions. RESULTS: Forty-seven out of a total of 85 contacted (55%) applicants completed some or all of the survey. Of those who completed the entirety of the survey, 39 (82.98%) completed the supplemental portion of the application and the available preference signaling. Applicants in this study were 2.95 (Odds ratio, 95% confidence interval [CI] 2.20 - 3.97, p<0.01) times as likely to receive an interview invitation from a program if they used a program signal. Applicants were 1.75 (odds ratio, 95% CI 1.38 - 2.21, p<0.01) times as likely to receive an interview invitation from a program in an indicated geographic preference division. Forty-seven percent (95% CI 31 - 64%) matched to a program they had sent a program signal to, and 97% (95% CI 78 - 100%) matched to a program in an indicated geographic preference division. CONCLUSIONS: The program signals and geographic preference division components of the supplemental application increased the likelihood of receiving an interview invitation but did not have a clear impact on match outcomes. Further research with larger sample sizes will be necessary to determine how these signals actually modify the outcomes of the NRMP.

Creation of an asynchronous faculty development curriculum on well-written narrative assessments that avoid bias.

BACKGROUND: The COVID-19 pandemic in parallel with concerns about bias in grading resulted in many medical schools adopting pass/fail clinical grading and relying solely on narrative assessments. However, narratives often contain bias and lack specificity. The purpose of this project was to develop asynchronous faculty development to rapidly educate/re-educate > 2000 clinical faculty spread across geographic sites and clinical disciplines on components of a well-written narrative and methods to minimize bias in the assessment of students. METHODS: We describe creation, implementation, and pilot data outcomes for an asynchronous faculty development curriculum created by a committee of volunteer learners and faculty. After reviewing the literature on the presence and impact of bias in clinical rotations and ways to mitigate bias in written narrative assessments, the committee developed a web-based curriculum using multimedia learning theory and principles of adult learning. Just-in-time supplemental materials accompanied the curriculum. The Dean added completion of the module by 90% of clinical faculty to the department chairperson's annual education metric. Module completion was tracked in a learning management system, including time spent in the module and the answer to a single text entry question about intended changes in behavior. Thematic analysis of the text entry question with grounded theory and inductive processing was used to define themes of how faculty anticipate future teaching and assessment as a result of this curricula. OUTCOMES: Between January 1, 2021, and December 1, 2021, 2166 individuals completed the online module; 1820 spent between 5 and 90 min on the module, with a median time of 17 min and an average time of 20.2 min. 15/16 clinical departments achieved completion by 90% or more faculty. Major themes included: changing the wording of future narratives, changing content in future narratives, and focusing on efforts to change how faculty teach and lead teams, including efforts to minimize bias. CONCLUSIONS: We developed a faculty development curriculum on mitigating bias in written narratives with high rates of faculty participation. Inclusion of this module as part of the chair's education performance metric likely impacted participation. Nevertheless, time spent in the module suggests that faculty engaged with the material. Other institutions could easily adapt this curriculum with provided materials.

Use of a structured approach and virtual simulation practice to improve diagnostic reasoning.

OBJECTIVES: In 2015, the National Academy of Medicine IOM estimated that 12 million patients were misdiagnosed annually. This suggests that despite prolonged training in medical school and residency there remains a need to improve diagnostic reasoning education. This study evaluates a new approach. METHODS: A total of 285 medical students were enrolled in this 8 center, IRB approved trial. Students were randomized to receive training in either abdominal pain (AP) or loss of consciousness (LOC). Baseline diagnostic accuracy of the two different symptoms was assessed by completing a multiple-choice question (MCQ) examination and virtual patient encounters. Following a structured educational intervention, including a lecture on the diagnostic approach to that symptom and three virtual patient practice cases, each student was re-assessed. RESULTS: The change in diagnostic accuracy on virtual patient encounters was compared between (1) baseline and post intervention and (2) post intervention students trained in the prescribed symptom vs. the alternate symptom (controls). The completeness of the student's differential diagnosis was also compared. Comparison of proportions were conducted using χ2-tests. Mixed-effects regressions were used to examine differences accounting for case and repeated measures. Compared with baseline, both the AP and LOC groups had marked post-intervention improvements in obtaining a correct final diagnosis; a 27% absolute improvement in the AP group (p<0.001) and a 32% absolute improvement in the LOC group (p<0.001). Compared with controls (the groups trained in the alternate symptoms), the rate of correct diagnoses increased by 13% but was not statistically significant (p=0.132). The completeness and efficiency of the differential diagnoses increased by 16% (ß=0.37, p<0.001) and 17% respectively (ß=0.45, p<0.001). CONCLUSIONS: The study showed that a virtual patient platform combined with a diagnostic reasoning framework could be used for education and diagnostic assessment and improved correct diagnosis compared with baseline performance in a simulated platform.

Internal Medicine Residency Program Responses to the Increase of Residency Applications: Differences by Program Type and Characteristics.

BACKGROUND: Over the past decade, the number of residency applications has increased substantially, causing many residency programs to change their recruitment practices. OBJECTIVE: We determined how internal medicine (IM) residency programs have responded to increased applications by program type (community-based, community-based/university-affiliated, and university-based) and characteristics (percentage of international medical graduates, program size, and program director [PD] tenure). METHODS: The Association of Program Directors in Internal Medicine conducted a national survey of 363 IM PDs in 2017. Five questions assessed IM program responses to the increased number of residency applications in 3 areas: changes in recruitment strategies, impact on ability to perform holistic review, and interest in 5 potential solutions. We performed a subgroup analysis to measure differences by program type and characteristics. RESULTS: The response rate was 64% (233 of 363). There were no differences by program type or characteristics for experiencing an increase in the number of applicants, altering recruitment practices, or conducting holistic reviews. There were moderate differences in alterations of recruitment practices by program characteristics and moderate differences in interest in proposed solutions by program type. Community-based programs had the greatest interest in a program-specific statement (59%, P = .032) and the lowest percentage in a national database of matched applicants (44%, P = .034). CONCLUSIONS: IM residency programs are experiencing an increasing number of applications and are accommodating by adjusting recruitment practices in a variety of ways. A majority of IM PDs supported 4 of the 5 solutions, although the level of interest differed by program type.

Digital chalk-talk videos improve knowledge and satisfaction in renal physiology.

The authors began a curriculum reform project to improve the experience in a Renal Physiology course for first-year medical students. Taking into account both the variety of learning preferences among students and the benefits of student autonomy, the authors hypothesized that adding digital chalk-talk videos to lecture notes and live lectures would improve student knowledge, course satisfaction, and engagement. The authors measured performance on the renal physiology exam before (the traditional curriculum) and for 2 yr after implementation of the new curriculum. During the traditional and subsequent years, students took a Q-sort survey before and after the Renal Physiology course. Satisfaction was assessed based on ranked statements in the Q sort, as well as through qualitative analysis of student commentary. Compared with the traditional curriculum, mean scores on the renal physiology final exam were higher after implementation of the new curriculum: 65.3 vs. 74.4 ( P < 0.001) with year 1 and 65.3 vs. 79.4 ( P < 0.001) in the second year. After the new curriculum, students were more likely to agree with the statement, "I wish other courses were taught like this one." Qualitative analysis revealed how the video-based curriculum improved student engagement and satisfaction. Adding digital chalk-talk videos to a traditional Renal Physiology course that included active learning led to improved exam performance and high levels of student satisfaction. Other preclinical courses in medical school may benefit from such an intervention.

Is it a match? a novel method of evaluating medical school success.

BACKGROUND: Medical education program evaluation allows for curricular improvements to both Undergraduate (UME) and Graduate Medical Education (GME). UME programs are left with little more than match rates and self-report to evaluate success of graduates in The Match. OBJECTIVE: This manuscript shares a novel method of program evaluation through a systematic assessment of Match outcomes. DESIGN: Surveys were developed and distributed to Program Training Directors (PTDs) at our institution to classify residency programs into which our UME graduates matched using an ordinal response scale and open-ended responses. Outcomes-based measures for UME graduates were collected and analyzed. The relationship between PTD survey data and UME graduates' outcomes were explored. Open-ended response data were qualitatively analyzed using iterative cycles of coding and identifying themes. RESULTS: The PTD survey response rate was 100%. 71% of our graduates matched to programs ranked as 'elite' (36%) or 'top' (35%) tier. The mean total number of 'Honors' grades achieved by UME graduates was 2.6. Data showed that graduates entering elite and top GME programs did not consistently earn Honors in their associated clerkships. A positive correlation was identified between USMLE Step 1 score, number of honors, and residency program rankings for a majority of the programs. Qualitative analysis identified research, faculty, and clinical exposure as necessary characteristics of 'elite' programs:. Factors considered by PTDs in the rating of programs included reputation, faculty, research, national presence and quality of graduates. CONCLUSIONS: This study describes a novel outcomes-based method of evaluating the success of UME programs. Results provided useful feedback about the quality of our UME program and its ability to produce graduates who match in highly-regarded GME programs. The findings from this study can benefit Clerkship Directors, Student Affairs and Curriculam Deans, and residency PTDs as they help students determine their competitiveness forspecialties and specific residency programs.

An Entrustable Professional Activity (EPA)-Based Framework to Prepare Fourth-Year Medical Students for Internal Medicine Careers.

The purpose of the fourth year of medical school remains controversial. Competing demands during this transitional phase cause confusion for students and educators. In 2014, the Association of American Medical Colleges (AAMC) released 13 Core Entrustable Professional Activities for Entering Residency (CEPAERs). A committee comprising members of the Clerkship Directors in Internal Medicine and the Association of Program Directors in Internal Medicine applied these principles to preparing students for internal medicine residencies. The authors propose a curricular framework based on five CEPAERs that were felt to be most relevant to residency preparation, informed by prior stakeholder surveys. The critical areas outlined include entering orders, forming and answering clinical questions, conducting patient care handovers, collaborating interprofessionally, and recognizing patients requiring urgent care and initiating that care. For each CEPAER, the authors offer suggestions about instruction and assessment of competency. The fourth year of medical school can be rewarding for students, while adequately preparing them to begin residency, by addressing important elements defined in the core entrustable activities. Thus prepared, new residents can function safely and competently in supervised postgraduate settings.

How Prepared Are Medical and Nursing Students to Identify Common Hazards in the Intensive Care Unit?

RATIONALE: Care in the hospital is hazardous. Harm in the hospital may prolong hospitalization, increase suffering, result in death, and increase costs of care. Although the interprofessional team is critical to eliminating hazards that may result in adverse events to patients, professional students' formal education may not prepare them adequately for this role. OBJECTIVES: To determine if medical and nursing students can identify hazards of hospitalization that could result in harm to patients and to detect differences between professions in the types of hazards identified. METHODS: Mixed-methods observational study of graduating nursing (n = 51) and medical (n = 93) students who completed two "Room of Horrors" simulations to identify patient safety hazards. Qualitative analysis was used to extract themes from students' written hazard descriptions. Fisher's exact test was used to determine differences in frequency of hazards identified between groups. RESULTS: Identification of hazards by students was low: 66% did not identify missing personal protective equipment for a patient on contact isolation, and 58% did not identify a medication administration error (medication hanging for a patient with similar name). Interprofessional differences existed in how hazards were identified: medical students noted that restraints were not indicated (73 vs. 2%, P < 0.001), whereas nursing students noted that there was no order for the restraints (58.5 vs. 0%, P < 0.0001). Nursing students discovered more issues with malfunctioning or incorrectly used equipment than medical students. Teams performed better than individuals, especially for hazards in the second simulation that were similar to those in the first: need to replace a central line with erythema (73% teams identified) versus need to replace a peripheral intravenous line (10% individuals, P < 0.0001). Nevertheless, teams of students missed many intensive care unit-specific hazards: 54% failed to identify the presence of pressure ulcers; 85% did not notice high tidal volumes on the ventilator; and 90% did not identify the absence of missing spontaneous awakening/breathing trials and absent stress ulcer prophylaxis. CONCLUSIONS: Graduating nursing and medical students missed several hazards of hospitalization, especially those related to the intensive care unit. Orientation for residents and new nurses should include education on hospitalization hazards. Ideally, this orientation should be interprofessional to allow appreciation for each other's roles and responsibilities.

CaPOW! Using Problem Sets in a Capstone Course to Improve Fourth-Year Medical Students' Confidence in Self-Directed Learning.

PROBLEM: Despite the importance of self-directed learning (SDL) in the field of medicine, individuals are rarely taught how to perform SDL or receive feedback on it. Trainee skill in SDL is limited by difficulties with self-assessment and goal setting. APPROACH: Ninety-two graduating fourth-year medical students from Duke University School of Medicine completed an individualized learning plan (ILP) for a transition-to-residency Capstone course in spring 2015 to help foster their skills in SDL. Students completed the ILP after receiving a personalized report from a designated faculty coach detailing strengths and weaknesses on specific topics (e.g., pulmonary medicine) and clinical skills (e.g., generating a differential diagnosis). These were determined by their performance on 12 Capstone Problem Sets of the Week (CaPOWs) compared with their peers. Students used transitional-year milestones to self-assess their confidence in SDL. OUTCOMES: SDL was successfully implemented in a Capstone course through the development of required clinically oriented problem sets. Coaches provided guided feedback on students' performance to help them identify knowledge deficits. Students' self-assessment of their confidence in SDL increased following course completion. However, students often chose Capstone didactic sessions according to factors other than their CaPOW performance, including perceived relevance to planned specialty and session timing. NEXT STEPS: Future Capstone curriculum changes may further enhance SDL skills of graduating students. Students will receive increased formative feedback on their CaPOW performance and be incentivized to attend sessions in areas of personal weakness.

Mystery behind the match: an undergraduate medical education-graduate medical education collaborative approach to understanding match goals and outcomes.

BACKGROUND: There is a paucity of information regarding institutional targets for the number of undergraduate medical education (UME) graduates being matched to graduate medical education (GME) programs at their home institutions. At our institution, the Duke University, the number of UME graduates matched to GME programs declined dramatically in 2011. To better understand why this decline may have happened, we sought to identify perceived quality metrics for UME and GME learners, evaluate trends in match outcomes and educational program characteristics, and explore whether there is an ideal retention rate for UME graduates in their home institutions' GME programs. METHODS: We analyzed the number of Duke University UME graduates remaining at Duke for GME training over the past 5 years. We collected data to assess for changing characteristics of UME and GME, and performed descriptive analysis of trends over time to investigate the potential impact on match outcomes. RESULTS: A one-sample t-test analysis showed no statistically significant difference in the number of Duke UME graduates who stayed for GME training. For both UME and GME, no significant changes in the characteristics of either program were found. DISCUSSION: We created a process for monitoring data related to the characteristics or perceived quality of UME and GME programs and developed a shared understanding of what may impact match lists for both UME graduates and GME programs, leaving the Match somewhat less mysterious. While we understand the trend of graduates remaining at their home institutions for GME training, we are uncertain whether setting a goal for retention is reasonable, and so some mystery remains. We believe there is an invaluable opportunity for collaboration between UME and GME stakeholders to facilitate discussion about setting shared institutional goals.

A meaningful MESS (Medical Education Scholarship Support).

BACKGROUND: Graduate medical education faculty bear the responsibility of demonstrating active research and scholarship; however, faculty who choose education-focused careers may face unique obstacles related to the lack of promotion tracks, funding, career options, and research opportunities. Our objective was to address education research and scholarship barriers by providing a collaborative peer-mentoring environment and improve the production of research and scholarly outputs. METHODS: We describe a Medical Education Scholarship Support (MESS) group created in 2013. MESS is an interprofessional, multidisciplinary peer-mentoring education research community that now spans multiple institutions. This group meets monthly to address education research and scholarship challenges. Through this process, we develop new knowledge, research, and scholarly products, in addition to meaningful collaborations. RESULTS: MESS originated with eight founding members, all of whom still actively participate. MESS has proven to be a sustainable unfunded local community of practice, encouraging faculty to pursue health professions education (HPE) careers and fostering scholarship. We have met our original objectives that involved maintaining 100% participant retention; developing increased knowledge in at least seven content areas; and contributing to the development of 13 peer-reviewed publications, eight professional presentations, one Masters of Education project, and one educational curriculum. DISCUSSION: The number of individuals engaged in HPE research continues to rise. The MESS model could be adapted for use at other institutions, thereby reducing barriers HPE researchers face, providing an effective framework for trainees interested in education-focused careers, and having a broader impact on the education research landscape.

Teaching crucial skills: An electrocardiogram teaching module for medical students.

BACKGROUND: Medical student performance in electrocardiogram (ECG) interpretation at our institution could be improved. Varied resources exist to teach students this essential skill. METHODS: We created an ECG teaching module (ECGTM) of 75 cases representing 15 diagnoses to improve medical students' performance and confidence in ECG interpretation. Students underwent pre- and post-clerkship testing to assess ECG interpretation skills and confidence and also end-of-clinical-year testing in ECG and laboratory interpretation. Performance was compared for the years before and during ECGTM availability. RESULTS: Eighty-four percent of students (total n=101) reported using the ECGTM; 98% of those who used it reported it was useful. Students' performance and confidence were higher on the post-test. Students with access to the ECGTM (n=101) performed significantly better than students from the previous year (n=90) on the end-of-year ECG test. CONCLUSIONS: The continuous availability of an ECGTM was associated with improved confidence and ability in ECG interpretation. The ECGTM may be another available tool to help students as they learn to read ECGs.

How We Engage Graduating Professional Students in Interprofessional Patient Safety.

PROBLEM: Interprofessional curricula on patient safety do not acknowledge the culture and vulnerabilities of the student experience and often do not engage students. METHODS: We describe a patient safety collaboration between graduating nursing and medical students during their Capstone courses that fostered conversations about the similarities and differences in professional school experiences around patient safety. Students wrote reflections about an unanticipated patient outcome. Qualitative content analysis was used to characterize themes within student reflections, and to create audience response system questions to highlight differences in each profession's reflections and to facilitate discussion about those differences during the collaboration. FINDINGS: Medical students identified events in which perceived patient outcomes were worse than events identified by nursing students. Nursing students identified more near-miss events. Nursing students positively impacted the event and attributed action to the presence of a clinical instructor and personal responsibility for patient care. Medical students described themselves as "only a witness" and attributed inaction to hierarchy and concern about grades. CONCLUSIONS: Students felt the session would change their future attitudes and behaviors. Stevenson Chudgar Molloy Phillips Engle Clay.

Using Transitional Year Milestones to Assess Graduating Medical Students' Skills During a Capstone Course.

BACKGROUND: Undergraduate medical education (UME) follows the lead of graduate medical education (GME) in moving to competency-based assessment. The means for and the timing of competency-based assessments in UME are unclear. OBJECTIVE: We explored the feasibility of using the Accreditation Council for Graduate Medical Education Transitional Year (TY) Milestones to assess student performance during a mandatory, fourth-year capstone course. METHODS: Our single institution, observational study involved 99 medical students who completed the course in the spring of 2014. Students' skills were assessed by self, peer, and faculty assessment for 6 existing course activities using the TY Milestones. Evaluation completion rates and mean scores were calculated. RESULTS: Students' mean milestone levels ranged between 2.2 and 3.6 (on a 5-level scoring rubric). Level 3 is the performance expected at the completion of a TY. Students performed highest in breaking bad news and developing a quality improvement project, and lowest in developing a learning plan, working in interdisciplinary teams, and stabilizing acutely ill patients. Evaluation completion rates were low for some evaluations, and precluded use of the data for assessing student performance in the capstone course. Students were less likely to complete separate online evaluations. Faculty were less likely to complete evaluations when activities did not include dedicated time for evaluations. CONCLUSIONS: Assessment of student competence on 9 TY Milestones during a capstone course was useful, but achieving acceptable evaluation completion rates was challenging. Modifications are necessary if milestone scores from a capstone are intended to be used as a handoff between UME and GME.

GME Concentrations: A Collaborative Interdisciplinary Approach to Learner-Driven Education.

BACKGROUND: Graduate medical education (GME) programs may struggle to provide the knowledge, skills, and experiences necessary to meet trainee career interests and goals beyond a clinical focus. Sponsoring institutions can partner with programs to deliver content not included in typical clinical experiences of GME programs. OBJECTIVE: To perform a needs assessment and to develop, implement, and measure acceptability and feasibility of an institution-wide GME Concentrations curriculum. METHODS: In response to the needs assessment, GME leadership developed 4 concentrations: (1) Resident-as-Teacher; (2) Patient Safety and Quality Improvement; (3) Law, Ethics, and Health Policy; and (4) Leaders in Medicine. We formed advisory committees that developed curricula for each concentration, including didactics, experiential learning, and individual project mentoring. Participants completed pre- and postassessments. We assessed feasibility and evaluated participant presentations and final projects. RESULTS: Over the course of 3 years, 91 trainees (of approximately 1000 trainees each year) from 36 GME programs (of 82 accredited programs) have participated in the program. The number of participants has increased each year, and 22 participants have completed the program overall. Cost for each participant is estimated at $500. Participant projects addressed a variety of education and health care areas, including curriculum development, quality improvement, and national needs assessments. Participants reported that their GME Concentrations experience enhanced their training and fostered career interests. CONCLUSIONS: The GME Concentrations program provides a feasible, institutionally based approach for educating trainees in additional interest areas. Institutional resources are leveraged to provide and customize content important to participants' career interests beyond their specialty.

The Impact of a Mandatory Immersion Curriculum in Integrative Medicine for Graduating Medical Students.

BACKGROUND: Although many Americans utilize complementary and alternative medicine (CAM) services and products, few medical school curricula consistently provide instruction in counseling patients on the use of CAM or strategies for identifying credible evidence on the safety and effectiveness of CAM therapies. METHODS: This is a mixed methods study. A sustainable, mandatory, half-day CAM immersion curriculum for graduating medical students is described. Student comfort talking with patients about CAM, their willingness to refer patients to a CAM provider, and adequacy of the CAM curriculum was assessed. RESULTS: Students who participated in this mandatory curriculum, rated the medical school curriculum in CAM as more adequate than students at other medical schools without a mandatory curriculum. Students' narrative comments indicate the curriculum impacts students knowledge about CAM, patient use of CAM, and personal practice with CAM in the future. CONCLUSIONS: The timing of the CAM curriculum near to graduation, students' personal exploration of several CAM modalities through immersion, and student interaction with community CAM providers are aspects of the curriculum that make the curriculum successful and memorable.

Poor Positive Predictive Value of Lyme Disease Serologic Testing in an Area of Low Disease Incidence.

BACKGROUND: Lyme disease is diagnosed by 2-tiered serologic testing in patients with a compatible clinical illness, but the significance of positive test results in low-prevalence regions has not been investigated. METHODS: We reviewed the medical records of patients who tested positive for Lyme disease with standardized 2-tiered serologic testing between 2005 and 2010 at a single hospital system in a region with little endemic Lyme disease. Based on clinical findings, we calculated the positive predictive value of Lyme disease serology. Next, we reviewed the outcome of serologic testing in patients with select clinical syndromes compatible with disseminated Lyme disease (arthritis, cranial neuropathy, or meningitis). RESULTS: During the 6-year study period 4723 patients were tested for Lyme disease, but only 76 (1.6%) had positive results by established laboratory criteria. Among 70 seropositive patients whose medical records were available for review, 12 (17%; 95% confidence interval, 9%-28%) were found to have Lyme disease (6 with documented travel to endemic regions). During the same time period, 297 patients with a clinical illness compatible with disseminated Lyme disease underwent 2-tiered serologic testing. Six of them (2%; 95% confidence interval, 0.7%-4.3%) were seropositive, 3 with documented travel and 1 who had an alternative diagnosis that explained the clinical findings. CONCLUSIONS: In this low-prevalence cohort, fewer than 20% of positive Lyme disease tests are obtained from patients with clinically likely Lyme disease. Positive Lyme disease test results may have little diagnostic value in this setting.