Which Internal Medicine Clerkship Characteristics Are Associated With Students' Performance on the NBME Medicine Subject Exam? A Multi-Institutional Analysis.

PURPOSE: To identify which internal medicine clerkship characteristics may relate to NBME Medicine Subject Examination scores, given the growing trend toward earlier clerkship start dates. METHOD: The authors used linear mixed effects models (univariable and multivariable) to determine associations between medicine exam performance and clerkship characteristics (longitudinal status, clerkship length, academic start month, ambulatory clinical experience, presence of a study day, involvement in a combined clerkship, preclinical curriculum type, medicine exam timing). Additional covariates included number of NBME clinical subject exams used, number of didactic hours, use of a criterion score for passing the medicine exam, whether medicine exam performance was used to designate clerkship honors, and United States Medical Licensing Examination Step 1 performance. The sample included 24,542 examinees from 62 medical schools spanning 3 academic years (2011-2014). RESULTS: The multivariable analysis found no significant association between clerkship length and medicine exam performance (all pairwise P > .05). However, a small number of examinees beginning their academic term in January scored marginally lower than those starting in July (P < .001). Conversely, examinees scored higher on the medicine exam later in the academic year (all pairwise P < .001). Examinees from schools that used a criterion score for passing the medicine exam also scored higher than those at schools that did not (P < .05). Step 1 performance remained positively associated with medicine exam performance even after controlling for all other variables in the model (P < .001). CONCLUSIONS: In this sample, the authors found no association between many clerkship variables and medicine exam performance. Instead, Step 1 performance was the most powerful predictor of medicine exam performance. These findings suggest that medicine exam performance reflects the overall medical knowledge students accrue during their education rather than any specific internal medicine clerkship characteristics.

Use of Filters for Residency Application Review: Results From the Internal Medicine In-Training Examination Program Director Survey.

BACKGROUND: The increase in applications to residency programs, known as "application inflation," creates challenges for program directors (PDs). Prior studies have shown that internal medicine (IM) PDs utilize criteria, such as United States Medical Licensing Examination (USMLE) Step examination performance, when reviewing applications. However, little is known about how early these filters are utilized in the application review cycle. OBJECTIVE: This study sought to assess the frequency and types of filters utilized by IM PDs during initial residency application screening and prior to more in-depth application review. METHODS: A web-based request for the 2016 Internal Medicine In-Training Examination (IM-ITE) PD Survey was sent to IM PDs. Responses from this survey were analyzed, excluding non-US programs. RESULTS: With a 50% response rate (214 of 424), IM PDs responded that the most commonly used data points to filter applicants prior to in-depth application review were the USMLE Step 2 Clinical Knowledge score (32%, 67 of 208), USMLE Step 1 score (24%, 50 of 208), and medical school attended (12%, 25 of 208). Over half of US IM PD respondents (55%, 114 of 208) indicated that they list qualifying interview criteria on their program website, and 31% of respondents (50 of 160) indicated that more than half of their applicant pool does not meet the program's specified interview criteria. CONCLUSIONS: Results from the 2016 IM-ITE PD Survey indicate many IM PDs use filters for initial application screening, and that these filters, when available to applicants, do not affect many applicants' decisions to apply.

The New Internal Medicine Subinternship Curriculum Guide: a Report from the Alliance for Academic Internal Medicine.

The internal medicine (IM) subinternship has been a long-established clinical experience in the final phase of medical school deemed by key stakeholders as a crucial rotation to prepare senior medical students for internship. Medical education has changed greatly since the first national curriculum for this course was developed in 2002 by the Clerkship Directors in Internal Medicine (CDIM). Most notably, competency-based medical education (CBME) has become a fixture in graduate medical education and has gradually expanded into medical school curricula. Still, residency program directors and empirical studies have identified gaps and inconsistencies in knowledge and skills among new interns. Recognizing these gaps, the Association of Program Directors in Internal Medicine (APDIM) surveyed its members in 2010 and identified four core skills essential for intern readiness. The Association of American Medical Colleges (AAMC) also published 13 core entrustable professional activities (EPAs) for entering residency to be expected of all medical school graduates. Results from the APDIM survey along with the widespread adoption of CBME informed this redesign of the IM subinternship curriculum. The authors provide an overview of this new guide developed by the Alliance for Academic Internal Medicine (AAIM) Medical Student-to-Resident Interface Committee (MSRIC).

Validity Evidence for a Brief Online Key Features Examination in the Internal Medicine Clerkship.

PURPOSE: Medical educators use key features examinations (KFEs) to assess clinical decision making in many countries, but not in U.S. medical schools. The authors developed an online KFE to assess third-year medical students' decision-making abilities during internal medicine (IM) clerkships in the United States. They used Messick's unified validity framework to gather validity evidence regarding response process, internal structure, and relationship to other variables. METHOD: From February 2012 through January 2013, 759 students (at eight U.S. medical schools) had 75 minutes to complete one of four KFE forms during their IM clerkship. They also completed a survey regarding their experiences. The authors performed item analyses and generalizability studies, comparing KFE scores with prior clinical experience and National Board of Medical Examiners Subject Examination (NBME-SE) scores. RESULTS: Five hundred fifteen (67.9%) students consented to participate. Across KFE forms, mean scores ranged from 54.6% to 60.3% (standard deviation 8.4-9.6%), and Phi-coefficients ranged from 0.36 to 0.52. Adding five cases to the most reliable form would increase the Phi-coefficient to 0.59. Removing the least discriminating case from the two most reliable forms would increase the alpha coefficient to, respectively, 0.58 and 0.57. The main source of variance came from the interaction of students (nested in schools) and cases. Correlation between KFE and NBME-SE scores ranged from 0.24 to 0.47 (P < .01). CONCLUSIONS: These results provide strong evidence for response-process and relationship-to-other-variables validity and moderate internal structure validity for using a KFE to complement other assessments in U.S. IM clerkships.

Experiences of Female and Male Medical Students With Death, Dying, and Palliative Care: One Size Does Not Fit All.

BACKGROUND: Medical students learn about death, dying, and palliative care (DDPC) through formal curricular offerings and informal clinical experiences; however, the lessons learned in the clinic may be at odds with the formal curriculum. Reflective writing is a means for students to "bracket" their DDPC experiences and reconcile conflicts between the formal and informal curriculum. OBJECTIVES: The aim of this study is to compare the level of reflection demonstrated in medical students' narratives on DDPC with other experiences and to examine the domains of professionalism that students perceive to be prevalent in their DDPC experiences. METHODS: Third-year medical students submitted professionalism narratives during their internal medicine clerkship. We identified a subset of narratives related to DDPC (n = 388) and randomly selected control narratives (n = 153). We assessed the level of reflection demonstrated in the narratives using a validated rubric and analyzed the professionalism domains that students identified as relevant to their experience. RESULTS: There was no difference in reflective level between DDPC and control narratives. Within the DDPC group, female students demonstrated higher reflection (2.24 ± 0.71) than male students (2.01 ± 0.77; P < .001). Caring, compassion and communication, and honor and integrity were prominent among DDPC narratives. More females identified caring, compassion, and communication as relevant to their DDPC experiences, whereas more males identified altruism. CONCLUSION: Males and females have different perceptions of DDPC experiences, and female students appear to be more deeply impacted. These findings can help clinical faculty engage students more effectively with this challenging topic.

The Association Between Peer and Self-Assessments and Professionalism Lapses Among Medical Students.

Peer and self-assessments are widely used to assess professionalism during medical school as part of a multisource feedback model. The purpose of this study was to examine the associations between peer and self-assessments and professionalism lapses at a large medical school. A retrospective case-control study design was used to compare peer and self-assessment scores from Years 1 to 3 of medical school for students who had been cited for professionalism lapses during medical school (case group; n = 78) with those of a randomly selected control group ( n = 230). Students in the case group had significantly lower peer assessment scores than students in the control group during all 3 years. Year 3 peer assessment scores showed the greatest difference (cases = 7.81 ± 0.65, controls = 8.22 ± 0.34, p < .01). Students with lower peer assessment scores were also significantly more likely to have been cited for a professionalism lapse (odds ratio = 6.25, 95% CI [3.13, 11.11], p < .01). This study reinforces the value of peer assessments of professionalism, which may be useful to help identify students who may be at risk for professionalism lapses during medical school.

An Entrustable Professional Activity (EPA)-Based Framework to Prepare Fourth-Year Medical Students for Internal Medicine Careers.

The purpose of the fourth year of medical school remains controversial. Competing demands during this transitional phase cause confusion for students and educators. In 2014, the Association of American Medical Colleges (AAMC) released 13 Core Entrustable Professional Activities for Entering Residency (CEPAERs). A committee comprising members of the Clerkship Directors in Internal Medicine and the Association of Program Directors in Internal Medicine applied these principles to preparing students for internal medicine residencies. The authors propose a curricular framework based on five CEPAERs that were felt to be most relevant to residency preparation, informed by prior stakeholder surveys. The critical areas outlined include entering orders, forming and answering clinical questions, conducting patient care handovers, collaborating interprofessionally, and recognizing patients requiring urgent care and initiating that care. For each CEPAER, the authors offer suggestions about instruction and assessment of competency. The fourth year of medical school can be rewarding for students, while adequately preparing them to begin residency, by addressing important elements defined in the core entrustable activities. Thus prepared, new residents can function safely and competently in supervised postgraduate settings.

Internal Medicine Residency Program Directors' Views of the Core Entrustable Professional Activities for Entering Residency: An Opportunity to Enhance Communication of Competency Along the Continuum.

PURPOSE: To examine internal medicine (IM) residency program directors' (PDs') perspectives on the Core Entrustable Professional Activities for Entering Residency (Core EPAs)-introduced into undergraduate medical education to further competency-based assessment-and on communicating competency-based information during transitions. METHOD: A spring 2015 Association of Program Directors in Internal Medicine survey asked PDs of U.S. IM residency programs for their perspectives on which Core EPAs new interns must or should possess on day 1, which are most essential, and which have the largest gap between expected and observed performance. Their views and preferences were also requested regarding communicating competency-based information at transitions from medical school to residency and residency to fellowship/employment. RESULTS: The response rate was 57% (204/361 programs). The majority of PDs felt new interns must/should possess 12 of the 13 Core EPAs. PDs' rankings of Core EPAs by relative importance were more varied than their rankings by the largest gaps in performance. Although preferred timing varied, most PDs (82%) considered it important for medical schools to communicate Core EPA-based information to PDs; nearly three-quarters (71%) would prefer a checklist format. Many (60%) would be willing to provide competency-based evaluations to fellowship directors/employers. Most (> 80%) agreed that there should be a bidirectional communication mechanism for programs/employers to provide feedback on competency assessments. CONCLUSIONS: The gaps identified in Core EPA performance may help guide medical schools' curricular and assessment tool design. Sharing competency-based information at transitions along the medical education continuum could help ensure production of competent, practice-ready physicians.

Is Reflective Ability Associated With Professionalism Lapses During Medical School?

PURPOSE: Recently, many have argued that learning to reflect on one's experiences is a critical component of professional identity formation and of professionalism. However, little empirical evidence exists to support this claim. This study explored the association between reflective ability and professionalism lapses among medical students. METHOD: The authors conducted a retrospective case-control study of all students who matriculated at Indiana University School of Medicine from 2001 to 2009. The case group (n = 70) included those students who had been cited for a professionalism lapse during medical school; the students in the control group (n = 230) were randomly selected from the students who had not been cited for a professionalism lapse. Students' professionalism journal entries were scored using a validated rubric to assess reflective ability. Mean reflection scores were compared across groups using t tests, and logistic regression analysis was used to assess the relationship between reflective ability and professionalism lapses. RESULTS: Reflection scores for students in the case group (2.46 ± 1.05) were significantly lower than those for students in the control group (2.82 ± 0.83) (P = .01). A lower reflection score was associated with an increased likelihood that the student had been cited for a professionalism lapse (odds ratio = 1.56; P < .01). CONCLUSIONS: This study revealed a significant relationship between reflective ability and professionalism, although further study is needed to draw any conclusions regarding causation. These findings provide quantitative evidence to support current anecdotal claims about the relationship between reflection and professionalism.

The Internal Medicine Subinternship--Now More Important than Ever: A Joint CDIM-APDIM Position Paper.

For decades, the internal medicine (IM) subinternship has served as a critical interface between undergraduate and graduate medical education. As such, the vast majority of U.S. medical schools offer this rotation to help students prepare for post-graduate training. Historically an experiential rotation, a formal curriculum with specific learning objectives was eventually developed for this course in 2002. Since then, graduate medical education (GME) has changed significantly with the regulation of duty hours, adoption of competency-based education, and development of training milestones and entrustable professional activities. In response to these and many other changes to residency training and medical practice, in 2010, the Association of Program Directors in Internal Medicine (APDIM) surveyed its members-with input from the Clerkship Directors in Internal Medicine (CDIM) Subinternship Task Force-to determine which core skills program directors expected from new medical school graduates. The results of that survey helped to inform a joint CDIM-APDIM committee's decision to re-evaluate the goals of the IM subinternship in an effort to enhance the transition from medical school to residency. This joint committee defined the minimum expectations of what constitutes an IM subinternship rotation, proposed recommended skills for IM subinterns, and discussed challenges and future directions for this crucial course.

What skills should new internal medicine interns have in july? A national survey of internal medicine residency program directors.

PROBLEM: The transition from medical student to intern may cause stress and burnout in new interns and the delivery of suboptimal patient care. Despite a formal set of subinternship curriculum guidelines, program directors have expressed concern regarding the skill set of new interns and the lack of standardization in that skill set among interns from different medical schools. To address these issues, the Accreditation Council for Graduate Medical Education's Next Accreditation System focuses on the development of a competency-based education continuum spanning undergraduate, graduate, and continuing medical education. APPROACH: In 2010, the Clerkship Directors in Internal Medicine subinternship task force, in collaboration with the Association of Program Directors in Internal Medicine survey committee, surveyed internal medicine residency program directors to determine which competencies or skills they expected from new medical school graduates. The authors summarized the results using categories of interest. OUTCOMES: In both an item rank list and free-text responses, program directors were nearly uniform in ranking the skills they deemed most important for new interns-organization and time management and prioritization skills; effective communication skills; basic clinical skills; and knowing when to ask for assistance. NEXT STEPS: Stakeholders should use the results of this survey as they develop a milestone-based curriculum for the fourth year of medical school and for the internal medicine subinternship. By doing so, they should develop a standardized set of skills that meet program directors' expectations, reduce the stress of transitions across the educational continuum, and improve the quality of patient care.

Which experiences in the hidden curriculum teach students about professionalism?

PURPOSE: To examine the relationship between learner experience in the "hidden curriculum" and student attribution of such experiences to professionalism categories. METHOD: Using the output of a thematic analysis of 272 consecutive narratives recorded by 135 students on a medical clerkship from June through November 2007, the authors describe the frequency of these experiences within and across student-designated Association of American Medical Colleges-National Board of Medical Examiners professionalism categories and employ logistic regression to link varieties of experience to specific professionalism categories. RESULTS: Thematic analysis uncovered two main domains of student experience: medical-clinical interaction and teaching-and-learning experiences. From a student perspective the critical incident stories evoked all professionalism categories. Most frequently checked off categories were caring/compassion/communication (77%) and respect (69%). Logistic regression suggested that student experiences within the teaching-and-learning environment were associated with professionalism categories of excellence, leadership, and knowledge and skills, whereas those involving medical-clinical interactions were associated with respect, responsibility and accountability, altruism, and honor and integrity. Experiences of communicating and working within teams had the broadest association with learning about professionalism. CONCLUSIONS: Student narratives touched on all major professionalism categories as well as illuminating the contexts in which critical experiences emerged. Linked qualitative and quantitative analysis identified those experiences that were associated with learning about particular aspects of professionalism. Experiences of teamwork were especially relevant to student learning about professionalism in action.

Exploring the meaning of respect in medical student education: an analysis of student narratives.

BACKGROUND: Respect for others is recognized in the medical literature and society as an essential attribute of the good medical professional. However, the specific meaning of respect varies widely and is underexplored as a lived experience of physicians-in-training. OBJECTIVE: To describe third-year medical students' narratives of respect and disrespect [(dis)respect] during their internal medicine clerkship. DESIGN: Qualitative thematic analysis of 152 third-year student narratives that 'taught them something about professionalism,' focusing on (dis)respect. APPROACH: Immersion/crystallization narrative analysis. RESULTS: We reviewed 595 professionalism narratives and found that one in four narratives involved (dis)respect. We then found that 2/3 of these narratives were negative (describing instances of disrespect rather than respect). In the other coded categories, the proportion of negative narratives was significantly lower. In order to better understand these results, we analyzed the content of the (dis)respect narratives and identified six primary themes: (1) content and manner of communication (including, appreciating or belittling, being sensitive or blunt and respecting privacy); (2) conduct: behaviors expressing (dis)respect; (3) patient centeredness: honoring others' preferences, decisions and needs; (4) treating others as equals; (5) valuing the other and their experience and/or problem; and (6) nurturing students' learning. CONCLUSIONS: Focusing on the lived experience of (dis)respect on wards broadens the concept of respect beyond any one type of act, behavior or attitude. Students perceive respect as a way of being that applies in all settings (private and public), with all participants (patients, family members, nurses, colleagues and students) and under all circumstances (valuing others' time, needs, preferences, choices, opinions and privacy). Respect seems to entail responding to a need, while disrespect involves ignoring the need or bluntly violating it.

Medical students' professionalism narratives: a window on the informal and hidden curriculum.

PURPOSE: The aim of this study was to use medical students' critical incident narratives to deepen understanding of the informal and hidden curricula. METHOD: The authors conducted a thematic analysis of 272 stories of events recorded by 135 third-year medical students that "taught them something about professionalism and professional values." Students wrote these narratives in a "professionalism journal" during their internal medicine clerkships at Indiana University School of Medicine, June through November 2007. RESULTS: The majority of students' recorded experiences involved witnessing positive embodiment of professional values, rather than breaches. Attending physicians and residents were the central figures in the incidents. Analyses revealed two main thematic categories. The first focused on medical-clinical interactions, especially on persons who were role models interacting with patients, families, coworkers, and colleagues. The second focused on events in the teaching-and-learning environment, particularly on students' experiences as learners in the clinical setting. CONCLUSIONS: The findings strongly suggest that students' reflective narratives are a rich source of information about the elements of both the informal and hidden curricula, in which medical students learn to become physicians. Experiences with both positive and negative behaviors shaped the students' perceptions of the profession and its values. In particular, interactions that manifest respect and other qualities of good communication with patients, families, and colleagues taught powerfully.

The subinternship curriculum in internal medicine: a national survey of clerkship directors.

BACKGROUND: The 4th-year internal medicine subinternship (subI), with its strong emphasis on experience-based learning and increased patient responsibilities, is an important component of undergraduate medical education. Discussions have begun amongst educational leaders on the importance of standardizing curriculum and evaluation tools utilized during the subinternship. PURPOSE: The objective of this survey was to describe the current state of educational practices regarding the subI curriculum, use of Clerkship Directors in Internal Medicine (CDIM) curricular guidelines, evaluation methodologies, and subintern call frequency and duty hours. METHODS: The survey was a part of the CDIM annual survey sent to 109 institutional members in 2005. The subinternship survey included 17 questions that addressed the prevalence of formal curricula in the subinternship, use of CDIM curricular guidelines and resources, and the evaluation tools utilized for assessment of the subinternship. Two questions examined the call frequency and duty hours of the subinterns. RESULTS: Eighty-eight members responded (81%). Of the respondents, 37% have a formal curriculum and 35% used CDIM curricular guidelines for the subinternship. More than half of the respondents agreed that the curricular guide helped organize their subI rotations (53%), and 48% agreed that it enhanced the quality of subI education. A global rating summative evaluation form (56%) was the most commonly used evaluation tool in the subinternship. Objective Structured Clinical Examinations (1%) and Mini-Clinical Evaluation Exercise (mini-CEX)(2%) were much less utilized during the subinternship. The respondents expressed an interest in the development of a standardized exam (47%) and clinical assessment tools such as the mini-CEX (59%). The vast majority of IM subinterns participate in 3 to 9 calls per rotation and spend less than 80 hr in the hospital per week. CONCLUSIONS: A formal curriculum is offered in slightly more than one third of subIs. Clerkship directors expressed interest in developing standardized evaluation tools to assess subintern competencies. The majority of IM subinterns spend less than 80 hr in the hospital per week. Further studies are needed to assess barriers to incorporating a formal curriculum and develop standardized tools to assess subintern competencies.