Structures and Processes of Grading Committees in Internal Medicine Clerkships: Results of a National Survey.

PURPOSE: Clerkship grades are important in the residency selection process but can be influenced by individual bias and grading tendencies. Although clinical competency committees are standard in graduate medical education, in undergraduate medical education, they have not gained widespread traction. This study describes structures and processes of grading committees in internal medicine (IM) clerkships and strategies used to mitigate grading bias. METHOD: From September to December 2022, the Clerkship Directors in Internal Medicine conducted its annual survey of IM core clerkship directors at 140 U.S. and U.S. territory-based medical schools. This study was based on 23 questions about grading committees in IM clerkships. RESULTS: The survey response rate was 80% (n = 112/140). Forty-seven respondents (42%) reported using grading committees in their IM clerkship (median committee size, 7; range, 3-20) (primarily clerkship leadership and faculty). Responsibilities included determining grades for all students (31 [66%]) and students at borderline of failing (30 [64%]), adjudicating on students with professionalism concerns (25 [53%]), and reconciling discordant clinical evaluations (24 [51%]). To mitigate deliberation bias, committees most frequently used multisource assessments (38 [81%]) and adoption of a shared mental model (36 [77%]). Approximately one-third of grading committees "rarely" discussed gender (14 [30%]) and race or ethnicity (15 [32%]), and 7 committees (15%) "never" discussed gender and race or ethnicity. Clerkship directors perceived developing a shared mental model (60 [92%]), promoting consistency (59 [91%]) and transparency (57 [88%]) in the process, mitigating assessment bias (58 [89%]), improving student satisfaction (54 [83%]), and sharing grading responsibility (44 [68%]) as potential benefits. CONCLUSIONS: This study found that grading committees in IM clerkships are not widely used and that existing committees vary in structure and process. These findings highlight an opportunity for medical schools to consider using grading committees to improve grade assignment and address grading inconsistencies.

Changes in Gender and Racial/Ethnic Diversity in US Residency Program Applications From 2018 to 2022.

Background Residency application patterns by gender and race/ethnicity offer important insights about diversity in residency recruitment. It is unknown how the COVID-19 pandemic and virtual interviewing affected these patterns. Objective We hypothesized that the introduction of virtual interviews caused an increase in applications submitted per applicant and that there may be differences by gender and race/ethnicity. Methods We extracted publicly reported Electronic Residency Application Service application data from 2018 to 2022 for 14 residency specialties with 1000 or more applicants in 2022 by self-reported gender and underrepresented in medicine (UIM) status. We compared patterns before and after virtual interviews were introduced in 2021. Results Among 401 480 residency applicants, the average number of applications submitted per applicant increased for all specialties between 2018 and 2022 across gender and race/ethnicity. Across all years, women applied to more programs than men in 5 specialties (dermatology, neurology, obstetrics/gynecology, pediatrics, and surgery), whereas men applied to more programs than women in 3 (anesthesia, family medicine, and physical medicine and rehabilitation). Across all years, non-UIM applicants applied to more programs than UIM applicants in all 14 specialties. There were no clear changes in application patterns by gender and race/ethnicity during in-person versus virtual interview years. Conclusions The average number of applications submitted per applicant increased over time across gender and race/ethnicity. In some specialties, women applied to more programs than men, and in others vice-versa, whereas non-UIM applicants applied to more programs than UIM applicants in all specialties. Virtual interviews did not change these patterns.

Strategies for Advancing Equity in Frontline Clinical Assessment.

Educational equity in medicine cannot be achieved without addressing assessment bias. Assessment bias in health professions education is prevalent and has extensive implications for learners and, ultimately, the health care system. Medical schools and educators desire to minimize assessment bias, but there is no current consensus on effective approaches. Frontline teaching faculty have the opportunity to mitigate bias in clinical assessment in real time. Based on their experiences as educators, the authors created a case study about a student to illustrate ways bias affects learner assessment. In this paper, the authors use their case study to provide faculty with evidence-based approaches to mitigate bias and promote equity in clinical assessment. They focus on 3 components of equity in assessment: contextual equity, intrinsic equity, and instrumental equity. To address contextual equity, or the environment in which learners are assessed, the authors recommend building a learning environment that promotes equity and psychological safety, understanding the learners' contexts, and undertaking implicit bias training. Intrinsic equity, centered on the tools and practices used during assessment, can be promoted by using competency-based, structured assessment methods and employing frequent, direct observation to assess multiple domains. Instrumental equity, focused on communication and how assessments are used, includes specific, actionable feedback to support growth and use of competency-based narrative descriptors in assessments. Using these strategies, frontline clinical faculty members can actively promote equity in assessment and support the growth of a diverse health care workforce.

Reviewing Internal Medicine Clerkship Grading Through a Proequity Lens: Results of a National Survey.

PURPOSE: Equity in assessment and grading has become imperative across medical education. Although strategies to promote equity exist, there may be variable penetrance across institutions. The objectives of this study were to identify strategies internal medicine (IM) clerkship directors (CDs) use to reduce inequities in assessment and grading and explore IM CDs' perceptions of factors that impede or facilitate the implementation of these strategies. METHOD: From October to December 2021, the Clerkship Directors in Internal Medicine of the Alliance for Academic Internal Medicine conducted its annual survey of IM core CDs at 137 U.S. and U.S. territory-based medical schools. This study is based on 23 questions from the survey about equity in IM clerkship assessment and grading. RESULTS: The survey response rate was 73.0% (100 of 137 medical school CDs). Use of recommended evidence-based strategies to promote equity in clerkship assessment and grading varied among IM clerkships. Only 30 respondents (30.0%) reported that their clerkships had incorporated faculty development on implicit bias for clinical supervisors of students; 31 (31.0%) provided education to faculty on how to write narrative assessments that minimize bias. Forty respondents (40.0%) provided guidance to clerkship graders on how to minimize bias when writing final IM clerkship summaries, and 41 (41.0%) used grading committees to determine IM clerkship grades. Twenty-three CDs (23.0%) received formal education by their institution on how to generate clerkship grades and summaries in a way that minimized bias. CONCLUSIONS: This national survey found variability among medical schools in the application of evidence-based strategies to promote equity in assessment and grading within their IM clerkships. Opportunities exist to adopt and optimize proequity grading strategies, including development of programs that address bias in clerkship assessment and grading, reevaluation of the weight of standardized knowledge exam scores on grades, and implementation of grading committees.

Medical student advising during virtual residency recruitment: results of a national survey of internal medicine clerkship and sub-internship directors.

INTRODUCTION: The residency application process is a critical time for medical students. The COVID-19 pandemic prompted changes to the residency recruitment procedures with the conversion of interviews to a virtual format. For medical school advisors guiding students on an all-virtual residency application process brought uncertainty to their advising practices. Thus, this study aimed to identify advising practices during the 2021 virtual application cycle. METHODS: We administered an IRB-exempt national survey through the Clerkship Directors in Internal Medicine to 186 internal medicine core/co-/associate/assistant clerkship directors and sub-internship directors representing 140 Liaison Committee on Medical Education-accredited U.S./U.S.-territory-based medical schools in spring 2021. The 23-question survey was designed and pilot-tested by faculty-educators and leaders with expertise in undergraduate medical education. Data analysis included paired t- and z-tests and thematic analysis of open-ended questions. RESULTS: The institutional response rate was 67% (93/140) and individual rate 55% (103/186). Half of the respondents felt prepared/very prepared (40% and 13% respectively) for their advising roles. Compared to pre-pandemic cycles, respondents advised a typical student in the middle-third of their class at their institution to apply to more residency programs (mean 24 programs vs 20, p < 0.001) and accept more interviews (mean 14 interviews vs 12, p < 0.001). Sixty-three percent (64/101) of respondents spent more time on student advising; 51% (51/101) reported more students asked them for informal advice. Fifty-nine percent (60/101) of respondents reported their advisees were able to assess a residency program 'somewhat well;' 31% (31/101) expressed that residency recruitment should remain entirely virtual in the future. CONCLUSION: The transition to virtual residency recruitment due to COVID-19 prompted advising practices that may have contributed to application inflation and increased advising workload. Future studies should explore longitudinal outcomes of virtual interviews on student success to guide best practices in how to advise students during residency recruitment.

The Impact of Internal Medicine Clerkship Characteristics and NBME Subject Exams on USMLE Step 2 Clinical Knowledge Exam Performance.

BACKGROUND: Residency program directors will likely emphasize the United States Medical Licensing Exam (USMLE) Step 2 clinical knowledge (CK) exam more during residency application given the recent USMLE Step 1 transition to pass/fail scoring. We examined how internal medicine clerkship characteristics and NBME subject exam scores affect USMLE Step 2 CK performance. DESIGN: The authors used univariable and multivariable generalized estimating equations to determine associations between Step 2 CK performance and internal medicine clerkship characteristics and NBME subject exams. The sample had 21,280 examinees' first Step 2 CK scores for analysis. RESULTS: On multivariable analysis, Step 1 performance (standardized ß = 0.45, p < .001) and NBME medicine subject exam performance (standardized ß = 0.40, p < .001) accounted for approximately 60% of the variance in Step 2 CK performance. Students who completed the internal medicine clerkship last in the academic year scored lower on Step 2 CK (Mdiff = -3.17 p < .001). Students who had a criterion score for passing the NBME medicine subject exam scored higher on Step 2 CK (Mdiff = 1.10, p = .03). There was no association between Step 2 CK performance and other internal medicine clerkship characteristics (all p > 0.05) nor with the total NBME subject exams completed (ß=0.05, p = .78). CONCLUSION: Despite similarities between NBME subject exams and Step 2 CK, the authors did not identify improved Step 2 CK performance for students who had more NBME subject exams. The lack of association of Step 2 CK performance with many internal medicine clerkship characteristics and more NBME subject exams has implications for future clerkship structure and summative assessment. The improved Step 2 CK performance in students that completed their internal medicine clerkship earlier warrants further study given the anticipated increase in emphasis on Step 2 CK.

Innovation and Missed Opportunities in Internal Medicine Undergraduate Education During COVID-19: Results from a National Survey.

BACKGROUND: COVID-19 disrupted undergraduate clinical education when medical schools removed students from clinical rotations following AAMC recommendations. Clerkship directors (CDs) had to adapt rapidly and modify clerkship curricula. However, the scope and effects of these modifications are unknown. OBJECTIVE: To examine the effects of the initial phase of COVID-19 on the internal medicine (IM) undergraduate clinical education. DESIGN: A nationally representative web survey. PARTICIPANTS: IM CDs from 137 LCME-accredited US medical schools in 2020. MAIN MEASURES: Items (80) assessed clerkship structure and curriculum, assessment in clerkships, post-clerkship IM clinical experiences, and CD roles and support. The framework of Understanding Crisis Response (Royal Society for Encouragement of Arts, Manufactures, and Commerce) was used to determine whether curricular modifications were "amplified," "restarted," "let go," or "ended." KEY RESULTS: Response rate was 74%. In response to COVID-19, 32% (32/101) of clerkships suspended all clinical activities and 66% (67/101) only in-person. Prior to clinical disruption, students spent a median of 8.0 weeks (IQR: 2) on inpatient and 2.0 weeks (IQR: 4) on ambulatory rotations; during clinical re-entry, students were spending 5.0 (IQR: 3) and 1.0 (IQR: 2) weeks, respectively. Bedside teaching and physical exam instruction were "let go" during the early phase. Students were removed from direct patient care for a median of 85.5 days. The sub-internship curriculum remained largely unaffected. Before the pandemic, 11% of schools were using a pass/fail grading system; at clinical re-entry 47% and during the survey period 23% were using it. Due to the pandemic, 78.2% of CDs assumed new roles or had expanded responsibilities; 51% reported decreased scholarly productivity. CONCLUSIONS: Curricular adaptations occurred in IM clerkships across US medical schools as a result of COVID-19. More research is needed to explore the long-term implications of these changes on medical student education and clinical learning environments.

Patient Advocacy Assessment in the Medicine Clerkship: A Qualitative Study of Definition, Context, and Impact.

BACKGROUND: Advocacy is a core value of the medical profession. However, patient advocacy (advocacy) is not uniformly assessed and there are no studies of the behaviors clinical supervisors consider when assessing advocacy. OBJECTIVE: To explore how medical students and supervisors characterize advocacy during an internal medicine clerkship, how assessment of advocacy impacted students and supervisors, and elements that support effective implementation of advocacy assessment. DESIGN: A constructivist qualitative paradigm was used to understand advocacy assessment from the perspectives of students and supervisors. PARTICIPANTS: Medical students who completed the internal medicine clerkship at UCSF during the 2018 and 2019 academic years and supervisors who evaluated students during this period. APPROACH: Supervisor comments from an advocacy assessment item in the medicine clerkship and transcripts of focus groups were used to explore which behaviors students and supervisors deem to be advocacy. Separate focus groups with both students and supervisors examined the impact that advocacy assessment had on students' and supervisors' perceptions of advocacy and what additional context was necessary to effectively implement advocacy assessment. KEY RESULTS: Students and supervisors define advocacy as identifying and addressing social determinants of health, recognizing and addressing patient wishes and concerns, navigating the health care system, conducting appropriate evaluation and treatment, and creating exceptional therapeutic alliances. Effective implementation of advocacy assessment requires the creation of non-hierarchical team environments, supervisor role modeling, and pairing assessment with teaching of advocacy skills. Inclusion of advocacy assessment reflects and dictates institutional priorities, shapes professional identity formation, and enhances advocacy skill development for students and their supervisors. CONCLUSIONS: Students and supervisors consider advocacy to be a variety of behaviors beyond identifying and addressing social determinants of health. Effectively implementing advocacy assessment shapes students' professional identity formation, underscoring the critical importance of formally focusing on this competency in the health professions education.

Moving toward Mastery: Changes in Student Perceptions of Clerkship Assessment with Pass/Fail Grading and Enhanced Feedback.

ProblemClerkship grades contribute to a summative assessment culture in clerkships and can therefore interfere with students' learning. For example, by focusing on summative, tiered clerkship grades, students often discount accompanying feedback that could inform future learning. This case report seeks to explore whether an assessment system intervention which eliminated tiered grades and enhanced feedback was associated with changes in student perceptions of clerkship assessment and perceptions of the clinical learning environment. Intervention: In January 2019, our institution eliminated tiered clerkship grading (honors/pass/fail) for medical students during the core clerkship year and implemented pass/fail clerkship grading along with required twice weekly, work-based assessments for formative feedback. Context: In this single institution, cross-sectional survey study, we collected data from fourth-year medical students one year after an assessment system intervention. The intervention entailed changing from honors/pass/fail to pass/fail grading in all eight core clerkships and implementing an electronic system to record twice-weekly real-time formative work-based assessments. The survey queried student perceptions on the fairness and accuracy of grading and the clinical learning environment-including whether clerkships were mastery- or performance-oriented. We compared responses from students one year after the assessment intervention to those from the class one year before the intervention. Comparisons were made using unpaired, two-tailed t-tests or chi-squared tests as appropriate with Cohen's d for effect size estimation for score differences. Content analysis was used to analyze responses from two open-ended questions about feedback and grading. Impact: Survey response rates were similar before and after intervention (76% (127/168) vs. 72% (118/163), respectively) with no between-group differences in demographics. The after-intervention group showed statistically significant increases in the following factors: "grades are transparent and fair" (Cohen's d = 0.80), "students receive useful feedback" (d = 0.51), and "resident evaluation procedures are fair" (d = 0.40). After-intervention respondents perceived the clerkship learning environment to be more mastery-oriented (d = 0.52), less performance approach-oriented (d = 0.63), and less performance avoid-oriented (d = 0.49). There were no statistical differences in the factors "attending evaluation procedures are fair," "evaluations are accurate," "evaluations are biased," or "perception of stereotype threat." Open-ended questions revealed student recommendations to improve clerkship summary narratives, burden of work-based assessment, and in-person feedback. Lessons Learned: After an assessment system change to pass/fail grading with work-based assessments, we observed moderate to large improvements in student perceptions of clerkship grading and the mastery orientation of the learning environment. Our intervention did not improve perceptions around bias in assessment in clerkships. Other medical schools may consider similar interventions to begin to address student concerns with clerkship assessment and promote a more adaptive learning environment.

A is for advocacy: How introducing student advocacy assessment impacts longitudinal integrated clerkship students and clinical supervisors.

To improve health, physicians are increasingly called to advocate. Yet medical schools have only recently focused on health advocacy skill-building. Limited work to date addresses assessing medical student advocacy on behalf of patients. We describe how students and clinical supervisors (CS) in two urban longitudinal integrated clerkships (LIC) experience patient advocacy and how introducing a new advocacy assessment impacts them. Using a thematic approach, we analyzed transcripts of focus groups during 2018-2019. Seventeen of 24 (71%) students and 15 of 21 (71%) CS participated in the focus groups. We describe how students perceive their advocacy role as they accompany the patient, amplify their voice, and facilitate connection. The rationale for advocacy assessment includes that it (1) adds a novel dimension to the written and verbal assessment, (2) drives student learning, (3) aligns with the institutional goal to promote equity, and (4) impacts CS teaching and clinical practice. Challenges are the ambiguity of expectations, pressure to 'perform,' and a moral overlay to advocacy assessment. Findings demonstrate how educational alliances between students and CS and longitudinal relationships between LIC students and patients offer a constructive opportunity for advocacy assessment. We describe suggestions to hone and expand the reach of advocacy assessment.

Roles and Responsibilities of Medicine Subinternship Directors : Medicine Subinternship Director Roles.

BACKGROUND: The internal medicine (IM) subinternship (also referred to as acting internship) plays a crucial part in preparing medical students for residency. The roles, responsibilities, and support provided to subinternship directors have not been described. OBJECTIVE: We sought to describe the current role of IM subinternship directors with respect to their responsibilities, salary support, and reporting structure. DESIGN: Nationally representative, annually recurring thematic survey of IM core clerkship directors with membership in an academic professional association as of September 2017. PARTICIPANTS: A total of 129 core clinical medicine clerkship directors at Liaison Committee on Medical Education fully accredited U.S./U.S.-territory-based medical schools. MAIN MEASURES: Responsibilities, salary support, and reporting structure of subinternship directors. KEY RESULTS: The survey response rate was 83.0% (107/129 medical schools). Fifty-one percent (54/107) of respondents reported overseeing both core clerkship inpatient experiences and/or one or more subinternships. For oversight, 49.1% (28/53) of subinternship directors also reported that they were the clerkship director, 26.4% (14/53) that another faculty member directed all medicine subinternships, and 18.9% (10/53) that each subinternship had its own director. The most frequently reported responsibilities for the subinternship directors were administration, including scheduling, and logistics of student schedules (83.0%, 44/53), course evaluation (81.1%, 43/53), and setting grades 79.2% (42/53). The modal response for estimated FTE per course was 10-20% FTE, with 33.3% (16/48) reporting this level of support and 29.2% (14/54) reporting no FTE support. CONCLUSIONS: The role of the IM subinternship director has become increasingly complex. Since the IM subinternship is critical to preparing students for residency, IM subinternship directors require standard expectations and adequate support. Future studies are needed to determine the appropriate level of support for subinternship directors and to define essential roles and responsibilities.

Antibody status and cumulative incidence of SARS-CoV-2 infection among adults in three regions of France following the first lockdown and associated risk factors: a multicohort study.

BACKGROUND: We aimed to estimate the seropositivity to anti-SARS-CoV-2 antibodies in May-June 2020 after the first lockdown period in adults living in three regions in France and to identify the associated risk factors. METHODS: Between 4 May 2020 and 23 June 2020, 16 000 participants in a survey on COVID-19 from an existing consortium of three general adult population cohorts living in the Ile-de-France (IDF) or Grand Est (GE) (two regions with high rate of COVID-19) or in the Nouvelle-Aquitaine (NA) (with a low rate) were randomly selected to take a dried-blood spot for anti-SARS-CoV-2 antibodies assessment with three different serological methods (ClinicalTrial Identifier #NCT04392388). The primary outcome was a positive anti-SARS-CoV-2 ELISA IgG result against the spike protein of the virus (ELISA-S). Estimates were adjusted using sampling weights and post-stratification methods. Multiple imputation was used to infer the cumulative incidence of SARS-CoV-2 infection with adjustments for imperfect tests accuracies. RESULTS: The analysis included 14 628 participants, 983 with a positive ELISA-S. The weighted estimates of seropositivity and cumulative incidence were 10.0% [95% confidence interval (CI): 9.1%, 10.9%] and 11.4% (95% CI: 10.1%, 12.8%) in IDF, 9.0% (95% CI: 7.7%, 10.2%) and 9.8% (95% CI: 8.1%, 11.8%) in GE and 3.1% (95% CI: 2.4%, 3.7%) and 2.9% (95% CI: 2.1%, 3.8%) in NA, respectively. Seropositivity was higher in younger participants [odds ratio (OR) = 1.84 (95% CI: 1.79, 6.09) in <40 vs 50-60 years old and OR = 0.56 (95% CI: 0.42, 0.74) in ≥70 vs 50-60 years old)] and when at least one child or adolescent lived in the same household [OR = 1.30 (95% CI: 1.11, 1.53)] and was lower in smokers compared with non-smokers [OR = 0.71 (95% CI: 0.57, 0.89)]. CONCLUSIONS: Seropositivity to anti-SARS-CoV-2 antibodies in the French adult population was ≤10% after the first wave. Modifiable and non-modifiable risk factors were identified.