Do You See What I See? A Comparison of CCC and Self-Assigned Milestones Across Military Medical Specialties.

BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to complete competency-based assessments of medical trainees based on nationally established Milestones. Previous research demonstrates a strong correlation between CCC and resident scores on the Milestones in surgery, but little is known if this is true between specialties. In this study, we investigated a variety of specialties and sought to determine what factors affect self-assessment of milestones. In addition, a post-hoc analysis was completed on the COVID-19 pandemic effects on self-evaluation. METHODS: This is an IRB approved observational study on prospectively collected self-evaluation milestone data that is used within each ACGME program's Clinical Competency Committees. Medical trainees within the San Antonio Uniformed Services Health Education Consortium were approached for possible participation in this study with permission from program directors. RESULTS: There was no significant difference between self-assessments and CCC-assessments based on self-identified gender or residency type (surgical versus nonsurgical) for any milestone domain. Within the postgraduate year (PGY) groups, the PGY5 and PGY6 tended to rate themselves higher than CCC. Chiefs (Internal Medicine PGY2/3, and General Surgery PGY5/6) tended to be more accurate in scoring themselves than the interns (PGY1) within the milestone of Interpersonal Skills and Communication (chiefs 0.5 vs. interns 0.62, p = 0.03). On post hoc analysis of self-rating, during the first wave of the COVID 19 pandemic, Post-Covid residents were more likely to underrate themselves in Systems-Based Practice compared to the Pre-Covid cohort (-0.49 vs 0.10; p = 0.007) and more likely to rate themselves higher in Professionalism (-0.54 vs. -0.10, p = 0.012). CONCLUSION: Unique to this study and our institution, there was no gender difference found in self vs CCC evaluations. With the change in learning environment from COVID, there was also a change in ability for some learners to self-assess accurately. As medical educators, we should understand the importance of both encouraging learners to practice self-assessment as well as give feedback to trainees on their progress. We also need to educate our faculty on the use of milestones for assessment to create a true gold standard in the CCC.

Divergence and Dissonance in Residents' Reported and Actual Feedback to Faculty in a Military Context.

INTRODUCTION: Trainees (e.g., residents) are an obvious and common source of feedback for faculty; however, gaps exist in our understanding of their experiences and practices of providing such feedback. To gain a deeper understanding, this study examined residents' beliefs about what feedback is important to provide, the kinds of feedback they report giving, and the feedback they actually gave. MATERIALS AND METHODS: Descriptive statistics were used to analyze residents' perceptions and feedback behaviors (n = 42/96). Thematic analysis was used to analyze end-of-rotation faculty assessments from 2018 to 2019 (n = 559) to explore the actual written feedback residents provided to the faculty. RESULTS: The findings suggest that residents experience workload constraints (e.g., too many feedback requests), feel that their feedback is not valuable or relevant, and place conditions on when and what feedback is given (e.g., faculty agreeableness, prefer giving positively oriented feedback, and uncomfortable giving negative feedback). When comparing what feedback residents rated as important with the kinds of feedback they reported giving and actually gave, the findings also suggest that there were consistencies (e.g., clinical instruction and professionalism) and inconsistencies (e.g., evidence-based practice and medical knowledge) that may limit constructive feedback for faculty. CONCLUSIONS: Taken together, the findings suggest that trainee assessments of faculty may be insufficient as a primary source of feedback to support the improvement of faculty performance. Potential solutions are discussed.