Development and evaluation of standardized narrative cases depicting the general surgery professionalism milestones.

ABSTRACT

PURPOSE: Residency programs now are required to use educational milestones, which has led to the need for new methods of assessment. The literature suggests that narrative cases are a promising tool to track residents' progress. This study demonstrates the process for developing and evaluating narrative cases representing the five levels of the professionalism milestones. METHOD: In 2013, the authors identified 28 behaviors in the Accreditation Council for Graduate Medical Education general surgery professionalism milestones. They modified previously published narrative cases to fit these behaviors. To evaluate the quality of these cases, the authors developed a 28-item, five-point scale instrument, which 29 interdisciplinary faculty completed. The authors compared the faculty ratings by narrative case and specialty with the authors' initial rankings of the cases by milestone level. They used t tests and analysis of variance to compare mean scores across specialties. RESULTS: The authors developed 10 narrative cases, 2 for each of the 5 milestone levels. Each case contained at least 20 of the 28 behaviors identified in the milestones. Mean faculty ratings matched the milestone levels. Reliability was good (G coefficient = 0.86, phi coefficient = 0.85), indicating consistency in raters' ability to determine the proper milestone level for each case. CONCLUSIONS: The authors demonstrate a process for using specialty-specific milestones to develop narrative cases that map to a spectrum of professionalism behaviors. This process can be applied to other competencies and specialties to facilitate faculty awareness of resident performance descriptors and provide a frame of reference for milestones assessment.