Identifying areas for curricular program improvement based on perceptions of skills, competencies, and performance.

ABSTRACT

OBJECTIVE: Educational program evaluation and program improvement are processes that can be enhanced by involving multiple stakeholders and measurement tools. The purpose of this study was to compare faculty and resident physician perceptions of both teaching quality and resident competence for 13 core psychiatric skills and the six Accreditation Council for Graduate Medical Education general competencies. Additionally, resident performance as assessed by portfolios was descriptively examined relative to these perceptions of skills and competencies. Using this combination of data, the authors propose a useful approach for identifying areas for improvement within a medical residency's program curriculum. METHODS: Ten teaching faculty members (100%) and 18 residents (82%) within a psychiatry department completed parallel surveys. Trained raters scored resident-completed portfolio entries. Nonparametric and descriptive analyses were used to compare the various data sources. RESULTS: In general, faculty and residents showed congruence in their perceptions about resident performance. They agreed on the quality of teaching skills and general competencies. General competency scores from the portfolios were congruent with this perspective. However, there were mismatches in perceptions and performance on portfolios for some areas. Of the 13 core psychiatric skills examined, only one area, neuropsychiatry, was rated below average by both faculty and residents, while the median portfolio performance score was below competent for four skills, including neuropsychiatry. Faculty rated resident competence significantly higher than residents rated themselves for biopsychosocial formulation and medical psychiatry. CONCLUSION: This study describes and highlights the usefulness of a multidimensional approach that includes resident and faculty perspectives and externally rated performance measures to assist in identifying potential target areas for curriculum improvement.