Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters.

ABSTRACT

BACKGROUND: A major challenge with Competency Based Medical Education (CBME) is that of increased assessment burden on faculty. To reduce this burden, the accuracy and reliability of peer-assessment for surgical skills requires further exploration. METHODS: Forty-two second year medical students were video recorded while performing a simple interrupted suture and an instrument tie. Four novice raters underwent a short training session on the use of the Objective Structured Assessment of Technical Skills (OSATS) checklists. Videos of the suturing task were then independently assessed by the four novice raters and two expert raters on two occasions. Agreement between novice and expert rater scores was calculated using the intraclass correlation coefficient (ICC). RESULTS: For both simple interrupted suturing (ICC = 0.78, CI = 0.66-0.86, p < 0.001) and instrument ties (ICC = 0.87, CI = 0.80-0.92, p < 0.001), there was good agreement between novice and expert raters. CONCLUSIONS: Novice raters can be taught to use the OSATS checklists to assess peers on simple suturing and instrument tying tasks.