Synthesis versus imitation: evaluation of a medical student simulation curriculum via Objective Structured Assessment of Technical Skill.

ABSTRACT

BACKGROUND: Given the investment that programs make to simulation training, it is important to evaluate its effects on student learning. Tasks (e.g., gowning and gloving, suturing) are typically taught in isolation over a series of linked sessions. This study assessed students' ability to integrate such tasks while executing an unrehearsed procedure before and after a new simulation curriculum was introduced. METHODS: An Objective Structured Assessment of Technical Skill (OSATS) was administered to 26 students in the 2007 clerkship who received a 3-hour orientation to the operating room followed by a 3-hour animate laboratory, and to 167 students in the 2008 clerkship who received a 9-hour simulation skills curriculum. The OSATS task involved a live volunteer "patient" with an arm laceration. Students had 40 minutes to explain the procedure, start an intravenous line, administer a local anesthetic, prepare the wound (pig's foot), gown and glove, and suture the wound. The OSATS was scored by trained raters using a tool with 57 checklist and 7 global rating items. Its internal consistency reliability was 0.82. Independent sample t tests were used to analyze differences between "pre" and "post" groups. RESULTS: Mean scores were significantly higher for the post group for the checklist score (83% vs 62%, p < 0.001), the average global item score (3.62 vs 3.07, p = 0.003) and the OSATS total score (79% correct vs 62%, p < 0.001). Students from both groups were weakest in maintaining a sterile field, motion, and flow. Although superior, post group students still struggled with organizing a plan of action when faced with an unrehearsed procedure. CONCLUSIONS: The revised curriculum had a positive impact on students' mastery of basic surgical skills, despite the loss of the animal laboratory. Implications for instruction include greater use of discovery-learning techniques to teach productive versus reproductive skills.