Simulation in laparoscopic surgery: a concurrent validity study for FLS.

BACKGROUND: Current assessments using the fundamentals of laparoscopic surgery (FLS) tasks are labour intensive and depend heavily on expert raters. Hand motion analysis may offer an alternative method of objective evaluation of FLS performance. PURPOSE: The aim of this study was to assess whether a correlation exists between the expert rated assessments of the FLS tasks and computer-based assessment of motion efficiency using the Imperial College surgical assessment device (ICSAD). METHODS: We recruited 26 volunteer subjects who were stratified into three experience groups: juniors (PGY1-3) (N = 13), seniors (PGY4,5) (N = 7) and staff surgeons (N = 6). All subjects performed four FLS tasks: (1) peg transfer, (2) pattern cut, (3) endoloop and (4) intracorporeal suturing. Performance was assessed by both standard FLS expert rating and motion analysis using ICSAD. Group differences were analyzed using the Kruskal-Wallis test, and Spearman coefficient analyses were employed to compare FLS and ICSAD scores. RESULTS: FLS expert-derived scores discriminated effectively between experience groups for all tasks (p < 0.05). Motion efficiency scores discriminated between experience groups for tasks 1, 3 and 4 for number of movements (p < 0.05), tasks 1 and 4 for total distance (p < 0.05) and tasks 1, 2, 3 and 4 for total time (p < 0.005). There was a significant correlation between total FLS expert scores and the motion efficiency metrics of total distance, number of movements and total time (Spearman coefficient and p values of 0.81, < 0.001; 0.76; < 0.001; and 0.93, < 0.001, respectively). CONCLUSION: There is a high correlation between FLS standard scoring and motion efficiency metrics. The use of ICSAD for the objective assessment of FLS tasks may in the future offer an adjunctive method of evaluation. ICSAD metrics are potentially less labour intensive due to the instant and fully automated computerized scoring that it provides.

Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback.

BACKGROUND: We carried out a prospective, randomized, 4-arm study including control arm, blinding of examiners to determine effectiveness of computer-based video instruction (CBVI) and different types of expert feedback (concurrent and summary) on learning of a basic technical skill. METHODS: Using bench models, participants were pre-tested on a suturing and instrument knot-tying skill after viewing an instructional video. The students were subsequently assigned randomly to 4 practice conditions: no additional intervention (control), self study with CBVI, expert feedback during practice trials (concurrent feedback), and expert feedback after practice trials (summary feedback). All participants underwent 19 trials of practice, over 1 hour, in their assigned training condition. The effectiveness of training was assessed both at an immediate post-test and 1 month later at a retention test. Performance was evaluated using both expert-based (Global Rating Scores) and computer-based assessment (Hand Motion Analysis). Data were analyzed using repeated-measures ANOVA. RESULTS: There were no differences in GRS between groups at pre-test. The CBVI, concurrent feedback and summary feedback methods were equally effective initially for the instruction of this basic technical skill to naive medical students and displayed better performance than control (control, 12.71 [10.79 to 14.62]; CBVI, 16.39 [14.38 to 18.40]; concurrent, 16.97 [15.79 to 18.15]; summary, 16.09 [13.57 to 18.62]; P < .001 each). At retention. however, only CBVI and summary feedback groups retained superior suturing and knot-tying performance versus control (control, 8.13 [6.94 to 9.85]; CBVI, 11.92 [10.19 to 14.99] P = .037; concurrent, 9.80 [8.55 to 13.45] P = .635; summary, 111.19 [10.27 to 14.29] P = .037). Hand motion data displayed a similar pattern of results. There were no group differences in the rate of learning (P > .05). CONCLUSION: Our study showed that CBVI can be as effective as summary expert feedback in the instruction of basic technical skills to medical students. Thoughtfully incorporated into technical curricula, CBVI can make efficient use of faculty time and serve as a useful pedagogic adjunct for basic skills training. Additionally, our study provides evidence supporting an increased role of summary feedback to effectively train novices in technical skills.

Computer-assisted assessment of one-handed knot tying skills performed within various contexts: a construct validity study.

BACKGROUND: The widespread use of computer-assisted assessment of technical proficiency in surgical residents shows the need for further investigations of the construct validity of these devices (eg, hand motion analysis) before implementation into competency testing. METHODS: Thirty general surgery residents performed 1-handed knot tying in 2 contexts: superficial and deep. The Imperial College Surgical Assessment Device (ICSAD) was used to evaluate performance. It was hypothesized that senior residents (postgraduate year [PGY] 4-5) would perform better then junior residents (PGY 1-3) and that the superficial version of the skill would be easier than the deep. RESULTS: Technical efficiency scores were better for seniors than for juniors (P < .001) and on the superficial versus the deep model (P < .001). Both groups were equally affected by the contextual changes to the skill, suggesting a consistent impact on the skill-specific movement patterns. CONCLUSIONS: Additional evidence for the validity of ICSAD as a competency assessment tool has been shown. First, it distinguished senior and junior residents. Second, it discerned differences on the same skill performed in 2 different contexts.

Do not teach me while I am working!

BACKGROUND: The aim of this study was to determine if technical surgical experience enhances the learning of new cognitive information under multitasking conditions. METHODS: Junior (years 1-3) and experienced (years 4 and 5) general surgery residents performed a Nissen fundoplication on a synthetic model (the primary task). While performing the primary task, they listened to and memorized information describing the steps of a computer-assisted hip replacement surgery (the secondary task). Performance on the primary and secondary tasks was assessed using performance metrics and multiple-choice questions. RESULTS: The primary task was performed better by the senior than the junior trainees (P = .001, P = .007). The senior trainees also scored higher on the secondary task than the junior trainees (P = .001). CONCLUSIONS: Senior trainees have superior capacity to multitask. This may have direct implications on both clinical and simulation-based education, such that educators need to adjust the amount of information presented in accordance to trainees' levels of training.

Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills.

BACKGROUND: Teaching of technical surgical skills to undergraduate medical students in a laboratory setting away from the patient is not common practice. Because of the large volume of students and shortage of available teaching faculty new methods of teaching must be developed for this group of trainees. In this study we examined the effectiveness of computer-based video training, different types of computer-based motion efficiency feedback (with and without expert criteria), and expert feedback on learning of a basic technical skill in medical students. METHODS: Forty-five junior medical students were randomized into 3 groups and learned suturing and knot-tying skills. Group A received computer-generated feedback about the economy of their movements. Group B received the same motion economy feedback, as well as expert reference values. Group C received verbal feedback from an expert. All groups were pre-tested, allowed 18 practice trials, and post-tested, and their skill retention was retested after 1 month. Performance was assessed by expert analysis using an objective structured analysis of technical skill and by computer analysis (Imperial College Surgical Assessment Device [ICSAD]). RESULTS: All groups showed improvement from pre-test to post-test. However, only group C showed retention of skill on delayed performance testing. CONCLUSIONS: Verbal feedback from an expert instructor led to lasting improvements in technical skills performance. Providing information about motion efficiency did not lead to similar improvements.

Surgical skill acquisition with self-directed practice using computer-based video training.

BACKGROUND: Computer-based video training (CBVT) provides flexible opportunities for surgical trainees to learn fundamental technical skills, but may be ineffective in self-directed practice settings because of poor trainee self-assessment. This study examined whether CBVT is effective in a self-directed learning environment among novice trainees. METHODS: Thirty novice trainees used CBVT to learn the 1-handed square knot while self-assessing their proficiency every 3 minutes. On reaching self-assessed skill proficiency, trainees were randomized to either cease practice or to complete additional practice. Performance was evaluated with computer and expert-based measures during practice and on pretests, posttests, and 1-week retention tests. RESULTS: Analyses revealed performance improvements for both groups (all P < .05), but no differences between the 2 groups (all P > .05) on all tests. CONCLUSIONS: CBVT for the 1-handed square knot is effective in a self-directed learning environment among novices. This lends support to the implementation of self-directed digital media-based learning within surgical curricula.

Computer-based video instructions for acquisition of technical skills.

This study aimed to assess which type of information presented in an interactive computer-based video instruction was most frequently used by novice medical students during a 1-hour training session in instrument suturing and knot-tying skills. Custom-designed instructional software enabled tracking when a given segment of the video was accessed. The results suggest that, in the early stages of learning, trainees require guidance in proper looping techniques and placement of the knots. In accordance with motor learning theory, when setting up CD-ROM or Web-based curricula, instructors should, therefore, emphasize these steps during early stages of learning.

Human immunodeficiency virus, hepatitis B, and hepatitis C seroprevalence in a Canadian trauma population.

BACKGROUND: The current seroprevalence of human immunodeficiency virus (HIV), hepatitis B, and hepatitis C in the Canadian trauma population is unknown. Establishing the seroprevalence of these diseases is vital for education, postexposure prophylaxis, and counseling, and to establish potential screening guidelines. The purpose of this study was to determine the seroprevalence of HIV, hepatitis B, and hepatitis C in the trauma population of London, Ontario, Canada. METHODS: All adult (aged > or = 18 years) trauma patients treated by the trauma team at London Health Sciences Centre were prospectively studied from January to December 2003. The study was conducted as a linked, confidential serosurvey with delayed full disclosure. Serum was analyzed for HIV, hepatitis C antibody, and Hepatitis B surface antigen. RESULTS: A total of 287 (76%) of 377 consecutive trauma patients had blood testing completed. Of the 287 patients tested, 1 (0.3%) was positive for hepatitis B, 8 (2.8%) were positive for hepatitis C, and no patients tested positive for HIV. Hepatitis C-positive patients were predominantly men (63%) with a mean age of 46 years and a mean Injury Severity Score of 19; 63% were injured in a motor vehicle crash, and 88% were discharged alive. There were no statistically significant differences in the demographic and injury profiles from the hepatitis C-negative patients (p > 0.2 for all). CONCLUSION: This is the first study to determine the rates of HIV, hepatitis B, and hepatitis C in the Canadian trauma population. Our trauma population demonstrated a threefold higher hepatitis C seroprevalence rate compared with the general population. Hepatitis C poses the highest risk to the trauma team of the three bloodborne diseases studied. With no vaccine or postexposure prophylaxis currently available for hepatitis C, this study highlights the importance of prevention and the strict use of universal precautions in the setting of trauma.