Can both residents and chief physicians assess surgical skills?

BACKGROUND: It is known that structured assessment of an operation can provide trainees with useful knowledge and potentially shorten their learning curve. However, methods for objective assessment have not been widely adopted into the clinical setting. This might be because of a lack of expertise using an assessment tool. The aim of this present study was to investigate if a validated laparoscopic procedure-specific assessment tool could be used by doctors with different levels of experience. METHODS: The study was conducted as an observer-blinded, prospective cohort study. Three video recordings of a right-side laparoscopic salpingectomy were distributed to ten chief physicians, eight residents (fourth year trainees), and two expert assessors (all in gynecology) in order to be assessed using a validated procedure-specific assessment tool. The three salpingectomies were selected because they easily showed the different operational levels: novice, intermediate, and expert. The two expert assessors, i.e., our gold standard, were familiar with the OSA-LS assessment scale, but the chief physicians and the residents were not. All participants were blinded to the fact that surgeons with different experience had performed the salpingectomies. RESULTS: No significant differences between the residents and chief physicians were observed in any of the three assessed operations: novice, p = 0.63; intermediate, p = 0.93; and expert, p = 0.93. The chief physicians and residents matched our gold standard in assessing the intermediate operation (p = 0.177), but not the novice operation (p = 0.005) or the expert operation (p = 0.001). CONCLUSIONS: Residents and chief physicians generated similar performance scores when assessing operations using a laparoscopic procedure-specific assessment scale, and they could distinguish performance levels between the surgeons. They matched the assessment score of our expert on the intermediate operation. We conclude that a procedure-specific assessment scale can be used by both residents and chief physicians when giving formative feedback.

The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials.

BACKGROUND: Virtual reality (VR) simulators for surgical training might possess the properties needed for basic training in laparoscopy. Evidence for training efficacy of VR has been investigated by research of varying quality over the past decade. OBJECTIVE: To review randomized controlled trials regarding VR training efficacy compared with traditional or no training, with outcome measured as surgical performance in humans or animals. DATA SOURCES: In June 2011 Medline, Embase, the Cochrane Central Register of Controlled Trials, Web of Science and Google Scholar were searched using the following medical subject headings (MeSh) terms: Laparoscopy/standards, Computing methodologies, Programmed instruction, Surgical procedures, Operative, and the following free text terms: Virtual real* OR simulat* AND Laparoscop* OR train* LIMITS: Controlled trials. STUDY ELIGIBILITY CRITERIA: All randomized controlled trials investigating the effect of VR training in laparoscopy, with outcome measured as surgical performance. METHODS: A total of 98 studies were screened, 26 selected and 12 included, with a total of 241 participants. RESULTS: Operation time was reduced by 17-50% by VR training, depending on simulator type and training principles. Proficiency-based training appeared superior to training based on fixed time or fixed numbers of repetition. Simulators offering training for complete operative procedures came out as more efficient than simulators offering only basic skills training. CONCLUSIONS: Skills in laparoscopic surgery can be increased by proficiency-based procedural VR simulator training. There is substantial evidence (grade IA - IIB) to support the use of VR simulators in laparoscopic training.

Vacuum extraction: development and test of a procedure-specific rating scale.

OBJECTIVES: To develop and validate an Objective Structured Assessment of Technical Skills (OSATS) scale for vacuum extraction. DESIGN: Two-part study design: Primarily, development of a procedure-specific checklist for vacuum extraction. Hereafter, validation of the developed OSATS scale for vacuum extraction in a prospective observational study. SETTING: Rigshospitalet, University Hospital of Copenhagen. POPULATION: For development, an obstetric expert from each labor ward in Denmark (28 departments) was invited to participate. For validation, nine first-year residents and 10 chief physicians with daily work in the obstetric field were tested. METHODS: The Delphi method was used for development of the scale. In a simulated vacuum extraction scenario, first-year residents and obstetric chief physicians were rated using the developed OSATS scale for vacuum extraction to test construct validity of the scale. MAIN OUTCOME MEASURES: Consensus for the content of the scale. To test the scale of Cronbach's alpha, interclass correlation and differential item function was calculated in the prospective study. RESULTS: 89% completed the first and 61% completed the second Delphi round. Hereafter, consensus was obtained. There was a significant difference between residents' and experts' performance for total score and for the score of the separate parts of the scale. Cronbach's alpha for total score and for the separate parts of the scale was 0.91-0.95 and interclass correlation 0.84-0.9. CONCLUSIONS: The OSATS scale for vacuum extraction is a reliable test for differentiating between competence levels in a simulated setting.

Instructor feedback versus no instructor feedback on performance in a laparoscopic virtual reality simulator: a randomized educational trial.

BACKGROUND: Several studies have found a positive effect on the learning curve as well as the improvement of basic psychomotor skills in the operating room after virtual reality training. Despite this, the majority of surgical and gynecological departments encounter hurdles when implementing this form of training. This is mainly due to lack of knowledge concerning the time and human resources needed to train novice surgeons to an adequate level. The purpose of this trial is to investigate the impact of instructor feedback regarding time, repetitions and self-perception when training complex operational tasks on a virtual reality simulator. METHODS/DESIGN: The study population consists of medical students on their 4th to 6th year without prior laparoscopic experience. The study is conducted in a skills laboratory at a centralized university hospital. Based on a sample size estimation 98 participants will be randomized to an intervention group or a control group. Both groups have to achieve a predefined proficiency level when conducting a laparoscopic salpingectomy using a surgical virtual reality simulator. The intervention group receives standardized instructor feedback of 10 to 12 min a maximum of three times. The control group receives no instructor feedback. Both groups receive the automated feedback generated by the virtual reality simulator. The study follows the CONSORT Statement for randomized trials. Main outcome measures are time and repetitions to reach the predefined proficiency level on the simulator. We include focus on potential sex differences, computer gaming experience and self-perception. DISCUSSION: The findings will contribute to a better understanding of optimal training methods in surgical education. TRIAL REGISTRATION: NCT01497782.

Retention of laparoscopic procedural skills acquired on a virtual-reality surgical trainer.

BACKGROUND: Virtual-reality (VR) simulator training has been shown to improve surgical performance in laparoscopic procedures in the operating room. We have, in a randomised controlled trial, demonstrated transferability to real operations. The validity of the LapSim virtual-reality simulator as an assessment tool has been demonstrated in several reports. However, an unanswered question regarding simulator training is the durability, or retention, of skills acquired during simulator training. The aim of the present study is to assess the retention of skills acquired using the LapSim VR simulator, 6 and 18 months after an initial training course. METHODS AND MATERIALS: The investigation was designed as a 6- and 18-month follow-up on a cohort of participants who earlier participated in a skills training programme on the LapSim VR. The follow-up cohort consisted of trainees and senior consultants allocated to two groups: (1) novices (experience < 5 procedures, n = 9) and (2) experts (experience > 200 procedures during the past 3 years, n = 10). Each participant performed ten sessions. Assessment of skills was based on time, economy of movement and the error parameter "bleeding". The novice group were re-tested after 6 and 18 months, whereas the expert group were only retested once, after 6 months. None of the novices performed laparoscopic surgery in the follow-up period. The experts continued their daily work with laparoscopic surgery. RESULTS AND CONCLUSIONS: Novices showed retention of skills after 6 months. After 18 months, novices' laparoscopic skills had returned to the pre-training level. This indicates that laparoscopic skills seemed to deteriorate in the period between 6 and 18 months without training. Experts showed consistent performance over time. This information can be included when planning training curricula in minimal invasive surgery.