Development and prospective validation of a scoring system for the Basic Endoscopic Skills Training (BEST) box.

AIMS: The increased use of endoscopy as a minimally invasive therapeutic technique has created a great demand for endoscopic training. The Basic Endoscopic Skills Training (BEST) box provides a low-cost solution by adapting the Fundamentals of Laparoscopic Surgery (FLS) box for flexible endoscopic simulation. The BEST box consists of six endoscopic tasks with a 5-min time limit per task. This study aims to develop a scoring system for objective evaluation of user performance. METHODS: A total of 165 participants were tested on the BEST box. Participants were divided into two groups: retrospective analysis (n = 100) and prospective analysis (n = 65). From the retrospective group, 55 individuals were also scored on the Global Assessment of Gastrointestinal Endoscopic Skills-Upper Endoscopy (GAGES-UE). Linear regression between user performance on BEST box and GAGES-UE was performed to develop the scoring system. Receiver Operating Characteristic curve was used to determine a threshold score to help users appreciate their endoscopic expertise. Prospective scoring of 65 individuals was then performed using the formula developed (20 experts and 45 trainees). RESULTS: The minimum and maximum possible scores are 30 and 110, respectively. Retrospective analysis showed that the scoring system was able to distinguish between experts and trainees (p < 0.001), correlated with GAGES-UE (p < 0.001), and had a reliability constant of r = 0.765 (p < 0.001). On prospective testing using the scoring system the expert group received a final average score of 92, whereas the average score for the trainee group was 61 (p < 0.001). CONCLUSIONS: The developed BEST box scoring system correlates with the experience level of the test taker as well as with the GAGES-UE scoring system. The results of this study add further evidence to the validity of the BEST box as an effective, low-cost endoscopic simulator with the scores used by trainees to track their performance level overtime.

Correlation of surgical trainee performance on laparoscopic versus endoscopic simulation.

BACKGROUND: Both laparoscopy and endoscopy are image-based procedures, which are less intuitive than traditional open surgery and require extensive training to reach adequate proficiency. Currently, there is lack of understanding as to how the skills in one image-based procedure translate to another, such as endoscopy to laparoscopy and vice versa. The aim of our study was to explore the relationship between endoscopic and laparoscopic skills using a Fundamentals of Laparoscopic Surgery (FLS) trainer, a traditional virtual reality endoscopic trainer and a "desk-top" endoscopic physical simulator. METHODS: Senior surgical residents from across Canada participating in an advanced laparoscopic foregut training course were enrolled in the study. Participants were assessed performing the FLS laparoscopic suturing task, the Endobubble 2 task (Simbionix, GI Mentor), and a forward viewing peg transfer on the novel Basics in Endoscopic Skills Training Box (BEST Box). RESULTS: There was significant correlation between the participant's skill in simulated laparoscopic suturing and simulated endoscopic skill using the BEST box (Pearson coefficient (r) was 0.551 (p = 0.033) and the coefficient of determination (r2) was 0.304). There was a trend towards correlation between laparoscopic suturing time and Endobubble 2 score, but this did not reach statistical significance (r = 0.458, p = 0.086; r2 = 0.210). CONCLUSIONS: Performance in the two physical simulators, laparoscopic suturing and simulated flexible endoscopy using the BEST box, showed a correlation. This study adds to the growing body of evidence that laparoscopic and endoscopic skills are complementary and has the potential to impact simulation training involving both skill sets.

Adaptation of the fundamentals of laparoscopic surgery box for endoscopic simulation: performance evaluation of the first 100 participants.

BACKGROUND: The paucity of readily accessible, cost-effective models for the simulation, practice, and evaluation of endoscopic skills present an ongoing barrier for resident training. We have previously described a system for conversion of the Fundamentals of Laparoscopic Surgery box (FLS) for flexible endoscopic simulation. Six endoscopic tasks focusing on scope manipulation, and other clinically relevant endoscopic skills are performed within a 5-min time limit per task. This study describes our experience and validation results with the first 100 participants. METHODS: A total of 100 participants were evaluated on the simulator. Thirty individuals were classified as experts (having done over 200 endoscopic procedures), and 70 were classified as trainees (39 individuals reported having no prior endoscopy experience). Of the 100 participants, 55 individuals were retested on the simulator within a period of 4 months. These 55 individuals were also evaluated using the "Global Assessment of Gastrointestinal Endoscopic Skills" (GAGES). T-tests and Pearson correlations were used where appropriate, values less than 0.05 were considered significant. RESULTS: Experts completed all six tasks significantly faster than trainees. For the 55 participants who were retested on the simulator, all tasks demonstrated evidence of test-retest reliability for both experts and trainees who did not practice in between tests. Moderate correlations between lower completion times and higher GAGES scores were observed for all tasks except the clipping task. CONCLUSIONS: The results from the first 100 participants provide evidence for the simulator's validity. Based on task completion times, we found that experts perform significantly better than trainees. Additionally, preliminary data demonstrate evidence of test-retest reliability, as well as GAGES score correlation. Additional studies to determine and validate a scoring system for this simulator are ongoing.

Feasibility of adapting the fundamentals of laparoscopic surgery trainer box to endoscopic skills training tool.

BACKGROUND: The fundamentals of laparoscopic surgery (FLS) training box is a validated tool, already accessible to surgical trainees to hone their laparoscopic skills. We aim to investigate the feasibility of adapting the FLS box for the practice and assessment of endoscopic skills. This would allow for a highly available, reusable, low-cost, mechanical trainer. METHODS: The design and development process was based on a user-centered design, which is a combination of the design thinking method and cognitive task analysis. The process comprises four phases: empathy, cognitive, prototyping/adaptation, and end user testing. The underlying idea was to utilize as many of the existing components of FLS training to maintain simplicity and cost effectiveness while allowing for the practice of clinically relevant endoscopic skills. A sample size of 18 participants was calculated to be sufficient to detect performance differences between experts and trainees using a two tailed t test with alpha set at 0.05, standard deviation of 5.5, and a power of 80%. RESULTS: Adaptation to the FLS box included two fundamental attachments: a front panel with an insertion point for an endoscope and a shaft which provides additional support and limits movement of the scope. The panel also allows for mounting of retroflexion tasks. Six endoscopic tasks inspired by FLS were designed (two of which utilize existing FLS components). Pilot testing with 38 participants showed high user's satisfaction and demonstrated that the trainer was robust and reliable. Task performance times was able to discriminate between trainees and experts for all six tasks. CONCLUSIONS: A mechanical, reusable, low-cost adaptation of the FLS training box for endoscopic skills is feasible and has high user satisfaction. Preliminary testing shows that the simulator is able to discriminate between trainees and experts. Following further validation, this adaptation may act as a supplement to the FES program.