Validity and reliability evidence support task-specific metrics for laparoscopic fundoplication.

BACKGROUND: Laparoscopic hiatal hernia repair (LHHR) is a complex operation requiring advanced surgical training. Surgical simulation offers a potential solution for learning complex operations without the need for high surgical volume. Our goal is to develop a virtual reality (VR) simulator for LHHR; however, data supporting task-specific metrics for this procedure are lacking. The purpose of this study was to develop and assess validity and reliability evidence of task-specific metrics for the fundoplication phase of LHHR. METHODS: In phase I, structured interviews with expert foregut surgeons were conducted to develop task-specific metrics (TSM). In phase II, participants with varying levels of surgical expertise performed a laparoscopic Nissen fundoplication procedure on a porcine stomach explant. Video recordings were independently assessed by two blinded graders using global and TSM. An intraclass correlation coefficient (ICC) was used to assess interrater reliability (IRR). Performance scores were compared using a Kruskal-Wallis test. Spearman's rank correlation was used to evaluate the association between global and TSM. RESULTS: Phase I of the study consisted of 12 interviews with expert foregut surgeons. Phase II engaged 31 surgery residents, a fellow, and 6 attendings in the simulation. Phase II results showed high IRR for both global (ICC = 0.84, p < 0.001) and TSM (ICC = 0.75, p < 0.001). Significant between-group differences were detected for both global (χ2 = 24.01, p < 0.001) and TSM (χ2 = 18.4, p < 0.001). Post hoc analysis showed significant differences in performance between the three groups for both metrics (p < 0.05). There was a strong positive correlation between the global and TSM (rs = 0.86, p < 0.001). CONCLUSION: We developed task-specific metrics for LHHR and using a fundoplication model, we documented significant reliability and validity evidence. We anticipate that these LHHR task-specific metrics will be useful in our planned VR simulator.

Automated segmentation of phases, steps, and tasks in laparoscopic cholecystectomy using deep learning.

BACKGROUND: Video-based review is paramount for operative performance assessment but can be laborious when performed manually. Hierarchical Task Analysis (HTA) is a well-known method that divides any procedure into phases, steps, and tasks. HTA requires large datasets of videos with consistent definitions at each level. Our aim was to develop an AI model for automated segmentation of phases, steps, and tasks for laparoscopic cholecystectomy videos using a standardized HTA. METHODS: A total of 160 laparoscopic cholecystectomy videos were collected from a publicly available dataset known as cholec80 and from our own institution. All videos were annotated for the beginning and ending of a predefined set of phases, steps, and tasks. Deep learning models were then separately developed and trained for the three levels using a 3D Convolutional Neural Network architecture. RESULTS: Four phases, eight steps, and nineteen tasks were defined through expert consensus. The training set for our deep learning models contained 100 videos with an additional 20 videos for hyperparameter optimization and tuning. The remaining 40 videos were used for testing the performance. The overall accuracy for phases, steps, and tasks were 0.90, 0.81, and 0.65 with the average F1 score of 0.86, 0.76 and 0.48 respectively. Control of bleeding and bile spillage tasks were most variable in definition, operative management, and clinical relevance. CONCLUSION: The use of hierarchical task analysis for surgical video analysis has numerous applications in AI-based automated systems. Our results show that our tiered method of task analysis can successfully be used to train a DL model.

The development of a deep learning model for automated segmentation of the robotic pancreaticojejunostomy.

BACKGROUND: Minimally invasive surgery provides an unprecedented opportunity to review video for assessing surgical performance. Surgical video analysis is time-consuming and expensive. Deep learning provides an alternative for analysis. Robotic pancreaticoduodenectomy (RPD) is a complex and morbid operation. Surgeon technical performance of pancreaticojejunostomy (PJ) has been associated with postoperative pancreatic fistula. In this work, we aimed to utilize deep learning to automatically segment PJ RPD videos. METHODS: This was a retrospective review of prospectively collected videos from 2011 to 2022 that were in libraries at tertiary referral centers, including 111 PJ videos. Each frame of a robotic PJ video was categorized based on 6 tasks. A 3D convolutional neural network was trained for frame-level visual feature extraction and classification. All the videos were manually annotated for the start and end of each task. RESULTS: Of the 100 videos assessed, 60 videos were used for the training the model, 10 for hyperparameter optimization, and 30 for the testing of performance. All the frames were extracted (6 frames/second) and annotated. The accuracy and mean per-class F1 scores were 88.01% and 85.34% for tasks. CONCLUSION: The deep learning model performed well for automated segmentation of PJ videos. Future work will focus on skills assessment and outcome prediction.

Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching.

BACKGROUND: Early introduction and distributed learning have been shown to improve student comfort with basic requisite suturing skills. The need for more frequent and directed feedback, however, remains an enduring concern for both remote and in-person training. A previous in-person curriculum for our second-year medical students transitioning to clerkships was adapted to an at-home video-based assessment model due to the social distancing implications of COVID-19. We aimed to develop an Artificial Intelligence (AI) model to perform video-based assessment. METHODS: Second-year medical students were asked to submit a video of a simple interrupted knot on a penrose drain with instrument tying technique after self-training to proficiency. Proficiency was defined as performing the task under two minutes with no critical errors. All the videos were first manually rated with a pass-fail rating and then subsequently underwent task segmentation. We developed and trained two AI models based on convolutional neural networks to identify errors (instrument holding and knot-tying) and provide automated ratings. RESULTS: A total of 229 medical student videos were reviewed (150 pass, 79 fail). Of those who failed, the critical error distribution was 15 knot-tying, 47 instrument-holding, and 17 multiple. A total of 216 videos were used to train the models after excluding the low-quality videos. A k-fold cross-validation (k = 10) was used. The accuracy of the instrument holding model was 89% with an F-1 score of 74%. For the knot-tying model, the accuracy was 91% with an F-1 score of 54%. CONCLUSIONS: Medical students require assessment and directed feedback to better acquire surgical skill, but this is often time-consuming and inadequately done. AI techniques can instead be employed to perform automated surgical video analysis. Future work will optimize the current model to identify discrete errors in order to supplement video-based rating with specific feedback.

Baseline performance in a robotic virtual reality platform predicts rate of skill acquisition in a proficiency-based curriculum: a cohort study of surgical trainees.

BACKGROUND: Residency programs must prepare to train the next generation of surgeons on the robotic platform. The purpose of this study was to determine if baseline skills of residents on a virtual reality (VR) robotic simulator before intern year predicted future performance in a proficiency-based curriculum. METHODS: Across two academic years, 21 general surgery PGY-1s underwent the robotic surgery boot camp at the University of Texas Southwestern. During boot camp, subjects completed five previously validated VR tasks, and their performance metrics (score, time, and economy of motion [EOM]) were extracted retrospectively from their Intuitive learning accounts. The same metrics were assessed during their residency until they reached previously validated proficiency benchmarks. Outcomes were defined as the score at proficiency, attempts to reach proficiency, and time to proficiency. Spearman's rho and Mann-Whitney U tests were used; median (IQR) was reported. Significance level was set at p < 0.05. RESULTS: Twenty-one residents completed at least three out of the five boot camp tasks and achieved proficiency in the former during residency. The median average score at boot camp was 12.3 (IQR: 5.14-18.5). The median average EOM at boot camp was 599.58 cm (IQR: 529.64-676.60). The average score at boot camp significantly correlated with lower time to achieve proficiency (p < 0.05). EOM at boot camp showed a significant correlation with attempts to proficiency and time to proficiency (p < 0.01). Residents with an average baseline EOM below the median showed a significant difference in attempts to proficiency (p < 0.05) and time to proficiency (p < 0.05) compared to those with EOMs above or equal to the median. CONCLUSION: Residents with an innate ability to perform tasks with better EOM may acquire robotic surgery skills faster. Future investigators could explore how these innate differences impact performance throughout residency.

Objective metrics for hand-sewn bowel anastomoses can differentiate novice from expert surgeons.

BACKGROUND: Assessing performance automatically in a virtual reality trainer or from recorded videos is advantageous but needs validated objective metrics. The purpose of this study is to obtain expert consensus and validate task-specific metrics developed for assessing performance in double-layered end-to-end anastomosis. MATERIALS AND METHODS: Subjects were recruited into expert (PGY 4-5, colorectal surgery residents, and attendings) and novice (PGY 1-3) groups. Weighted average scores of experts for each metric item, completion time, and the total scores computed using global and task-specific metrics were computed for assessment. RESULTS: A total of 43 expert surgeons rated our task-specific metric items with weighted averages ranging from 3.33 to 4.5 on a 5-point Likert scale. A total of 20 subjects (10 novices and 10 experts) participated in validation study. The novice group completed the task significantly more slowly than the experienced group (37.67 ± 7.09 vs 25.47 ± 7.82 min, p = 0.001). In addition, both the global rating scale (23.47 ± 4.28 vs 28.3 ± 3.85, p = 0.016) and the task-specific metrics showed a significant difference in performance between the two groups (38.77 ± 2.83 vs 42.58 ± 4.56 p = 0.027) following partial least-squares (PLS) regression. Furthermore, PLS regression showed that only two metric items (Stay suture tension and Tool handling) could reliably differentiate the performance between the groups (20.41 ± 2.42 vs 24.28 ± 4.09 vs, p = 0.037). CONCLUSIONS: Our study shows that our task-specific metrics have significant discriminant validity and can be used to evaluate the technical skills for this procedure.

What are essential laparoscopic skills these days? Results of the SAGES Fundamentals of Laparoscopic Surgery (FLS) Committee technical skills survey.

INTRODUCTION: The Fundamentals of Laparoscopic Surgery (FLS) program tests basic knowledge and skills required to perform laparoscopic surgery. Educational experiences in laparoscopic training and development of associated competencies have evolved since FLS inception, making it important to review the definition of fundamental laparoscopic skills. The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) assigned an FLS Technical Skills Working Group to characterize technical skills used in basic laparoscopic surgery in current practice contexts and their possible application to future FLS tests. METHODS: A group of subject matter experts defined an inventory of 65 laparoscopic skills using a Nominal Group Technique. From these, a survey was developed rating these items for importance, frequency of use, and priority for testing for FLS certification. This survey was distributed to SAGES members, recent recipients of FLS certification, and members of the Association of Program Directors in Surgery (APDS). Results were collected using a secure web-based survey platform. RESULTS: Complete data were available for 1742 surveys. Of these, 1143 comprised results for post-residency participants who performed advanced procedures. Seventeen competencies were identified for FLS testing prioritization by determining the proportion of respondents who identified them of highest priority, at median (50th percentile) of the maximum survey scale rating. These included basic peritoneal access, laparoscope and instrument use, tissue manipulation, and specific problem management skills. Sixteen could be used to show appropriateness of the domain construct by confirmatory factor analysis. Of these 8 could be characterized as manipulative tasks. Of these 5 mapped to current FLS tasks. CONCLUSIONS: This survey-identified competencies, some of which are currently assessed in FLS, with a high level of priority for testing. Further work is needed to determine if this should prompt consideration of changes or additions to the FLS technical skills test component.

Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy During Laparoscopic Cholecystectomy.

OBJECTIVE: The aim of this study was to develop and evaluate the performance of artificial intelligence (AI) models that can identify safe and dangerous zones of dissection, and anatomical landmarks during laparoscopic cholecystectomy (LC). SUMMARY BACKGROUND DATA: Many adverse events during surgery occur due to errors in visual perception and judgment leading to misinterpretation of anatomy. Deep learning, a subfield of AI, can potentially be used to provide real-time guidance intraoperatively. METHODS: Deep learning models were developed and trained to identify safe (Go) and dangerous (No-Go) zones of dissection, liver, gallbladder, and hepatocystic triangle during LC. Annotations were performed by 4 high-volume surgeons. AI predictions were evaluated using 10-fold cross-validation against annotations by expert surgeons. Primary outcomes were intersection- over-union (IOU) and F1 score (validated spatial correlation indices), and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, ± standard deviation. RESULTS: AI models were trained on 2627 random frames from 290 LC videos, procured from 37 countries, 136 institutions, and 153 surgeons. Mean IOU, F1 score, accuracy, sensitivity, and specificity for the AI to identify Go zones were 0.53 (±0.24), 0.70 (±0.28), 0.94 (±0.05), 0.69 (±0.20). and 0.94 (±0.03), respectively. For No-Go zones, these metrics were 0.71 (±0.29), 0.83 (±0.31), 0.95 (±0.06), 0.80 (±0.21), and 0.98 (±0.05), respectively. Mean IOU for identification of the liver, gallbladder, and hepatocystic triangle were: 0.86 (±0.12), 0.72 (±0.19), and 0.65 (±0.22), respectively. CONCLUSIONS: AI can be used to identify anatomy within the surgical field. This technology may eventually be used to provide real-time guidance and minimize the risk of adverse events.

The Operating Room Black Box: Understanding Adherence to Surgical Checklists.

OBJECTIVE: We report for the first time the use of the Operating Room Black Box (ORBB) to track checklist compliance, engagement, and quality. BACKGROUND: Implementation of operative checklists is associated with improved outcomes. Compliance is difficult to monitor. Most studies report either no assessment of checklist compliance or deployed in-person short-term assessment. The ORBB a novel artificially intelligence-driven data analytic platform affords the opportunity to assess checklist compliance without disrupting surgical workflow. METHODS: This was a retrospective review of prospectively collected ORBB data. Operative cases included elective surgery at a quaternary referral center. Cases were analyzed as prepolicy change (first 9 months) or as a postpolicy change (last 9 months). Measures of checklist compliance, engagement, and quality were assessed. RESULTS: There were 3879 cases that were performed and monitored for checklist compliance between August 15, 2020, and February 20, 2022. The overall scores for compliance, engagement, and quality were 81%, 84%, and 67% respectively. When broken down by phase, the scores for time-out were compliance 100%, engagement 98%, and quality 61%. Scores for the debrief phase were 81% for compliance, 98% for engagement, and 66% for quality. After a hospital policy change, the debrief scores improved significantly (85%; P <0.001 for compliance, 88%; P <0.001 for engagement and 71%; P <0.001 for quality). CONCLUSIONS: ORBB provides the unprecedented ability to assess not only compliance with surgical safety checklists but also engagement and quality. Utilization of this technology allows the assessment of compliance in near real time and to accurately address safety threats that may arise from noncompliance.

A contextual detector of surgical tools in laparoscopic videos using deep learning.

BACKGROUND: The complexity of laparoscopy requires special training and assessment. Analyzing the streaming videos during the surgery can potentially improve surgical education. The tedium and cost of such an analysis can be dramatically reduced using an automated tool detection system, among other things. We propose a new multilabel classifier, called LapTool-Net to detect the presence of surgical tools in each frame of a laparoscopic video. METHODS: The novelty of LapTool-Net is the exploitation of the correlations among the usage of different tools and, the tools and tasks-i.e., the context of the tools' usage. Towards this goal, the pattern in the co-occurrence of the tools is utilized for designing a decision policy for the multilabel classifier based on a Recurrent Convolutional Neural Network (RCNN), which is trained in an end-to-end manner. In the post-processing step, the predictions are corrected by modeling the long-term tasks' order with an RNN. RESULTS: LapTool-Net was trained using publicly available datasets of laparoscopic cholecystectomy, viz., M2CAI16 and Cholec80. For M2CAI16, our exact match accuracies (when all the tools in one frame are predicted correctly) in online and offline modes were 80.95% and 81.84% with per-class F1-score of 88.29% and 90.53%. For Cholec80, the accuracies were 85.77% and 91.92% with F1-scores if 93.10% and 96.11% for online and offline, respectively. CONCLUSIONS: The results show LapTool-Net outperformed state-of-the-art methods significantly, even while using fewer training samples and a shallower architecture. Our context-aware model does not require expert's domain-specific knowledge, and the simple architecture can potentially improve all existing methods.

Feasibility, effectiveness and transferability of a novel mastery-based virtual reality robotic training platform for general surgery residents.

BACKGROUND: The annual number of robotic surgical procedures is on the rise. Robotic surgery requires unique skills compared to other surgical approaches. Simulation allows basic robot skill acquisition and enhances patient safety. The purpose of this study was to evaluate the feasibility, effectiveness, and transferability of a mastery-based curriculum using a new virtual reality (VR) robotic simulator for surgery resident training. METHODS: Nineteen PGY2s and 22 PGY4s were enrolled. Residents completed a pretest and posttest consisting of five VR and three previously validated inanimate tasks. Training included practicing 33 VR tasks until a total score ≥ 90% ("mastery") was achieved using automated metrics (time, economy of motion). Inanimate performance was evaluated by two trained, blinded raters using video review metrics (time, errors, and modified OSATS). Outcomes were defined as: curriculum feasibility (completion rate, training time, repetitions), training effectiveness (pre/post training skill improvement), and skill transferability (skill transfer to validated inanimate drills). Wilcoxon signed-rank and Mann-Whitney U tests were used; median (IQR) reported. RESULTS: Thirty-four of 41 residents (83%) achieved mastery on all 33 VR tasks; median training time was 7 h (IQR: 5'26″-8'52″). Pretest vs. post-test performance improved (all p < 0.001) according to all VR and Inanimate metrics for both PGY2 and PGY4 residents. Significant pretest performance differences were observed between PGY2 and PGY4 residents for VR but not inanimate tasks; no PGY2 vs. PGY4 posttest performance differences were observed for both VR and inanimate tasks. CONCLUSION: This mastery-based VR curriculum was associated with a high completion rate and excellent feasibility. Significant performance improvements were noted for both the VR and inanimate tasks, supporting training effectiveness and skill transferability. Additional studies examining validity evidence may help further refine this curriculum.

Validation of the VBLaST pattern cutting task: a learning curve study.

BACKGROUND: Mastery of laparoscopic skills is essential in surgical practice and requires considerable time and effort to achieve. The Virtual Basic Laparoscopic Skill Trainer (VBLaST-PC©) is a virtual simulator that was developed as a computerized version of the pattern cutting (PC) task in the Fundamentals of Laparoscopic Surgery (FLS) system. To establish convergent validity for the VBLaST-PC©, we assessed trainees' learning curves using the cumulative summation (CUSUM) method and compared them with those on the FLS. METHODS: Twenty-four medical students were randomly assigned to an FLS training group, a VBLaST training group, or a control group. Fifteen training sessions, 30 min in duration per session per day, were conducted over 3 weeks. All subjects completed pretest, posttest, and retention test (2 weeks after posttest) on both the FLS and VBLaST© simulators. Performance data, including time, error, FLS score, learning rate, learning plateau, and CUSUM score, were analyzed. RESULTS: The learning curve for all trained subjects demonstrated increasing performance and a performance plateau. CUSUM analyses showed that five of the seven subjects reached the intermediate proficiency level but none reached the expert proficiency level after 150 practice trials. Performance was significantly improved after simulation training, but only in the assigned simulator. No significant decay of skills after 2 weeks of disuse was observed. Control subjects did not show any learning on the FLS simulator, but improved continually in the VBLaST simulator. CONCLUSIONS: Although VBLaST©- and FLS-trained subjects demonstrated similar learning rates and plateaus, the majority of subjects required more than 150 trials to achieve proficiency. Trained subjects demonstrated improved performance in only the assigned simulator, indicating specificity of training. The virtual simulator may provide better opportunities for learning, especially with limited training exposure.

Immersive virtual reality-based training improves response in a simulated operating room fire scenario.

BACKGROUND: SAGES FUSE curriculum provides didactic knowledge on OR fire prevention. The objective of this study is to evaluate the impact of an immersive virtual reality (VR)-based OR fire training simulation system in combination with FUSE didactics. METHODS: The study compared a control with a simulation group. After a pre-test questionnaire that assessed the baseline knowledge, both groups were given didactic material that consists of a 10-min presentation and reading materials about precautions and stopping an OR fire from the FUSE manual. The simulation group practiced on the OR fire simulation for one session that consisted of five trials within a week from the pre-test. One week later, both groups were reassessed using a questionnaire. A week after the post-test both groups also participated in a simulated OR fire scenario while their performance was videotaped for assessment. RESULTS: A total of 20 subjects (ten per group) participated in this IRB approved study. Median test scores for the control group increased from 5.5 to 9.00 (p = 0.011) and for the simulation group it increased from 5.0 to 8.5 (p = 0.005). Both groups started at the same baseline (pre-test, p = 0.529) and reached similar level in cognitive knowledge (post-test, p = 0.853). However, when tested in the mock OR fire scenario, 70% of the simulation group subjects were able to perform the correct sequence of steps in extinguishing the simulated fire whereas only 20% subjects in the control group were able to do so (p = 0.003). The simulation group was better than control group in correctly identifying the oxidizer (p = 0.03) and ignition source (p = 0.014). CONCLUSIONS: Interactive VR-based hands-on training was found to be a relatively inexpensive and effective mode for teaching OR fire prevention and management scenarios.

Learning Curve Associated With an Automated Laparoscopic Suturing Device Compared With Laparoscopic Suturing.

BACKGROUND: Laparoscopic suturing has proved to be a challenging skill to master which may prevent surgical procedures from being started, or completed, in a minimally invasive fashion. The aim of this study is to compare the learning curves between traditional laparoscopic techniques with a novel suturing device. METHODS: In this prospective single blinded nonrandomized controlled crossover study, we recruited 19 general surgery residents ranging from beginner (PGY1-2, n = 12) to advanced beginner (PGY3-5, n = 7). They were assigned to perform a knot tying and suturing task using either Endo360 or traditional laparoscopic technique (TLT) with needle holders before crossing over to the other method. The proficiency standards were developed by collecting the data for task completion time (TCT in seconds), dots on target (DoT in numbers), and total deviation (D in mm) on 5 expert attending surgeons (mean ± 2SD). The test subjects were "proficient" when they reached these standards 2 consecutive times. RESULTS: Number of attempts to complete the task was collected for Endo360 and TLT. A significant difference was observed between mean number of attempts to reach proficiency for Endo360 versus TLT ( P = .0027) in both groups combined, but this was not statistically significant in the advanced beginner group. TCT was examined for both methods and demonstrated significantly less time to complete the task for Endo360 versus TLT ( P < .0001). There were significantly less DoT for Endo360 as compared with TLT ( P < .0001), which was also associated with significantly less D ( P < .0001) indicating lower accuracy with Endo360. However, no significant difference was observed between the groups for increasing number of trials for both DoT and D. CONCLUSIONS: This novel suturing device showed a shorter learning curve with regard to number of attempts to complete a task for the beginner group in our study, but matched the learning curve in the advanced beginner group. With regard to time to complete the task, the device was faster in both groups.

Graphic and haptic simulation for transvaginal cholecystectomy training in NOTES.

BACKGROUND: Natural Orifice Transluminal Endoscopic Surgery (NOTES) provides an emerging surgical technique which usually needs a long learning curve for surgeons. Virtual reality (VR) medical simulators with vision and haptic feedback can usually offer an efficient and cost-effective alternative without risk to the traditional training approaches. Under this motivation, we developed the first virtual reality simulator for transvaginal cholecystectomy in NOTES (VTEST™). METHODS: This VR-based surgical simulator aims to simulate the hybrid NOTES of cholecystectomy. We use a 6DOF haptic device and a tracking sensor to construct the core hardware component of simulator. For software, an innovative approach based on the inner-spheres is presented to deform the organs in real time. To handle the frequent collision between soft tissue and surgical instruments, an adaptive collision detection method based on GPU is designed and implemented. To give a realistic visual performance of gallbladder fat tissue removal by cautery hook, a multi-layer hexahedral model is presented to simulate the electric dissection of fat tissue. RESULTS: From the experimental results, trainees can operate in real time with high degree of stability and fidelity. A preliminary study was also performed to evaluate the realism and the usefulness of this hybrid NOTES simulator. CONCLUSIONS: This prototyped simulation system has been verified by surgeons through a pilot study. Some items of its visual performance and the utility were rated fairly high by the participants during testing. It exhibits the potential to improve the surgical skills of trainee and effectively shorten their learning curve.

Natural orifice translumenal endoscopic surgery (NOTES): emerging trends and specifications for a virtual simulator.

INTRODUCTION AND STUDY AIM: A virtual translumenal endoscopic surgical trainer (VTEST) is being developed to accelerate the development of natural orifice translumenal endoscopic surgery (NOTES) procedures and devices in a safe and risk-free environment. For a rapidly developing field such as NOTES, a needs analysis must be conducted regularly to discover emerging research trends and areas of potential high impact for a virtual simulator. This paper presents a survey-based study which follows a similar study conducted by this group in 2011 (Sankaranarayanan et al. in Surg Endosc 27:1607-1616, 2013). METHODS: A 32-point questionnaire was distributed at the 2012 Natural Orifice Surgery Consortium for Assessment and Research annual meeting. These data were subsequently augmented by an identical online survey, targeted at the members of the American Society for Gastrointestinal Endoscopy and the Society of American Gastrointestinal and Endoscopic Surgeons, and analyzed. RESULTS: Twenty-eight NOTES experts participated in the 2012 study. Cholecystectomy (CE) procedure remained the most commonly performed NOTES technique, with 18 positive responses (64%). In contrast to 2011, the popularity of the NOTES appendectomy (AE) was significantly lower, with only 2 (7%) instances (CE vs. AE, p < 0.001), while the number of peroral endoscopic myotomy (POEM, PE) cases had increased significantly, with 11 (39%) positive responses, respectively (PE vs. AE, p = 0.013). Strong preference toward hybrid rather than pure NOTES techniques (82 vs. 11%, p < 0.001) was also expressed. Other responses were similar to those in the 2011 study, with the VTEST™ utility in developing and testing new techniques and instruments ranked particularly high. CONCLUSION: Based on the results of this study, a decision was made to focus exclusively on the transvaginal hybrid NOTES cholecystectomy procedure, including both rigid and flexible scope techniques. The importance of developing a virtual NOTES simulator was reaffirmed, with POEM identified as a promising candidate for future simulator development.

Face validation of the Virtual Electrosurgery Skill Trainer (VEST©).

BACKGROUND: Electrosurgery is a modality that is widely used in surgery, whose use has resulted in injuries, OR fires and even death. The SAGES has established the FUSE program to address the knowledge gap in the proper and safe usage of electrosurgical devices. Complementing it, we have developed the Virtual Electrosurgery Skill Trainer (VEST(©)), which is designed to train subjects in both cognitive and motor skills necessary to safely operate electrosurgical devices. The objective of this study is to asses the face validity of the VEST(©) simulator. METHODS: Sixty-three subjects were recruited at the 2014 SAGES Learning Center. They all completed the monopolar electrosurgery module on the VEST(©) simulator. At the end of the study, subjects assessed the face validity with questions that were scored on a 5-point Likert scale. RESULTS: The subjects were divided into two groups; FUSE experience (n = 15) and no FUSE experience (n = 48). The median score for both the groups was 4 or higher on all questions and 5 on questions on effectiveness of VEST(©) in aiding learning electrosurgery fundamentals. Questions on using the simulator in their own skills lab and recommending it to their peers also scored at 5. Mann-Whitney U test showed no significant difference (p > 0.05) indicating a general agreement. 46% of the respondents preferred VEST compared with 52% who preferred animal model and 2% preferred both for training in electrosurgery. CONCLUSION: This study demonstrated the face validity of the VEST(©) simulator. High scores showed that the simulator was visually realistic and reproduced lifelike tissue effects and the features were adequate enough to provide high realism. The self-learning instructional material was also found to be very useful in learning the fundamentals of electrosurgery. Adding more modules would increase the applicability of the VEST(©) simulator.

Face and construct validation of a next generation virtual reality (Gen2-VR) surgical simulator.

INTRODUCTION: Surgical performance is affected by distractors and interruptions to surgical workflow that exist in the operating room. However, traditional surgical simulators are used to train surgeons in a skills laboratory that does not recreate these conditions. To overcome this limitation, we have developed a novel, immersive virtual reality (Gen2-VR) system to train surgeons in these environments. This study was to establish face and construct validity of our system. METHODS AND PROCEDURES: The study was a within-subjects design, with subjects repeating a virtual peg transfer task under three different conditions: Case I: traditional VR; Case II: Gen2-VR with no distractions and Case III: Gen2-VR with distractions and interruptions. In Case III, to simulate the effects of distractions and interruptions, music was played intermittently, the camera lens was fogged for 10 s and tools malfunctioned for 15 s at random points in time during the simulation. At the completion of the study subjects filled in a 5-point Likert scale feedback questionnaire. A total of sixteen subjects participated in this study. RESULTS: Friedman test showed significant difference in scores between the three conditions (p < 0.0001). Post hoc analysis using Wilcoxon signed-rank tests with Bonferroni correction further showed that all the three conditions were significantly different from each other (Case I, Case II, p < 0.0001), (Case I, Case III, p < 0.0001) and (Case II, Case III, p = 0.009). Subjects rated that fog (mean 4.18) and tool malfunction (median 4.56) significantly hindered their performance. CONCLUSION: The results showed that Gen2-VR simulator has both face and construct validity and that it can accurately and realistically present distractions and interruptions in a simulated OR, in spite of limitations of the current HMD hardware technology.

Face and content validation of a Virtual Translumenal Endoscopic Surgery Trainer (VTEST™).

BACKGROUND: Natural orifice translumenal endoscopic surgery (NOTES) is an emerging surgical paradigm, where peritoneal access is achieved through one of the natural orifices of the body. It is being reported as a safe and feasible surgical technique with significantly reduced external scarring. Virtual Translumenal Endoscopic Surgical Trainer (VTEST™) is the first virtual reality simulator for the NOTES. The VTEST™ simulator was developed to train surgeons in the hybrid transvaginal NOTES cholecystectomy procedure. The initial version of the VTEST™ simulator underwent face validation at the 2013 Natural Orifice Surgery Consortium for Assessment and Research (NOSCAR) summit. Several areas of improvement were identified as a result, and the corresponding modifications were implemented in the simulator. This manuscript outlines the results of the subsequent evaluation study, performed in order to assess the face and content validity of the latest VTEST™ simulator. METHODS: Twelve subjects participated in an institutional review board-approved study that took place at the 2014 NOSCAR summit. Six of the 12 subjects, who are experts with NOTES experience, were used for face and content validation. The subjects performed the hybrid transvaginal NOTES cholecystectomy procedure on VTEST™ that included identifying the Calot's triangle, clipping and cutting the cystic duct/artery, and detaching the gallbladder. The subjects then answered five-point Likert scale feedback questionnaires for face and content validity. RESULTS: Overall, subjects rated 12/15 questions as 3.0 or greater (60 %), for face validity questions regarding the realism of the anatomical features, interface, and the tasks. Subjects also highly rated the usefulness of the simulator in learning the fundamental NOTES technical skills (3.50 ± 0.84). Content validity results indicate a high level of usefulness of the VTEST™ for training prior to operating room experience (4.17 ± 0.75).

Identifying Opportunities for Virtual Reality Simulation in Surgical Education: A Review of the Proceedings from the Innovation, Design, and Emerging Alliances in Surgery (IDEAS) Conference: VR Surgery.

OBJECTIVES: To conduct a review of the state of virtual reality (VR) simulation technology, to identify areas of surgical education that have the greatest potential to benefit from it, and to identify challenges to implementation. BACKGROUND DATA: Simulation is an increasingly important part of surgical training. VR is a developing platform for using simulation to teach technical skills, behavioral skills, and entire procedures to trainees and practicing surgeons worldwide. Questions exist regarding the science behind the technology and most effective usage of VR simulation. A symposium was held to address these issues. METHODS: Engineers, educators, and surgeons held a conference in November 2013 both to review the background science behind simulation technology and to create guidelines for its use in teaching and credentialing trainees and surgeons in practice. RESULTS: Several technologic challenges were identified that must be overcome in order for VR simulation to be useful in surgery. Specific areas of student, resident, and practicing surgeon training and testing that would likely benefit from VR were identified: technical skills, team training and decision-making skills, and patient safety, such as in use of electrosurgical equipment. CONCLUSIONS: VR simulation has the potential to become an essential piece of surgical education curriculum but depends heavily on the establishment of an agreed upon set of goals. Researchers and clinicians must collaborate to allocate funding toward projects that help achieve these goals. The recommendations outlined here should guide further study and implementation of VR simulation.