Developing artificial intelligence models for medical student suturing and knot-tying video-based assessment and coaching.

BACKGROUND: Early introduction and distributed learning have been shown to improve student comfort with basic requisite suturing skills. The need for more frequent and directed feedback, however, remains an enduring concern for both remote and in-person training. A previous in-person curriculum for our second-year medical students transitioning to clerkships was adapted to an at-home video-based assessment model due to the social distancing implications of COVID-19. We aimed to develop an Artificial Intelligence (AI) model to perform video-based assessment. METHODS: Second-year medical students were asked to submit a video of a simple interrupted knot on a penrose drain with instrument tying technique after self-training to proficiency. Proficiency was defined as performing the task under two minutes with no critical errors. All the videos were first manually rated with a pass-fail rating and then subsequently underwent task segmentation. We developed and trained two AI models based on convolutional neural networks to identify errors (instrument holding and knot-tying) and provide automated ratings. RESULTS: A total of 229 medical student videos were reviewed (150 pass, 79 fail). Of those who failed, the critical error distribution was 15 knot-tying, 47 instrument-holding, and 17 multiple. A total of 216 videos were used to train the models after excluding the low-quality videos. A k-fold cross-validation (k = 10) was used. The accuracy of the instrument holding model was 89% with an F-1 score of 74%. For the knot-tying model, the accuracy was 91% with an F-1 score of 54%. CONCLUSIONS: Medical students require assessment and directed feedback to better acquire surgical skill, but this is often time-consuming and inadequately done. AI techniques can instead be employed to perform automated surgical video analysis. Future work will optimize the current model to identify discrete errors in order to supplement video-based rating with specific feedback.

Development and Validation of Task-Specific Metrics for the Assessment of Linear Stapler-Based Small Bowel Anastomosis.

INTRODUCTION: Task-specific metrics facilitate the assessment of surgeon performance. This 3-phased study was designed to (1) develop task-specific metrics for stapled small bowel anastomosis, (2) obtain expert consensus on the appropriateness of the developed metrics, and (3) establish its discriminant validity. METHODS: In Phase I, a hierarchical task analysis was used to develop the metrics. In Phase II, a survey of expert colorectal surgeons established the importance of the developed metrics. In Phase III, to establish discriminant validity, surgical trainees and surgeons, divided into novice and experienced groups, constructed a side-to-side anastomosis on porcine small bowel using a linear cutting stapler. The participants' performances were videotaped and rated by 2 independent observers. Partial least squares regression was used to compute the weights for the task-specific metrics to obtain weighted total score. RESULTS: In Phase II, a total of 45 colorectal surgeons were surveyed: 28 with more than 15 years, 13 with 5 to 15 years, and 4 with less than 5 years of experience. The consensus was obtained on all the task-specific metrics in the more experienced groups. In Phase III, 20 subjects participated equally in both groups. The experienced group performed better than the novice group regardless of the rating scale used: global rating scale (p = 0.009) and the task-specific metrics (p = 0.012). After partial least squares regression, the weighted task-specific metric score continued to show that the experienced group performed better (p < 0.001). CONCLUSION: Task-specific metric items were developed based on expert consensus and showed good discriminant validity compared with a global rating scale between experienced and novice operators. These items can be used for evaluating technical skills in a stapled small bowel anastomosis model.

Needs analysis for developing a virtual-reality NOTES simulator.

UNLABELLED: INTRODUCTION AND STUDY AIM: Natural orifice translumenal endoscopic surgery (NOTES) is an emerging surgical technique that requires a cautious adoption approach to ensure patient safety. High-fidelity virtual-reality-based simulators allow development of new surgical procedures and tools and train medical personnel without risk to human patients. As part of a project funded by the National Institutes of Health, we are developing the virtual transluminal endoscopic surgery trainer (VTEST) for this purpose. The objective of this study is to conduct a structured needs analysis to identify the design parameters for such a virtual-reality-based simulator for NOTES. METHODS: A 30-point questionnaire was distributed at the 2011 National Orifice Surgery Consortium for Assessment and Research meeting to obtain responses from experts. Ordinal logistic regression and the Wilcoxon rank-sum test were used for analysis. RESULTS: A total of 22 NOTES experts participated in the study. Cholecystectomy (CE, 68 %) followed by appendectomy (AE, 63 %) (CE vs AE, p = 0.0521) was selected as the first choice for simulation. Flexible (FL, 47 %) and hybrid (HY, 47 %) approaches were equally favorable compared with rigid (RI, 6 %) with p < 0.001 for both FL versus RI and HY versus RI. The transvaginal approach was preferred 3 to 1 to the transgastric. Most participants preferred two-channel (2C) scopes (65 %) compared with single (1C) or three (3C) or more channels with p < 0.001 for both 2C versus 1C and 2C versus 3C. The importance of force feedback and the utility of a virtual NOTES simulator in training and testing new tools for NOTES were rated very high by the participants. CONCLUSION: Our study reinforces the importance of developing a virtual NOTES simulator and clearly presents expert preferences. The results of this analysis will direct our initial development of the VTEST platform.

ToolTrack™: a compact, low-cost system for measuring surgical tool motion.

Accurate measurement of tool motion in minimally invasive procedures is an essential component of proficiency assessment. This is particularly important to automate the scoring procedures in, e.g., the Fundamentals of Laparoscopic Surgery (FLS) trainer box. Existing techniques are bulky, require extensive modifications to the surgical tool or trocar mechanism and are expensive. In this work, we propose a new system - the ToolTrack™ consisting of an optical mouse sensor and a MEMS inertial unit that can be easily integrated with any laparoscopic tool, provide accurate and reliable sensing, is light weight and low cost.

Cost-efficient suturing simulation with pre-computed models.

Suturing is currently one of the most common procedures in minimally invasive surgery (MIS). We present a suturing simulation paradigm with pre-computed finite element models which include detailed needle-tissue and thread-tissue interaction. The interaction forces are derived through a reanalysis technique for haptic feedback. Besides providing deformation updates and high fidelity forces, our simulation is computationally less costly.

Objective assessment of telesurgical robot systems: Telerobotic FLS.

The Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Fundamentals of Laparoscopic Surgery (FLS) program contains curriculum that includes both a cognitive and psychomotor skills. In this research the use of FLS Block Transfer task is used to evaluate the performance of surgeons' teleoperating the University of Washington Surgical robot. The use of the FLS Trainer Box and accessories kit provides a well-defined series of tasks that can be repeated by any researchers working in the field of surgical robotics so that systems can be evaluated using a common method.