Automatic assessment of laparoscopic surgical skill competence based on motion metrics.

The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.

Motion analysis for better understanding of psychomotor skills in laparoscopy: objective assessment-based simulation training using animal organs.

BACKGROUND: Our aim was to characterize the motions of multiple laparoscopic surgical instruments among participants with different levels of surgical experience in a series of wet-lab training drills, in which participants need to perform a range of surgical procedures including grasping tissue, tissue traction and dissection, applying a Hem-o-lok clip, and suturing/knotting, and digitize the level of surgical competency. METHODS: Participants performed tissue dissection around the aorta, dividing encountered vessels after applying a Hem-o-lok (Task 1), and renal parenchymal closure (Task 2: suturing, Task 3: suturing and knot-tying), using swine cadaveric organs placed in a box trainer under a motion capture (Mocap) system. Motion-related metrics were compared according to participants' level of surgical experience (experts: 50 ≤ laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test, and significant metrics were subjected to principal component analysis (PCA). RESULTS: A total of 15 experts, 12 intermediates, and 18 novices participated in the training. In Task 1, a shorter path length and faster velocity/acceleration/jerk were observed using both scissors and a Hem-o-lok applier in the experts, and Hem-o-lok-related metrics markedly contributed to the 1st principal component on PCA analysis, followed by scissors-related metrics. Higher-level skills including a shorter path length and faster velocity were observed in both hands of the experts also in tasks 2 and 3. Sub-analysis showed that, in experts with 100 ≤ cases, scissors moved more frequently in the "close zone (0 ≤ to < 2.0 cm from aorta)" than those with 50-99 cases. CONCLUSION: Our novel Mocap system recognized significant differences in several metrics in multiple instruments according to the level of surgical experience. "Applying a Hem-o-lok clip on a pedicle" strongly reflected the level of surgical experience, and zone-metrics may be a promising tool to assess surgical expertise. Our next challenge is to give completely objective feedback to trainees on-site in the wet-lab.

Reliability and validity of pelvic floor muscle strength assessment using the MizCure perineometer.

BACKGROUND: The purpose of this study was to clarify the reliability and validity of pelvic floor muscle (PFM) strength assessment using the MizCure perineometer in healthy women. METHODS: Twenty healthy women (age 20-45 years) participated in this study. The vaginal pressure measured using the MizCure and validated Peritron perineometers were repeated during PFM contraction in the supine and standing positions. All women were evaluated twice by examiners 1 and 2. Following the measurements in the first session (Test 1), they were repeated after an interval of between 2 and 6 weeks (Test 2). Within- and between-session intra- and inter-rater reliabilities in vaginal pressure were analyzed using intraclass correlation coefficients (ICC) (1, 1) and (2, 1), respectively. Validity was assessed by Pearson's product-moment correlation coefficient and Spearman's rank correlation analysis. RESULTS: Within-session intra-rater reliabilities for both examiners 1 and 2 for all vaginal pressures in Tests 1 and 2 were 0.90-0.96 for both perineometers. Between-session intra-rater reliability for the MizCure was 0.72-0.79 for both positions for examiner 1, and 0.63 in the supine position and 0.80 in the standing position for examiner 2. Inter-rater reliability for Test 1 was 0.91 in the supine position and 0.87 in the standing position for the MizCure. The vaginal pressures using the MizCure and Peritron were significantly associated with the supine position (r = 0.68, P < .001) and the standing position (rs = 0.82, P < .001). CONCLUSION: MizCure perineometer is a validated tool to measure PFM strength in both supine and standing positions in healthy nulliparous women.

Validity assessment of the laparoscopic radical nephrectomy module of the LapVision virtual reality simulator.

BACKGROUND: Virtual reality simulators allow trainees to perform repeated practice and provide objective dexterity metrics regarding their performance, which means that virtual reality-based surgical training is becoming a vital part of initial learning of basic laparoscopic surgical skills. However, its educational role in learning advanced procedures remains undetermined. We evaluated the validity of the laparoscopic radical nephrectomy module of the LapVision virtual reality simulator. METHODS: Urologists, medical students, and a junior resident voluntarily participated in the present study, and they performed training with a laparoscopic left radical nephrectomy module. For construct validation, dexterity metrics calculated in the simulator and the mean score of Global Operative Assessment of Laparoscopic Skills evaluated by 2 experts' video review were compared according to the certification of Japanese Endoscopic Surgical Skill Qualification or previous surgical experience. RESULTS: Ten experts (≥ 50 laparoscopic surgeries), 9 intermediates (11-49), and 14 novices (0-10) voluntarily participated in the present study. Regarding the construct validity, there was a significant difference in the total number of errors, blood loss, and Global Operative Assessment of Laparoscopic Skills score among the groups for both the Endoscopic Surgical Skill Qualification status and previous surgical experience. CONCLUSION: The present study demonstrated good construct validity for the LapVision nephrectomy module. Furthermore, global skill assessment was possible by experts' reviews, which indicates the usefulness of the virtual reality procedural module as a skill assessment tool. Virtual reality-based procedural simulation has marked potential to become a vital part of integrated laparoscopic training programs.