Gap analysis between trainees' subjective competencies and the competencies expected by instructors in urology: A need assessment survey in Japan.

OBJECTIVE: According to the rapid progress in surgical techniques, a growing number of procedures should be learned during postgraduate training periods. This study aimed to clarify the current situation regarding urological surgical training and identify the perception gap between trainees' competency and the competency expected by instructors in Japan. METHODS: Regarding the 40 urological surgical procedures selected via the Delphi method, we collected data on previous caseloads, current subjective autonomy, and confidence for future skill acquisition from trainees (<15 post-graduate years [PGY]), and the competencies when trainees became attending doctors expected by instructors (>15 PGY), according to a 5-point Likert scale. In total, 174 urologists in Hokkaido Prefecture, Japan were enrolled in this study. RESULTS: The response rate was 96% (165/174). In a large proportion of the procedures, caseloads grew with accumulation of years of clinical practice. However, trainees had limited caseloads of robotic and reconstructive surgeries even after 15 PGY. Trainees showed low subjective competencies at present and low confidence for future skill acquisition in several procedures, such as open cystectomy, ureteroureterostomy, and ureterocystostomy, while instructors expected trainees to be able to perform these procedures independently when they became attending doctors. CONCLUSION: Trainees showed low subjective competencies and low confidence for future skill acquisition in several open and reconstructive procedures, while instructors considered that these procedures should be independently performable by attending doctors. We believe that knowledge of these perception gaps is helpful to develop a practical training program.

Automatic assessment of laparoscopic surgical skill competence based on motion metrics.

The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.

Motion analysis for better understanding of psychomotor skills in laparoscopy: objective assessment-based simulation training using animal organs.

BACKGROUND: Our aim was to characterize the motions of multiple laparoscopic surgical instruments among participants with different levels of surgical experience in a series of wet-lab training drills, in which participants need to perform a range of surgical procedures including grasping tissue, tissue traction and dissection, applying a Hem-o-lok clip, and suturing/knotting, and digitize the level of surgical competency. METHODS: Participants performed tissue dissection around the aorta, dividing encountered vessels after applying a Hem-o-lok (Task 1), and renal parenchymal closure (Task 2: suturing, Task 3: suturing and knot-tying), using swine cadaveric organs placed in a box trainer under a motion capture (Mocap) system. Motion-related metrics were compared according to participants' level of surgical experience (experts: 50 ≤ laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test, and significant metrics were subjected to principal component analysis (PCA). RESULTS: A total of 15 experts, 12 intermediates, and 18 novices participated in the training. In Task 1, a shorter path length and faster velocity/acceleration/jerk were observed using both scissors and a Hem-o-lok applier in the experts, and Hem-o-lok-related metrics markedly contributed to the 1st principal component on PCA analysis, followed by scissors-related metrics. Higher-level skills including a shorter path length and faster velocity were observed in both hands of the experts also in tasks 2 and 3. Sub-analysis showed that, in experts with 100 ≤ cases, scissors moved more frequently in the "close zone (0 ≤ to < 2.0 cm from aorta)" than those with 50-99 cases. CONCLUSION: Our novel Mocap system recognized significant differences in several metrics in multiple instruments according to the level of surgical experience. "Applying a Hem-o-lok clip on a pedicle" strongly reflected the level of surgical experience, and zone-metrics may be a promising tool to assess surgical expertise. Our next challenge is to give completely objective feedback to trainees on-site in the wet-lab.

Validity assessment of the laparoscopic radical nephrectomy module of the LapVision virtual reality simulator.

BACKGROUND: Virtual reality simulators allow trainees to perform repeated practice and provide objective dexterity metrics regarding their performance, which means that virtual reality-based surgical training is becoming a vital part of initial learning of basic laparoscopic surgical skills. However, its educational role in learning advanced procedures remains undetermined. We evaluated the validity of the laparoscopic radical nephrectomy module of the LapVision virtual reality simulator. METHODS: Urologists, medical students, and a junior resident voluntarily participated in the present study, and they performed training with a laparoscopic left radical nephrectomy module. For construct validation, dexterity metrics calculated in the simulator and the mean score of Global Operative Assessment of Laparoscopic Skills evaluated by 2 experts' video review were compared according to the certification of Japanese Endoscopic Surgical Skill Qualification or previous surgical experience. RESULTS: Ten experts (≥ 50 laparoscopic surgeries), 9 intermediates (11-49), and 14 novices (0-10) voluntarily participated in the present study. Regarding the construct validity, there was a significant difference in the total number of errors, blood loss, and Global Operative Assessment of Laparoscopic Skills score among the groups for both the Endoscopic Surgical Skill Qualification status and previous surgical experience. CONCLUSION: The present study demonstrated good construct validity for the LapVision nephrectomy module. Furthermore, global skill assessment was possible by experts' reviews, which indicates the usefulness of the virtual reality procedural module as a skill assessment tool. Virtual reality-based procedural simulation has marked potential to become a vital part of integrated laparoscopic training programs.