Gap analysis between trainees' subjective competencies and the competencies expected by instructors in urology: A need assessment survey in Japan.

OBJECTIVE: According to the rapid progress in surgical techniques, a growing number of procedures should be learned during postgraduate training periods. This study aimed to clarify the current situation regarding urological surgical training and identify the perception gap between trainees' competency and the competency expected by instructors in Japan. METHODS: Regarding the 40 urological surgical procedures selected via the Delphi method, we collected data on previous caseloads, current subjective autonomy, and confidence for future skill acquisition from trainees (<15 post-graduate years [PGY]), and the competencies when trainees became attending doctors expected by instructors (>15 PGY), according to a 5-point Likert scale. In total, 174 urologists in Hokkaido Prefecture, Japan were enrolled in this study. RESULTS: The response rate was 96% (165/174). In a large proportion of the procedures, caseloads grew with accumulation of years of clinical practice. However, trainees had limited caseloads of robotic and reconstructive surgeries even after 15 PGY. Trainees showed low subjective competencies at present and low confidence for future skill acquisition in several procedures, such as open cystectomy, ureteroureterostomy, and ureterocystostomy, while instructors expected trainees to be able to perform these procedures independently when they became attending doctors. CONCLUSION: Trainees showed low subjective competencies and low confidence for future skill acquisition in several open and reconstructive procedures, while instructors considered that these procedures should be independently performable by attending doctors. We believe that knowledge of these perception gaps is helpful to develop a practical training program.

Automatic assessment of laparoscopic surgical skill competence based on motion metrics.

The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.

Motion analysis for better understanding of psychomotor skills in laparoscopy: objective assessment-based simulation training using animal organs.

BACKGROUND: Our aim was to characterize the motions of multiple laparoscopic surgical instruments among participants with different levels of surgical experience in a series of wet-lab training drills, in which participants need to perform a range of surgical procedures including grasping tissue, tissue traction and dissection, applying a Hem-o-lok clip, and suturing/knotting, and digitize the level of surgical competency. METHODS: Participants performed tissue dissection around the aorta, dividing encountered vessels after applying a Hem-o-lok (Task 1), and renal parenchymal closure (Task 2: suturing, Task 3: suturing and knot-tying), using swine cadaveric organs placed in a box trainer under a motion capture (Mocap) system. Motion-related metrics were compared according to participants' level of surgical experience (experts: 50 ≤ laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test, and significant metrics were subjected to principal component analysis (PCA). RESULTS: A total of 15 experts, 12 intermediates, and 18 novices participated in the training. In Task 1, a shorter path length and faster velocity/acceleration/jerk were observed using both scissors and a Hem-o-lok applier in the experts, and Hem-o-lok-related metrics markedly contributed to the 1st principal component on PCA analysis, followed by scissors-related metrics. Higher-level skills including a shorter path length and faster velocity were observed in both hands of the experts also in tasks 2 and 3. Sub-analysis showed that, in experts with 100 ≤ cases, scissors moved more frequently in the "close zone (0 ≤ to < 2.0 cm from aorta)" than those with 50-99 cases. CONCLUSION: Our novel Mocap system recognized significant differences in several metrics in multiple instruments according to the level of surgical experience. "Applying a Hem-o-lok clip on a pedicle" strongly reflected the level of surgical experience, and zone-metrics may be a promising tool to assess surgical expertise. Our next challenge is to give completely objective feedback to trainees on-site in the wet-lab.

Validity assessment of the laparoscopic radical nephrectomy module of the LapVision virtual reality simulator.

BACKGROUND: Virtual reality simulators allow trainees to perform repeated practice and provide objective dexterity metrics regarding their performance, which means that virtual reality-based surgical training is becoming a vital part of initial learning of basic laparoscopic surgical skills. However, its educational role in learning advanced procedures remains undetermined. We evaluated the validity of the laparoscopic radical nephrectomy module of the LapVision virtual reality simulator. METHODS: Urologists, medical students, and a junior resident voluntarily participated in the present study, and they performed training with a laparoscopic left radical nephrectomy module. For construct validation, dexterity metrics calculated in the simulator and the mean score of Global Operative Assessment of Laparoscopic Skills evaluated by 2 experts' video review were compared according to the certification of Japanese Endoscopic Surgical Skill Qualification or previous surgical experience. RESULTS: Ten experts (≥ 50 laparoscopic surgeries), 9 intermediates (11-49), and 14 novices (0-10) voluntarily participated in the present study. Regarding the construct validity, there was a significant difference in the total number of errors, blood loss, and Global Operative Assessment of Laparoscopic Skills score among the groups for both the Endoscopic Surgical Skill Qualification status and previous surgical experience. CONCLUSION: The present study demonstrated good construct validity for the LapVision nephrectomy module. Furthermore, global skill assessment was possible by experts' reviews, which indicates the usefulness of the virtual reality procedural module as a skill assessment tool. Virtual reality-based procedural simulation has marked potential to become a vital part of integrated laparoscopic training programs.

[Initial assessment criteria for urination problems of the elderly--comparison of criteria for patients/caregivers/nurses, general practitioners, and urologists].

As the population ages rapidly, we need to establish a cost-effective system to assess and treat urination problems of the elderly. Recently, criteria have been developed for urologists to assess benign prostatic hyperplasia using the International Prostate Symptom Score (I-PSS), QoL index, urination function (maximal flow rates and residual urine volume) and prostate volume. It is suggested that patients with moderate to severe urination problems as measured by these criteria need to be treated but those with mild problems have no need for treatment. We examined whether these criteria can be applied to the general urination problems of elderly men and women, with 112 men and 21 women aged 50 and over as subjects. Furthermore, we set Level I criteria for the elderly themselves, their families or nurses taking care of them, using I-PSS and QoL index, and Level II criteria for general practitioners, using I-PSS, QoL index and residual urine volume, in addition to Level III criteria for urologists (omitting prostate volume in women). Correspondence rates among severities of urination problems at Level I, II and III, and sensitivities and specificities of Level I and II criteria were investigated. One hundred twenty-one patients of 124 with moderate to severe urination problems at Level III needed to be treated and eight of nine with mild urination problems did not. As a whole, our findings suggested that 129 of 133 patients (97.0%) were correctly assessed at Level III. We found that Level III criteria were valid for general urination problems of the elderly, not just benign prostatic hyperplasia. While 102 and 111 patients were judged as having moderate or severe urination problems, respectively, at Level I and Level II, and all of these were diagnosed as having moderate or severe problems at Level III, 22 (71.0%) of 31 and 13 (59.1%) of 22 patients judged as having mild urination problems at Level I and Level II were diagnosed as having moderate problems at Level III. Although specificities of Level I and Level II criteria were 100%, sensitivities improved from 82.3% (102/124) to 89.5% (111/124), adding residual urine measurement to the I-PSS and QoL scores. Polyuria of more than 2,000 ml per day was noted in 33 patients (24.8%). At Level I, residual urine of 50 ml or more was noted in 9 (29.0%) patients with mild urination problems, in 22 (24.2%) with moderate problems and in 4 (36.4%) with severe problems. Assessment criteria for general practitioners should include a frequency-volume chart and measurement of residual urine. When the elderly are judged as having moderate or severe urination problems according to either Level I or II criteria, they need treatment. However, more than half of the elderly, judged as having mild problems according to those criteria, need treatment from the standpoint of urologists.