Non-technical skills for urological surgeons (NoTSUS): development and evaluation of curriculum and assessment scale.

OBJECTIVE: In the last decade non-technical skills (NTS) have emerged as a vital area for improvement within surgery. This study aims to develop and evaluate a Non-technical Skills for Urological Surgeons (NoTSUS) training curriculum and assessment scale. METHODS: This international, longitudinal and observational study began with a 3-round Delphi methodology to refine curriculum contents and rating scale. Sessions with up to four participants were delivered where each candidate undertook an independent scenario within the validated full immersion simulation environment. Candidates were assessed using both the NoTSS (Non-technical Skills for Surgeons) and NoTSUS rating scales by NTS-trained and non-trained experts. A post-training evaluation survey was distributed. RESULTS: 62 participants comprising trainees (n = 43) and specialists (n = 19) undertook the NoTSUS course. The NoTSS and NoTSUS scales correlated well, with a mean difference of 3.3 in the overall total (p = 0.10, r = 0.53). However, there was significant differences in scores between the NoTSS-trained and non-trained raters (n = 28, p = 0.03). A one-way ANOVA test revealed significant improvement throughout the four simulation scenarios in each session (p = 0.02). The NoTSUS curriculum received positive feedback from participants and demonstrated educational value and acceptability. CONCLUSIONS: The NoTSUS curriculum has demonstrated high educational value for NTS training aimed at urologists, with marked improvement throughout sessions. Correlation of NoTSUS and NoTSS scales proves its suitability for evaluating NTS in future training. Demonstration of inter-rater reliability indicates that the scale is reliable for use in assessment by expert faculty members. Furthermore, qualitative feedback from participants suggests gain of transferrable skills over the course.

Development and content validation of the percutaneous nephrolithotomy assessment score.

OBJECTIVES: To develop and content validate a percutaneous nephrolithotomy assessment score, taking into consideration the procedure-specific risks. METHODS: This prospective international study utilized the Healthcare Failure Mode and Effect Analysis to systematically outline percutaneous nephrolithotomy and failure modes for each step. A total of 25 h was spent observing percutaneous nephrolithotomy carried out by six expert surgeons. Hazard analysis scoring was carried out by 11 experts. It was determined if the steps were single point weaknesses. Single point weaknesses and those assigned a hazard score ≥4 were included in the percutaneous nephrolithotomy assessment score. The tool was then content validated by 16 experts from 10 countries. RESULTS: Application of the Healthcare Failure Mode and Effect Analysis identified 64 failure modes; 37 failure modes had a hazard score ≥4. After adaptations based on expert feedback the final percutaneous nephrolithotomy assessment score was developed containing 10 phases, 21 processes and 47 subprocesses. All participants agreed that the tool contained pertinent procedural steps. CONCLUSIONS: This study has developed and shown the international content validity of a novel percutaneous nephrolithotomy assessment score. The tool can be utilized in modular operating room training to quantify operator progress, and can be used in conjunction with other modules as part of a complete percutaneous nephrolithotomy curriculum for trainees.

Current Status of Technical Skills Assessment Tools in Surgery: A Systematic Review.

BACKGROUND: Tools for assessment of technical skills are a crucial part of surgical education. They provide trainees with quantitative feedback highlighting both proficiency and areas for improvement. For this to be relevant to day-to-day practice, the tools used have to be validated and relevant to each surgical situation. This study aims to evaluate the validity of assessment tools used within surgery. MATERIALS AND METHODS: Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines, a systematic review was conducted searching the MEDLINE and Embase databases (International Prospective Register of Systematic Reviews: CRD42018104674). Studies utilizing any assessment tool in any surgical specialty were included. Messick's criteria were used for literature evaluation, and the Modified Educational Oxford Centre for Evidence-Based Medicine was used to assess levels of recommendation. RESULTS: A total of 303 studies and 76 tools were included. The most commonly used tool was Objective Structured Assessment Tool Skills (OSATS; n = 137, 45.2%). OSATS was used in conjunction with another tool or tools in an additional 55 studies (18.2%). Seven further tools were used in at least 3 studies. A total of five studies evaluated contained all five aspects of Messick's validity. CONCLUSIONS: There are several widely validated tools for assessing technical skills, the most common of which is OSATS. There is an emerging trend for crowdsourcing as a quick, cheap method for assessment of technical skills. This technique has been validated using both GEARS and GOALS. Numerous tools were found to be used only once and demonstrate a tendency for units to create their own tools for a specific task or specialty.

Validity assessment of a simulation module for robot-assisted thoracic lobectomy.

BACKGROUND: Training for robot-assisted thoracic lobectomy remains an issue, prompting the development of virtual reality simulators. Our aim was to assess the construct and face validity of a new thoracic lobectomy module on the RobotiX Mentor, a robotic surgery simulator. We also aimed to determine the acceptability and feasibility of implementation into training. METHODS: This prospective, observational, and comparative study recruited novice (n = 16), intermediate (n = 9), and expert (n = 5) participants from King's College London, the 25th European Conference on General Thoracic Surgery, and the Society of Robotic Surgery conference 2018. Each participant completed two familiarization tasks followed by the Guided Robotic Lobectomy module and an evaluation questionnaire. Outcome measures were compared using Mann-Whitney U tests. RESULTS: Construct validity was demonstrated in 12/21 performance evaluation metrics. Significant differences between groups were found in all metrics including: time taken to complete module, vascular injury, respect for tissue, number of stapler firings, time instruments out of view, number of instrument collisions, and number of movements. Participants deemed aspects of the simulator (mean 3/5) and module (3/5) as realistic and rated the simulator as both acceptable (3.8/5) and feasible (3.8/5) for robotic surgical training. CONCLUSIONS: Face validity, acceptability, and feasibility were established for the thoracic lobectomy module of the RobotiX Mentor simulator. Moderate evidence of construct validity was also demonstrated. With further work, this simulation module could help to reduce the initial part of the learning curve for trainees and decrease the risk of errors during live training.

European Association of Urology Section of Urolithiasis (EULIS) Consensus Statement on Simulation, Training, and Assessment in Urolithiasis.

BACKGROUND: Simulation-based training offers an acceptable adjunct to the traditional mentor-apprentice model in helping trainees to traverse the early stages of the learning curve for ureteroscopy and percutaneous renal surgery. In addition, nontechnical skills are increasingly important in preventing adverse events in the operating room, and simulation-based training can be used for training in such skills. Incorporation of simulation into formalised, standardised, and validated curricula offers an applicable method for training residents. OBJECTIVE: To develop a curriculum for urolithiasis procedures incorporating technical and nontechnical skills training for implementation across Europe. DESIGN, SETTING, AND PARTICIPANTS: An international panel of experts from EULIS, EUREP, ESU and ESUT was consulted in five stages. The study incorporated a mix of qualitative and quantitative data for collection and analysis. Responses were drawn out in (1) an opinion survey and (2) a curriculum development survey, which were discussed in (3) a focus group meeting. Group responses from this meeting were analysed for themes, which were discussed at (4) a focus group meeting, where consensus was reached among the group. Data analysis and integration at this stage were used to draft the curriculum. RESULTS AND LIMITATIONS: All group meetings were transcribed from the focus group discussion. Eight themes were generated, into which all data were categorised. These were: need for a training curriculum; curriculum objectives; curriculum structure; curriculum content; teaching platforms and tools; assessment and certification; validation and implementation; and global integration of the curriculum. A curriculum, including recommended simulators for use, was subsequently proposed. CONCLUSIONS: We propose a comprehensive curriculum for training in urolithiasis. Additional planning is required for full validation and implementation before it can be used to train residents. PATIENT SUMMARY: Stone disease accounts for a major proportion of surgical interventions worldwide. We describe a consensus guideline for effective training of stone surgeons.

Simulation-based training and assessment in urological surgery.

Simulation has become widely accepted as a supplementary method of training. Within urology, the greatest number of procedure-specific models and subsequent validation studies have been carried out in the field of endourology. Many generic-skills simulators have been created for laparoscopic and robot-assisted surgery, but only a limited number of procedure-specific models are available. By contrast, open urological simulation has only seen a handful of validated models. Of the available modalities, virtual reality (VR) simulators are most commonly used for endourology and robotic surgery training, the former also employing many high-fidelity bench models. Smaller dry-lab and ex vivo animal models have been used for laparoscopic and robotic training, whereas live animals and human cadavers are widely used for full procedural training. Newer concepts such as augmented-reality (AR) models and patient-specific simulators have also been introduced. Several curricula, including one recommended within, have been produced, incorporating various different training modalities and nontechnical skills training techniques. Such curricula and validated models should be used in a structured fashion to supplement operating room training.

Current Status of Simulation and Training Models in Urological Surgery: A Systematic Review.

PURPOSE: Increased awareness of patient safety, advances in surgical technology and reduced working times have led to the adoption of simulation enhanced training. However, the simulators available need to be scientifically evaluated before integration into curricula. We identify the currently available training models for urological surgery, their status of validation and the evidence behind each model. MATERIALS AND METHODS: MEDLINE®, Embase® and the Cochrane Library databases were searched for English language articles published between 1990 and 2015 describing urological simulators and/or validation studies of these models. All studies were assessed for level of evidence, and each model was subsequently awarded a level of recommendation using a modified Oxford Centre for Evidence-Based Medicine classification, adapted for education by the European Association of Endoscopic Surgeons. RESULTS: A total of 91 validation studies were identified pertaining to training models in endourology (63), laparoscopic surgery (17), robot-assisted surgery (8) and open urological surgery (6), with a total of 55 models. Of the included studies 6 were classified Level 1b, 9 Level 2a, 39 Level 2b and 19 Level 2c. Of all the training models the URO Mentor™ was the only one to receive a level of recommendation of 1. CONCLUSIONS: UroSimulation is a growing field and increasing numbers of models are being produced. However, there are still too few validation studies with a high level of evidence demonstrating the transferability of skills. Nevertheless, efforts should be made to use the currently available models in curriculum based training programs.

A novel cadaveric simulation program in urology.

OBJECTIVE: To evaluate the urology human cadaver training program developed by the British Association of Urological Surgeons. DESIGN: This prospective, observational comparative study recruited urology residents, with different levels of experience, in 2 sessions of a 3-day modular cadaveric operative urology training. Participants performed various procedures on fresh-frozen cadaveric specimens, as per module, supervised by certified urological surgeons. At the conclusion of each module, all residents and faculty were invited to complete an evaluation survey. SETTING: The training days were hosted by the British Association of Urological Surgeons at the University of Manchester Surgical Skills and Simulation Centre. PARTICIPANTS: A total of 81 urology residents were recruited, with a maximum of 14 participants attending each module, over 2 sessions. We allocated 2 participants to each cadaver with access to all necessary equipment and guidance. RESULTS: A total of 102 evaluation surveys were received from the trainees and faculty; a response rate of 94%. All procedures scored a mean of 3 on 5 for face validity, which is higher than the acceptability range. Regarding content validity, participants and faculty rated all aspects ≥3 on 5. Respondents held a positive view of the cadaver sessions and believed them to be useful for learning anatomy and steps of an operation (mean = 4.54) and as a confidence booster for performing a procedure (mean = 4.33). Furthermore, it was thought that the training program significantly improved skills (mean = 4.11), gave transferrable skills for the operating room (mean = 4.21), and was feasible to be incorporated into training programs (mean = 4.29). Human cadaveric simulation was rated as the best mode of simulation-based training for all the procedures in the curriculum. CONCLUSIONS: This study on cadaveric simulation training demonstrated face and content validities. It also showed feasibility, acceptability, a high value for educational influence and cost-effectiveness for cadaveric simulation. A simulation-based training pathway has been proposed for effective procedural training in urology.