Non-technical skills for urological surgeons (NoTSUS): development and evaluation of curriculum and assessment scale.

OBJECTIVE: In the last decade non-technical skills (NTS) have emerged as a vital area for improvement within surgery. This study aims to develop and evaluate a Non-technical Skills for Urological Surgeons (NoTSUS) training curriculum and assessment scale. METHODS: This international, longitudinal and observational study began with a 3-round Delphi methodology to refine curriculum contents and rating scale. Sessions with up to four participants were delivered where each candidate undertook an independent scenario within the validated full immersion simulation environment. Candidates were assessed using both the NoTSS (Non-technical Skills for Surgeons) and NoTSUS rating scales by NTS-trained and non-trained experts. A post-training evaluation survey was distributed. RESULTS: 62 participants comprising trainees (n = 43) and specialists (n = 19) undertook the NoTSUS course. The NoTSS and NoTSUS scales correlated well, with a mean difference of 3.3 in the overall total (p = 0.10, r = 0.53). However, there was significant differences in scores between the NoTSS-trained and non-trained raters (n = 28, p = 0.03). A one-way ANOVA test revealed significant improvement throughout the four simulation scenarios in each session (p = 0.02). The NoTSUS curriculum received positive feedback from participants and demonstrated educational value and acceptability. CONCLUSIONS: The NoTSUS curriculum has demonstrated high educational value for NTS training aimed at urologists, with marked improvement throughout sessions. Correlation of NoTSUS and NoTSS scales proves its suitability for evaluating NTS in future training. Demonstration of inter-rater reliability indicates that the scale is reliable for use in assessment by expert faculty members. Furthermore, qualitative feedback from participants suggests gain of transferrable skills over the course.

Current Status of Technical Skills Assessment Tools in Surgery: A Systematic Review.

BACKGROUND: Tools for assessment of technical skills are a crucial part of surgical education. They provide trainees with quantitative feedback highlighting both proficiency and areas for improvement. For this to be relevant to day-to-day practice, the tools used have to be validated and relevant to each surgical situation. This study aims to evaluate the validity of assessment tools used within surgery. MATERIALS AND METHODS: Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines, a systematic review was conducted searching the MEDLINE and Embase databases (International Prospective Register of Systematic Reviews: CRD42018104674). Studies utilizing any assessment tool in any surgical specialty were included. Messick's criteria were used for literature evaluation, and the Modified Educational Oxford Centre for Evidence-Based Medicine was used to assess levels of recommendation. RESULTS: A total of 303 studies and 76 tools were included. The most commonly used tool was Objective Structured Assessment Tool Skills (OSATS; n = 137, 45.2%). OSATS was used in conjunction with another tool or tools in an additional 55 studies (18.2%). Seven further tools were used in at least 3 studies. A total of five studies evaluated contained all five aspects of Messick's validity. CONCLUSIONS: There are several widely validated tools for assessing technical skills, the most common of which is OSATS. There is an emerging trend for crowdsourcing as a quick, cheap method for assessment of technical skills. This technique has been validated using both GEARS and GOALS. Numerous tools were found to be used only once and demonstrate a tendency for units to create their own tools for a specific task or specialty.

Development of a technical checklist for the assessment of suturing in robotic surgery.

BACKGROUND: With the increased use of simulation for surgical training, there is a need for objective forms of assessment to evaluate trainees. The Global Evaluative Assessment of Robotic Skills (GEARS) is widely used for assessing skills in robotic surgery, but there are no recognised checklist scoring systems. This study aimed to develop a checklist for suturing in robotic surgery. METHODS: A suturing checklist for needle driving and knot tying was constructed following evaluation of participants performing urethrovesical anastomoses. Key procedural steps were identified from expert videos, while assessing novice videos allowed identification of common technical errors. 22 novice and 13 expert videos were marked on needle driving, while 18 novices and 10 experts were assessed on knot tying. Validation of the finalised checklist was performed with the assessment of 39 separate novices by an expert surgeon and compared to GEARS scoring. RESULTS: The internal consistency of the preliminary checklist was high (Cronbach's alpha = 0.870 for needle driving items; 0.736 for knot tying items), and after removal of poorly correlating items, the final checklist contained 23 steps. Both the needle driving and knot tying categories discriminated between novices and experts, p < 0.005. While the GEARS score demonstrated construct validity for needle driving, it could not significantly differentiate between novices and experts for knot tying, p = 0.286. The needle driving category significantly correlated with the corresponding GEARS scores (rs = 0.613, p < 0.005), but the correlation for knot tying was insignificant (rs = 0.296, p = 0.127). The pilot data indicates the checklist significantly correlated with the GEARS score (p < 0.005). CONCLUSION: This study reports the development of a valid assessment tool for suturing in robotic surgery. Given that checklists are simple to use, there is significant scope for this checklist to be used in surgical training.

Simulation-based training and assessment in urological surgery.

Simulation has become widely accepted as a supplementary method of training. Within urology, the greatest number of procedure-specific models and subsequent validation studies have been carried out in the field of endourology. Many generic-skills simulators have been created for laparoscopic and robot-assisted surgery, but only a limited number of procedure-specific models are available. By contrast, open urological simulation has only seen a handful of validated models. Of the available modalities, virtual reality (VR) simulators are most commonly used for endourology and robotic surgery training, the former also employing many high-fidelity bench models. Smaller dry-lab and ex vivo animal models have been used for laparoscopic and robotic training, whereas live animals and human cadavers are widely used for full procedural training. Newer concepts such as augmented-reality (AR) models and patient-specific simulators have also been introduced. Several curricula, including one recommended within, have been produced, incorporating various different training modalities and nontechnical skills training techniques. Such curricula and validated models should be used in a structured fashion to supplement operating room training.