Validation of the endoscopic stone treatment step 1 (EST-s1): a novel EAU training and assessment tool for basic endoscopic stone treatment skills-a collaborative work by ESU, ESUT and EULIS.

INTRODUCTION: The endoscopic stone treatment step 1 (EST s1) protocol has been developed after 2 years of collaborative work between different European Association of Urology (EAU) sections. OBJECTIVES: In this study, we added construct validity evidence to the EST s1 curriculum. MATERIALS AND METHODS: The EST-s1 curriculum includes four standardized tasks: flexible cystoscopy, rigid cystoscopy, semi-rigid URS and flexible URS. Validation was performed during the annual 2016 EUREP meeting in Prague. 124 participants provided information on their endoscopic logbook and carried out these 4 tasks during a DVD recorded session. Recordings were anonymized and blindly assessed independently by five proctors. Inter-rater reliability was checked on a sample of five videos by the calculation of intra-class correlation coefficient. Task-specific clinical background of participants was correlated with their personal performance on the simulator. Breakpoint analysis was used to define the minimum number of performed cases, to be considered "proficient". "Proficient" and "Non-proficient" groups were compared for construct validity assessment. Likert scale-based questionnaires were used to test content and to comment on when the EST-s1 exams should be undertaken within the residency program. RESULTS: 124 participants (105 final-year residents and 19 faculty members) took part in this study. The breakpoint analysis showed a significant change in performance curve at 36, 41, 67 and 206 s, respectively, corresponding to 30, 60, 25 and 120 clinical cases for each of the 4 tasks. EST-s1 was scored as a valid training tool, correctly representing the procedures performed in each task. Experts felt that this curriculum is best used during the third year of residency training. CONCLUSION: Our validation study successfully demonstrated correlation between clinical expertise and EST-s1 tasks, adding construct validity evidence to it. Our work also demonstrates the successful collaboration established within various EAU sections.

Development Methodology of the Novel Endoscopic Stone Treatment Step 1 Training/Assessment Curriculum: An International Collaborative Work by European Association of Urology Sections.

Background: Simulation-based technical skill assessment is a core topic of debate, especially in high-risk environments. After the introduction of the E-BLUS (European Basic Laparoscopic Urological Skills) exam for basic laparoscopy, no more technical training/assessment urological protocols have been developed in Europe. Objective: We describe the methodology used in the development of the novel Endoscopic Stone Treatment step 1 (EST s1) assessment curriculum. Materials and Methods: The "full life cycle curriculum development" template was followed for curriculum development. A cognitive task analysis was run to define the most important steps and details of retrograde intrarenal surgery, in accordance with European Association of Urology (EAU) Urolithiasis guidelines. Training tasks were created between April 2015 and September 2015. Tasks and metrics were further analyzed by a consensus meeting with the European Section of Urolithiasis (EULIS) board in February 2016. A review, aimed to study available simulators and their accordance with task requirements, was subsequently run in London in March 2016. After initial feedback and further tests, content validity of this protocol was achieved during European Urology Residents Education Programme (EUREP) 2016. Results: The EST s1 curriculum development, took 23 months. Seventy-two participants tested the five preliminary tasks during EUREP 2015, with sessions of 45 minutes each. Likert-scale questionnaires were filled out to score the quality of training. The protocol was modified accordingly and 25 participants tested the four tasks during the hands-on training sessions of the European Section of Uro-Technology (ESUT) 2016 congress. One hundred thirty-four participants finally participated in the validation study in EUREP 2016. During the same event, 10 experts confirmed content validity by filling out a Likert-scale questionnaire. Conclusion: We described a reliable and replicable methodology that can be followed to develop training/assessment protocols for surgical procedures. The expert consensus meetings, strict adherence to guidelines, and updated literature search toward an Endourology curriculum allowed correct training and assessment protocol development. It is the first step toward standardized simulation training in Endourology with a potential for worldwide adoption.