Crowdsourced assessment of surgical skills: A systematic review.

INTRODUCTION: Crowdsourced assessment utilizes a large group of untrained individuals from the general population to solve tasks in the medical field. The aim was to examine the correlation between crowd workers and expert surgeons for the use of crowdsourced assessments of surgical skills. MATERIAL AND METHODS: A systematic literature review was performed on April 14th, 2021 from inception to the present. Two reviewers screened all articles with eligibility criteria of inclusion and assessed for quality using The Medical Education Research Study Quality Instrument (MERSQI) and Newcastle-Ottawa Scale-Education (NOS-E)(Holst et al., 2015).7General information was extracted for each article. RESULTS: 250 potential studies were identified, and 32 articles were included. There appeared to be a generally moderate to very strong correlation between crowd workers and experts (Cronbach's alpha 0.72-0.95, Pearson's r 0.7-0.95, Spearman Rho 0.7-0.89, linear regression 0.45-0.89). Six studies had either questionable or no significant correlation between crowd workers and experts. CONCLUSION: Crowdsourced assessment can provide accurate, rapid, cost-effective, and objective feedback across different specialties and types of surgeries in dry lab, simulation, and live surgeries.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients.

Background: Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve. Objective: To implement a national training programme for simulation-based mastery learning in TURBT and explore operating theatre performance after training. Design setting and participants: From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients. Outcome measurements and statistical analyses: Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample t tests were used to compare pretraining and post-training analyses and independent t tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302. Results and limitations: Before training, novices had significantly lower performance scores in comparison to those with intermediate experience (p = 0.017) and experienced doctors (p < 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; p = 0.049, n = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training (p = 0.9 and p = 0.8, respectively). Conclusions: Novices improved their TURBT performance in the operating theatre after completing a proficiency-based training programme on a virtual reality simulator. Patient summary: We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists.