Am I doing this right? Structured self-assessment during simulation training of mastoidectomy improves cadaver dissection performance: a prospective educational study.

PURPOSE: Temporal bone surgery requires excellent surgical skills and simulation-based training can aid novices' skills acquisition. However, simulation-based training is challenged by early stagnation of performance after few performances. Structured self-assessment during practice might enhance learning by inducing reflection and engagement in the learning task. In this study, structured self-assessment was introduced during virtual reality (VR) simulation of mastoidectomy to investigate the effects on subsequent performance during cadaveric dissection. METHODS: A prospective educational study with comparison with historical controls (reference cohort). At a temporal bone dissection course, eighteen participants performed structured self-assessment during 3 h of VR simulation mastoidectomy training before proceeding to cadaver dissection (intervention cohort). At a previous course, eighteen participants received identical VR simulation training but without the structured self-assessment (reference cohort). Final products from VR simulation and cadaveric dissection were recorded and assessed by two blinded raters using a 19-point modified Welling Scale. RESULTS: The intervention cohort completed fewer procedures (average 4.2) during VR simulation training than the reference cohort (average 5.7). Nevertheless, the intervention cohort achieved a significantly higher average performance score both in VR simulation (11.1 points, 95% CI [10.6-11.5]) and subsequent cadaveric dissection (11.8 points, 95% CI [10.7-12.8]) compared with the reference cohort, who scored 9.1 points (95% CI [8.7-9.5]) during VR simulation and 5.8 points (95% CI [4.8-6.8]) during cadaveric dissection. CONCLUSIONS: Structured self-assessment is a valuable learning support during self-directed VR simulation training of mastoidectomy and the positive effect on performance transfers to subsequent cadaveric dissection performance.

Content validity evidence for a simulation-based test of handheld otoscopy skills.

PURPOSE: At graduation from medical school, competency in otoscopy is often insufficient. Simulation-based training can be used to improve technical skills, but the suitability of the training model and assessment must be supported by validity evidence. The purpose of this study was to collect content validity evidence for a simulation-based test of handheld otoscopy skills. METHODS: First, a three-round Delphi study was conducted with a panel of nine clinical teachers in otorhinolaryngology (ORL) to determine the content requirements in our educational context. Next, the authenticity of relevant cases in a commercially available technology-enhanced simulator (Earsi, VR Magic, Germany) was evaluated by specialists in ORL. Finally, an integrated course was developed for the simulator based on these results. RESULTS: The Delphi study resulted in nine essential diagnoses of normal variations and pathologies that all junior doctors should be able to diagnose with a handheld otoscope. Twelve out of 15 tested simulator cases were correctly recognized by at least one ORL specialist. Fifteen cases from the simulator case library matched the essential diagnoses determined by the Delphi study and were integrated into the course. CONCLUSION: Content validity evidence for a simulation-based test of handheld otoscopy skills was collected. This informed a simulation-based course that can be used for undergraduate training. The course needs to be further investigated in relation to other aspects of validity and for future self-directed training.