3-D-Printed Models for Temporal Bone Training: A Validity Study.

OBJECTIVE: 3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model. STUDY DESIGN: A prospective educational study gathering validity evidence using Messick's validity framework. SETTING: Seven Danish otorhinolaryngology training institutions. PARTICIPANTS: Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts). INTERVENTION: Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS). MAIN OUTCOME MEASURE: Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory. RESULTS: Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment. CONCLUSION: Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.

3D-printing a cost-effective model for mastoidectomy training.

BACKGROUND: 3D-printed temporal bone models can potentially provide a cost-effective alternative to cadaver surgery that can be manufactured locally at the training department. The objective of this study was to create a cost-effective 3D-printed model suitable for mastoidectomy training using entry level and commercially available print technologies, enabling individuals, without prior experience on 3D-printing, to manufacture their own models for basic temporal bone training. METHODS: Expert technical professionals and an experienced otosurgeon identified the best material for replicating the temporal bone and created a cost-effective printing routine for the model using entry-level print technologies. Eleven participants at a temporal bone dissection course evaluated the model using a questionnaire. RESULTS: The 3D-printed temporal bone model was printed using a material extrusion 3D-printer with a heat resistant filament, reducing melting during drilling. After printing, a few simple post-processing steps were designed to replicate the dura, sigmoid sinus and facial nerve. Modifying the 3D-printer by installing a direct-drive and ruby nozzle resulted in more successful prints and less need for maintenance. Upon evaluation by otorhinolaryngology trainees, unanimous feedback was that the model provided a good introduction to the mastoidectomy procedure, and supplementing practice to cadaveric temporal bones. CONCLUSION: In-house production of a cost-effective 3D-printed model for temporal bone training is feasible and enables training institutions to manufacture their own models. Further, this work demonstrates the feasibility of creating new temporal bone models with anatomical variation to provide ample training opportunity.

Are Video Recordings Reliable for Assessing Surgical Performance? A Prospective Reliability Study Using Generalizability Theory.

INTRODUCTION: Reliability is pivotal in surgical skills assessment. Video-based assessment can be used for objective assessment without physical presence of assessors. However, its reliability for surgical assessments remains largely unexplored. In this study, we evaluated the reliability of video-based versus physical assessments of novices' surgical performances on human cadavers and 3D-printed models-an emerging simulation modality. METHODS: Eighteen otorhinolaryngology residents performed 2 to 3 mastoidectomies on a 3D-printed model and 1 procedure on a human cadaver. Performances were rated by 3 experts evaluating the final surgical result using a well-known assessment tool. Performances were rated both hands-on/physically and by video recordings. Interrater reliability and intrarater reliability were explored using κ statistics and the optimal number of raters and performances required in either assessment modality was determined using generalizability theory. RESULTS: Interrater reliability was moderate with a mean κ score of 0.58 (range 0.53-0.62) for video-based assessment and 0.60 (range, 0.55-0.69) for physical assessment. Video-based and physical assessments were equally reliable (G coefficient 0.85 vs. 0.80 for 3D-printed models and 0.86 vs 0.87 for cadaver dissections). The interaction between rater and assessment modality contributed to 8.1% to 9.1% of the estimated variance. For the 3D-printed models, 2 raters evaluating 2 video-recorded performances or 3 raters physically assessing 2 performances yielded sufficient reliability for high-stakes assessment (G coefficient >0.8). CONCLUSIONS: Video-based and physical assessments were equally reliable. Some raters were affected by changing from physical to video-based assessment; consequently, assessment should be either physical or video based, not a combination.

Am I doing this right? Structured self-assessment during simulation training of mastoidectomy improves cadaver dissection performance: a prospective educational study.

PURPOSE: Temporal bone surgery requires excellent surgical skills and simulation-based training can aid novices' skills acquisition. However, simulation-based training is challenged by early stagnation of performance after few performances. Structured self-assessment during practice might enhance learning by inducing reflection and engagement in the learning task. In this study, structured self-assessment was introduced during virtual reality (VR) simulation of mastoidectomy to investigate the effects on subsequent performance during cadaveric dissection. METHODS: A prospective educational study with comparison with historical controls (reference cohort). At a temporal bone dissection course, eighteen participants performed structured self-assessment during 3 h of VR simulation mastoidectomy training before proceeding to cadaver dissection (intervention cohort). At a previous course, eighteen participants received identical VR simulation training but without the structured self-assessment (reference cohort). Final products from VR simulation and cadaveric dissection were recorded and assessed by two blinded raters using a 19-point modified Welling Scale. RESULTS: The intervention cohort completed fewer procedures (average 4.2) during VR simulation training than the reference cohort (average 5.7). Nevertheless, the intervention cohort achieved a significantly higher average performance score both in VR simulation (11.1 points, 95% CI [10.6-11.5]) and subsequent cadaveric dissection (11.8 points, 95% CI [10.7-12.8]) compared with the reference cohort, who scored 9.1 points (95% CI [8.7-9.5]) during VR simulation and 5.8 points (95% CI [4.8-6.8]) during cadaveric dissection. CONCLUSIONS: Structured self-assessment is a valuable learning support during self-directed VR simulation training of mastoidectomy and the positive effect on performance transfers to subsequent cadaveric dissection performance.

Ten Tips for Performing Your First Peer Review: The Next Step for the Aspiring Academic Plastic Surgeon.

Performing the first peer review of a plastic surgical research article can be an overwhelming task. However, it is an essential scholarly skill and peer review is used in a multitude of settings: evaluation of journal articles, conference abstracts, and research proposals. Furthermore, peer reviewing provides more than just the opportunity to read and help improve other's work: peer reviewing can improve your own scientific writing. A structured approach is possible and recommended. In these ten tips, we provide guidance on how to successfully conduct the first peer reviews. The ten tips on peer reviewing concern: 1) Appropriateness: are you qualified and prepared to perform the peer review? 2) Familiarization with the journal and its reviewing guidelines; 3) Gathering first impressions of the paper followed by specific tips for reviewing; 4) the abstract and introduction; 5) Materials, methods, and results (including statistical considerations); and 6) discussion, conclusion, and references. Tip 7 concerns writing and structuring the review; Tips 7 and 8 describe how to provide constructive criticism and understanding the limits of your expertise. Finally, Tip 10 details why-and how-you become a peer reviewer. Peer review can be done by any plastic surgeon, not just those interested in an academic career. These ten tips provide useful insights for both the aspiring and the experienced peer reviewer. In conclusion, a systematic approach to peer reviewing is possible and recommended, and can help you getting started to provide quality peer reviews that contribute to moving the field of plastic surgery forward.

Mastoidectomy Training: Is Anatomical Variation Needed? A Randomized, Controlled Trial on Performance and Skills Transfer From Virtual Reality to a Three-Dimensional Printed Model.

OBJECTIVE: Virtual reality (VR) simulation-based training effectively improves novices' mastoidectomy skills. Unfortunately, learning plateaus at an insufficient level and knowledge on optimizing mastoidectomy training to overcome this plateau is needed. In this study, we aim to investigate how training on anatomically different temporal bone cases affects learning, including the effect on retention and transfer of skills. STUDY DESIGN: Randomized controlled trial of an educational intervention. SETTING: The Simulation Center at Copenhagen Academy for Medical Education and Simulation. PARTICIPANTS: Twenty-four medical students from the University of Copenhagen. INTERVENTION: Participants were randomized to practice mastoidectomy on either 12 anatomically varying (intervention group) or 12 identical (control group) cases in a VR simulator. At the end of training and again ~ 3 weeks after training (retention), learners were tested on a new VR patient case and a three-dimensional printed model. MAIN OUTCOME MEASURE: Mastoidectomy performance evaluated by blinded expert raters using a 26-item modified Welling Scale. RESULTS: The intervention and control groups' performance results were comparable at the end of training. Likewise, retention and transfer performances were similar between groups. The overall mean score at the end of training corresponded to approximately 70% of the possible maximum score. CONCLUSIONS: Simulation-based training using anatomical variation was equivalent to training on a single case with respect to acquisition, retention, and transfer of mastoidectomy skills. This suggests that efforts to expose novices to variation during initial training are unnecessary as this variation has limited effect, and-conversely-that educators can expose novices to naturally different anatomical variations without worry of hindered learning.

Cochlear Implant Surgery: Virtual Reality Simulation Training and Transfer of Skills to Cadaver Dissection-A Randomized, Controlled Trial.

BACKGROUND: Cochlear implantation requires excellent surgical skills; virtual reality simulation training is an effective method for acquiring basic competency in temporal bone surgery before progression to cadaver dissection. However, cochlear implantation virtual reality simulation training remains largely unexplored and only one simulator currently supports the training of the cochlear implantation electrode insertion. Here, we aim to evaluate the effect of cochlear implantation virtual reality simulation training on subsequent cadaver dissection performance and self-directedness. METHODS: This was a randomized, controlled trial. Eighteen otolaryngology residents were randomized to either mastoidectomy including cochlear implantation virtual reality simulation training (intervention) or mastoidectomy virtual reality simulation training alone (controls) before cadaver cochlear implantation surgery. Surgical performance was evaluated by two blinded expert raters using a validated, structured assess- ment tool. The need for supervision (reflecting self-directedness) was assessed via post-dissection questionnaires. RESULTS: The intervention group achieved a mean score of 22.9 points of a maximum of 44 points, which was 5.4% higher than the control group's 21.8 points (P = .51). On average, the intervention group required assistance 1.3 times during cadaver drilling; this was 41% more frequent in the control group who received assistance 1.9 times (P = .21). CONCLUSION: Cochlear implantation virtual reality simulation training is feasible in the context of a cadaver dissection course. The addition of cochlear implantation virtual reality training to basic mastoidectomy virtual reality simulation training did not lead to a significant improvement of performance or self-directedness in this study. Our findings suggest that learning an advanced temporal bone procedure such as cochlear implantation surgery requires much more training than learning mastoidectomy.

The cutting edge of customized surgery: 3D-printed models for patient-specific interventions in otology and auricular management-a systematic review.

PURPOSE: 3D-printing (three-dimensional printing) is an emerging technology with promising applications for patient-specific interventions. Nonetheless, knowledge on the clinical applicability of 3D-printing in otology and research on its use remains scattered. Understanding these new treatment options is a prerequisite for clinical implementation, which could improve patient outcomes. This review aims to explore current applications of 3D-printed patient-specific otologic interventions, including state of the evidence, strengths, limitations, and future possibilities. METHODS: Following the PRISMA statement, relevant studies were identified through Pubmed, EMBASE, the Cochrane Library, and Web of Science. Data on the manufacturing process and interventions were extracted by two reviewers. Study quality was assessed using Joanna Briggs Institute's critical appraisal tools. RESULTS: Screening yielded 590 studies; 63 were found eligible and included for analysis. 3D-printed models were used as guides, templates, implants, and devices. Outer ear interventions comprised 73% of the studies. Overall, optimistic sentiments on 3D-printed models were reported, including increased surgical precision/confidence, faster manufacturing/operation time, and reduced costs/complications. Nevertheless, study quality was low as most studies failed to use relevant objective outcomes, compare new interventions with conventional treatment, and sufficiently describe manufacturing. CONCLUSION: Several clinical interventions using patient-specific 3D-printing in otology are considered promising. However, it remains unclear whether these interventions actually improve patient outcomes due to lack of comparison with conventional methods and low levels of evidence. Further, the reproducibility of the 3D-printed interventions is compromised by insufficient reporting. Future efforts should focus on objective, comparative outcomes evaluated in large-scale studies.

Assessing competence in cochlear implant surgery using the newly developed Cochlear Implant Surgery Assessment Tool.

PURPOSE: To develop and gather validity evidence for a novel tool for assessment of cochlear implant (CI) surgery, including virtual reality CI surgery training. METHODS: Prospective study gathering validity evidence according to Messick's framework. Four experts developed the CI Surgery Assessment Tool (CISAT). A total of 35 true novices (medical students), trained novices (residents) and CI surgeons performed two CI-procedures each in the Visible Ear Simulator, which were rated by three blinded experts. Classical test theory and generalizability theory were used for reliability analysis. RESULTS: The CISAT significantly discriminated between the three groups (p < 0.001). The generalizability coefficient was 0.76 and most of the score variance (53.3%) was attributable to the participant and only 6.8% to the raters. When exploring a standard setting for CI surgery, the contrasting groups method suggested a pass/fail score of 36.0 points (out of 55), but since the trained novices performed above this, we propose using the mean CI surgeon performance score (45.3 points). CONCLUSION: Validity evidence for simulation-based assessment of CI performance supports the CISAT. Together with the standard setting, the CISAT might be used to monitor progress in competency-based training of CI surgery and to determine when the trainee can advance to further training.

Cochlear implantation: Exploring the effects of 3D stereovision in a digital microscope for virtual reality simulation training - A randomized controlled trial.

In cochlear implantation (CI), excellent surgical technique is critical for hearing outcomes. Recent advances in temporal bone Virtual Reality (VR) training allow for specific training of CI and through introduction of new digital microscopes with ultra-high-fidelity (UHF) graphics. This study aims to investigate whether UHF increases performance in VR simulation training of CI electrode insertion compared with conventional, screen-based VR (cVR). METHODS: Twenty-four medical students completed a randomized, controlled trial of an educational intervention. They performed a total of eight CI electrode insertions each in blocks of four using either UHF-VR or cVR, in randomized order. CI electrode insertion performances were rated by two blinded expert raters using a structured assessment tool supported by validity evidence. RESULTS: Performance scores in cVR were higher than in the UHF-VR simulation although this was not significant (19.8 points, 95% CI [19.3-20.3] vs. 18.8 points, 95% CI [18.2-19.4]; P = 0.09). The decisive factor for performance was participants' ability to achieve stereovision (mean difference = 1.1 points, 95% CI [0.15-2.08], P = 0.02). DISCUSSION: No additional benefit was found from UHF-VR over cVR training of CI electrode insertion for novices. Consequently, standard cVR simulation should be used for novices' basic skills acquisition in CI surgery. Future studies should instead explore the effects of other improvements in CI surgery training and if the lacking benefit of UHF-VR also applies for more experienced learners. CONCLUSION: The increased graphical perception and the superior lifelikeness of UHF-VR does not improve early skills acquisition of CI insertion for novices.

Effect of 3D-Printed Models on Cadaveric Dissection in Temporal Bone Training.

OBJECTIVE: Mastoidectomy is a cornerstone in the surgical management of middle and inner ear diseases. Unfortunately, training is challenged by insufficient access to human cadavers. Three-dimensional (3D) printing of temporal bones could alleviate this problem, but evidence on their educational effectiveness is lacking. It is largely unknown whether training on 3D-printed temporal bones improves mastoidectomy performance, including on cadavers, and how this training compares with virtual reality (VR) simulation. To address this knowledge gap, this study investigated whether training on 3D-printed temporal bones improves cadaveric dissection performance, and it compared this training with the already-established VR simulation. STUDY DESIGN: Prospective cohort study of an educational intervention. SETTING: Tertiary university hospital, cadaver dissection laboratory, and simulation center in Copenhagen, Denmark. METHODS: Eighteen otorhinolaryngology residents (intervention) attending the national temporal bone dissection course received 3 hours of mastoidectomy training on 3D-printed temporal bones. Posttraining cadaver mastoidectomy performances were rated by 3 experts using a validated assessment tool and compared with those of 66 previous course participants (control) who had received time-equivalent VR training prior to dissection. RESULTS: The intervention cohort outperformed the controls during cadaver dissection by 29% (P < .001); their performances were largely similar across training modalities but remained at a modest level (~50% of the maximum score). CONCLUSION: Mastoidectomy skills improved from training on 3D-printed temporal bone and seemingly more so than on time-equivalent VR simulation. Importantly, these skills transferred to cadaveric dissection. Training on 3D-printed temporal bones can effectively supplement cadaver training when learning mastoidectomy.

Cochlear implant surgery: Learning curve in virtual reality simulation training and transfer of skills to a 3D-printed temporal bone - A prospective trial.

OBJECTIVE: Mastering Cochlear Implant (CI) surgery requires repeated practice, preferably initiated in a safe - i.e. simulated - environment. Mastoidectomy Virtual Reality (VR) simulation-based training (SBT) is effective, but SBT of CI surgery largely uninvestigated. The learning curve is imperative for understanding surgical skills acquisition and developing competency-based training. Here, we explore learning curves in VR SBT of CI surgery and transfer of skills to a 3D-printed model. METHODS: Prospective, single-arm trial. Twenty-four novice medical students completed a pre-training CI inserting test on a commercially available pre-drilled 3D-printed temporal bone. A training program of 18 VR simulation CI procedures was completed in the Visual Ear Simulator over four sessions. Finally, a post-training test similar to the pre-training test was completed. Two blinded experts rated performances using the validated Cochlear Implant Surgery Assessment Tool (CISAT). Performance scores were analyzed using linear mixed models. RESULTS: Learning curves were highly individual with primary performance improvement initially, and small but steady improvements throughout the 18 procedures. CI VR simulation performance improved 33% (p < 0.001). Insertion performance on a 3D-printed temporal bone improved 21% (p < 0.001), demonstrating skills transfer. DISCUSSION: VR SBT of CI surgery improves novices' performance. It is useful for introducing the procedure and acquiring basic skills. CI surgery training should pivot on objective performance assessment for reaching pre-defined competency before cadaver - or real-life surgery. Simulation-based training provides a structured and safe learning environment for initial training. CONCLUSION: CI surgery skills improve from VR SBT, which can be used to learn the fundamentals of CI surgery.

3D-Printed Models for Temporal Bone Surgical Training: A Systematic Review.

OBJECTIVE: 3D-printed models hold great potential for temporal bone surgical training as a supplement to cadaveric dissection. Nevertheless, critical knowledge on manufacturing remains scattered, and little is known about whether use of these models improves surgical performance. This systematic review aims to explore (1) methods used for manufacturing and (2) how educational evidence supports using 3D-printed temporal bone models. DATA SOURCES: PubMed, Embase, the Cochrane Library, and Web of Science. REVIEW METHODS: Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, relevant studies were identified and data on manufacturing and validation and/or training extracted by 2 reviewers. Quality assessment was performed using the Medical Education Research Study Quality Instrument tool; educational outcomes were determined according to Kirkpatrick's model. RESULTS: The search yielded 595 studies; 36 studies were found eligible and included for analysis. The described 3D-printed models were based on computed tomography scans from patients or cadavers. Processing included manual segmentation of key structures such as the facial nerve; postprocessing, for example, consisted of removal of print material inside the model. Overall, educational quality was low, and most studies evaluated their models using only expert and/or trainee opinion (ie, Kirkpatrick level 1). Most studies reported positive attitudes toward the models and their potential for training. CONCLUSION: Manufacturing and use of 3D-printed temporal bones for surgical training are widely reported in the literature. However, evidence to support their use and knowledge about both manufacturing and the effects on subsequent surgical performance are currently lacking. Therefore, stronger educational evidence and manufacturing knowhow are needed for widespread implementation of 3D-printed temporal bones in surgical curricula.

Current Status of Handheld Otoscopy Training: A Systematic Review.

OBJECTIVE: Otoscopy is a frequently performed procedure and competency in this skill is important across many specialties. We aim to systematically review current medical educational evidence for training of handheld otoscopy skills. METHODS: Following the PRISMA guideline, studies reporting on training and/or assessment of handheld otoscopy were identified searching the following databases: PubMed, Embase, OVID, the Cochrane Library, PloS Medicine, Directory of Open Access Journal (DOAJ), and Web of Science. Two reviewers extracted data on study design, training intervention, educational outcomes, and results. Quality of educational evidence was assessed along with classification according to Kirkpatrick's model of educational outcomes. RESULTS: The searches yielded a total of 6064 studies with a final inclusion of 33 studies for the qualitative synthesis. Handheld otoscopy training could be divided into workshops, physical simulators, web-based training/e-learning, and smartphone-enabled otoscopy. Workshops were the most commonly described educational intervention and typically consisted of lectures, hands-on demonstrations, and training on peers. Almost all studies reported a favorable effect on either learner attitude, knowledge, or skills. The educational quality of the studies was reasonable but the educational outcomes were mostly evaluated on the lower Kirkpatrick levels with only a single study determining the effects of training on actual change in the learner behavior. CONCLUSION: Overall, it seems that any systematic approach to training of handheld otoscopy is beneficial in training regardless of learner level, but the heterogeneity of the studies makes comparisons between studies difficult and the relative effect sizes of the interventions could not be determined.

Ultra-high-fidelity virtual reality mastoidectomy simulation training: a randomized, controlled trial.

PURPOSE: Ultra-high-fidelity (UHF) graphics in virtual reality (VR) simulation might improve surgical skill acquisition in temporal bone training. This study aims to compare UHF VR simulation training with conventional, screen-based VR simulation training (cVR) with respect to performance and cognitive load (CL). METHODS: In a randomized trial with a cross-over design, 24 students completed a total of four mastoidectomies in a VR temporal bone surgical simulator: two performances under UHF conditions using a digital microscope and two performances under conventional conditions using screen-based VR simulation. Performances were assessed by two blinded raters using an established assessment tool. In addition, CL was estimated as the relative change in secondary-task reaction time during simulation when compared with individual baseline measurements. Data were analyzed using linear mixed model analysis for repeated measurements. RESULTS: The mean final-product performance score was significantly lower in UHF VR simulation compared to cVR simulation [mean difference 1.0 points out of 17 points, 95% CI (0.2-1.7), p = 0.02]. The most important factor for performance during UHF simulation was the ability to achieve stereovision (mean difference = 3.4 points, p < 0.001). Under the UHF VR condition, CL was significantly higher than during cVR (28% vs. 18%, respectively, p < 0.001). CONCLUSION: UHF graphics in VR simulation training reduced performance and induced a higher CL in novices than conventional, screen-based VR simulation training. Consequently, UHF VR simulation training should be preceded by cVR training and might be better suited for the training of intermediates or experienced surgeons.

European status on temporal bone training: a questionnaire study.

PURPOSE: In otorhinolaryngology training, introduction to temporal bone surgery through hands-on practice on cadaveric human temporal bones is the gold-standard training method before commencing supervised surgery. During the recent decades, the availability of such specimens and the necessary laboratory facilities for training seems to be decreasing. Alternatives to traditional training can consist of drilling artificial models made of plaster or plastic but also virtual reality (VR) simulation. Nevertheless, the integration and availability of these alternatives into specialist training programs remain unknown. METHODS: We conducted a questionnaire study mapping current status on temporal bone training and included responses from 113 departments from 23 countries throughout Europe. RESULTS: In general, temporal bone training during residency in ORL is organized as in-house training, or as participation in national or international temporal bone courses or some combination hereof. There are considerable differences in the availability of training facilities for temporal bone surgery and the number of drillings each ORL trainee can perform. Cadaveric dissection is still the most commonly used training modality. CONCLUSIONS: VR simulation and artificial models are reported to be used at many leading training departments already. Decreasing availability of cadavers, lower costs of VR simulation and artificial models, in addition to established evidence for a positive effect on the trainees' competency, were reported as the main reasons. Most remaining departments expect to implement VR simulation and artificial models for temporal bone training into their residency programs in the near future.