Comparing the impact of surgical expert versus non-ophthalmologist instructors on virtual-reality surgical performance: A randomized controlled trial.

PURPOSE: To compare Manual Small Incision Cataract Surgery (MSICS) microsurgical performance in course participants who received virtual reality simulation-based training by either a surgical expert or a non-ophthalmologist instructor. SETTING: Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark. DESIGN: Randomized controlled trial. METHODS: Residents and specialists in ophthalmology with no prior MSICS experience were included to receive virtual reality simulation training in MSICS using the HelpMeSee simulator. The participants were randomly allocated to receive training from either an experienced MSICS surgeon or a non-ophthalmologist, also known as near-peer teaching. The performances of the participants were evaluated at baseline and post-training using a MSICS proficiency-based test with evidence of validity. RESULTS: Thirty participants were included in the study and 29 completed the course. There was no significant difference in final test score between the two groups (p = 0.13). The performance score of both groups of participants increased significantly after receiving the training (p < 0.001). All participants passed the proficiency-based test after receiving the training. CONCLUSION: We found no significant difference in surgical proficiency-level whether the participants were trained by a surgical expert or a non-ophthalmologist instructor for MSICS in a virtual-reality based setting. The findings of this study suggest that near-peer teaching within microsurgical performance potentially could be applied with teaching outcomes comparable to a surgical expert-instructor.

Lack of stereopsis does not reduce surgical performance but prolongs the learning curve of vitreoretinal surgery.

PURPOSE: To investigate whether individuals with long-term reduced stereopsis were able to obtain the same level of surgical skills in simulated vitreoretinal surgery on the Eyesi Surgical Simulator as individuals with normal stereopsis. METHODS: Twenty-four medical students were recruited and divided into two groups according to their degree of stereopsis: Group 1 (n = 12) included subjects with normal stereopsis (60 arcsec or lower) and Group 2 (n = 12) included subjects with reduced stereopsis (120 arcsec or higher). Stereopsis was tested using the TNO test (Lameris Ootech BV, Nieuwegein, the Netherlands). The participants were trained in virtual reality-simulated vitreoretinal surgery and continuously measured using a test with solid validity evidence and a pre-defined pass-fail score. All data were analysed using the Wilcoxon rank sum test. RESULTS: We observed no differences in overall performance scores for any of the four modules. The participants with reduced stereopsis used 5.8 more attempts in bimanual training (p = 0.04), 8.8 more attempts in removal of posterior hyaloid (p = 0.04), 9.1 more attempts in navigation training (p = 0.20) and 0.3 fewer attempts in removal of internal limiting membrane (p = 0.69). CONCLUSION: The final performance scores on the Eyesi Surgical Simulator were independent of the degree of stereopsis. However, the number of attempts to achieve the pre-defined pass-fail score increased significantly with reduced stereopsis in two of four modules. These results indicate that a high degree of stereopsis is not necessary to become proficient in microsurgery but may prolong the learning curve.

Ocular tumours.

Ocular tumours may arise from various tissues and therefore present as a heterogeneous group of diseases with unspecific symptoms. Some of the tumours carry a high mortality with a life expectancy less than 50% after ten years. Early diagnosis and treatment are essential for a good outcome, and centralization has led to a decreased morbidity and increased survival in Denmark. Tumour-specific somatic mutations can be used for personalized follow-up programmes and may lead to new treatment modalities, as argued in this review.

A validated test has been developed for assessment of manual small incision cataract surgery skills using virtual reality simulation.

This study investigates the validity evidence of metrics used for the assessment of surgical skills for Manual Small Incision Cataract Surgery (MSICS) in a virtual reality simulator. MSICS surgery is a low-cost, low-technology cataract surgery technique, which is widely used in low- and middle-income countries. However, there is a lack of cataract surgeons globally, and efficient and evidence-based training of new surgeons is needed. In order to investigate the validity of simulator metrics, we included three groups of participants: (1) MSICS novices who were ophthalmologists with no cataract surgery experience, (2) MSICS novices who were experienced phacoemulsification cataract surgeons, but with no MSICS experience, and (3) experienced phacoemulsification and MSICS surgeons. The evaluation included 11 steps of the MSICS procedure, and all simulator metrics for those steps were reviewed. Of the 55 initial metrics, 30 showed high positive discriminative ability. A test passing score of 20 out of 30 was established, and one of 15 novices with no MSICS experience (mean score 15.5) and 7 out of 10 experienced MSICS surgeons (mean score 22.7) passed the test. We have developed and established validity evidence for a test for MSICS skills in a virtual reality simulator for future use in proficiency-based training and evidence-based testing of training interventions.

Quantifying surgical skill in macular surgery.

PURPOSE: To evaluate the relationship between patient outcome and surgical experience by developing an objective quality measure of macular hole surgery based on forceps damage to the inner retina. METHODS: We retrospectively examined 3 macular hole case series >1 year after pars plana vitrectomy, internal limiting membrane peeling and gas tamponade. The patients were operated by (1) a novice surgeon (<20 cases), (2) an intermediate (150+ cases) and (3) an experienced surgeon (2000+ cases). Primary outcome was inner retinal volume defect as segmented from optical coherence tomography (GCL++: thickness from internal limiting membrane to inner plexiform layer). Secondary outcome was retinal function measured by confocal microperimetry using a custom scanning protocol. RESULTS: Thirty-two patients were examined: 11, 10 and 11 patients in the novice, intermediate and experienced surgeon group, respectively. Median GCL++ volume defect was 23.68 × 106 µm3 (IQR: 22.77 × 106 -44.81 × 106 µm3 ), 8.42 × 106 µm3 (IQR: 4.86 × 106 -10.03 × 106 µm3 ) and 3.55 × 106 µm3 (IQR: 1.44 × 103 -7.94 × 106 µm3 ) in the novice, intermediate and experienced surgeon group, respectively (p = 0.0004). The novice surgeon volume defect differed significantly from the intermediate and experienced surgeon (p = 0.016 and p = 0.0002, respectively). A subset of 12 patients underwent microperimetry measurements demonstrating correlation between inner retinal volume defect and reduced retinal sensitivity (p = 0.02). CONCLUSIONS: Forceps induced inner retinal damage commonly occurs during initiation of internal limiting membrane peeling in macular hole surgery. Damage to the structure and function of the inner retina seems to correlate to surgical experience.

Pretraining of basic skills on a virtual reality vitreoretinal simulator: A waste of time.

PURPOSE: To investigate whether pretraining of basic skills in virtual vitreoretinal surgery affected the performance curve when proceeding to procedure-specific modules. METHODS: This study was a prospective, randomized, controlled, two-centre study. Medical students were randomized into two groups: Group 1 pretrained basic psycho-motor skills (Navigation Training level 2 and Bimanual Training level 3) until they reached their performance curve plateau. Hereafter, both groups trained on the procedure-specific modules (Posterior Hyaloid level 3 and ILM Peeling level 3) until they reached their performance curve plateau. Plateau was defined as three consecutive sessions with the same score with an acceptable variation. The primary outcome was time used to reach performance curve plateau in the procedure-specific modules. RESULTS: A total of 68 medical students were included, and equally randomized into two groups. The participants in Group 1 used a median time of 88 minutes to reach plateau in the basic skills modules but did not differ from Group 2 in time to reach plateau on the procedure-specific modules (183 min versus 210 min, p = 0.40) or in the amplitude of plateau. Group 1 and 2 differed significantly in the starting score of ILM peeling level 3 (0 (0-0) versus 3.5 (0-75), p = 0.03). CONCLUSION: We were not able to show positive skill transfer from basic skills training to the procedure-specific modules in time, starting score or amplitude of plateau. Thus, we recommend that aspiring vitreoretinal surgeons proceed directly to simulation-based training of procedures instead of spending valuable training time on basic skills training.

Simulation of advanced cataract surgery - validation of a newly developed test.

PURPOSE: To develop and investigate an Eyesi simulator-based test for the more experienced cataract surgeon for evidence of validity. METHODS: The study was a prospective interventional cohort study and carried out at the Copenhagen Academy for Medical Education and Simulation. The Eyesi Simulator was used for the test which was developed by three expert cataract surgeons. Ten cataract surgeons (>250 surgeries performed) and ten ophthalmic residents performed two repetitions of the test. The test consisted of four modules: Iris Expansion Ring insertion - level 1, Iris Expansion Ring extraction - level 2, Capsulorhexis - level 3 and Anterior Vitrectomy - level 6. RESULTS: Internal consistency reliability showed Cronbach's alpha of 0.63. Test-retest reliabilities were significant for Iris Expansion Ring extraction - level 2 (p = 0.012) and Capsulorhexis - level 3 (p = 0.018). Differences between the two groups were only significant in both repetitions for the Iris Expansion Ring extraction - level 2 (p < 0.001 and p = 0.041, respectively). Furthermore, we found a statistically significant difference between the mean module scores for novices and the more experienced surgeons for Iris Expansion Ring insertion - level 1 (p = 0.021) and Capsulorhexis - level 3 (p = 0.019) in the first repetition. CONCLUSION: The investigated modules show evidence of validity within several aspects of Messick's framework. However, the evidence is not strong enough to apply the test for certification purposes of cataract surgeons, but the modules may still be relevant in the training of advanced cataract surgical procedures.

ROBOT-ASSISTED VITREORETINAL SURGERY IMPROVES SURGICAL ACCURACY COMPARED WITH MANUAL SURGERY: A Randomized Trial in a Simulated Setting.

PURPOSE: To compare manual and robot-assisted vitreoretinal surgery using a virtual-reality surgical simulator. METHODS: Randomized controlled crossover study. Ten experienced vitreoretinal surgeons and 10 novice ophthalmic surgeons were included. The participants were randomized to start with either manual or robot-assisted surgery. Participants completed a test session consisting of three vitreoretinal modules on the Eyesi virtual-reality simulator. The automated metrics of performance supplied by the Eyesi simulator were used as outcome measures. Primary outcome measures were time with instruments inserted (seconds), instrument movement (mm), and tissue treatment (mm). RESULTS: Robot-assisted surgery was slower than manual surgery for both novices and vitreoretinal surgeons, 0.24 SD units (P = 0.024) and 0.73 SD units (P < 0.001), respectively. Robot-assisted surgery allowed for greater precision in novices and vitreoretinal surgeons, -0.96 SD units (P < 0.001) and -0.47 SD units (P < 0.001), respectively. Finally, novices using robot-assisted surgery inflicted less tissue damage when compared with that using manual surgery, -0.59 SD units (P = 0.009). CONCLUSION: At the cost of time, robot-assisted vitreoretinal surgery seems to improve precision and limit tissue damage compared with that of manual surgery. In particular, the performance of novice surgeons is enhanced with robot-assisted vitreoretinal surgery.

Simulators in the training of surgeons: is it worth the investment in money and time? 2018 Jules Gonin lecture of the Retina Research Foundation.

This paper describes transfer of skills obtained from training with the EyeSI virtual reality simulator of ophthalmic surgery to real-life surgical performance. Skills in real-life phacoemulsification surgery were assessed by systematic blinded evaluation of surgical videos based on the OSACCS system. Nineteen Danish cataract surgeons with varying clinical experience levels had their cataract surgery skills evaluated before and after completing a standardized mastery learning program on the EyeSI. It was found that transfer of skills could be demonstrated only for surgeons with a real-life experience of less than 75 completed, independent cases. We could not demonstrate transfer of skills from the EyeSI cataract module to the EyeSI vitreoretinal module, so each subspecialty seems to require specific training. Finally, the discriminative power of EyeSI simulation between emerging surgeons and experts was found to reside only in the first training sessions. The EyeSI simulator in its current state of development, and our implementation of it, seems to require further development before it can be used as a tool to select residents for surgical training and to re-certify more senior surgeons.

Computer analysis of individual cataract surgery segments in the operating room.

PURPOSE: Objective feedback is important for the continuous development of surgical skills. Motion tracking, which has previously been validated across an entire cataract procedure, can be a useful adjunct. We aimed to measure quantitative differences between junior and senior surgeons' performance in three distinct segments. We further explored whether automated analysis of trainee surgical videos through PhacoTracking could be aligned with metrics from the EyeSi virtual reality simulator, allowing focused improvement of these areas in a controlled environment. METHODS: Prospective cohort analysis, comparing junior vs. senior surgeons' real-life performance in distinct segments of cataract surgery: continuous curvilinear capsulorhexis (CCC), phacoemulsification, and irrigation and aspiration (I&A). EyeSi metrics that could be aligned with motion tracking parameters were identified. Motion tracking parameters (instrument path length, number of movements and total time) were measured. t-test used between the two cohorts for each component to check for any significance (p < 0.05). RESULTS: A total of 120 segments from videos of 20 junior and 20 senior surgeons were analysed. Significant differences between junior and senior surgeons were found during CCC (path length p = 0.0004; number of movements p < 0.0001; time taken p < 0.0001), phacoemulsification (path length p < 0.0001; number of movements p < 0.0001; time taken p < 0.0001), and irrigation and aspiration (path length p = 0.006; number of movements p = 0.013; time taken p = 0.036). CONCLUSION: Individual segments of cataract surgery analysed using motion tracking appear to discriminate between junior and senior surgeons. Alignment of motion tracking and EyeSi parameters could enable independent, task specific, objective and quantitative feedback for each segment of surgery thus mirroring the widely utilised modular training.

Surgical simulation: Current practices and future perspectives for technical skills training.

Simulation-based training (SBT) has become a standard component of modern surgical education, yet successful implementation of evidence-based training programs remains challenging. In this narrative review, we use Kern's framework for curriculum development to describe where we are now and what lies ahead for SBT within surgery with a focus on technical skills in operative procedures. Despite principles for optimal SBT (proficiency-based, distributed, and deliberate practice) having been identified, massed training with fixed time intervals or a fixed number of repetitions is still being extensively used, and simulators are generally underutilized. SBT should be part of surgical training curricula, including theoretical, technical, and non-technical skills, and be based on relevant needs assessments. Furthermore, training should follow evidence-based theoretical principles for optimal training, and the effect of training needs to be evaluated using relevant outcomes. There is a larger, still unrealized potential of surgical SBT, which may be realized in the near future as simulator technologies evolve, more evidence-based training programs are implemented, and cost-effectiveness and impact on patient safety is clearly demonstrated.

Consensus on procedures to include in a simulation-based curriculum in ophthalmology: a national Delphi study.

PURPOSE: The number of available simulation-based models for technical skills training in ophthalmology is rapidly increasing, and development of training programmes around these procedures should follow a structured approach. The aim of this study was to identify all technical procedures that should be integrated in a simulation-based curriculum in ophthalmology. METHODS: Key opinion leaders involved in the education of ophthalmologists in Denmark including heads of departments, heads of clinical education, professors and board members of the society were invited to participate in a three-round Delphi process. Round 1 aimed at identifying technical procedures that physicians should be able to perform competently when completing specialty training; round 2 involved characterization of each procedure including frequency, number of operators, risk and/or discomfort for patients associated with an inexperienced physician, and feasibility of simulation-based training; round 3 included a priority ranking of procedures. RESULTS: The response rate for each round was 71%, 64% and 64%, respectively. Sixty-five procedures were reduced to 25 prioritized procedures during the three rounds. Two-thirds of the procedures that were identified and highly prioritized were therapeutic procedures such as intravitreal injection therapy, yttrium-aluminium-garnet laser iridotomy/capsulotomy, minor ocular surface procedures and retinal argon laser therapy. The diagnostic procedures that were prioritized were ocular ultrasound, superficial keratectomy and optical coherence tomography (OCT). CONCLUSION: The Delphi process identified and prioritized 25 procedures that should be practised in a simulation-based environment to achieve competency before working with patients. The list may be used to guide the development of future training programmes for ophthalmologists.

Predictors of visual outcome in patients operated for craniopharyngioma - a Danish national study.

PURPOSE: Craniopharyngioma often causes visual loss due to the close relation to the anterior visual pathways. This study investigates the incidence and predictors of visual outcomes in patients with craniopharyngioma. METHODS: Data from sixty-six patients who underwent surgery for craniopharyngioma from 2009 to 2013 in Denmark were reviewed. Primary outcomes were visual acuity (VA) and visual field (VF) defects from pre-and postoperative visits. Secondary outcomes were optic nerve atrophy (OA) and papilledema. RESULTS: Fifty-eight patients were included. The VA of the patients 1-year after surgery improved by -0.16 log(MAR) (95%CI: -0.30 to -0.02; p = 0.0266). Visual field (VF) defects worsened in 17 eyes (30%), remained stable in 21 eyes (37%) and improved in 19 eyes (33%). The presence of papilledema and the absence of OA were significantly correlated with an improvement in VA postoperatively (p = 0.011 and p = 0.011, respectively). Patients undergoing surgery within a week or less after their first ophthalmological examination had a significant improvement in VA (-0.36; 95%CI: -0.62 to -0.09; p = 0.0099). Patients undergoing surgery using a subfrontal approach also showed improvement in VA (p = 0.048). Tumour recurrence had a significantly worse VA outcome (p = 0.0074). CONCLUSION: Patients show a slight improvement in VA 1-year after operation for craniopharyngioma. The presence of papilledema and early surgical intervention is associated with a significant improvement in VA. Early involvement of a dedicated ophthalmologist is recommended to secure an early detection of a visual decline and potential tumour recurrence.

Is there inter-procedural transfer of skills in intraocular surgery? A randomized controlled trial.

PURPOSE: To investigate how experience in simulated cataract surgery impacts and transfers to the learning curves for novices in vitreoretinal surgery. METHODS: Twelve ophthalmology residents without previous experience in intraocular surgery were randomized to (1) intensive training in cataract surgery on a virtual-reality simulator until passing a test with predefined validity evidence (cataract trainees) or to (2) no cataract surgery training (novices). Possible skill transfer was assessed using a test consisting of all 11 vitreoretinal modules on the EyeSi virtual-reality simulator. All participants repeated the test of vitreoretinal surgical skills until their performance curve plateaued. Three experienced vitreoretinal surgeons also performed the test to establish validity evidence. Analysis with independent samples t-tests was performed. RESULTS: The vitreoretinal test on the EyeSi simulator demonstrated evidence of validity, given statistically significant differences in mean test scores for the first repetition; experienced surgeons scored higher than novices (p = 0.023) and cataract trainees (p = 0.003). Internal consistency for the 11 modules of the test was acceptable (Cronbach's α = 0.73). Our findings did not indicate a transfer effect with no significant differences found between cataract trainees and novices in their starting scores (mean ± SD 381 ± 129 points versus 455 ± 82 points, p = 0.262), time to reach maximum performance level (10.7 ± 3.0 hr versus 8.7 ± 2.8 hr, p = 0.265), or maximum scores (785 ± 162 points versus 805 ± 73 points, p = 0.791). CONCLUSION: Pretraining in cataract surgery did not demonstrate any measurable effect on vitreoretinal procedural performance. The results of this study indicate that we should not anticipate extensive transfer of surgical skills when planning training programmes in intraocular surgery.

High correlation between performance on a virtual-reality simulator and real-life cataract surgery.

PURPOSE: To investigate the correlation in performance of cataract surgery between a virtual-reality simulator and real-life surgery using two objective assessment tools with evidence of validity. METHODS: Cataract surgeons with varying levels of experience were included in the study. All participants performed and videorecorded three standard cataract surgeries before completing a proficiency-based test on the EyeSi virtual-reality simulator. Standard cataract surgeries were defined as: (1) surgery performed under local anaesthesia, (2) patient age >60 years, and (3) visual acuity >1/60 preoperatively. A motion-tracking score was calculated by multiplying average path length and average number of movements from the three real-life surgical videos of full procedures. The EyeSi test consisted of five abstract and two procedural modules: intracapsular navigation, antitremor training, intracapsular antitremor training, forceps training, bimanual training, capsulorhexis and phaco divide and conquer. RESULTS: Eleven surgeons were enrolled. After a designated warm-up period, the proficiency-based test on the EyeSi simulator was strongly correlated to real-life performance measured by motion-tracking software of cataract surgical videos with a Pearson correlation coefficient of -0.70 (p = 0.017). CONCLUSION: Performance on the EyeSi simulator is significantly and highly correlated to real-life surgical performance. However, it is recommended that performance assessments are made using multiple data sources.

Operating Room Performance Improves after Proficiency-Based Virtual Reality Cataract Surgery Training.

PURPOSE: To investigate the effect of virtual reality proficiency-based training on actual cataract surgery performance. The secondary purpose of the study was to define which surgeons benefit from virtual reality training. DESIGN: Multicenter masked clinical trial. PARTICIPANTS: Eighteen cataract surgeons with different levels of experience. METHODS: Cataract surgical training on a virtual reality simulator (EyeSi) until a proficiency-based test was passed. MAIN OUTCOME MEASURES: Technical performance in the operating room (OR) assessed by 3 independent, masked raters using a previously validated task-specific assessment tool for cataract surgery (Objective Structured Assessment of Cataract Surgical Skill). Three surgeries before and 3 surgeries after the virtual reality training were video-recorded, anonymized, and presented to the raters in random order. RESULTS: Novices (non-independently operating surgeons) and surgeons having performed fewer than 75 independent cataract surgeries showed significant improvements in the OR-32% and 38%, respectively-after virtual reality training (P = 0.008 and P = 0.018). More experienced cataract surgeons did not benefit from simulator training. The reliability of the assessments was high with a generalizability coefficient of 0.92 and 0.86 before and after the virtual reality training, respectively. CONCLUSIONS: Clinically relevant cataract surgical skills can be improved by proficiency-based training on a virtual reality simulator. Novices as well as surgeons with an intermediate level of experience showed improvement in OR performance score.

Simulation-based certification for cataract surgery.

PURPOSE: To evaluate the EyeSi(™) simulator in regard to assessing competence in cataract surgery. The primary objective was to explore all simulator metrics to establish a proficiency-based test with solid evidence. The secondary objective was to evaluate whether the skill assessment was specific to cataract surgery. METHODS: We included 26 ophthalmic trainees (no cataract surgery experience), 11 experienced cataract surgeons (>4000 cataract procedures) and five vitreoretinal surgeons. All subjects completed 13 different modules twice. Simulator metrics were used for the assessments. RESULTS: Total module score on seven of 13 modules showed significant discriminative ability between the novices and experienced cataract surgeons. The intermodule reliability coefficient was 0.76 (p < 0.001). A pass/fail level was defined from the total score on these seven modules using the contrasting-groups method. The test had an overall ability to discriminate between novices and experienced cataract surgeons, as 21 of 26 novices (81%) versus one of 11 experienced surgeons (9%) did not pass the test. The vitreoretinal surgeons scored significantly higher than the novices (p = 0.006), but not significantly lower than the experienced cataract surgeons (p = 0.32). CONCLUSION: We have established a performance test, consisting of seven modules on the EyeSi(™) simulator, which possess evidence of validity. The test is a useful and reliable tool for assessment of both cataract surgical and general microsurgical skills in vitro.