Training on Congenital 3D Cardiac Models - Will Models Improve Surgical Performance?

Technical skill development in congenital heart surgery (CHS) is challenging due to numerous factors which potentially limit the hands-on operative exposure in surgical training. These challenges have stimulated the growth of simulation-based training through the development of 3D-printed models, providing hands-on surgical training (HOST). From its inception in 2015, the models used in the HOST program have constantly improved, and now include valvar/subvalvar apparatus and better materials that mimic real tissue. Evidence shows that deliberate, regular simulation practice can improve a surgeon's technical skills across the spectrum of CHS. Furthermore, surgical trainees who undergo simulation training are able to translate this improved performance into the operative environment with improved patient outcomes. Despite evidence to support the incorporation of simulation methods into congenital training, its widespread adoption into training curricula remains low. This is due to numerous factors including funding, lack of dedicated time or proctorship and access to models-all of which can be overcome with the newer generation of models and committed trainers. Training programs should consider incorporating simulation-methods as a routine component of congenital training programs.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training.

BACKGROUND: Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum. METHODS: Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA. RESULTS: Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion. CONCLUSION: A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Remote training in laparoscopy: a randomized trial comparing home-based self-regulated training to centralized instructor-regulated training.

BACKGROUND: Simulation-based surgical training (SBST) is key to securing future surgical expertise. Proficiency-based training (PBT) in laparoscopy has shown promising results on skills transfer. However, time constraints and limited possibilities for distributed training constitute barriers to effective PBT. Home-based training may provide a solution to these barriers and may be a feasible alternative to centralized training in times of assembly constraints. METHODS: We randomly assigned first-year trainees in abdominal surgery, gynecology, and urology to either centralized instructor-regulated training (CIRT) or home-based self-regulated training (HSRT) in laparoscopy. All participants trained on portable box trainers providing feedback on metrics and possibility for video reviewing. Training in both groups was structured as PBT with graded proficiency exercises adopted from the Fundamentals of Laparoscopic Surgery (FLS). The HSRT group trained at home guided by online learning materials, while the CIRT group attended two training sessions in the simulation center with feedback from experienced instructors. Performance tests consisted of hand-eye and bimanual coordination, suture and knot-tying, and FLS exercises. We analyzed passing rates, training time and distribution, and test performances. RESULTS: Passing rates were 87% and 96% in the CIRT and HSRT group, respectively. HSRT facilitated distributed training and resulted in greater variation in training times. Task times for hand-eye and bimanual coordination were significantly reduced between pretest and posttest in both groups. Trainees maintained their posttest performances at the 6-month retention test. Our analyses revealed no significant inter-group differences in performances at pretest, posttest, or retention test. Performance improvements in the two groups followed similar patterns. CONCLUSION: CIRT and HSRT in laparoscopy result in comparable performance improvements. HSRT in laparoscopy is a feasible and effective alternative to CIRT when offered inside a supportive instructional design. Further research is needed to clarify trainees' preferences and explore facilitators and barriers to HSRT.

Objective assessment of intraoperative skills for robot-assisted radical prostatectomy (RARP): results from the ERUS Scientific and Educational Working Groups Metrics Initiative.

OBJECTIVE: To develop and seek consensus from procedure experts on the metrics that best characterise a reference robot-assisted radical prostatectomy (RARP) and determine if the metrics distinguished between the objectively assessed RARP performance of experienced and novice urologists, as identifying objective performance metrics for surgical training in robotic surgery is imperative for patient safety. MATERIALS AND METHODS: In Study 1, the metrics, i.e. 12 phases of the procedure, 81 steps, 245 errors and 110 critical errors for a reference RARP were developed and then presented to an international Delphi panel of 19 experienced urologists. In Study 2, 12 very experienced surgeons (VES) who had performed >500 RARPs and 12 novice urology surgeons performed a RARP, which was video recorded and assessed by two experienced urologists blinded as to subject and group. Percentage agreement between experienced urologists for the Delphi meeting and Mann-Whitney U- and Kruskal-Wallis tests were used for construct validation of the newly identified RARP metrics. RESULTS: At the Delphi panel, consensus was reached on the appropriateness of the metrics for a reference RARP. In Study 2, the results showed that the VES performed ~4% more procedure steps and made 72% fewer procedure errors than the novices (P = 0.027). Phases VIIa and VIIb (i.e. neurovascular bundle dissection) best discriminated between the VES and novices. LIMITATIONS: VES whose performance was in the bottom half of their group demonstrated considerable error variability and made five-times as many errors as the other half of the group (P = 0.006). CONCLUSIONS: The international Delphi panel reached high-level consensus on the RARP metrics that reliably distinguished between the objectively scored procedure performance of VES and novices. Reliable and valid performance metrics of RARP are imperative for effective and quality assured surgical training.

Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks.

BACKGROUND: Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator. METHOD: Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0-100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages. RESULTS: Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%). CONCLUSION: This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.

Development of a train-to-proficiency curriculum for the technical skills component of the fundamentals of endoscopic surgery exam.

BACKGROUND: The demonstration of competency in endoscopy is required prior to obtaining American Board of Surgery Certification. To demonstrate competency, the resident must pass a national high-stakes cognitive test and a technical skills exam on a virtual reality simulator. The purpose of this preliminary study was to design a proficiency-based endoscopy simulation curriculum to meet this competency requirement. METHODS: This is a mixed methods prospective cohort study at a single academic medical institution. Prior to taking the national exam, surgery residents were required to participate in a skills lab and demonstrate proficiency on 10 simulation tasks. Proficiency was based on time and percent of objects targeted/mucosa seen. Simulation practice time, number of task repetitions to proficiency, and prior endoscopic experience were recorded. Resident's self-reported confidence scores in endoscopic skills prior to and following simulation lab training were obtained. RESULTS: From January 1, 2016 through August 1, 2017, 20 surgical residents (8 PGY2, 8 PGY3, 4 PGY4) completed both a faculty-supervised endoscopy skills lab and independent learning with train-to-proficiency simulation tasks. Median overall simulator time per resident was 306 min (IQR: 247-405 min). Median overall time to proficiency in all tasks was 235 min (IQR: 208-283 min). The median time to proficiency decreased with increasing PGY status (r = 0.4, P = 0.05). There was no correlation between prior real-time endoscopic experience and time to proficiency. Reported confidence in endoscopic skills increased significantly from mean of 5.75 prior to 7.30 following the faculty-supervised endoscopy skills lab (P = 0.0002). All 20 residents passed the national exam. CONCLUSIONS: In this preliminary study, a train-to-proficiency curriculum in endoscopy improved surgical resident's confidence in their endoscopic skills and 100% of residents passed the FES technical skills test on their first attempt. Our findings also indicate that uniform proficiency was not achieved by real-time experience alone.

Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure.

BACKGROUND: The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study's objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum. METHODS: Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module 'Guided Vaginal Cuff Closure' six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach's alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups' method. RESULTS: The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach's alpha). The experienced surgeons' mean composite score for all six repetitions were significantly better than the novice surgeons' (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives). CONCLUSION: Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.

Impact of 3D in the training of basic laparoscopic skills and its transferability to 2D environment: a prospective randomized controlled trial.

BACKGROUND: Various studies have shown the benefit of three-dimensional (3D) systems over two-dimensional (2D) systems in laparoscopic surgery. However, a few studies have shown the impact of proficiency-based training and transferability of the skills from 2D to 3D system and vice versa. PURPOSE: The purpose of this study was to investigate the impact of 2D and 3D environments on the time taken by novice trainees to learn basic laparoscopic skills and the transferability of the skills learnt in one environment to another. METHODS: Medical students with no prior experience in training of laparoscopic tasks were randomly assigned to 2D or 3D groups. Both groups were trained in peg transfer and intracorporeal suture of Fundamentals of Laparoscopic Surgery (FLS) program until they attained predefined benchmarks. The practice times taken to reach the goal were compared. After they had reached the goal, all the participants were asked to repeat these tasks under both 2D and 3D environments. Their scores in both of the environments were compared. RESULTS: Eighty-eight participants were randomly assigned to 2D (n = 44) or 3D (n = 44) training groups. There was no difference between the backgrounds of both groups. The 3D training group was faster in reaching its goal in both tasks (p < 0.001 and p = 0.007, respectively). After reaching the goal, both groups were faster in the 3D environment in peg transfer (p = 0.04, p = 0.012). In intracorporeal suture, the 2D training group had similar scores in both the environments (p = 0.32), but the 3D group was slower in the 2D environment (p = <0.001). CONCLUSION: The 3D environment helped shorten the training time of basic laparoscopic skills to novices. However, the novices trained in the 3D environment failed to transfer their skills to the 2D environment.

Towards standardized robotic surgery in gastrointestinal oncology.

PURPOSE: Minimally invasive techniques have revolutionized the field of surgery over the past several decades. Specifically, robotic surgery is increasingly being used for complex operations, although the appreciable learning curve required to become proficient has deterred many surgeons. We describe how use of a proficiency-based training program can decrease the learning curve and lead to standardized robotic surgery. METHODS: The steps of a proficiency-based robotic training program are described, including (1) a proficiency-based virtual reality simulation curriculum, (2) an inanimate biotissue curriculum, (3) a video library training, (4) intraoperative evaluation, and (5) skill maintenance with ongoing assessment. The learning curve for robotic gastrointestinal surgery is explored, as well as outcomes compared to laparoscopic and open techniques. RESULTS: The implementation of a proficiency-based robotic training program is feasible. Surgical oncology fellows who participated in the program demonstrated improvement in both the simulation and biotissue curricula. Analyzed as a group, the participants improved in time and errors after the biotissue curriculum. CONCLUSIONS: Published outcomes from robotic gastrointestinal surgery have demonstrated safety, feasibility, and preserved oncologic resections. A proficiency-based robotic curriculum is ideal to enable surgeons to achieve mastery in robotic surgery while minimizing the learning curve required.

Transferability of Virtual Reality, Simulation-Based, Robotic Suturing Skills to a Live Porcine Model in Novice Surgeons: A Single-Blind Randomized Controlled Trial.

STUDY OBJECTIVE: To assess whether a robotic simulation curriculum for novice surgeons can improve performance of a suturing task in a live porcine model. DESIGN: Randomized controlled trial (Canadian Task Force classification I). SETTING: Academic medical center. PATIENTS: Thirty-five medical students without robotic surgical experience. INTERVENTIONS: Participants were enrolled in an online session of training modules followed by an in-person orientation. Baseline performance testing on the Mimic Technologies da Vinci Surgical Simulator (dVSS) was also performed. Participants were then randomly assigned to the completion of 4 dVSS training tasks (camera clutching 1, suture sponge 1 and 2, and tubes) versus no further training. The intervention group performed each dVSS task until proficiency or up to 10 times. A final suturing task was performed on a live porcine model, which was video recorded and blindly assessed by experienced surgeons. The primary outcomes were Global Evaluative Assessment of Robotic Skills (GEARS) scores and task time. The study had 90% power to detect a mean difference of 3 points on the GEARS scale, assuming a standard deviation (SD) of 2.65, and 80% power to detect a mean difference of 3 minutes, assuming an SD of 3 minutes. MEASUREMENTS AND MAIN RESULTS: There were no differences in demographics and baseline skills between the 2 groups. No significant differences in task time in minutes or GEARS scores were seen for the final suturing task between the intervention and control groups, respectively (9.2 [2.65] vs 9.9 [2.07] minutes, p = .406; and 15.37 [2.51] vs 15.25 [3.38], p = .603). The 95% confidence interval for the difference in mean task times was -2.36 to .96 minutes and for mean GEARS scores -1.91 to 2.15 points. CONCLUSIONS: Live suturing task performance was not improved with a proficiency-based virtual reality simulation suturing curriculum compared with standard orientation to the da Vinci robotic console in a group of novice surgeons.

Proficiency Levels and Validity Evidence for Scoring Metrics for a Virtual Reality and Inanimate Robotic Surgery Simulation Curriculum.

OBJECTIVE: Our institution recently implemented a virtual reality (VR) skills curriculum for general surgery residents using the SimNow simulator. Based on a content alignment study, we revised the curriculum to include only 20 of 33 VR tasks and we added 3 previously validated inanimate tasks. The purpose of this study was to establish expert-derived proficiency levels for all tasks and to evaluate the validity of the scoring for the VR tasks. DESIGN: Two expert robotic surgeons performed 5 repetitions of each VR and inanimate task. The trimmed mean (lowest scoring attempt and outliers [>2 standard deviations] were eliminated) was defined as the expert level for each task. For the VR tasks, expert levels were compared to resident performance to evaluate validity. SETTING: This study was conducted at the University of Texas Southwestern Medical Center (Dallas, TX), a tertiary care academic teaching hospital. PARTICIPANTS: Two expert robotic surgeons participated in this study. The data from 42 residents (PGY2-4) who completed the original curriculum was used to represent novice performance. RESULTS: Comparison of expert levels and resident performance was statistically significant for 15 VR tasks (supporting validity) and approached significance (p = 0.06, 0.09) for 2 VR tasks; expert levels were designated as proficiency levels for these 17 tasks. Group comparisons were clearly not significant (p = 0.2-0.8) for 3 VR tasks; 2 of these 3 tasks were retained as introductory exercises (with 3 repetitions required) and 1 was excluded. For the 3 inanimate tasks, expert levels minus 2 standard deviations were designated as proficiency levels. CONCLUSIONS: This analysis generated validity evidence for 15 VR tasks and established expert-derived proficiency levels for 17 VR tasks and 3 inanimate tasks. Our proposed curriculum now consists of 19 VR and 3 inanimate tasks using the selected proficiency levels. We anticipate that this design will maximize curriculum efficiency and effectiveness.

Evaluation of different approaches to define expert benchmark scores for new robotic training simulators based on the Medtronic HUGO™ RAS surgical robot experience.

New robot-assisted surgery platforms being developed will be required to have proficiency-based simulation training available. Scoring methodologies and performance feedback for trainees are currently not consistent across all robotic simulator platforms. Also, there are virtually no prior publications on how VR simulation passing benchmarks have been established. This paper compares methods evaluated to determine the proficiency-based scoring thresholds (a.k.a. benchmarks) for the new Medtronic Hugo™ RAS robotic simulator. Nine experienced robotic surgeons from multiple disciplines performed the 49 skills exercises 5 times each. The data were analyzed in 3 different ways: (1) include all data collected, (2) exclude first sessions, (3) exclude outliers. Eliminating the first session discounts becoming familiar with the exercise. Discounting outliers allows removal of potentially erroneous data that may be due to technical issues, unexpected distractions, etc. Outliers were identified using a common statistical technique involving the interquartile range of the data. Using each method above, mean and standard deviations were calculated, and the benchmark was set at a value of 1 standard deviation above the mean. In comparison to including all the data, when outliers are excluded, fewer data points are removed than just excluding first sessions, and the metric benchmarks are made more difficult by an average of 11%. When first sessions are excluded, the metric benchmarks are made easier by an average of about 2%. In comparison with benchmarks calculated using all data points, excluding outliers resulted in the biggest change making the benchmarks more challenging. We determined that this method provided the best representation of the data. These benchmarks should be validated with future clinical training studies.

Development of a proficiency-based training curriculum for beginners in endoscopic endonasal surgery.

OBJECTIVE: Precise endoscopic skills are critical for endoscopic endonasal surgery (EES) to ensure patient safety, as well as to succeed as a surgeon. Among these skills, basic skills such as hand-eye coordination and bimanual dexterity are the most important skills for novice surgeons. However, there is no proficiency-based training for the basic skills in EES. In this study, we aimed to develop a proficiency-based training curriculum for beginners in EES using tractable tasks and to assess the curriculum's validity and educational benefits. METHODS: Training tasks for basic surgical skills of EES were created by experts, using a low-cost and simple simulator. Subsequently, a proficiency-based training curriculum was developed through a preliminary study. The face and content validity of the curriculum was evaluated by expert surgeons. The construct validity was confirmed by comparing the scores of the medical students with those of the experts. In addition, the educational benefits of the curriculum were assessed by comparing the scores of the medical students before and after the implementation of the curriculum. RESULTS: A proficiency-based training curriculum using six tractable tasks was developed in the preliminary study. Replication of real surgical situation, camera navigation, and instrument operability under endoscopy were all highly appreciated (4.6, 4.6, and 4.8 out of 5 points, respectively), which indicates face validity. The content validity of the curriculum was demonstrated by the results of the questionnaire (4.4 out of 5 points). The curriculum was implemented by 12 medical students. The initial student scores were significantly lower than the expert scores (mean score: 61.9 vs. 100.0, p < 0.001), and each coefficient of variation of the student scores was higher than that of the expert scores (p < 0.01), supporting the construct validity of the curriculum. The mean student composite scores were statistically improved after the training (61.9 vs. 93.3, p < 0.0001). In addition, improvement of five other medical students' EES skills through this training curriculum was confirmed in cadaver dissection (1.7 vs. 3.7, p < 0.0001). CONCLUSION: The validity of the proficiency-based training curriculum developed through the preliminary study was established based on the questionnaire of experts and the difference between student and expert scores. In addition, the educational benefits were demonstrated by the students' learning curves. The low-cost and simple simulator seems appropriate as an entry model for beginners in EES, and this curriculum can provide rapid intervention and objective assessment of basic skills in EES.

International Expert Consensus on Metric-based Characterization of Robot-assisted Partial Nephrectomy.

BACKGROUND: Robot-assisted partial nephrectomy (RAPN) training usually takes place in vivo, and methods vary across countries/institutions. No common system exists to objectively assess trainee ability to perform RAPN at predetermined performance levels prior to in vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. OBJECTIVE: We sought to identify objective performance metrics that best characterize a reference approach to RAPN, and obtain face and content validity from procedure experts through a modified Delphi meeting. DESIGN, SETTING, AND PARTICIPANTS: During a series of online meetings, a core metrics team of three RAPN experts and a senior behavioral scientist performed a detailed task deconstruction of a transperitoneal left-sided RAPN procedure. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Based on published guidelines, manufacturers' instructions, and unedited videos of RAPN, the team identified performance metrics that constitute an optimal approach for training purposes. The metrics were then subjected to an in-person modified international Delphi panel meeting with 19 expert surgeons. RESULTS AND LIMITATIONS: Eleven procedure phases, with 64 procedure steps, 43 errors, and 39 critical errors, were identified. After the modified Delphi process, the international expert panel added 13 metrics (two steps), six were deleted, and three were modified; 100% panel consensus on the resulting metrics was obtained. Limitations are that the metrics are applicable only to left-sided RAPN cases and some might have been excluded. CONCLUSIONS: Performance metrics that accurately characterize RAPN procedure were developed by a core group of experts. The metrics were then presented to and endorsed by an international panel of very experienced peers. Reliable and valid metrics underpin effective, quality-assured, structured surgical training for RAPN. PATIENT SUMMARY: We organize a meeting among robot-assisted partial nephrectomy (RAPN) experts to identify and reach consensus on objective performance metrics for RAPN training. The metrics are a crucial starting point to improve and quality assure surgical training and patients' clinical outcomes.

What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature.

A crucial element of any surgical training program is the ability to provide procedure-specific, objective, and reliable measures of performance. During robotic surgery, objective clinically relevant performance metrics (CRPMs) can provide tailored contextual feedback and correlate with clinical outcomes. This review aims to define CRPMs, assess their validity in robotic surgical training and compare CRPMs to existing measures of robotic performance. A systematic search of Medline and Embase databases was conducted in May 2022 following the PRISMA guidelines. The search terms included Clinically Relevant Performance Metrics (CRPMs) OR Clinically Relevant Outcome Measures (CROMs) AND robotic surgery. The study settings, speciality, operative context, study design, metric details, and validation status were extracted and analysed. The initial search yielded 116 citations, of which 6 were included. Citation searching identified 3 additional studies, resulting in 9 studies included in this review. Metrics were defined as CRPMs, CROMs, proficiency-based performance metrics and reference-procedure metrics which were developed using a modified Delphi methodology. All metrics underwent both contents and construct validation. Two studies found a strong correlation with GEARS but none correlated their metrics with patient outcome data. CRPMs are a validated and objective approach for assessing trainee proficiency. Evaluating CRPMs with other robotic-assessment tools will facilitate a multimodal metric evaluation approach to robotic surgery training. Further studies should assess the correlation with clinical outcomes. This review highlights there is significant scope for the development and validation of CRPMs to establish proficiency-based progression curricula that can be translated from a simulation setting into clinical practice.

Objective assessment of intraoperative skills for robot-assisted partial nephrectomy (RAPN).

RAPN training usually takes place in-vivo and methods vary across countries/institutions. No common system exists to objectively assess trainee capacity to perform RAPN at predetermined performance levels prior to in-vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. The authors sought to examine the reliability, construct and discriminative validity of objective intraoperative performance metrics which best characterize the optimal and suboptimal performance of a reference approach for training novice RAPN surgeons. Seven Novice and 9 Experienced RAPN surgeons video recorded one or two independently performed RAPN procedures in the human. The videos were anonymized and two experienced urology surgeons were trained to reliably score RAPN performance, using previously developed metrics. The assessors were blinded to the performing surgeon, hospital and surgeon group. They independently scored surgeon RAPN performance. Novice and Experienced group performance scores were compared for procedure steps completed and errors made. Each group was divided at the median for Total Errors score, and subgroup scores (i.e., Novice HiErrs and LoErrs, Experienced HiErrs and LoErrs) were compared. The mean inter-rater reliability (IRR) for scoring was 0.95 (range 0.84-1). Compared with Novices, Experienced RAPN surgeons made 69% fewer procedural Total Errors. This difference was accentuated when the LoErr Expert RAPN surgeon's performance was compared with the HiErrs Novice RAPN surgeon's performance with an observed 170% fewer Total Errors. GEARS showed poor reliability (Mean IRR = 0.44; range 0.0-0.8), for scoring RAPN surgical performance. The RAPN procedure metrics reliably distinguish Novice and Experienced surgeon performances. They further differentiated performance levels within a group with similar experiences. Reliable and valid metrics will underpin quality-assured novice RAPN surgical training.

Developing procedural metrics for training in class II posterior composite resin restorations: Outcomes from an international Delphi panel.

OBJECTIVE: The objective of this study was to develop and operationally define procedural performance metrics for the Class II posterior composite resin restoration procedure and to obtain face and content validity through a consensus meeting. METHODS: A Metrics Group consisting of 4 experienced Consultants in Restorative Dentistry, an experienced member of staff from the Restorative Dentistry department in CUDSH and a senior behavioural scientist and education/training expert deconstructed the performance of the Class II posterior composite resin restoration and defined performance metrics. At a modified Delphi meeting, 20 experts in the field of Restorative Dentistry from 11 different Dental Institutions critiqued these metrics and their operational definitions before reaching consensus. RESULTS: Initially performance metrics consisting of 15 Phases, 45 steps, 42 errors and 34 critical errors were identified that characterize the performance of the Class II posterior resin composite procedure. During the Delphi panel these were modified and consensus was reached on 15 Phases (with a change to the initial sequence) with 46 Steps (1 added, 13 modified), 37 Errors (2 added, 1 deleted, 6 reclassified as Critical Error), and 43 Critical Errors (9 added). Consensus on the resulting metrics was obtained and face and content validity verified. CONCLUSIONS: It is possible to develop and objectively define performance metrics that comprehensively characterize a Class II posterior composite resin restoration. It is also possible to achieve consensus on the metrics from a Delphi panel of experts and to confirm the face and content validity of those procedure metrics.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients.

Background: Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve. Objective: To implement a national training programme for simulation-based mastery learning in TURBT and explore operating theatre performance after training. Design setting and participants: From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients. Outcome measurements and statistical analyses: Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample t tests were used to compare pretraining and post-training analyses and independent t tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302. Results and limitations: Before training, novices had significantly lower performance scores in comparison to those with intermediate experience (p = 0.017) and experienced doctors (p < 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; p = 0.049, n = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training (p = 0.9 and p = 0.8, respectively). Conclusions: Novices improved their TURBT performance in the operating theatre after completing a proficiency-based training programme on a virtual reality simulator. Patient summary: We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists.

Demonstrating the effectiveness of the fundamentals of robotic surgery (FRS) curriculum on the RobotiX Mentor Virtual Reality Simulation Platform.

Fundamentals of robotic surgery (FRS) is a proficiency-based progression curriculum developed by robotic surgery experts from multiple specialty areas to address gaps in existing robotic surgery training curricula. The RobotiX Mentor is a virtual reality training platform for robotic surgery. Our aims were to determine if robotic surgery novices would demonstrate improved technical skills after completing FRS training on the RobotiX Mentor, and to compare the effectiveness of FRS across training platforms. An observational, pre-post design, multi-institutional rater-blinded trial was conducted at two American College of Surgeons Accredited Education Institutes-certified simulation centers. Robotic surgery novices (n = 20) were enrolled and trained to expert-derived benchmarks using FRS on the RobotiX Mentor. Participants' baseline skill was assessed before (pre-test) and after (post-test) training on an avian tissue model. Tests were video recorded and graded by blinded raters using the Global Evaluative Assessment of Robotic Skills (GEARS) and a 32-criteria psychomotor checklist. Post hoc comparisons were conducted against previously published comparator groups. On paired-samples T tests, participants demonstrated improved performance across all GEARS domains (p < 0.001 to p = 0.01) and for time (p < 0.001) and errors (p = 0.003) as measured by psychometric checklist. By ANOVA, improvement in novices' skill after FRS training on the RobotiX Mentor was not inferior to improvement reported after FRS training on previously published platforms. Completion of FRS on the RobotiX Mentor resulted in improved robotic surgery skills among novices, proving effectiveness of training. These data provide additional validity evidence for FRS and support use of the RobotiX Mentor for robotic surgery skill acquisition.

Development and validation of a curriculum for laparoscopic supracervical hysterectomy.

STUDY OBJECTIVE: To develop and validate a three-step curriculum for laparoscopic supracervical hysterectomy (LSH) designed for a busy clinical setting. NETHODS: Single-centre, prospective, cohort study. Twelve eligible gynaecological trainees were included (group 1). The theoretical part (step 1) was a validated multiple-choice test. The practical part (step 2) consisted of five tasks on a virtual reality simulator. The participants had to reach a pre-defined proficiency level before advancing to performing a LSH (step 3). The validation of the curriculum was based on the surgical performance. The surgical procedure was recorded and assessed by two experts using Global Operative Assessment of Laparoscopic Skills (GOALS) and Competence Assessment Tool - Laparoscopic Supracervical Hysterectomy (CAT-LSH). The scores were compared with scores from gynaecological trainees who performed their first LSH without virtual reality simulator training (group 2). RESULTS: Ten trainees completed the curriculum and performed a LSH that was recorded and evaluated. Mean duration of the training period (step 1 and 2) was 57 days (SD 26.0), and mean training time spent on the simulator to reach the pre-set proficiency level was 173 min (SD 49). The mean GOALS score was 18.5 (SD 5.8) in group 1 and 13.6 (SD 3.3) in group 2, p=0.027. The mean CAT-LSH score of the performance of the hysterectomy was 42.1 (SD 6.9) in group 1 and 34.8 (SD 4.3) in group 2, p= 0.009. CONCLUSIONS: Trainees who completed the curriculum appeared to have a higher performance score compared with trainees who did not perform structured training.