Experts in Minimally Invasive Surgery are Outperformed by Trained Novices on Suturing Skills.

INTRODUCTION: Learning minimally invasive suturing can be challenging, creating a barrier to further implementation, especially with the development of easier methods. Nevertheless, mastering intracorporeal knot tying is crucial when alternative techniques prove inadequate. Therefore, the minimally invasive surgery (MIS) suturing skills of MIS experts are compared with a group of novices during their learning curve on a simulator. METHODS: The novice participants repeatedly performed the intracorporeal suturing task on the EoSim MIS simulator (up to a maximum of 20 repetitions). The experts (>50 MIS procedures and advanced MIS experience) completed the same task once. The first and last exercises of the novices and the expert tasks were all blindly recorded and assessed by two independent assessors using the Laparoscopic Suturing Competency Assessment Tool (LS-CAT). Additionally, objective assessment parameters, "time" and "distance", using instrument tracking, were collected. The scores of the experts were then compared with the novices. RESULTS: At the end of the training, novices significantly outperformed the experts on both the expert assessment (LS-CAT: 16.8 versus 26.8, P = 0.001) and objective parameters (median time: 190 s versus 161 s, P < 0.001; median distance: 6.1 m versus 3.6 m, P < 0.001). Although the experts showed slightly better performance than the novices during their first task, the difference was not significant on the expert assessment (LS-CAT experts 16.8, novices 20.5, P = 0.057). CONCLUSIONS: Our findings underscore the significance of continued MIS suturing training for both residents and surgeons. In this study, trained novices demonstrated a significant outperformance of experts on both quantitative and qualitative outcome parameters within a simulated setting.

Robot assisted versus laparoscopic suturing learning curve in a simulated setting.

BACKGROUND: Compared to conventional laparoscopy, robot assisted surgery is expected to have most potential in difficult areas and demanding technical skills like minimally invasive suturing. This study was performed to identify the differences in the learning curves of laparoscopic versus robot assisted suturing. METHOD: Novice participants performed three suturing tasks on the EoSim laparoscopic augmented reality simulator or the RobotiX robot assisted virtual reality simulator. Each participant performed an intracorporeal suturing task, a tilted plane needle transfer task and an anastomosis needle transfer task. To complete the learning curve, all tasks were repeated up to twenty repetitions or until a time plateau was reached. Clinically relevant and comparable parameters regarding time, movements and safety were recorded. Intracorporeal suturing time and cumulative sum analysis was used to compare the learning curves and phases. RESULTS: Seventeen participants completed the learning curve laparoscopically and 30 robot assisted. Median first knot suturing time was 611 s (s) for laparoscopic versus 251 s for robot assisted (p < 0.001), and this was 324 s versus 165 (sixth knot, p < 0.001) and 257 s and 149 s (eleventh knot, p < 0.001) respectively on base of the found learning phases. The percentage of 'adequate surgical knots' was higher in the laparoscopic than in the robot assisted group. First knot: 71% versus 60%, sixth knot: 100% versus 83%, and eleventh knot: 100% versus 73%. When assessing the 'instrument out of view' parameter, the robot assisted group scored a median of 0% after repetition four. In the laparoscopic group, the instrument out of view increased from 3.1 to 3.9% (left) and from 3.0 to 4.1% (right) between the first and eleventh knot (p > 0.05). CONCLUSION: The learning curve of minimally invasive suturing shows a shorter task time curve using robotic assistance compared to the laparoscopic curve. However, laparoscopic outcomes show good end results with rapid outcome improvement.

Competency assessment tool for laparoscopic suturing: development and reliability evaluation.

BACKGROUND: Laparoscopic suturing can be technically challenging and requires extensive training to achieve competency. To date no specific and objective assessment method for laparoscopic suturing and knot tying is available that can guide training and monitor performance in these complex surgical skills. In this study we aimed to develop a laparoscopic suturing competency assessment tool (LS-CAT) and assess its inter-observer reliability. METHODS: We developed a bespoke CAT tool for laparoscopic suturing through a structured, mixed methodology approach, overseen by a steering committee with experience in developing surgical assessment tools. A wide Delphi consultation with over twelve experts in laparoscopic surgery guided the development stages of the tool. Following, subjects with different levels of laparoscopic expertise were included to evaluate this tool, using a simulated laparoscopic suturing task which involved placing of two surgical knots. A research assistant video recorded and anonymised each performance. Two blinded expert surgeons assessed the anonymised videos using the developed LS-CAT. The LS-CAT scores of the two experts were compared to assess the inter-observer reliability. Lastly, we compared the subjects' LS-CAT performance scores at the beginning and end of their learning curve. RESULTS: This study evaluated a novel LS-CAT performance tool, comprising of four tasks. Thirty-six complete videos were analysed and evaluated with the LS-CAT, of which the scores demonstrated excellent inter-observer reliability. Cohen's Kappa analysis revealed good to excellent levels of agreement for almost all tasks of both instrument handling and tissue handling (0.87; 0.77; 0.75; 0.86; 0.85, all with p < 0.001). Subjects performed significantly better at the end of their learning curve compared to their first attempt for all LS-CAT items (all with p < 0.001). CONCLUSIONS: We developed the LS-CAT, which is a laparoscopic suturing grading matrix, with excellent inter-rater reliability and to discriminate between experience levels. This LS-CAT has a potential for wider use to objectively assess laparoscopic suturing skills.

Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.

Extra-corporeal membrane oxygenation (ECMO) cannulation can be a stressful procedure because a fast cannulation is vital for the patient's survival. Therefore, it is important to train the steps of cannulation outside the clinical setting. A relatively low budget, easy to use model, was developed to train the most important steps of an ECMO cannulation. Following this, it was evaluated by experts and target group participants. They all completed a questionnaire regarding their experience and opinions on the ECMO model on general aspects and the training of the component steps, rated on a 5-point Likert scale. Twenty-one participants completed the questionnaire. The features and steps of the model were rated with a mean of 3.9 on average. The haptics of the landscape scored least, with a mean of 3.6, although the haptics of the vessels scored highest with 4.0. The rating of the component steps showed that only 'opening of the vessels' was scored significantly different between the expertise levels (means experts: 4.0, target group: 3.4, p = 0.032). This low budget model is considered to be a valid tool to train the component steps of the ECMO cannulation, which could reduce the learning curve in the a stressful clinical setting. Level of evidence: II prospective comparative study.

Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks.

BACKGROUND: Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator. METHOD: Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0-100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages. RESULTS: Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%). CONCLUSION: This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.

Construct, content and face validity of the eoSim laparoscopic simulator on advanced suturing tasks.

BACKGROUND: The purpose of this study was to validate the eoSim, an affordable and mobile inanimate laparoscopic simulator with instrument tracking capabilities, regarding face, content and construct validity on complex suturing tasks. METHODS: Participants recruited for this study were novices (no laparoscopic experience), target group for this training (surgical/gynaecologic/urologic residents, > 10 basic and < 20 advanced laparoscopic procedures) and experts (> 20 advanced laparoscopic procedures). Each participant performed the intracorporeal suturing exercise (Task 1), an upside down needle transfer (Task 2, developed for this study) and an anastomosis needle transfer (Task 3). Following, the participants completed a questionnaire regarding their demographics and opinion on the eoSim in terms of realism, didactic value and usability. Measured outcome parameters were time, distance, percentage of instrument tip off-screen, working area, speed, acceleration and smoothness. RESULTS: In total, 104 participants completed the study, of which 60 novices, 31 residents and 13 experts. Face and content validity results showed a mean positive opinion on realism (3.9 Task 1, 3.6 Task 2 and 3.7 Task 3), didactic value (4.0, 3.4 and 3.7, respectively) and usability (4.2. 3.7 and 4.0, respectively). There were no significant differences in these outcomes between the specified expertise groups. Construct validity results showed significant differences between experts, target group or novices for Task 1 in terms of time (means 339, 607 and 1224 s, respectively, p < 0.001) and distance (means 8.1, 15.6 and 21.7 m, respectively, p < 0.001). Task 2 showed significant differences between groups regarding time (p < 0.001), distance (p 0.003), off-screen (p < 0.001) and working area (p < 0.001). Task 3 showed significant differences between groups, after subanalyses, on total number of stitches (p < 0.001), time per stitch (p < 0.001) and distance per stitch (p < 0.001). CONCLUSIONS: The results of this study indicate that the eoSim is a potential meaningful and valuable simulator in the training of suturing tasks.

Assessment of Minimally Invasive Suturing Skills: Is Instrument Tracking an Accurate Prediction?

Introduction: Minimally invasive surgery (MIS) suturing demands advanced surgical skills. Therefore, it is important these skills are adequately trained and assessed. Assessment and feedback can consist of judgments and scores of expert observers or objective parameters using instrument tracking. The aim of this study was to determine to what extent objective parameters correspond to expert assessment. Methods: Participants performed an intracorporeal suturing task on the EoSim simulator repeatedly (maximum 20 repetitions) during training. The best discriminating parameters, which previously shown construct validation, were combined into a composite score, using regression analysis. All videos were blinded and assessed by 2 independent reviewers using the validated laparoscopic suturing competency assessment tool (LS-CAT). These scores were compared with the composite score. Results: A 100 videos of 16 trainees, during separate points on their learning curve, and 8 experts were used. The parameters "time" and "distance" were statistically significantly correlated with all LS-CAT domains. The composite score (calculated from "time" and "distance") showed improvement between the first and the last knot (57% versus 94%, P < .001). Also the LS-CAT score improved (28 versus 17, P < .001). However, the correlation of the composite score with the LS-CAT score was weak (R: 0.351), with an accuracy of 55/100 when pooling the outcomes based on inadequate, adequate, or good performance. Conclusion: Instrument tracking parameters (using Surgtrac) could give an indication of the skill level, however, it missed important elements, essential for reliable assessment. Therefore, expert assessment remains superior to determine the skill level in MIS suturing skills.

Rare and Complex Urology: Clinical Overview of ERN eUROGEN.

BACKGROUND: In 2017, the European Commission launched 24 European Reference Networks (ERNs). ERN eUROGEN is the network for urorectogenital diseases and complex conditions, and started with 29 full member healthcare providers (HCPs) in 11 countries. It then covered 19 different disease areas distributed over three workstreams (WSs). OBJECTIVE: To provide an overview and identify challenges in data collection at European level of the ERN eUROGEN patient population treated by HCPs in the network. DESIGN, SETTING, AND PARTICIPANTS: A retrospective cohort study was conducted of the 29 HCPs who were full members between 2013 and 2019. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Data were extracted from the original HCP applications and the ERN continuous monitoring system. Patient volumes, new patient numbers, and procedures were compared between different WSs, countries, and HCPs. Discrepancies between monitoring and application data were identified. RESULTS AND LIMITATIONS: Between 2013 and 2019, 122 040 patients required long-term care within the 29 HCPs. The volume of patients treated and procedures undertaken per year increased over time. Large discrepancies were found between patient numbers contained in the application forms and those reported in the continuous monitoring system (0-1357% deviation). CONCLUSIONS: Patient numbers and procedures increased across ERN eUROGEN HCPs. Reliable data extraction appeared challenging, illustrated by the patient volume discrepancies between application forms and the continuous monitoring data. Improved disease definitions, re-evaluation of affiliated HCPs, and valid data extraction are needed for future improvements. PATIENT SUMMARY: We analysed the patient population with rare urorectogenital diseases or complex conditions within the ERN eUROGEN network between 2013 and 2019. Clinical activity was found to increase, but differences in patient numbers were evident between healthcare providers. In order to acquire valid patient numbers, both improved definitions of diagnostic codes and greater insight into the data-gathering process are required.

The Effect of an Interval Training on Skill Retention of High-Complex Low-Volume Minimal Invasive Pediatric Surgery Skills: A Pilot Study.

Background: Current training programs for complex pediatric minimal invasive surgery (MIS) are usually bulk training, consisting of 1- or 2-day courses. The aim of this study was to examine the effects of bulk training versus interval training on the preservation of high-complex, low-volume MIS skills. Materials and Methods: Medical students, without prior surgical experience, were randomly assigned to either a bulk or interval training program for complex MIS (congenital diaphragmatic hernia [CDH] and esophageal atresia [EA] repair). Both groups trained for 5 hours; the bulk group twice within 3 days and the interval groups five times in 3 weeks. Skills retention was assessed at 2 weeks, 6 weeks, and 6 months posttraining, using a composite score (0%-100%) based on the objective parameters tracked by SurgTrac. Results: Seventeen students completed the training sessions (bulk n = 9, interval n = 8) and were assessed accordingly. Retention of the skills for EA repair was significantly better for the interval training group than for the bulk group at 6 weeks (P = .004). However, at 6 months, both groups scored significantly worse than after the training sessions for EA repair (bulk 60 versus 67, P = .176; interval 63 versus 74, P = .028) and CDH repair (bulk 32 versus 67, P = .018; interval 47 versus 62, P = .176). Conclusion: This pilot study suggests superior retention of complex pediatric MIS skills after interval training, during a longer period of time, than bulk training. However, after 6 months, both groups scored significantly worse than after their training, indicating the need for continuous training.

Training benchmarks based on validated composite scores for the RobotiX robot-assisted surgery simulator on basic tasks.

The RobotiX robot-assisted virtual reality simulator aims to aid in the training of novice surgeons outside of the operating room. This study aimed to determine the validity evidence on multiple levels of the RobotiX simulator for basic skills. Participants were divided in either the novice, laparoscopic or robotic experienced group based on their minimally invasive surgical experience. Two basic tasks were performed: wristed manipulation (Task 1) and vessel energy dissection (Task 2). The performance scores and a questionnaire regarding the realism, didactic value, and usability were gathered (content). Composite scores (0-100), pass/fail values, and alternative benchmark scores were calculated. Twenty-seven novices, 21 laparoscopic, and 13 robotic experienced participants were recruited. Content validity evidence was scored positively overall. Statistically significant differences between novices and robotic experienced participants (construct) was found for movements left (Task 1 p = 0.009), movements right (Task 1 p = 0.009, Task 2 p = 0.021), path length left (Task 1 p = 0.020), and time (Task 1 p = 0.040, Task 2 p < 0.001). Composite scores were statistically significantly different between robotic experienced and novice participants for Task 1 (85.5 versus 77.1, p = 0.044) and Task 2 (80.6 versus 64.9, p = 0.001). The pass/fail score with false-positive/false-negative percentage resulted in a value of 75/100, 46/9.1% (Task 1) and 71/100, 39/7.0% (Task 2). Calculated benchmark scores resulted in a minority of novices passing multiple parameters. Validity evidence on multiple levels was assessed for two basic robot-assisted surgical simulation tasks. The calculated benchmark scores can be used for future surgical simulation training.

Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.

BACKGROUND: Minimal invasive surgery (MIS) is increasingly used for the correction of congenital diaphragmatic hernia (CDH) and esophageal atresia (EA). It is important to master these complex procedures, preferably preclinically, to avoid complications. The aim of this study was to validate recently developed models to train these MIS procedures preclinically. METHODS: Two low cost, reproducible models (one for CDH and one for EA) were validated during several pediatric surgical conferences and training sessions (January 2017-December 2018), used in either the LaparoscopyBoxx or EoSim simulator. Participants used one or both models and completed a questionnaire regarding their opinion on realism (face validity) and didactic value (content validity), rated on a five-point-Likert scale. RESULTS: Of all 60 participants enrolled, 44 evaluated the EA model. All items were evaluated as significantly better than neutral, with means ranging from 3.7 to 4.1 (p < 0.001). The CDH model was evaluated by 48 participants. All items scored significantly better than neutral (means 3.5-3.9, p < 0.001), with exception of the haptics of the simulated diaphragm (mean 3.3, p = 0.054). Both models were considered a potent training tool (means 3.9). CONCLUSION: These readily available and low budget models are considered a valid and potent training tool by both experts and target group participants. TYPE OF STUDY: Prospective study. LEVEL OF EVIDENCE: Level II.

Take-Home Laparoscopy Simulators in Pediatric Surgery: Is More Expensive Better?

Background: To increase complex minimally invasive skills (MIS), frequent training outside the clinical setting is of uttermost importance. This study compares two low-cost pediatric MIS simulators, which can easily be used preclinically. Materials and Methods: The LaparoscopyBoxx is a portable simulator without a tracking system, with costs ranging from €90 to €315. The EoSim simulator has a built-in camera and tracking system and costs range from €780 to €1800. During several pediatric surgical conferences and workshops (January 2017-December 2018), participants were asked to use both simulators. Afterward, they completed a questionnaire regarding their opinion on realism and didactic value, scored on a five-point Likert scale. Results: A total of 50 participants (24 experts and 25 target group, one unknown) evaluated one or both simulators. Both simulators scored well on the questionnaire. The LaparoscopyBoxx scored significantly better regarding the "on screen representation of the instrument actions" (mean 4.2 versus 3.5, P = .001), "training tool for pediatric surgery" (mean 4.4 versus 3.9, P = .005), and "appealing take-home simulator" (mean 4.6 versus 4.0, P = .002). Conclusion: The simulators tested in this study were both regarded an appealing take-home simulator. The LaparoscopyBoxx scored significantly better than the EoSim, even though this is a low budget simulator without tracking capabilities.

A cost-effectiveness modeling study of robot-assisted (RARC) versus open radical cystectomy (ORC) for bladder cancer to inform future research.

BACKGROUND: Open radical cystectomy (ORC) is regarded the standard treatment for muscle-invasive bladder cancer, but robot-assisted radical cystectomy (RARC) is increasingly used in practice. However, it is unclear whether RARC provides value for money. OBJECTIVE: To identify the main evidence gaps and main drivers of cost-effectiveness, comparing RARC to ORC. DESIGN, SETTING, AND PARTICIPANTS: A decision analytical model was developed to study the 30d and 90d postoperative complications with RARC versus ORC and their related cost in bladder cancer patients. Input data were derived from systematic literature searches, meta-analyses, internal databases and expert opinion. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Cost per saved complication (in Clavien-Dindo grading) was determined. Deterministic sensitivity analyses was performed to search for threshold values for RARC to become cost saving. Uncertainty was addressed using probabilistic sensitivity analyses. RESULTS: The expected 30d and 90d risk for a minor complication was lower for RARC than ORC (37% vs. 45% and 32% vs. 36%). The expected 30d and 90d risk of RARC versus ORC for a major complication was 18% vs. 23% and 16% vs. 25%. The 30d and 90d extra costs needed to prevent one major complication were €62,582 and €37,007, respectively. Data on the impact of complications on quality of life were lacking. Three scenarios resulted in cost savings for RARC: operating time (threshold: ≤175min), length of stay (≤4d), and RARC equipment (≤€281). CONCLUSION: Current evidence suggests that it is unlikely that RARC will become less expensive than ORC. However, RARC might result in fewer complications. To determine value for money, research is needed into the consequences of these complications in terms of quality of life. PATIENT SUMMARY: Economic modeling showed that RARC might result in fewer complications, but is more expensive than ORC. Future research should focus on the impact on quality of life.

Face, Content, and Construct Validity of the Take-Home EoSim Augmented Reality Laparoscopy Simulator for Basic Laparoscopic Tasks.

Background: The eoSim® laparoscopic augmented reality (AR) simulator has instrument tracking capabilities that may be suitable for implementation in laparoscopic training. The objective is to assess face, content, and construct validity of this simulator for basic laparoscopic skills training. Methods: Participants were divided into three groups: novices (no training), intermediates (<50 laparoscopic procedures), and experts (>50 laparoscopic procedures). Three basic tasks were completed on the simulator: thread transfer (1), cyst dissection (2), and tube ligation (3). A questionnaire was completed on realism, didactic value, and usability of the simulator. Measured outcome parameters were as follows: time, distance, time off screen, average speed, acceleration, and smoothness. Results: Mean ± standard deviation scores on realism were positive (Task 1 or T1; 3.9 ± 0.7, P = .13, T2; 3.7 ± 0.7, P = .07, T3; 3.7 ± 0.07), as well as didactic value (T1; 3.9 ± 0.8, P = .71, T2; 3.9 ± 0.8, P = .31, T3; 4.0 ± 0.8, P = .40). Usability was valued the highest, with mean scores between 3.9 and 4.3 (T1; P = .71, T2; P = .80, T3; P = .85). Scores did not differ significantly between groups. Experts were significantly faster (Task 1; P < .001, Task 2; P = .042, Task 3: P < .001) with higher handling speed for tasks 2 and 3 (Task 1; P = .20, task 2; P = .034, task 3; P = .049). Results for other outcome parameters also indicated experts had better instrument control and efficiency than novices, although these differences did not reach statistical significance. Conclusions: The eoSim laparoscopic AR simulator is regarded as a realistic, accessible, and useful tool for the training of basic laparoscopic skills, with good face validity. Construct validity of the eoSim AR simulator was demonstrated on several core variables, but not all.