Evaluation of different approaches to define expert benchmark scores for new robotic training simulators based on the Medtronic HUGO™ RAS surgical robot experience.

New robot-assisted surgery platforms being developed will be required to have proficiency-based simulation training available. Scoring methodologies and performance feedback for trainees are currently not consistent across all robotic simulator platforms. Also, there are virtually no prior publications on how VR simulation passing benchmarks have been established. This paper compares methods evaluated to determine the proficiency-based scoring thresholds (a.k.a. benchmarks) for the new Medtronic Hugo™ RAS robotic simulator. Nine experienced robotic surgeons from multiple disciplines performed the 49 skills exercises 5 times each. The data were analyzed in 3 different ways: (1) include all data collected, (2) exclude first sessions, (3) exclude outliers. Eliminating the first session discounts becoming familiar with the exercise. Discounting outliers allows removal of potentially erroneous data that may be due to technical issues, unexpected distractions, etc. Outliers were identified using a common statistical technique involving the interquartile range of the data. Using each method above, mean and standard deviations were calculated, and the benchmark was set at a value of 1 standard deviation above the mean. In comparison to including all the data, when outliers are excluded, fewer data points are removed than just excluding first sessions, and the metric benchmarks are made more difficult by an average of 11%. When first sessions are excluded, the metric benchmarks are made easier by an average of about 2%. In comparison with benchmarks calculated using all data points, excluding outliers resulted in the biggest change making the benchmarks more challenging. We determined that this method provided the best representation of the data. These benchmarks should be validated with future clinical training studies.

Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum.

STUDY OBJECTIVE: Evaluate a universal proficiency metric for Robotic Surgery Virtual Reality (VR) simulation that will allow comparison of all users across any VR curriculum. DESIGN: Retrospective analysis of VR Simulation metrics. SETTING: Two training institutions. PATIENTS OR PARTICIPANTS: Residents, fellows and practicing surgeons. INTERVENTIONS: Analysis of the Mimic robotic Virtual Reality (VR)-Simulation database of over 600,000 sessions was utilized to calculate Mean scores for each exercise. Those Mean scores were then normalized to 100. Subject's scores were also averaged and normalized to 100. We called this Index score the MScore Proficiency Index (MPI©). Scores above 100 were better than average; Less than 100 were worse than average. MEASUREMENTS AND MAIN RESULTS: Seventeen thousand six hundred and forty eight sessions were analyzed (2017-2020) comparing 77 students (residents to practicing surgeons) working in 7 different curriculums. On average, each student spent 8 h and 24 min on simulation, attempted 26.5 different exercises, and became proficient in 20.6 exercises per user. The MPI© mean score for all participants in all curricula was an MPI© of 104.9 (SD: 15.5). Thirteen students were 1 standard deviation below the norm with an average MPI© of 80.15. This group averaged 9 h 27 min each on the simulator attempting 23.46 exercises but becoming proficient in only 10.38 (47%) of them in 224 sessions. Twelve students were 1 standard deviation above the norm with an average MPI© of 127.05. This group averaged 6 h 31 min each on the simulator attempting 29.08 exercises but becoming proficient in 27.5 (95%) of them in 196 sessions. CONCLUSION: A universal skill-based performance index (MPI©) was calculated and found to be a reliable tool that could be used to identify relative proficiency among students in different robotic surgery VR Simulation curriculums. An individual user's proficiency can be utilized to identify a student's progress in a given curriculum. Future studies of MPI© will determine if machine learning can provide timely personalized feedback to the user.

The impact of different surgical modalities for hysterectomy on satisfaction and patient reported outcomes.

BACKGROUND: There is an ongoing debate regarding the cost-benefit of different surgical modalities for hysterectomy. Studies have relied primarily on evaluation of clinical outcomes and medical expenses. Thus, a paucity of information on patient-reported outcomes including satisfaction, recovery, and recommendations exists. OBJECTIVE: The objective of this study was to identify differences in patient satisfaction and recommendations by approach to a hysterectomy. METHODS: We recruited a large, geographically diverse group of women who were members of an online hysterectomy support community. US women who had undergone a benign hysterectomy formed this retrospective study cohort. Self-reported characteristics and experiences were compared by surgical modality using chi-square tests. Outcomes over time were assessed with the Jonkheere-Terpstra trend test. Logistic regression identified independent predictors of patient satisfaction and recommendations. RESULTS: There were 6262 women who met the study criteria; 41.74% (2614/6262) underwent an abdominal hysterectomy, 10.64% (666/6262) were vaginal, 27.42% (1717/6262) laparoscopic, 18.94% (1186/6262) robotic, and 1.26% (79/6262) single-incision laparoscopic. Most women were at least college educated (56.37%, 3530/6262), and identified as white, non-Hispanic (83.17%, 5208/6262). Abdominal hysterectomy rates decreased from 68.2% (152/223) to 24.4% (75/307), and minimally invasive surgeries increased from 31.8% (71/223) to 75.6% (232/307) between 2001 or prior years and 2013 (P<.001 all trends). Trends in overall patient satisfaction and recommendations showed significant improvement over time (P<.001).There were differences across the surgical modalities in all patient-reported experiences (ie, satisfaction, time to walking, driving and working, and whether patients would recommend or use the same technique again; P<.001). Significantly better outcomes were evident among women who had vaginal, laparoscopic, and robotic procedures than among those who had an abdominal procedure. However, robotic surgery was the only approach that was an independent predictor of better patient experience; these patients were more satisfied overall (odds ratio [OR] 1.31, 95% CI 1.13-1.51) and on six other satisfaction measures, and more likely to recommend (OR 1.64, 95% CI 1.39-1.94) and choose the same modality again (OR 2.07, 95% CI 1.67-2.57). Abdominal hysterectomy patients were more dissatisfied with outcomes after surgery and less likely to recommend (OR 0.36, 95% CI 0.31-0.40) or choose the same technique again (OR 0.29, 95% CI 0.25-0.33). Quicker return to normal activities and surgery after 2007 also were independently associated with better overall satisfaction, willingness to recommend, and to choose the same surgery again. CONCLUSIONS: Consistent with other US data, laparoscopic and robotic hysterectomy rates increased over time, with a concomitant decline in abdominal hysterectomy. While inherent shortcomings of this retrospective Web-based study exist, findings show that patient experience was better for each of the major minimally invasive approaches than for abdominal hysterectomy. However, robotic-assisted hysterectomy was the only modality that independently predicted greater satisfaction and willingness to recommend and have the same procedure again.