Evaluation of different approaches to define expert benchmark scores for new robotic training simulators based on the Medtronic HUGO™ RAS surgical robot experience.

New robot-assisted surgery platforms being developed will be required to have proficiency-based simulation training available. Scoring methodologies and performance feedback for trainees are currently not consistent across all robotic simulator platforms. Also, there are virtually no prior publications on how VR simulation passing benchmarks have been established. This paper compares methods evaluated to determine the proficiency-based scoring thresholds (a.k.a. benchmarks) for the new Medtronic Hugo™ RAS robotic simulator. Nine experienced robotic surgeons from multiple disciplines performed the 49 skills exercises 5 times each. The data were analyzed in 3 different ways: (1) include all data collected, (2) exclude first sessions, (3) exclude outliers. Eliminating the first session discounts becoming familiar with the exercise. Discounting outliers allows removal of potentially erroneous data that may be due to technical issues, unexpected distractions, etc. Outliers were identified using a common statistical technique involving the interquartile range of the data. Using each method above, mean and standard deviations were calculated, and the benchmark was set at a value of 1 standard deviation above the mean. In comparison to including all the data, when outliers are excluded, fewer data points are removed than just excluding first sessions, and the metric benchmarks are made more difficult by an average of 11%. When first sessions are excluded, the metric benchmarks are made easier by an average of about 2%. In comparison with benchmarks calculated using all data points, excluding outliers resulted in the biggest change making the benchmarks more challenging. We determined that this method provided the best representation of the data. These benchmarks should be validated with future clinical training studies.

The MEMORY Study: MulticentEr study of Minimally invasive surgery versus Open Radical hYsterectomy in the management of early-stage cervical cancer: Survival outcomes.

OBJECTIVE: The Laparoscopic Approach to Cervical Cancer (LACC) trial found that minimally invasive radical hysterectomy compared to open radical hysterectomy compromised oncologic outcomes and was associated with worse progression-free survival (PFS) and overall survival (OS) in early-stage cervical carcinoma. We sought to assess oncologic outcomes at multiple centers between minimally invasive (MIS) radical hysterectomy and OPEN radical hysterectomy. METHODS: This is a multi-institutional, retrospective cohort study of patients with 2009 FIGO stage IA1 (with lymphovascular space invasion) to IB1 cervical carcinoma from 1/2007-12/2016. Patients who underwent preoperative therapy were excluded. Squamous cell carcinoma, adenocarcinoma, and adenosquamous carcinomas were included. Appropriate statistical tests were used. RESULTS: We identified 1093 cases for analysis-715 MIS (558 robotic [78%]) and 378. OPEN procedures. The OPEN cohort had more patients with tumors >2 cm, residual disease in the hysterectomy specimen, and more likely to have had adjuvant therapy. Median follow-up for the MIS and OPEN cohorts were 38.5 months (range, 0.03-149.51) and 54.98 months (range, 0.03-145.20), respectively. Three-year PFS rates were 87.9% (95% CI: 84.9-90.4%) and 89% (95% CI: 84.9-92%), respectively (P = 0.6). On multivariate analysis, the adjusted HR for recurrence/death was 0.70 (95% CI: 0.47-1.03; P = 0.07). Three-year OS rates were 95.8% (95% CI: 93.6-97.2%) and 96.6% (95% CI: 93.8-98.2%), respectively (P = 0.8). On multivariate analysis, the adjusted HR for death was 0.81 (95% CI: 0.43-1.52; P = 0.5). CONCLUSION: This multi-institutional analysis showed that an MIS compared to OPEN radical hysterectomy for cervical cancer did not appear to compromise oncologic outcomes, with similar PFS and OS.

Surgical Science-Simbionix Robotic Hysterectomy Simulator: Validating a New Tool.

STUDY OBJECTIVE: To gather validity evidence for and determine acceptability of Surgical Science-Simbionix Hysterectomy Modules for the DaVinci Xi console simulation system (software; 3D Systems by Simbionix [now Surgical Science-Simbionix], Littleton, CO, and hardware; Intuitive Surgical, Inc., Sunnyvale, CA) and evaluate performance benchmarks between novice and experienced or expert surgeons. DESIGN: Prospective education study (Messick validity framework). SETTING: Multicenter, academic medical institutions. PARTICIPANTS: Residents, fellows, and faculty in obstetrics and gynecology were invited to participate at 3 institutions. Participants were categorized by experience level: fewer than 10 hysterectomies (novice), 10 to 50 hysterectomies (experienced), and more than 50 hysterectomies (expert). A total of 10 novice, 10 experienced, and 14 expert surgeons were included. INTERVENTIONS: Participants completed 4 simulator modules (ureter identification, bladder flap development, colpotomy, complete hysterectomy) and a qualitative survey. Simulator recordings were reviewed in duplicate by educators in minimally invasive gynecologic surgery using the Modified Global Evaluative Assessment of Robotic Skills (GEARS) rating scale. MEASUREMENTS AND MAIN RESULTS: Most participants felt that the simulator realistically simulated robotic hysterectomy (64.7%) and that feedback provided by the simulator was as or more helpful than feedback from previous simulators (88.2%) but less helpful than feedback provided in the operating room (73.5%). Participants felt that this simulator would be helpful for teaching junior residents. Simulator-generated metrics correlated with GEARS performance for the bladder flap and ureter identification modules in multiple domains including total movements and total time for completion. GEARS performance for the bladder flap module correlated with experience level (novice vs experienced/expert) in the domains of interest and total score but did not consistently correlate for the other procedural modules. Performance benchmarks were evaluated for the bladder flap module for each GEARS domain and total score. CONCLUSION: The modules were well received by participants of all experience levels. Individual simulation modules appear to better discriminate between novice and experienced/expert users than overall simulator performance. Based on these data and participant feedback, the use of individual modules in early residency education may be helpful for providing feedback and may ultimately serve as 1 component of determining readiness to perform robotic hysterectomy.

Expert Consensus Recommendations for Robotic Surgery Credentialing.

OBJECTIVE: To define criteria for robotic credentialing using expert consensus. BACKGROUND: A recent review of institutional robotic credentialing policies identified significant variability and determined current policies are largely inadequate to ensure surgeon proficiency and may threaten patient safety. METHODS: Twenty-eight national robotic surgery experts were invited to participate in a consensus conference. After review of available institutional policies and discussion, the group developed a 91 proposed criteria. Using a modified Delphi process the experts were asked to indicate their agreement with the proposed criteria in three electronic survey rounds after the conference. Criteria that achieved 80% or more in agreement (consensus) in all rounds were included in the final list. RESULTS: All experts agreed that there is a need for standardized robotic surgery credentialing criteria across institutions that promote surgeon proficiency. Forty-nine items reached consensus in the first round, 19 in the second, and 8 in the third for a total of 76 final items. Experts agreed that privileges should be granted based on video review of surgical performance and attainment of clearly defined objective proficiency benchmarks. Parameters for ongoing outcome monitoring were determined and recommendations for technical skills training, proctoring, and performance assessment were defined. CONCLUSIONS: Using a systematic approach, detailed credentialing criteria for robotic surgery were defined. implementation of these criteria uniformly across institutions will promote proficiency of robotic surgeons and has the potential to positively impact patient outcomes.

Enfermedad trofoblástica gestacional en el posparto tardío

La enfermedad trofoblástica gestacional (ETG) es un término utilizado para un grupo de tumores relacionados con el embarazo que surgen de la proliferación defectuosa del tejido trofoblástico. Su incidencia después de un embarazo a término no es frecuente, siendo diagnosticada en la mayoría de los casos de forma incidental, por síntomas de metástasis. Presentamos un caso de ETG que ocurrió 9 semanas después de un parto vaginal a término. El nivel sérico de gonadotropina coriónica humana beta (hCG-β) fue de 38 084 mUI/mL y una tomografía de tórax mostró múltiples nódulos pulmonares. La paciente recibió actinomicina D durante 3 ciclos. La hCG-β se normalizó después del tercer ciclo y la paciente permaneció libre de enfermedad durante 12 meses después del diagnóstico. Se describe la presentación clínica y los desafíos en el diagnóstico de ETG en el período posparto tardío.

Gestational trophoblastic disease (GTD) is a term used for a group of pregnancyrelated tumors arising from defective proliferation of trophoblastic tissue. Its incidence after term pregnancy is infrequent, being diagnosed in most cases incidentally, due to symptoms of metastasis. We present a case of GTD that occurred 9 weeks after a vaginal delivery at term. The serum beta-human chorionic gonadotropin (β-hCG) level was 38 084 mIU/mL and a chest CT scan showed multiple pulmonary nodules. The patient received actinomycin D for 3 cycles. The β-hCG normalized after the third cycle and the patient remained disease-free for 12 months after diagnosis. The clinical presentation and challenges in diagnosing GTD in the late postpartum period are described.

Prophylactic Negative Pressure Wound Therapy After Laparotomy for Gynecologic Surgery: A Randomized Controlled Trial.

OBJECTIVE: To estimate the effectiveness of prophylactic negative pressure wound therapy in patients undergoing laparotomy for gynecologic surgery. METHODS: We conducted a randomized controlled trial. Eligible, consenting patients, regardless of body mass index (BMI), who were undergoing laparotomy for presumed gynecologic malignancy were randomly allocated to standard gauze or negative pressure wound therapy. Patients with BMIs of 40 or greater and benign disease also were eligible. Randomization, stratified by BMI, occurred after skin closure. The primary outcome was wound complication within 30 (±5) days of surgery. A sample size of 343 per group (N=686) was planned. RESULTS: From March 1, 2016, to August 20, 2019, we identified 663 potential patients; 289 were randomized to negative pressure wound therapy (254 evaluable participants) and 294 to standard gauze (251 evaluable participants), for a total of 505 evaluable patients. The median age of the entire cohort was 61 years (range 20-87). Four hundred ninety-five patients (98%) underwent laparotomy for malignancy. The trial was eventually stopped for futility after an interim analysis of 444 patients. The rate of wound complications was 17.3% in the negative pressure wound therapy (NPWT) group and 16.3% in the gauze group, absolute risk difference 1% (90% CI -4.5 to 6.5%; P=.77). Adjusted odds ratio controlling for estimated blood loss and diabetes was 0.99 (90% CI 0.62-1.60). Skin blistering occurred in 33 patients (13%) in the NPWT group and in three patients (1.2%) in the gauze group (P<.001). CONCLUSION: Negative pressure wound therapy after laparotomy for gynecologic surgery did not lower the wound complication rate but did increase skin blistering. CLINICAL TRIAL REGISTRATION: ClinicalTrials.gov, NCT02682316. FUNDING SOURCE: The protocol was supported in part by KCI/Acelity.

Are current credentialing requirements for robotic surgery adequate to ensure surgeon proficiency?

BACKGROUND: Robotic surgery has seen unprecedented growth, requiring hospitals to establish or update credentialing policies regarding this technology. Concerns about verification of robotic surgeon proficiency and the adequacy of current credentialing criteria to maintain patient safety have arisen. The aim of this project was to examine existing institutional credentialing requirements for robotic surgery and evaluate their adequacy in ensuring surgeon proficiency. METHODS: Robotic credentialing policies for community and academic surgery programs were acquired and reviewed. Common criteria across institutions related to credentialing and recredentialing were identified and the average, standard deviation, and range of numeric requirements, if defined, was calculated. Criteria for proctors and assistants were also analyzed. RESULTS: Policies from 42 geographically dispersed US hospitals were reviewed. The majority of policies relied on a defined number of proctored cases as a surrogate for proficiency with an average of 3.24 ± 1.69 and a range of 1-10 cases required for initial credentialing. While 34 policies (81%) addressed maintenance of privileges requirements, there was wide variability in the average number of required robotic cases (7.19 ± 3.28 per year) and range (1-15 cases per year). Only 11 policies (26%) addressed the maximum allowable time gap between robotic cases. CONCLUSION: Significant variability in credentialing policies exists in a representative sample of US hospitals. Most policies require completion of a robotic surgery training course and a small number of proctored cases; however, ongoing objective performance assessments and patient outcome monitoring was rarely described. Existing credentialing policies are likely inadequate to ensure surgeon proficiency; therefore, development and wide implementation of robust credentialing guidelines is recommended to optimize patient safety and outcomes.

Robotic Surgery: The Impact of Simulation and Other Innovative Platforms on Performance and Training.

OBJECTIVE: To review the current status of robotic training and the impact of various training platforms on the performance of robotic surgical trainees. DATA SOURCES: Literature review of Google Scholar and PubMed. The search terms included a combination of the following: "robotic training," "simulation," "robotic curriculum," "obgyn residency robotic training," "virtual reality robotic training," "DaVinci training," "surgical simulation," "gyn surgical training." The sources considered for inclusion included peer-reviewed articles, literature reviews, textbook chapters, and statements from various institutions involved in resident training. METHODS OF STUDY SELECTION: A literature search of Google Scholar and PubMed using terms related to robotic surgery and robotics training, as mentioned in the "Data Sources" section. RESULTS: Multiple novel platforms that use machine learning and real-time video feedback to teach and evaluate robotic surgical skills have been developed over recent years. Various training curricula, virtual reality simulators, and other robotic training tools have been shown to enhance robotic surgical education and improve surgical skills. The integration of didactic learning, simulation, and intraoperative teaching into more comprehensive training curricula shows positive effects on robotic skills proficiency. Few robotic surgery training curricula have been validated through peer-reviewed study, and there is more work to be completed in this area. In addition, there is a lack of information about how the skills obtained through robotics curricula and simulation translate into operating room performance and patient outcomes. CONCLUSION: Data collected to date show promising advances in the training of robotic surgeons. A diverse array of curricula for training robotic surgeons continue to emerge, and existing teaching modalities are evolving to keep up with the rapidly growing demand for proficient robotic surgeons. Futures areas of growth include establishing competency benchmarks for existing training tools, validating existing curricula, and determining how to translate the acquired skills in simulation into performance in the operating room and patient outcomes. Many surgical training platforms are beginning to expand beyond discrete robotic skills training to procedure-specific and team training. There is still a wealth of research to be done to understand how to create an effective training experience for gynecologic surgical trainees and robotics teams.

Proving the Effectiveness of the Fundamentals of Robotic Surgery (FRS) Skills Curriculum: A Single-blinded, Multispecialty, Multi-institutional Randomized Control Trial.

OBJECTIVE: To demonstrate the noninferiority of the fundamentals of robotic surgery (FRS) skills curriculum over current training paradigms and identify an ideal training platform. SUMMARY BACKGROUND DATA: There is currently no validated, uniformly accepted curriculum for training in robotic surgery skills. METHODS: Single-blinded parallel-group randomized trial at 12 international American College of Surgeons (ACS) Accredited Education Institutes (AEI). Thirty-three robotic surgery experts and 123 inexperienced surgical trainees were enrolled between April 2015 and November 2016. Benchmarks (proficiency levels) on the 7 FRS Dome tasks were established based on expert performance. Participants were then randomly assigned to 4 training groups: Dome (n = 29), dV-Trainer (n = 30), and DVSS (n = 32) that trained to benchmarks and control (n = 32) that trained using locally available robotic skills curricula. The primary outcome was participant performance after training based on task errors and duration on 5 basic robotic tasks (knot tying, continuous suturing, cutting, dissection, and vessel coagulation) using an avian tissue model (transfer-test). Secondary outcomes included cognitive test scores, GEARS ratings, and robot familiarity checklist scores. RESULTS: All groups demonstrated significant performance improvement after skills training (P < 0.01). Participating residents and fellows performed tasks faster (DOME and DVSS groups) and with fewer errors than controls (DOME group; P < 0.01). Inter-rater reliability was high for the checklist scores (0.82-0.97) but moderate for GEARS ratings (0.40-0.67). CONCLUSIONS: We provide evidence of effectiveness for the FRS curriculum by demonstrating better performance of those trained following FRS compared with controls on a transfer test. We therefore argue for its implementation across training programs before surgeons apply these skills clinically.

Robotic surgery during the COVID pandemic: why now and why for the future.

Health care has changed in unprecedented ways since the first reported cases of COVID-19. With global case rates continuing to rise and government restrictions beginning to loosen, many worry that a second wave in our future. In many hospitals around the world, non-emergent surgeries were put on hold as hospitals were transformed into COVID centers. As surgeons and administrators do their best to reinstate non-emergent procedures, guidance is sought from any and all reliable sources. Robotic surgery has many known and demonstrated benefits over open surgery and often over conventional laparoscopy. In this commentary, we aim to highlight some of the advantages robotic surgery may offer during this uniquely challenging time in health care.

When to Operate, Hesitate and Reintegrate: Society of Gynecologic Oncology Surgical Considerations during the COVID-19 Pandemic.

The COVID-19 pandemic has challenged our ability to provide timely surgical care for our patients. In response, the U.S. Surgeon General, the American College of Srugeons, and other surgical professional societies recommended postponing elective surgical procedures and proceeding cautiously with cancer procedures that may require significant hospital resources and expose vulnerable patients to the virus. These challenges have particularly distressing for women with a gynecologic cancer diagnosis and their providers. Currently, circumstances vary greatly by region and by hospital, depending on COVID-19 prevalence, case mix, hospital type, and available resources. Therefore, COVID-19-related modifications to surgical practice guidelines must be individualized. Special consideration is necessary to evaluate the appropriateness of procedural interventions, recognizing the significant resources and personnel they require. Additionally, the pandemic may occur in waves, with patient demand for surgery ebbing and flowing accordingly. Hospitals, cancer centers and providers must prepare themselves to meet this demand. The purpose of this white paper is to highlight all phases of gynecologic cancer surgical care during the COVID-19 pandemic and to illustrate when it is best to operate, to hestitate, and reintegrate surgery. Triage and prioritization of surgical cases, preoperative COVID-19 testing, peri-operative safety principles, and preparations for the post-COVID-19 peak and surgical reintegration are reviewed.

Society of Robotic Surgery review: recommendations regarding the risk of COVID-19 transmission during minimally invasive surgery.

OBJECTIVES: To determine the risk of COVID-19 transmission during minimally invasive surgical (MIS) procedures METHODS: Surgical society statements regarding the risk of COVID transmission during MIS procedures were reviewed. In addition, the available literature on COVID-19 and other viral transmission in CO2 pneumoperitoneum, as well as the presence of virus in the plume created by electrocautery during MIS was reviewed. The society recommendations were compared to the available literature on the topic to create our review and recommendations to mitigate COVID-19 transmission. RESULTS: The recommendations promulgated by various surgical societies evolved over time as more information became available on COVID-19 transmission. Review of the available literature on the presence of COVID-19 in CO2 pneumoperitoneum was inconclusive. There is no clear evidence of the presence of COVID-19 in plume created by electrocautery. Technologies to reduce CO2 pneumoperitoneum release into the operating room as well as filter viral particles are available and should reduce the exposure risk to operating room personnel. CONCLUSION: There is no clear evidence of COVID-19 virus in the CO2 used during MIS procedures or in the plume created by electrocautery. Until the presence or absence of COVID-19 viral particles has been clearly established, measures to mitigate CO2 and surgical cautery plume release into the operating room should be performed. Further study on the presence of COVID-19 in MIS pneumoperitoneum and cautery plume is needed.

A Novel Technique to Minimize Contamination for Cervical Cancer Surgery Patients.

The Laparoscopic Approach to Cervical Cancer (LACC) trial changed the surgical management of cervical cancer worldwide. It was a multinational phase III clinical trial that reported lower survival and higher rate of abdominopelvic recurrences in minimally invasive surgery (MIS) than those of open surgery after hysterectomy. It is possible that tumor exposure to the peritoneal cavity in the MIS arm may account for these differences. We propose a novel technique to minimize peritoneal contamination of malignant cells present at the cervical os by placing a vaginal cerclage abdominally to create a seal at the apex of the vagina during MIS radical hysterectomy. The 2 patients in this work remain healthy and disease-free more than 18 months after surgery using this novel technique. We intend this work to serve as a platform both for offering a safe alternative to the open approach supported by the LACC trial and, most importantly, for promoting discussion of the results of the LACC trial and further research on surgical techniques in the treatment of cervical cancer. MIS has repeatedly been shown to have lower rates of infection, fewer complications, and shorter hospital stays while providing oncologic care that is noninferior to open approach.

Utilising an Accelerated Delphi Process to Develop Guidance and Protocols for Telepresence Applications in Remote Robotic Surgery Training.

CONTEXT: The role of robot-assisted surgery continues to expand at a time when trainers and proctors have travel restrictions during the coronavirus disease 2019 (COVID-19) pandemic. OBJECTIVE: To provide guidance on setting up and running an optimised telementoring service that can be integrated into current validated curricula. We define a standardised approach to training candidates in skill acquisition via telepresence technologies. We aim to describe an approach based on the current evidence and available technologies, and define the key elements within optimised telepresence services, by seeking consensus from an expert committee comprising key opinion leaders in training. EVIDENCE ACQUISITION: This project was carried out in phases: a systematic review of the current literature, a teleconference meeting, and then an initial survey were conducted based on the current evidence and expert opinion, and sent to the committee. Twenty-four experts in training, including clinicians, academics, and industry, contributed to the Delphi process. An accelerated Delphi process underwent three rounds and was completed within 72 h. Additions to the second- and third-round surveys were formulated based on the answers and comments from the previous rounds. Consensus opinion was defined as ≥80% agreement. EVIDENCE SYNTHESIS: There was 100% consensus regarding an urgent need for international agreement on guidance for optimised telepresence. Consensus was reached in multiple areas, including (1) infrastructure and functionality; (2) definitions and terminology; (3) protocols for training, communication, and safety issues; and (4) accountability including ethical and legal issues. The resulting formulated guidance showed good internal consistency among experts, with a Cronbach alpha of 0.90. CONCLUSIONS: Using the Delphi methodology, we achieved international consensus among experts for development and content validation of optimised telepresence services for robotic surgery training. This guidance lays the foundation for launching telepresence services in robotic surgery. This guidance will require further validation. PATIENT SUMMARY: Owing to travel restrictions during the coronavirus disease 2019 (COVID-19) pandemic, development of remote training and support via telemedicine is becoming increasingly important. We report a key opinion leader consensus view on a standardised approach to telepresence.

How Wearable Technology Can Facilitate AI Analysis of Surgical Videos.

Operative video has great potential to enable instant replays of critical surgical decisions for training and quality review. Recently, artificial intelligence (AI) has shown early promise as a method of enabling efficient video review, analysis, and segmentation. Despite the progress with AI analysis of surgical videos, more work needs to be done to improve the accuracy and efficiency of AI-driven video analysis. At a recent consensus conference held on July 10-11, 2020, 8 research teams shared their work using AI for surgical video analysis. Four of the teams showcased the utility of wearable technology in providing objective surgical metrics. Data from these technologies were shown to pinpoint important cognitive and motor actions during operative tasks and procedures. The results support the utility of wearable technology to facilitate efficient and accurate video analysis and segmentation.

Utilising the Delphi Process to Develop a Proficiency-based Progression Train-the-trainer Course for Robotic Surgery Training.

CONTEXT: As the role of robot-assisted surgery continues to expand, development of standardised and validated training programmes is becoming increasingly important. OBJECTIVE: To provide guidance on an optimised "train-the-trainer" (TTT) structured educational programme for surgical trainers, in which delegates learn a standardised approach to training candidates in skill acquisition. We aim to describe a TTT course for robotic surgery based on the current published literature and to define the key elements within a TTT course by seeking consensus from an expert committee formed of key opinion leaders in training. EVIDENCE ACQUISITION: The project was carried out in phases: a systematic review of the current evidence was conducted, a face-to-face meeting was held in Philadelphia, and then an initial survey was created based on the current literature and expert opinion and sent to the committee. Thirty-two experts in training, including clinicians, academics, and industry, contributed to the Delphi process. The Delphi process underwent three rounds of survey in total. Additions to the second- and third-round surveys were formulated based on the answers and comments from the previous rounds. Consensus opinion was defined as ≥80% agreement. EVIDENCE SYNTHESIS: There was 100% consensus that there was a need for a standardized TTT course in robotic surgery. A consensus was reached in multiple areas, including the following: (1) definitions and terminologies, (2) qualifications to attend, (3) course objectives, (4) precourse considerations, (5) requirement of e-learning, (6) theory and course content, and (7) measurement of outcomes and performance level verification. The resulting formulated curriculum showed good internal consistency among experts, with a Cronbach alpha of 0.90. CONCLUSIONS: Using the Delphi methodology, we achieved an international consensus among experts to develop and reach content validation for a standardised TTT curriculum for robotic surgery training. This defined content lays the foundation for developing a proficiency-based progression model for trainers in robotic surgery. This TTT curriculum will require further validation. PATIENT SUMMARY: As the role of robot-assisted surgery continues to expand, development of standardised and validated training programmes is becoming increasingly important. There is currently a lack of high-level evidence on how best to train trainers in robot-assisted surgery. We report a consensus view on a standardised "train-the trainer" curriculum focused on robotic surgery. It was formulated by training experts from the USA and Europe, combining current evidence for training with experts' knowledge of surgical training.

Crowdsourcing: a valid alternative to expert evaluation of robotic surgery skills.

BACKGROUND: Robotic-assisted gynecologic surgery is common, but requires unique training. A validated assessment tool for evaluating trainees' robotic surgery skills is Robotic-Objective Structured Assessments of Technical Skills. OBJECTIVE: We sought to assess whether crowdsourcing can be used as an alternative to expert surgical evaluators in scoring Robotic-Objective Structured Assessments of Technical Skills. STUDY DESIGN: The Robotic Training Network produced the Robotic-Objective Structured Assessments of Technical Skills, which evaluate trainees across 5 dry lab robotic surgical drills. Robotic-Objective Structured Assessments of Technical Skills were previously validated in a study of 105 participants, where dry lab surgical drills were recorded, de-identified, and scored by 3 expert surgeons using the Robotic-Objective Structured Assessments of Technical Skills checklist. Our methods-comparison study uses these previously obtained recordings and expert surgeon scores. Mean scores per participant from each drill were separated into quartiles. Crowdworkers were trained and calibrated on Robotic-Objective Structured Assessments of Technical Skills scoring using a representative recording of a skilled and novice surgeon. Following this, 3 recordings from each scoring quartile for each drill were randomly selected. Crowdworkers evaluated the randomly selected recordings using Robotic-Objective Structured Assessments of Technical Skills. Linear mixed effects models were used to derive mean crowdsourced ratings for each drill. Pearson correlation coefficients were calculated to assess the correlation between crowdsourced and expert surgeons' ratings. RESULTS: In all, 448 crowdworkers reviewed videos from 60 dry lab drills, and completed a total of 2517 Robotic-Objective Structured Assessments of Technical Skills assessments within 16 hours. Crowdsourced Robotic-Objective Structured Assessments of Technical Skills ratings were highly correlated with expert surgeon ratings across each of the 5 dry lab drills (r ranging from 0.75-0.91). CONCLUSION: Crowdsourced assessments of recorded dry lab surgical drills using a validated assessment tool are a rapid and suitable alternative to expert surgeon evaluation.

Establishing Benchmarks For Minimum Competence With Dry Lab Robotic Surgery Drills.

The Robotic Objective Structured Assessment of Technical Skills (R-OSATS) is a previously validated assessment tool that is used to assess 5 standardized inanimate robotic surgery drills. R-OSATS is used to evaluate performance on surgical drills, with scores of 0 to 20 for each drill. Our objective was to establish the minimum threshold score that denotes competence on these drills. Thus, we performed a standard setting study using data from surgeons and trainees in 8 academic medical centers. Cutoff scores for the minimal level of competence using R-OSATS were established using 2 techniques: the modified Angoff and the contrasting groups methods. For the modified Angoff method, 8 content experts met and, in an iterative process, derived the scores that a minimally competent trainee should receive. After 2 iterative rounds of scoring and discussion with the modified Angoff method, we established a minimum competence score per drill with high agreement (rWG range, 0.92-0.98). There was unanimous consensus that a trainee needs to achieve competence on each independent drill. A second method, the contrasting groups method, was used to verify our results. In this method, we compared R-OSATS scores from "inexperienced" (34 postgraduate year 1 and 2 trainees) with "experienced" (22 faculty and fellow) robotic surgeons. The distributions of scores from both groups were plotted, and a cutoff score for each drill was determined from the intersection of the 2 curves. Using this method, the minimum score for competence would be 14 per drill, which is slightly more stringent but confirms the results obtained from the modified Angoff approach. In conclusion, using 2 well-described standard setting techniques, we have established minimum benchmarks designating trainee competence for 5 dry lab robotic surgery drills.

Incorporating resident/fellow training into a robotic surgery program.

With the rapid uptake of the robotic approach in gynecologic surgery, a thorough understanding of the technology, including its uses and limitations, is critical to maximize patient outcomes and safety. This review discusses the role of training modalities and development of curricula for robotic surgery. Furthermore, methods for incorporating the entire surgical team and the process of credentialing/maintaining privileges are described.