Expert Consensus Recommendations for Robotic Surgery Credentialing.

OBJECTIVE: To define criteria for robotic credentialing using expert consensus. BACKGROUND: A recent review of institutional robotic credentialing policies identified significant variability and determined current policies are largely inadequate to ensure surgeon proficiency and may threaten patient safety. METHODS: Twenty-eight national robotic surgery experts were invited to participate in a consensus conference. After review of available institutional policies and discussion, the group developed a 91 proposed criteria. Using a modified Delphi process the experts were asked to indicate their agreement with the proposed criteria in three electronic survey rounds after the conference. Criteria that achieved 80% or more in agreement (consensus) in all rounds were included in the final list. RESULTS: All experts agreed that there is a need for standardized robotic surgery credentialing criteria across institutions that promote surgeon proficiency. Forty-nine items reached consensus in the first round, 19 in the second, and 8 in the third for a total of 76 final items. Experts agreed that privileges should be granted based on video review of surgical performance and attainment of clearly defined objective proficiency benchmarks. Parameters for ongoing outcome monitoring were determined and recommendations for technical skills training, proctoring, and performance assessment were defined. CONCLUSIONS: Using a systematic approach, detailed credentialing criteria for robotic surgery were defined. implementation of these criteria uniformly across institutions will promote proficiency of robotic surgeons and has the potential to positively impact patient outcomes.

Are current credentialing requirements for robotic surgery adequate to ensure surgeon proficiency?

BACKGROUND: Robotic surgery has seen unprecedented growth, requiring hospitals to establish or update credentialing policies regarding this technology. Concerns about verification of robotic surgeon proficiency and the adequacy of current credentialing criteria to maintain patient safety have arisen. The aim of this project was to examine existing institutional credentialing requirements for robotic surgery and evaluate their adequacy in ensuring surgeon proficiency. METHODS: Robotic credentialing policies for community and academic surgery programs were acquired and reviewed. Common criteria across institutions related to credentialing and recredentialing were identified and the average, standard deviation, and range of numeric requirements, if defined, was calculated. Criteria for proctors and assistants were also analyzed. RESULTS: Policies from 42 geographically dispersed US hospitals were reviewed. The majority of policies relied on a defined number of proctored cases as a surrogate for proficiency with an average of 3.24 ± 1.69 and a range of 1-10 cases required for initial credentialing. While 34 policies (81%) addressed maintenance of privileges requirements, there was wide variability in the average number of required robotic cases (7.19 ± 3.28 per year) and range (1-15 cases per year). Only 11 policies (26%) addressed the maximum allowable time gap between robotic cases. CONCLUSION: Significant variability in credentialing policies exists in a representative sample of US hospitals. Most policies require completion of a robotic surgery training course and a small number of proctored cases; however, ongoing objective performance assessments and patient outcome monitoring was rarely described. Existing credentialing policies are likely inadequate to ensure surgeon proficiency; therefore, development and wide implementation of robust credentialing guidelines is recommended to optimize patient safety and outcomes.

Proving the Effectiveness of the Fundamentals of Robotic Surgery (FRS) Skills Curriculum: A Single-blinded, Multispecialty, Multi-institutional Randomized Control Trial.

OBJECTIVE: To demonstrate the noninferiority of the fundamentals of robotic surgery (FRS) skills curriculum over current training paradigms and identify an ideal training platform. SUMMARY BACKGROUND DATA: There is currently no validated, uniformly accepted curriculum for training in robotic surgery skills. METHODS: Single-blinded parallel-group randomized trial at 12 international American College of Surgeons (ACS) Accredited Education Institutes (AEI). Thirty-three robotic surgery experts and 123 inexperienced surgical trainees were enrolled between April 2015 and November 2016. Benchmarks (proficiency levels) on the 7 FRS Dome tasks were established based on expert performance. Participants were then randomly assigned to 4 training groups: Dome (n = 29), dV-Trainer (n = 30), and DVSS (n = 32) that trained to benchmarks and control (n = 32) that trained using locally available robotic skills curricula. The primary outcome was participant performance after training based on task errors and duration on 5 basic robotic tasks (knot tying, continuous suturing, cutting, dissection, and vessel coagulation) using an avian tissue model (transfer-test). Secondary outcomes included cognitive test scores, GEARS ratings, and robot familiarity checklist scores. RESULTS: All groups demonstrated significant performance improvement after skills training (P < 0.01). Participating residents and fellows performed tasks faster (DOME and DVSS groups) and with fewer errors than controls (DOME group; P < 0.01). Inter-rater reliability was high for the checklist scores (0.82-0.97) but moderate for GEARS ratings (0.40-0.67). CONCLUSIONS: We provide evidence of effectiveness for the FRS curriculum by demonstrating better performance of those trained following FRS compared with controls on a transfer test. We therefore argue for its implementation across training programs before surgeons apply these skills clinically.

Evaluation of different approaches to define expert benchmark scores for new robotic training simulators based on the Medtronic HUGO™ RAS surgical robot experience.

New robot-assisted surgery platforms being developed will be required to have proficiency-based simulation training available. Scoring methodologies and performance feedback for trainees are currently not consistent across all robotic simulator platforms. Also, there are virtually no prior publications on how VR simulation passing benchmarks have been established. This paper compares methods evaluated to determine the proficiency-based scoring thresholds (a.k.a. benchmarks) for the new Medtronic Hugo™ RAS robotic simulator. Nine experienced robotic surgeons from multiple disciplines performed the 49 skills exercises 5 times each. The data were analyzed in 3 different ways: (1) include all data collected, (2) exclude first sessions, (3) exclude outliers. Eliminating the first session discounts becoming familiar with the exercise. Discounting outliers allows removal of potentially erroneous data that may be due to technical issues, unexpected distractions, etc. Outliers were identified using a common statistical technique involving the interquartile range of the data. Using each method above, mean and standard deviations were calculated, and the benchmark was set at a value of 1 standard deviation above the mean. In comparison to including all the data, when outliers are excluded, fewer data points are removed than just excluding first sessions, and the metric benchmarks are made more difficult by an average of 11%. When first sessions are excluded, the metric benchmarks are made easier by an average of about 2%. In comparison with benchmarks calculated using all data points, excluding outliers resulted in the biggest change making the benchmarks more challenging. We determined that this method provided the best representation of the data. These benchmarks should be validated with future clinical training studies.

Artificial Intelligence Methods and Artificial Intelligence-Enabled Metrics for Surgical Education: A Multidisciplinary Consensus.

BACKGROUND: Artificial intelligence (AI) methods and AI-enabled metrics hold tremendous potential to advance surgical education. Our objective was to generate consensus guidance on specific needs for AI methods and AI-enabled metrics for surgical education. STUDY DESIGN: The study included a systematic literature search, a virtual conference, and a 3-round Delphi survey of 40 representative multidisciplinary stakeholders with domain expertise selected through purposeful sampling. The accelerated Delphi process was completed within 10 days. The survey covered overall utility, anticipated future (10-year time horizon), and applications for surgical training, assessment, and feedback. Consensus was agreement among 80% or more respondents. We coded survey questions into 11 themes and descriptively analyzed the responses. RESULTS: The respondents included surgeons (40%), engineers (15%), affiliates of industry (27.5%), professional societies (7.5%), regulatory agencies (7.5%), and a lawyer (2.5%). The survey included 155 questions; consensus was achieved on 136 (87.7%). The panel listed 6 deliverables each for AI-enhanced learning curve analytics and surgical skill assessment. For feedback, the panel identified 10 priority deliverables spanning 2-year (n = 2), 5-year (n = 4), and 10-year (n = 4) timeframes. Within 2 years, the panel expects development of methods to recognize anatomy in images of the surgical field and to provide surgeons with performance feedback immediately after an operation. The panel also identified 5 essential that should be included in operative performance reports for surgeons. CONCLUSIONS: The Delphi panel consensus provides a specific, bold, and forward-looking roadmap for AI methods and AI-enabled metrics for surgical education.

AMPATH surgical app: Low-cost simulator for the open appendectomy.

BACKGROUND: Due to a shortage and maldistribution of surgeons within Kenya, doctors with limited formal surgical training often perform emergency surgical procedures such as appendectomy. This lack of training can compromise patient outcomes and complicate care delivery. Our aim was to develop a low-cost simulator and skills curriculum to effectively teach open appendectomy. METHODS: Surgeons from 4 countries participated in semi-structured interviews to define the steps and technique of open appendectomy using cognitive task analysis. Using this input, our Academic Model Providing Access to Healthcare surgical team developed a curriculum, including a simulator and feedback mechanism. Surgeons and surgical trainees from Kenya and the United States tested the simulator prototype and provided feedback for its refinement based on clarity, utility, and realism. RESULTS: Instructions for a self-constructed simulator were developed at the cost of 70 Kenyan shillings (0.64 US dollars). Fifteen surgeons and surgical residents gave feedback on the simulator and curriculum, and each was presented with an updated version based on feedback. Overall, the curriculum was clear, with each sub-step receiving a median score of ≥83.5 out of 100 for clarity; however, through iterative design, the utility of sub-steps on the simulator improved. CONCLUSION: A comprehensive open appendectomy curriculum, including a low-cost appendectomy simulator model, was developed and refined using surgeon feedback. Such curricula may benefit trainees in low-resource settings who may otherwise have limited access to quality training material.

Development of the fundamentals of thoracic robotic surgery curriculum.

A great technological revolution in surgery occurred with the introduction of laparoscopic and other minimally invasive procedures, with enormous patient benefits. Robotic-assisted surgery (RAS) is a form of minimally invasive surgery that overcomes some of the limitations of laparoscopic techniques. Until recently, there were few standardized curricula for RAS. The Fundamentals of Robotic (FRS) developed a process through expert consensus conferences to develop an effective and validated curriculum for basic robotic surgery. A specialty specific curriculum for thoracic robotic surgery is also needed. The Fundamentals of Thoracic Robotic Surgery (FTRS) Consensus Conference brought together expert thoracic surgeons to build upon the accomplishments of FRS and incorporate the experiences of thoracic societies and academic institutions to establish a standardized FTRS curriculum for the development and maintenance of specialty-specific robotic surgical skills. A task deconstruction was completed for the thoracic 'signature' procedure, the lobectomy and training items and potential errors were identified for each step of the procedure. A final outline of the FTRS curriculum was developed during the conference and physical and virtual reality thoracic surgery training models were discussed. Following the conference, the steering committee completed the FTRS curriculum with editorial review from all stakeholders. The FTRS consensus conference followed the validated FRS model but in an accelerated process due to important groundwork set by experts in the FRS consensus conferences. A full online curriculum and supporting psychomotor skills training and team communication has been developed for the lobectomy procedure.

Demonstrating the effectiveness of the fundamentals of robotic surgery (FRS) curriculum on the RobotiX Mentor Virtual Reality Simulation Platform.

Fundamentals of robotic surgery (FRS) is a proficiency-based progression curriculum developed by robotic surgery experts from multiple specialty areas to address gaps in existing robotic surgery training curricula. The RobotiX Mentor is a virtual reality training platform for robotic surgery. Our aims were to determine if robotic surgery novices would demonstrate improved technical skills after completing FRS training on the RobotiX Mentor, and to compare the effectiveness of FRS across training platforms. An observational, pre-post design, multi-institutional rater-blinded trial was conducted at two American College of Surgeons Accredited Education Institutes-certified simulation centers. Robotic surgery novices (n = 20) were enrolled and trained to expert-derived benchmarks using FRS on the RobotiX Mentor. Participants' baseline skill was assessed before (pre-test) and after (post-test) training on an avian tissue model. Tests were video recorded and graded by blinded raters using the Global Evaluative Assessment of Robotic Skills (GEARS) and a 32-criteria psychomotor checklist. Post hoc comparisons were conducted against previously published comparator groups. On paired-samples T tests, participants demonstrated improved performance across all GEARS domains (p < 0.001 to p = 0.01) and for time (p < 0.001) and errors (p = 0.003) as measured by psychometric checklist. By ANOVA, improvement in novices' skill after FRS training on the RobotiX Mentor was not inferior to improvement reported after FRS training on previously published platforms. Completion of FRS on the RobotiX Mentor resulted in improved robotic surgery skills among novices, proving effectiveness of training. These data provide additional validity evidence for FRS and support use of the RobotiX Mentor for robotic surgery skill acquisition.

How Wearable Technology Can Facilitate AI Analysis of Surgical Videos.

Operative video has great potential to enable instant replays of critical surgical decisions for training and quality review. Recently, artificial intelligence (AI) has shown early promise as a method of enabling efficient video review, analysis, and segmentation. Despite the progress with AI analysis of surgical videos, more work needs to be done to improve the accuracy and efficiency of AI-driven video analysis. At a recent consensus conference held on July 10-11, 2020, 8 research teams shared their work using AI for surgical video analysis. Four of the teams showcased the utility of wearable technology in providing objective surgical metrics. Data from these technologies were shown to pinpoint important cognitive and motor actions during operative tasks and procedures. The results support the utility of wearable technology to facilitate efficient and accurate video analysis and segmentation.

A consensus-based framework for design, validation, and implementation of simulation-based training curricula in surgery.

BACKGROUND: Simulation-based training can improve technical and nontechnical skills in surgery. To date, there is no consensus on the principles for design, validation, and implementation of a simulation-based surgical training curriculum. The aim of this study was to define such principles and formulate them into an interoperable framework using international expert consensus based on the Delphi method. METHODS: Literature was reviewed, 4 international experts were queried, and consensus conference of national and international members of surgical societies was held to identify the items for the Delphi survey. Forty-five international experts in surgical education were invited to complete the online survey by ranking each item on a Likert scale from 1 to 5. Consensus was predefined as Cronbach's α ≥0.80. Items that 80% of experts ranked as ≥4 were included in the final framework. RESULTS: Twenty-four international experts with training in general surgery (n = 11), orthopaedic surgery (n = 2), obstetrics and gynecology (n = 3), urology (n = 1), plastic surgery (n = 1), pediatric surgery (n = 1), otolaryngology (n = 1), vascular surgery (n = 1), military (n = 1), and doctorate-level educators (n = 2) completed the iterative online Delphi survey. Consensus among participants was achieved after one round of the survey (Cronbach's α = 0.91). The final framework included predevelopment analysis; cognitive, psychomotor, and team-based training; curriculum validation evaluation and improvement; and maintenance of training. CONCLUSIONS: The Delphi methodology allowed for determination of international expert consensus on the principles for design, validation, and implementation of a simulation-based surgical training curriculum. These principles were formulated into a framework that can be used internationally across surgical specialties as a step-by-step guide for the development and validation of future simulation-based training curricula.