Simulation training in urology.

PURPOSE OF REVIEW: This review outlines recent innovations in simulation technology as it applies to urology. It is essential for the next generation of urologists to attain a solid foundation of technical and nontechnical skills, and simulation technology provides a variety of safe, controlled environments to acquire this baseline knowledge. RECENT FINDINGS: With a focus on urology, this review first outlines the evidence to support surgical simulation, then discusses the strides being made in the development of 3D-printed models for surgical skill training and preoperative planning, virtual reality models for different urologic procedures, surgical skill assessment for simulation, and integration of simulation into urology residency curricula. SUMMARY: Simulation continues to be an integral part of the journey towards the mastery of skills necessary for becoming an expert urologist. Clinicians and researchers should consider how to further incorporate simulation technology into residency training and help future generations of urologists throughout their career.

Exploring the Use of Artificial Intelligence in the Management of Prostate Cancer.

PURPOSE OF REVIEW: This review aims to explore the current state of research on the use of artificial intelligence (AI) in the management of prostate cancer. We examine the various applications of AI in prostate cancer, including image analysis, prediction of treatment outcomes, and patient stratification. Additionally, the review will evaluate the current limitations and challenges faced in the implementation of AI in prostate cancer management. RECENT FINDINGS: Recent literature has focused particularly on the use of AI in radiomics, pathomics, the evaluation of surgical skills, and patient outcomes. AI has the potential to revolutionize the future of prostate cancer management by improving diagnostic accuracy, treatment planning, and patient outcomes. Studies have shown improved accuracy and efficiency of AI models in the detection and treatment of prostate cancer, but further research is needed to understand its full potential as well as limitations.

Tailored Feedback Based on Clinically Relevant Performance Metrics Expedites the Acquisition of Robotic Suturing Skills-An Unblinded Pilot Randomized Controlled Trial.

PURPOSE: Previously, we identified 8 objective suturing performance metrics highly predictive of urinary continence recovery after robotic-assisted radical prostatectomy. Here, we aimed to test the feasibility of providing tailored feedback based upon these clinically relevant metrics and explore the impact on the acquisition of robotic suturing skills. MATERIALS AND METHODS: Training surgeons were recruited and randomized to a feedback group or a control group. Both groups completed a baseline, midterm and final dry laboratory vesicourethral anastomosis (VUA) and underwent 4 intervening training sessions each, consisting of 3 suturing exercises. Eight performance metrics were recorded during each exercise: 4 automated performance metrics (derived from kinematic and system events data of the da Vinci® Robotic System) representing efficiency and console manipulation competency, and 4 suturing technical skill scores. The feedback group received tailored feedback (a visual diagram+verbal instructions+video examples) based on these metrics after each session. Generalized linear mixed model was used to compare metric improvement (Δ) from baseline to the midterm and final VUA. RESULTS: Twenty-three participants were randomized to the feedback group (11) or the control group (12). Demographic data and baseline VUA metrics were comparable between groups. The feedback group showed greater improvement than the control group in aggregate suturing scores at midterm (mean Δ feedback group 4.5 vs Δ control group 1.1) and final VUA (Δ feedback group 5.3 vs Δ control group 4.9). The feedback group also showed greater improvement in the majority of the included metrics at midterm and final VUA. CONCLUSIONS: Tailored feedback based on specific, clinically relevant performance metrics is feasible and may expedite the acquisition of robotic suturing skills.

An Objective Assessment of Performance during Robotic Partial Nephrectomy: Validation and Correlation of Automated Performance Metrics with Intraoperative Outcomes.

PURPOSE: Automated performance metrics provide a novel approach to the assessment of surgical performance. Herein, we present a construct validation of automated performance metrics during robotic assisted partial nephrectomy. MATERIALS AND METHODS: Automated performance metrics (instrument motion tracking/system events) and synchronized surgical videos from da Vinci® Si systems during robotic assisted partial nephrectomy were recorded using a system data recorder. Each case was segmented into 7 steps: colon mobilization, ureteral identification/dissection, hilar dissection, exposure of tumor within Gerota's fascia, intraoperative ultrasound/tumor scoring, tumor excision, and renorrhaphy. Automated performance metrics from each step were compared between expert (≥150 cases) and trainee (<150 cases) surgeons by Mann-Whitney U test (continuous variables) and Pearson's chi-squared test (categorical variables). Clinical outcomes were collected prospectively and correlated to automated performance metrics and R.E.N.A.L. (radius, exophytic/endophytic, nearness of tumor to collecting system, anterior/posterior, location relative to polar line) nephrometry score by Spearman's correlation coefficients (r). RESULTS: A total of 50 robotic assisted partial nephrectomy cases were included for analysis, performed by 7 expert and 10 trainee surgeons. Automated performance metric profiles significantly differed between experts and novices in the initial 5 steps (p <0.05). Specifically, experts exhibited faster dominant instrument movement and greater dominant instrument usage (bimanual dexterity) than trainees in select steps (p ≤0.045). Automated performance metrics during tumor excision and renorrhaphy were significantly correlated with R.E.N.A.L. score (r ≥0.364; p ≤0.041). These included metrics related to instrument efficiency, task duration, and dominant instrument use. CONCLUSIONS: Experts are more efficient and directed in their movement during robotic assisted partial nephrectomy. Automated performance metrics during key steps correlate with objective measures of tumor complexity and may serve as predictors of clinical outcomes. These data help establish a standardized metric for surgeon assessment and training during robotic assisted partial nephrectomy.

A Novel Dissection Gesture Classification to Characterize Robotic Dissection Technique for Renal Hilar Dissection.

PURPOSE: Deconstruction of robotic surgical gestures into semantic vocabulary yields an effective tool for surgical education. In this study we disassembled tissue dissection into basic gestures, created a classification system, and showed its ability to distinguish between experts and novices. MATERIALS AND METHODS: Videos of renal hilum preparation during robotic assisted partial nephrectomies were manually reviewed to identify all discrete surgical movements. Identified dissection movements were classified into distinct gestures based on the consensus of 6 expert surgeons. This classification system was then employed to compare expert and novice dissection patterns during the renal hilum preparation. RESULTS: A total of 40 robotic renal hilum preparation videos were reviewed, representing 16 from 6 expert surgeons (100 or more robotic cases) and 24 from 13 novice surgeons (fewer than 100 robotic cases). Overall 9,819 surgical movements were identified, including 5,667 dissection movements and 4,152 supporting movements. Nine distinct dissection gestures were identified and classified into the 3 categories of single blunt dissection (spread, peel/push, hook), single sharp dissection (cold cut, hot cut and burn dissect) and combination gestures (pedicalize, 2-hand spread, and coagulate then cut). Experts completed 5 of 9 dissection gestures more efficiently than novices (p ≤0.033). In consideration of specific anatomical locations, experts used more peel/push and less hot cut while dissecting the renal vein (p <0.001), and used more pedicalize while dissecting the renal artery (p <0.001). CONCLUSIONS: Using this novel dissection gesture classification system, key differences in dissection patterns can be found between experts/novices. This comprehensive classification of dissection gestures may be broadly applied to streamline surgical education.

Innovations in Urologic Surgical Training.

PURPOSE OF REVIEW: This review aims to summarize innovations in urologic surgical training in the past 5 years. RECENT FINDINGS: Many assessment tools have been developed to objectively evaluate surgical skills and provide structured feedback to urologic trainees. A variety of simulation modalities (i.e., virtual/augmented reality, dry-lab, animal, and cadaver) have been utilized to facilitate the acquisition of surgical skills outside the high-stakes operating room environment. Three-dimensional printing has been used to create high-fidelity, immersive dry-lab models at a reasonable cost. Non-technical skills such as teamwork and decision-making have gained more attention. Structured surgical video review has been shown to improve surgical skills not only for trainees but also for qualified surgeons. Research and development in urologic surgical training has been active in the past 5 years. Despite these advances, there is still an unfulfilled need for a standardized surgical training program covering both technical and non-technical skills.

Comparison of clinical outcomes and automated performance metrics in robot-assisted radical prostatectomy with and without trainee involvement.

PURPOSE: In this study, we investigate the effect of trainee involvement on surgical performance, as measured by automated performance metrics (APMs), and outcomes after robot-assisted radical prostatectomy (RARP). METHODS: We compared APMs (instrument tracking, EndoWrist® articulation, and system events data) and clinical outcomes for cases with varying resident involvement. Four of 12 standardized RARP steps were designated critical ("cardinal") steps. Comparison 1: cases where the attending surgeon performed all four cardinal steps (Group A) and cases where a trainee was involved in at least one cardinal step (Group B). Comparison 2, where Group A is split into Groups C and D: cases where attending performs the whole case (Group C) vs. cases where a trainee performed at least one non-cardinal step (Group D). Mann-Whitney U and Chi-squared tests were used for comparisons. RESULTS: Comparison 1 showed significant differences in APM profiles including camera movement time, third instrument usage, dominant instrument moving time, velocity, articulation, as well as non-dominant instrument moving time and articulation (all favoring Group A p < 0.05). There was a significant difference in re-admission rates (10.9% in Group A vs 0% in Group B, p < 0.02), but not for post-operative outcomes. Comparison 2 demonstrated a significant difference in dominant instrument articulation (p < 0.05) but not in post-operative outcomes. CONCLUSIONS: Trainee involvement in RARP is safe. The degree of trainee involvement does not significantly affect major clinical outcomes. APM profiles are less efficient when trainees perform at least one cardinal step but not during non-cardinal steps.

Machine learning in the optimization of robotics in the operative field.

PURPOSE OF REVIEW: The increasing use of robotics in urologic surgery facilitates collection of 'big data'. Machine learning enables computers to infer patterns from large datasets. This review aims to highlight recent findings and applications of machine learning in robotic-assisted urologic surgery. RECENT FINDINGS: Machine learning has been used in surgical performance assessment and skill training, surgical candidate selection, and autonomous surgery. Autonomous segmentation and classification of surgical data have been explored, which serves as the stepping-stone for providing real-time surgical assessment and ultimately, improve surgical safety and quality. Predictive machine learning models have been created to guide appropriate surgical candidate selection, whereas intraoperative machine learning algorithms have been designed to provide 3-D augmented reality and real-time surgical margin checks. Reinforcement-learning strategies have been utilized in autonomous robotic surgery, and the combination of expert demonstrations and trial-and-error learning by the robot itself is a promising approach towards autonomy. SUMMARY: Robot-assisted urologic surgery coupled with machine learning is a burgeoning area of study that demonstrates exciting potential. However, further validation and clinical trials are required to ensure the safety and efficacy of incorporating machine learning into surgical practice.

Body mass index as an independent risk factor for inferior vena cava resection during thrombectomy for venous tumor thrombus of renal cell carcinoma.

OBJECTIVE: To define preoperative clinical and radiographic risk factors for the need of inferior vena cava (IVC) resection in patients with renal cell carcinoma (RCC) and IVC tumor thrombus. METHODS: We reviewed data of 121 patients with renal cell carcinoma and venous tumor thrombus receiving radical nephrectomy and thrombectomy at our institution between 2015 and 2017, and 86 patients with Mayo I-IV level tumor thrombus were included in the final analysis. Clinical features, operation details, and pathology data were collected. Preoperative images were reviewed separately by two radiologists. Univariable and multivariable logistic regression analyses were applied to evaluate clinical and radiographic risk factors of IVC resection. RESULTS: Of the 86 patients, 44 (51.2%) received IVC resection during thrombectomy. In univariate analysis, we found that body mass index (BMI) (odds ratio [OR] = 1.22, P = 0.003), primary tumor diameter (OR = 0.84, P = 0.022), tumor thrombus width (OR = 1.08, P = 0.037), tumor thrombus level (OR = 1.57, P = 0.030), and IVC occlusion (OR = 2.67, P = 0.038) were associated with the need for resection of the IVC. After adjusting for the other factors, BMI (OR = 1.18, P = 0.019) was the only significant risk factor for IVC resection. Multivariable analysis in Mayo II-IV subgroups confirmed BMI as an independent risk factor (OR = 1.26, P = 0.024). A correlation between BMI and the width (Pearson's correlation coefficient [PCC] = 0.27, P = 0.014) and length (PCC = 0.23, P = 0.037) of the tumor thrombus was noticed. CONCLUSION: We identified BMI as an independent risk factor for IVC resection during thrombectomy of RCC with tumor thrombus in a Chinese population. More careful preoperative preparation for the IVC resection and/or reconstruction is warranted in patients with higher BMI.

A Contemporary High-Volume Center Analysis of Postoperative Complications Following Surgical Management of Renal Cell Carcinoma with Tumor Thrombus.

PURPOSE: To define preoperative predictors and construct a preoperative multivariable model for prediction of postoperative complications in patients with renal cell carcinoma (RCC) after nephrectomy with thrombectomy. MATERIALS AND METHODS: We identified patients with RCC and level I-IV venous tumor thrombus (VTT) who underwent concomitant radical nephrectomy and thrombectomy between February 2015 and March 2018. Univariate and multivariate logistic regression analyses were used to assess the effect of preoperative factors on the incidence of overall and major postoperative complications within 30 days postoperatively. A nomogram for prediction of postoperative complications was also developed using regression coefficients from the multivariable analyses. RESULTS: A total of 120 patients met inclusion criteria. We reported an overall complication rate of 39.2% and major complication rate of 12.5% within 30 days after surgery, with perioperative mortality rate of 2.5%. On multivariate analysis, independent predictors of overall complications included systemic symptoms, comorbidity, level III/IV VTT and serum creatine (SCr) level, while only SCr level was significantly associated with major complications. The internal validation result showed that the accuracy of our preoperative nomogram for overall complications measured by c-index was 0.794. CONCLUSIONS: We constructed an accurate preoperative model to predict overall postoperative complications in patients with level I-IV thrombus. External verification is still needed to evaluate its general application.

Clinical applications of artificial intelligence in robotic surgery.

Artificial intelligence (AI) is revolutionizing nearly every aspect of modern life. In the medical field, robotic surgery is the sector with some of the most innovative and impactful advancements. In this narrative review, we outline recent contributions of AI to the field of robotic surgery with a particular focus on intraoperative enhancement. AI modeling is allowing surgeons to have advanced intraoperative metrics such as force and tactile measurements, enhanced detection of positive surgical margins, and even allowing for the complete automation of certain steps in surgical procedures. AI is also Query revolutionizing the field of surgical education. AI modeling applied to intraoperative surgical video feeds and instrument kinematics data is allowing for the generation of automated skills assessments. AI also shows promise for the generation and delivery of highly specialized intraoperative surgical feedback for training surgeons. Although the adoption and integration of AI show promise in robotic surgery, it raises important, complex ethical questions. Frameworks for thinking through ethical dilemmas raised by AI are outlined in this review. AI enhancements in robotic surgery is some of the most groundbreaking research happening today, and the studies outlined in this review represent some of the most exciting innovations in recent years.

Surgical gestures to evaluate apical dissection of robot-assisted radical prostatectomy.

Previously, our group established a surgical gesture classification system that deconstructs robotic tissue dissection into basic surgical maneuvers. Here, we evaluate gestures by correlating the metric with surgeon experience and technical skill assessment scores in the apical dissection (AD) of robotic-assisted radical prostatectomy (RARP). Additionally, we explore the association between AD performance and early continence recovery following RARP. 78 AD surgical videos from 2016 to 2018 across two international institutions were included. Surgeons were grouped by median robotic caseload (range 80-5,800 cases): less experienced group (< 475 cases) and more experienced (≥ 475 cases). Videos were decoded with gestures and assessed using Dissection Assessment for Robotic Technique (DART). Statistical findings revealed more experienced surgeons (n = 10) used greater proportions of cold cut (p = 0.008) and smaller proportions of peel/push, spread, and two-hand spread (p < 0.05) than less experienced surgeons (n = 10). Correlations between gestures and technical skills assessments ranged from - 0.397 to 0.316 (p < 0.05). Surgeons utilizing more retraction gestures had lower total DART scores (p < 0.01), suggesting less dissection proficiency. Those who used more gestures and spent more time per gesture had lower efficiency scores (p < 0.01). More coagulation and hook gestures were found in cases of patients with continence recovery compared to those with ongoing incontinence (p < 0.04). Gestures performed during AD vary based on surgeon experience level and patient continence recovery duration. Significant correlations were demonstrated between gestures and dissection technical skills. Gestures can serve as a novel method to objectively evaluate dissection performance and anticipate outcomes.

Competency in Robotic Surgery: Standard Setting for Robotic Suturing Using Objective Assessment and Expert Evaluation.

OBJECTIVE: Surgical skill assessment tools such as the End-to-End Assessment of Suturing Expertise (EASE) can differentiate a surgeon's experience level. In this simulation-based study, we define a competency benchmark for intraoperative robotic suturing using EASE as a validated measure of performance. DESIGN: Participants conducted a dry-lab vesicourethral anastomosis (VUA) exercise. Videos were each independently scored by 2 trained, blinded reviewers using EASE. Inter-rater reliability was measured with prevalence-adjusted bias-adjusted Kappa (PABAK) using 2 example videos. All videos were reviewed by an expert surgeon, who determined if the suturing skills exhibited were at a competency level expected for residency graduation (pass or fail). The Contrasting Group (CG) method was then used to set a pass/fail score at the intercept of the pass and fail cohorts' EASE score distributions. SETTING: Keck School of Medicine, University of Southern California. PARTICIPANTS: Twenty-six participants: 8 medical students, 8 junior residents (PGY 1-2), 7 senior residents (PGY 3-5) and 3 attending urologists. RESULTS: After 1 round of consensus-building, average PABAK across EASE subskills was 0.90 (Range 0.67-1.0). The CG method produced a competency benchmark EASE score of >35/39, with a pass rate of 10/26 (38%); 27% were deemed competent by expert evaluation. False positives and negatives were defined as medical students who passed and attendings who failed the assessment, respectively. This pass/fail score produced no false positives or negatives, and fewer JR than SR were considered competent by both the expert and CG benchmark. CONCLUSIONS: Using an absolute standard setting method, competency scores were set to identify trainees who could competently execute a standardized dry-lab robotic suturing exercise. This standard can be used for high stakes decisions regarding a trainee's technical readiness for independent practice. Future work includes validation of this standard in the clinical environment through correlation with clinical outcomes.

Artificial Intelligence-Based Video Feedback to Improve Novice Performance on Robotic Suturing Skills: A Pilot Study.

Introduction: Automated skills assessment can provide surgical trainees with objective, personalized feedback during training. Here, we measure the efficacy of artificial intelligence (AI)-based feedback on a robotic suturing task. Materials and Methods: Forty-two participants with no robotic surgical experience were randomized to a control or feedback group and video-recorded while completing two rounds (R1 and R2) of suturing tasks on a da Vinci surgical robot. Participants were assessed on needle handling and needle driving, and feedback was provided via a visual interface after R1. For feedback group, participants were informed of their AI-based skill assessment and presented with specific video clips from R1. For control group, participants were presented with randomly selected video clips from R1 as a placebo. Participants from each group were further labeled as underperformers or innate-performers based on a median split of their technical skill scores from R1. Results: Demographic features were similar between the control (n = 20) and feedback group (n = 22) (p > 0.05). Observing the improvement from R1 to R2, the feedback group had a significantly larger improvement in needle handling score (0.30 vs -0.02, p = 0.018) when compared with the control group, although the improvement of needle driving score was not significant when compared with the control group (0.17 vs -0.40, p = 0.074). All innate-performers exhibited similar improvements across rounds, regardless of feedback (p > 0.05). In contrast, underperformers in the feedback group improved more than the control group in needle handling (p = 0.02). Conclusion: AI-based feedback facilitates surgical trainees' acquisition of robotic technical skills, especially underperformers. Future research will extend AI-based feedback to additional suturing skills, surgical tasks, and experience groups.

Capturing relationships between suturing sub-skills to improve automatic suturing assessment.

Suturing skill scores have demonstrated strong predictive capabilities for patient functional recovery. The suturing can be broken down into several substep components, including needle repositioning, needle entry angle, etc. Artificial intelligence (AI) systems have been explored to automate suturing skill scoring. Traditional approaches to skill assessment typically focus on evaluating individual sub-skills required for particular substeps in isolation. However, surgical procedures require the integration and coordination of multiple sub-skills to achieve successful outcomes. Significant associations among the technical sub-skill have been established by existing studies. In this paper, we propose a framework for joint skill assessment that takes into account the interconnected nature of sub-skills required in surgery. The prior known relationships among sub-skills are firstly identified. Our proposed AI system is then empowered by the prior known relationships to perform the suturing skill scoring for each sub-skill domain simultaneously. Our approach can effectively improve skill assessment performance through the prior known relationships among sub-skills. Through the proposed approach to joint skill assessment, we aspire to enhance the evaluation of surgical proficiency and ultimately improve patient outcomes in surgery.

The effect of enhanced recovery after surgery on oncologic outcome following radical cystectomy for urothelial bladder carcinoma.

INTRODUCTION: Limited data are available regarding the effect of enhanced recovery after surgery (ERAS) protocols on the long-term outcomes of radical cystectomy (RC) in bladder cancer patients. The aim of this study is to evaluate the oncological outcomes in patients who underwent RC with ERAS protocol. METHODS: We reviewed the records of patients who underwent RC for primary urothelial bladder carcinoma with curative intent from January 2003 to August 2022. The primary and secondary outcomes were recurrence-free (RFS) and overall survival (OS). Multivariable Cox regression analysis was performed to evaluate the effect of ERAS on oncological outcomes. RESULTS: A total of 967 ERAS patients and 1144 non-ERAS patients were included in this study. The RFS rates at 1, 3, and 5 years after RC were 81%, 71.5%, and 69% in the ERAS cohort, respectively. This rate in the non-ERAS group was 81%, 71%, and 67% at 1, 3, and 5 years after RC, respectively (P = 0.50). However, ERAS patients had significantly better OS with 86%, 73%, and 67% survival rates at 1, 3, and 5 years compared to 84%, 68%, and 59.5% survival rates in the non-ERAS group, respectively (P = 0.002). In multivariable analysis adjusting for other relevant factors, ERAS was no longer independently associated with recurrence-free (HR = 0.96, 95% CI 0.76-1.22, P = 0.75) or overall survival (HR = 0.84, 95% CI 0.66-1.09, P = 0.28) following RC. CONCLUSION: ERAS protocols are associated with a shorter hospital stay, yet with no impact on long-term oncologic outcomes in patients undergoing RC for bladder cancer.

STFuse: Infrared and Visible Image Fusion via Semisupervised Transfer Learning.

Infrared and visible image fusion (IVIF) aims to obtain an image that contains complementary information about the source images. However, it is challenging to define complementary information between source images in the lack of ground truth and without borrowing prior knowledge. Therefore, we propose a semisupervised transfer learning-based method for IVIF, termed STFuse, which aims to transfer knowledge from an informative source domain to a target domain, thus breaking the above limitations. The critical aspect of our method is to borrow supervised knowledge from the multifocus image fusion (MFIF) task and to filter out task-specific attribute knowledge by using a guidance loss Lg , which motivates its cross-task use in IVIF tasks. Using this cross-task knowledge effectively alleviates the limitation of the lack of ground truth on fusion performance, and the complementary expression ability under the constraint of supervised knowledge is more instructive than prior knowledge. Moreover, we designed a cross-feature enhancement module (CEM) that utilizes self-attention and mutual-attention features to guide each branch to refine features and then facilitate the integration of cross-modal complementary features. Extensive experiments demonstrate that our method has good advantages in terms of visual quality and statistical metrics, as well as the docking of high-level vision tasks, compared with other state-of-the-art methods.

Assessing the efficacy of dissection gestures in robotic surgery.

Our group previously defined a dissection gesture classification system that deconstructs robotic tissue dissection into its most elemental yet meaningful movements. The purpose of this study was to expand upon this framework by adding an assessment of gesture efficacy (ineffective, effective, or erroneous) and analyze dissection patterns between groups of surgeons of varying experience. We defined three possible gesture efficacies as ineffective (no meaningful effect on the tissue), effective (intended effect on the tissue), and erroneous (unintended disruption of the tissue). Novices (0 prior robotic cases), intermediates (1-99 cases), and experts (≥ 100 cases) completed a robotic dissection task in a dry-lab training environment. Video recordings were reviewed to classify each gesture and determine its efficacy, then dissection patterns between groups were analyzed. 23 participants completed the task, with 9 novices, 8 intermediates with median caseload 60 (IQR 41-80), and 6 experts with median caseload 525 (IQR 413-900). For gesture selection, we found increasing experience associated with increasing proportion of overall dissection gestures (p = 0.009) and decreasing proportion of retraction gestures (p = 0.009). For gesture efficacy, novices performed the greatest proportion of ineffective gestures (9.8%, p < 0.001), intermediates commit the greatest proportion of erroneous gestures (26.8%, p < 0.001), and the three groups performed similar proportions of overall effective gestures, though experts performed the greatest proportion of effective retraction gestures (85.6%, p < 0.001). Between groups of experience, we found significant differences in gesture selection and gesture efficacy. These relationships may provide insight into further improving surgical training.

A vision transformer for decoding surgeon activity from surgical videos.

The intraoperative activity of a surgeon has substantial impact on postoperative outcomes. However, for most surgical procedures, the details of intraoperative surgical actions, which can vary widely, are not well understood. Here we report a machine learning system leveraging a vision transformer and supervised contrastive learning for the decoding of elements of intraoperative surgical activity from videos commonly collected during robotic surgeries. The system accurately identified surgical steps, actions performed by the surgeon, the quality of these actions and the relative contribution of individual video frames to the decoding of the actions. Through extensive testing on data from three different hospitals located in two different continents, we show that the system generalizes across videos, surgeons, hospitals and surgical procedures, and that it can provide information on surgical gestures and skills from unannotated videos. Decoding intraoperative activity via accurate machine learning systems could be used to provide surgeons with feedback on their operating skills, and may allow for the identification of optimal surgical behaviour and for the study of relationships between intraoperative factors and postoperative outcomes.