First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery.

The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre-operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real-time de-occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to the authors' best knowledge, the first-in-human on edge deployment of a real-time binary segmentation pipeline during three robot-assisted surgeries: partial nephrectomy, migrated endovascular stent removal, and liver metastasectomy. To this end, a state-of-the-art real-time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real-time binary segmentation of 37 non-organic surgical items, which are never occluded during AR. The application features real-time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics, and acceptance of AR in minimally invasive surgery.

Improving Augmented Reality Through Deep Learning: Real-time Instrument Delineation in Robotic Renal Surgery.

Several barriers prevent the integration and adoption of augmented reality (AR) in robotic renal surgery despite the increased availability of virtual three-dimensional (3D) models. Apart from correct model alignment and deformation, not all instruments are clearly visible in AR. Superimposition of a 3D model on top of the surgical stream, including the instruments, can result in a potentially hazardous surgical situation. We demonstrate real-time instrument detection during AR-guided robot-assisted partial nephrectomy and show the generalization of our algorithm to AR-guided robot-assisted kidney transplantation. We developed an algorithm using deep learning networks to detect all nonorganic items. This algorithm learned to extract this information for 65 927 manually labeled instruments on 15 100 frames. Our setup, which runs on a standalone laptop, was deployed in three different hospitals and used by four different surgeons. Instrument detection is a simple and feasible way to enhance the safety of AR-guided surgery. Future investigations should strive to optimize efficient video processing to minimize the 0.5-s delay currently experienced. General AR applications also need further optimization, including detection and tracking of organ deformation, for full clinical implementation.

Feasibility of a telementoring approach as a practical training for transurethral enucleation of the benign prostatic hyperplasia using bipolar energy: a pilot study.

INTRODUCTION: Telementoring is one of the applications of telemedicine capable of bringing highly experienced surgeons to areas lacking expertise. In the current study, we aimed to assess a novel telementoring application during the learning curve of transurethral enucleation of the prostate using bipolar energy (TUEB). MATERIAL AND METHODS: A telementoring system was developed by our engineering department. This application was used to mentor ten prospective cases of TUEB performed by an expert endourologist (novice to the TUEB). A questionnaire was filled by the operating surgeon and the mentor to provide subjective evaluation of the telementoring system. Finally, the outcomes of these patients were compared to a control group consisting of ten consecutive patients performed by the mentor. RESULTS: Ten consecutive TUEB were performed using this telementoring application. Delayed and interrupted connection were experienced in two and one patients, respectively; however, their effect was minor, and they did not compromise the safety of the procedure. None of the patients required conversion to conventional transurethral resection of the prostate. Only one patient in our series experienced grade IIIb complication. CONCLUSION: The telementoring application for TUEB is promising. It is a simple and low-cost tool that could be a feasible option to ensure patients' safety during the initial phase of the learning curve without time and locations constraints for both the mentor and the trainee; However, it should be mentioned that telementoring cannot yet replace the traditional surgical training with the mentor and trainee being in the operative room. Further studies are required to confirm the current results.

An algorithm for planning the number and the pose of the iceballs in cryoablation.

We present an algorithm that computes the number and the pose (position and orientation) of iceballs in a cryoablation procedure, in order to completely cover the target region, i.e. the tumor. Constraints to needle insertion, such as regions that have to be avoided, are taken into account and satisfied. We developed a tool for cryosurgery planning in MATLAB and perform several simulations to extract information on the algorithm behavior and to verify that it always brings to a complete coverage.