Robust tracking of deformable anatomical structures with severe occlusions using deformable geometrical primitives.

BACKGROUND AND OBJECTIVE: Surgical robotics tends to develop cognitive control architectures to provide certain degree of autonomy to improve patient safety and surgery outcomes, while decreasing the required surgeons' cognitive load dedicated to low level decisions. Cognition needs workspace perception, which is an essential step towards automatic decision-making and task planning capabilities. Robust and accurate detection and tracking in minimally invasive surgery suffers from limited visibility, occlusions, anatomy deformations and camera movements. METHOD: This paper develops a robust methodology to detect and track anatomical structures in real time to be used in automatic control of robotic systems and augmented reality. The work focuses on the experimental validation in highly challenging surgery: fetoscopic repair of Open Spina Bifida. The proposed method is based on two sequential steps: first, selection of relevant points (contour) using a Convolutional Neural Network and, second, reconstruction of the anatomical shape by means of deformable geometric primitives. RESULTS: The methodology performance was validated with different scenarios. Synthetic scenario tests, designed for extreme validation conditions, demonstrate the safety margin offered by the methodology with respect to the nominal conditions during surgery. Real scenario experiments have demonstrated the validity of the method in terms of accuracy, robustness and computational efficiency. CONCLUSIONS: This paper presents a robust anatomical structure detection in present of abrupt camera movements, severe occlusions and deformations. Even though the paper focuses on a case study, Open Spina Bifida, the methodology is applicable in all anatomies which contours can be approximated by geometric primitives. The methodology is designed to provide effective inputs to cognitive robotic control and augmented reality systems that require accurate tracking of sensitive anatomies.

Robot assisted Fetoscopic Laser Coagulation: Improvements in navigation, re-location and coagulation.

Fetoscopic Laser Coagulation (FLC) for Twin to Twin Transfusion Syndrome is a challenging intervention due to the working conditions: low quality images acquired from a 3 mm fetoscope inside a turbid liquid environment, local view of the placental surface, unstable surgical field and delicate tissue layers. FLC is based on locating, coagulating and reviewing anastomoses over the placenta's surface. The procedure demands the surgeons to generate a mental map of the placenta with the distribution of the anastomoses, maintaining, at the same time, precision in coagulation and protecting the placenta and amniotic sac from potential damages. This paper describes a teleoperated platform with a cognitive-based control that provides assistance to improve patient safety and surgery performance during fetoscope navigation, target re-location and coagulation processes. A comparative study between manual and teleoperated operation, executed in dry laboratory conditions, analyzes basic fetoscopic skills: fetoscope navigation and laser coagulation. Two exercises are proposed: first, fetoscope guidance and precise coagulation. Second, a resolved placenta (all anastomoses are indicated) to evaluate navigation, re-location and coagulation. The results are analyzed in terms of economy of movement, execution time, coagulation accuracy, amount of coagulated placental surface and risk of placenta puncture. In addition, new metrics, based on navigation and coagulation maps evaluate robotic performance. The results validate the developed platform, showing noticeable improvements in all the metrics.

A Framework for the Evaluation of Human Machine Interfaces of Robot-Assisted Colonoscopy.

The Human Machine Interface (HMI) of intraluminal robots has a crucial impact on the clinician's performance. It increases or decreases the difficulty of the tasks, and is connected to the users' physical and mental stress. OBJECTIVE: This article presents a framework to compare and evaluate different HMIs for robotic colonoscopy, with the objective of identifying the optimal HMI that minimises the clinician's effort and maximises the clinical outcomes. METHODS: The framework comprises a 1) a virtual simulator (clinically validated), 2) wearable sensors measuring the cognitive load, 3) a data collection unit of metrics correlated to the clinical performance, and 4) questionnaires exploring the users' impressions and perceived stress. The framework was tested with 42 clinicians investigating the optimal device for tele-operated control of robotic colonoscopes. Two control devices were selected and compared: a haptic serial-kinematic device and a standard videogame joypad. RESULTS: The haptic device was preferred by the endoscopists, but the joypad enabled better clinical performance and reduced cognitive and physical load. CONCLUSION: The framework can be used to evaluate different aspects of a HMI, both hardware and software, and determine the optimal HMI that can reduce the burden on clinicians while improving the clinical outcome. SIGNIFICANCE: The findings of this study, and of future studies performed with this framework, can inform the design and development of HMIs for intraluminal robots, leading to improved clinical performance, reduced physical and mental stress for clinicians, and ultimately better patient outcomes.

EndoTrainer: a novel hybrid training platform for endoscopic surgery.

PURPOSE: Endoscopy implies high demanding procedures, and their practice requires structured formation curricula supported by adequate training platforms. Physical platforms are the most standardised solution for surgical training, but over the last few years, virtual platforms have been progressively introduced. This research work presents a new hybrid, physic-virtual, endoscopic training platform that exploits the benefits of the two kind of platforms combining realistic tools and phantoms together with the capacity of measuring all relevant parameters along the execution of the exercises and of providing an objective assessment performance. METHODS: The developed platform, EndoTrainer, has been designed to train and assess surgical skills in hysteroscopy and cystoscopy following a structured curricula. The initial development and validation is focused on hysteroscopic exercises proposed in the Gynaecological Endoscopic Surgical Education and Assessment (GESEA) Certification Programme from The Academy and European Society for Gynaecological Endoscopy (ESGE) and analyses the obtained results of an extensive study with 80 gynaecologists executing 30 trials of the standard 30 degree endoscope navigation exercise. RESULTS: The experiments demonstrate the benefits of the presented hybrid platform. Multi-variable statistical analysis points out that all subjects have obtained statistically significant improvement in all relevant parameters: shorter and safer trajectories, improved 30-degree endoscope navigation, accurate positioning over the targets and reduction of the execution time. CONCLUSION: This paper presents a new hybrid approach for training, and evaluating whether it provides an objectivable improvement of camera navigation endoscopic basic skills. The obtained results demonstrate the initial hypothesis: all subjects have improved their camera handling and navigation skills.

Intraamniotic sealing of fetoscopic membrane defects in ex vivo and in vivo sheep models using an integrated semirigid bioadhesive patch.

BACKGROUND: Preterm prelabor rupture of membranes is the most frequent complication of fetoscopic surgery. Strategies to seal the membrane defect created by fetoscopy have been attempted with little success. We previously developed an integrated semirigid bioadhesive patch composed of silicone and hydroxypropyl methylcellulose that achieved ex vivo sealing of membrane defects. OBJECTIVE: To evaluate the feasibility of the insertion of our integrated semirigid bioadhesive patches using a fetoscopic technique and to test the adhesion in ex vivo human membranes and in an in vivo ovine model. STUDY DESIGN: An experimental study involving 2 experiments: (1) ex vivo-human fetal membranes were mounted in a custom-designed model with saline solution simulating intraamniotic pressure. The insertion of 2 different bioadhesive patches made of silicone-hydroxypropyl methylcellulose and silicone-polyurethane-hydroxypropyl methylcellulose was performed through a 12-Fr cannula mimicking fetoscopic surgery technique. The experiment was repeated 10 times with membranes from different donors. Measures included insertion time, successful insertion, and adhesion at 5 minutes; (2) in vivo-16 patches of silicone-hydroxypropyl methylcellulose were inserted by fetoscopy in the amniotic cavity of pregnant sheep (4 bioadhesives per animal, in 4 ewes). Measures included successful insertion, adhesion at 5 minutes, and adhesion at the end of surgery. RESULTS: In the ex vivo insertion study, there was no difference in the insertion time between silicone-hydroxypropyl methylcellulose and silicone-polyurethane-hydroxypropyl methylcellulose patches (P=.49). Insertion was successful in all cases, but complete adhesion at 5 minutes was superior for silicone-hydroxypropyl methylcellulose (P=.02). In the in vivo study, insertion of silicone-hydroxypropyl methylcellulose by fetoscopy was feasible and successful in all cases, and no complications were reported. Adhesion persisted at 5 minutes and at the end of the surgery in 68.8% and 56.3% of the patches, respectively. CONCLUSION: We describe the feasibility of deploying through a fetoscopic trocar a semirigid silicone-hydroxypropyl methylcellulose patch that seals fetal membranes after an invasive fetal procedure. The results warrant further research for improving long-term adhesion and developing a clinically applicable system.

Training Simulators for Gastrointestinal Endoscopy: Current and Future Perspectives.

Gastrointestinal (GI) endoscopy is the gold standard in the detection and treatment of early and advanced GI cancers. However, conventional endoscopic techniques are technically demanding and require visual-spatial skills and significant hands-on experience. GI endoscopy simulators represent a valid solution to allow doctors to practice in a pre-clinical scenario. From the first endoscopy mannequin, developed in 1969, several simulation platforms have been developed, ranging from purely mechanical systems to more complex mechatronic devices and animal-based models. Considering the recent advancement of technologies (e.g., artificial intelligence, augmented reality, robotics), simulation platforms can now reach high levels of realism, representing a valid and smart alternative to standard trainee/mentor learning programs. This is particularly true nowadays, when the current demographic trend and the most recent pandemic demand, more than ever, the ability to cope with many patients. This review offers a broad view of the technology available for GI endoscopy training, including platforms currently in the market and the relevant advancements in this research and application field. Additionally, new training needs and new emerging technologies are discussed to understand where medical education is heading.

Vision Based Robot Assistance in TTTS Fetal Surgery.

This paper presents an accurate and robust tracking vision algorithm for Fetoscopic Laser Photo-coagulation (FLP) surgery for Twin-Twin Transfusion Syndrome (TTTS). The aim of the proposed method is to assist surgeons during anastomosis localization, coagulation and review using a tele-operated robotic system. The algorithm computes the relative position of the fetoscope tool tip with respect to the placenta, via local vascular structure registration. The algorithm uses image features (local superficial vascular structures of the placenta's surface) to automatically match consecutive fetoscopic images. It is composed of three sequential steps: image processing (filtering, binarization and vascular structures segmentation); relevant Points Of Interest (POIs) seletion; and image registration between consecutive images. The algorithm has to deal with the low quality of fetoscopic images, the liquid and dirty environment inside the placenta jointly with the thin diameter of the fetoscope optics and low amount of environment light reduces the image quality. The obtained images are blurred, noisy and with very poor color components. The tracking system has been tested using real video sequences of FLP surgery for TTTS. The computational performance enables real time tracking, locally guiding the robot over the placenta's surface with enough accuracy.

Enhancing perception in minimally invasive robotic surgery through self-calibration of surgical instruments.

In minimally invasive surgery (MIS) the 3D perception and hand-eye coordination is reduced. A high degree of sensory-motor skills is required by the surgeon to mediate action and perception. The incorporation of robotic assistants is capable of integrating the required egocentric information to aid the surgeon providing a sense of presence. In this work we propose a robotic vision system as a tool to locate surgical instruments with respect to the surgeon. It estimates the absolute position of the port of entry, in which surgical instruments are inserted to access the patient body. The proposed algorithm shows that it is possible to estimate the orientation of the instruments with respect to the laparoscopic camera applying determined movements through the robotic assistant.