µRALP and Beyond: Micro-Technologies and Systems for Robot-Assisted Endoscopic Laser Microsurgery.

Laser microsurgery is the current gold standard surgical technique for the treatment of selected diseases in delicate organs such as the larynx. However, the operations require large surgical expertise and dexterity, and face significant limitations imposed by available technology, such as the requirement for direct line of sight to the surgical field, restricted access, and direct manual control of the surgical instruments. To change this status quo, the European project µRALP pioneered research towards a complete redesign of current laser microsurgery systems, focusing on the development of robotic micro-technologies to enable endoscopic operations. This has fostered awareness and interest in this field, which presents a unique set of needs, requirements and constraints, leading to research and technological developments beyond µRALP and its research consortium. This paper reviews the achievements and key contributions of such research, providing an overview of the current state of the art in robot-assisted endoscopic laser microsurgery. The primary target application considered is phonomicrosurgery, which is a representative use case involving highly challenging microsurgical techniques for the treatment of glottic diseases. The paper starts by presenting the motivations and rationale for endoscopic laser microsurgery, which leads to the introduction of robotics as an enabling technology for improved surgical field accessibility, visualization and management. Then, research goals, achievements, and current state of different technologies that can build-up to an effective robotic system for endoscopic laser microsurgery are presented. This includes research in micro-robotic laser steering, flexible robotic endoscopes, augmented imaging, assistive surgeon-robot interfaces, and cognitive surgical systems. Innovations in each of these areas are shown to provide sizable progress towards more precise, safer and higher quality endoscopic laser microsurgeries. Yet, major impact is really expected from the full integration of such individual contributions into a complete clinical surgical robotic system, as illustrated in the end of this paper with a description of preliminary cadaver trials conducted with the integrated µRALP system. Overall, the contribution of this paper lays in outlining the current state of the art and open challenges in the area of robot-assisted endoscopic laser microsurgery, which has important clinical applications even beyond laryngology.

Stereo vision-based tracking of soft tissue motion with application to online ablation control in laser microsurgery.

Recent research has revealed that image-based methods can enhance accuracy and safety in laser microsurgery. In this study, non-rigid tracking using surgical stereo imaging and its application to laser ablation is discussed. A recently developed motion estimation framework based on piecewise affine deformation modeling is extended by a mesh refinement step and considering texture information. This compensates for tracking inaccuracies potentially caused by inconsistent feature matches or drift. To facilitate online application of the method, computational load is reduced by concurrent processing and affine-invariant fusion of tracking and refinement results. The residual latency-dependent tracking error is further minimized by Kalman filter-based upsampling, considering a motion model in disparity space. Accuracy is assessed in laparoscopic, beating heart, and laryngeal sequences with challenging conditions, such as partial occlusions and significant deformation. Performance is compared with that of state-of-the-art methods. In addition, the online capability of the method is evaluated by tracking two motion patterns performed by a high-precision parallel-kinematic platform. Related experiments are discussed for tissue substitute and porcine soft tissue in order to compare performances in an ideal scenario and in a setup mimicking clinical conditions. Regarding the soft tissue trial, the tracking error can be significantly reduced from 0.72 mm to below 0.05 mm with mesh refinement. To demonstrate online laser path adaptation during ablation, the non-rigid tracking framework is integrated into a setup consisting of a surgical Er:YAG laser, a three-axis scanning unit, and a low-noise stereo camera. Regardless of the error source, such as laser-to-camera registration, camera calibration, image-based tracking, and scanning latency, the ablation root mean square error is kept below 0.21 mm when the sample moves according to the aforementioned patterns. Final experiments regarding motion-compensated laser ablation of structurally deforming tissue highlight the potential of the method for vision-guided laser surgery.

Soft tissue motion tracking with application to tablet-based incision planning in laser surgery.

PURPOSE: Recent research has revealed that incision planning in laser surgery deploying stylus and tablet outperforms micromanipulator control. However, vision-based adaption to dynamic surgical scenes has not been addressed so far. In this study, scene motion compensation for tablet-based planning by means of tissue deformation tracking is discussed. METHODS: A stereo-based method for motion tracking with piecewise affine deformation modeling is presented. Proposed parametrization relies on the epipolar constraint to enforce left-right consistency in the energy minimization problem. Furthermore, the method implements illumination-invariant tracking and appearance-based occlusion detection. Performance is assessed on laparoscopic and laryngeal in vivo data. In particular, tracking accuracy is measured under various conditions such as occlusions and simulated laser cuttings. Experimental validation is extended to a user study conducted on a tablet-based interface that integrates the tracking for image stabilization. RESULTS: Tracking accuracy measurement reveals a root-mean-square error of 2.45 mm for the laparoscopic and 0.41 mm for the laryngeal dataset. Results successfully demonstrate stereoscopic tracking under changes in illumination, translation, rotation and scale. In particular, proposed occlusion detection scheme can increase robustness against tracking failure. Moreover, assessed user performance indicates significantly increased path tracing accuracy and usability if proposed tracking is deployed to stabilize the view during free-hand path definition. CONCLUSION: The presented algorithm successfully extends piecewise affine deformation tracking to stereo vision taking the epipolar constraint into account. Improved surgical performance as demonstrated for laser incision planning highlights the potential of presented method regarding further applications in computer-assisted surgery.

Comparative study on surface reconstruction accuracy of stereo imaging devices for microsurgery.

PURPOSE: Processing stereoscopic image data is an emerging field. Especially in microsurgery that requires sub-millimeter accuracy, application of stereo-based methods on endoscopic and microscopic scenarios is of major interest. In this context, direct comparison of stereo-based surface reconstruction applied to several camera settings is presented. METHODS: A method for stereo matching is proposed and validated on in-vitro data. Demonstrating suitability for surgical scenarios, this method is applied to two custom-made stereo cameras, a miniaturized, bendable surgical endoscope and an operating microscope. Reconstruction accuracy is assessed on a custom-made reference sample. Subsequent to its fabrication, a coordinate measuring arm is used to acquire ground truth. Next, the sample is positioned by a robot at varying distances to each camera. Surface estimation is performed, while the specimen is localized based on. markers. Finally, the error between estimated surface and ground truth is computed. RESULTS: Sample measurement with the coordinate measuring arm yields reliable ground truth data with a root-mean-square error of 11.2 µm. Overall surface reconstruction with analyzed cameras is quantified by a root-mean-square error of less than 0.18 mm. Microscope setting with the highest magnification yields the most accurate measurement, while the maximum deviation does not exceed 0.5 mm. Custom-made stereo cameras perform similar but with outliers of increased magnitude. Miniaturized, bendable surgical endoscope produces the maximum error of approximately 1.2 mm. CONCLUSIONS: Reconstruction results reveal that microscopic imaging outperforms investigated chip-on-the-tip solutions, i.e., at higher magnification. Nonetheless, custom-made cameras are suitable for application in microsurgery. Although reconstruction with the miniaturized endoscope is more inaccurate, it provides a good trade-off between accuracy, outer dimensions and accessibility to hard-to-reach surgical sites.

Tissue surface information for intraoperative incision planning and focus adjustment in laser surgery.

PURPOSE: Introducing computational methods to laser surgery are an emerging field. Focusing on endoscopic laser interventions, a novel approach is presented to enhance intraoperative incision planning and laser focusing by means of tissue surface information obtained by stereoscopic vision. METHODS: Tissue surface is estimated with stereo-based methods using nonparametric image transforms. Subsequently, laser-to-camera registration is obtained by ablating a pattern on tissue substitutes and performing a principle component analysis for precise laser axis estimation. Furthermore, a virtual laser view is computed utilizing trifocal transfer. Depth-based laser focus adaptation is integrated into a custom experimental laser setup in order to achieve optimal ablation morphology. Experimental validation is conducted on tissue substitutes and ex vivo animal tissue. RESULTS: Laser-to-camera registration gives an error between planning and ablation of less than 0.2 mm. As a result, the laser workspace can accurately be highlighted within the live views and incision planning can directly be performed. Experiments related to laser focus adaptation demonstrate that ablation geometry can be kept almost uniform within a depth range of 7.9 mm, whereas cutting quality significantly decreases when the laser is defocused. CONCLUSIONS: An automatic laser focus adjustment on tissue surfaces based on stereoscopic scene information is feasible and has the potential to become an effective methodology for optimal ablation. Laser-to-camera registration facilitates advanced surgical planning for prospective user interfaces and augmented reality extensions.