Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument.

PURPOSE: Complications related to vascular damage such as intra-operative bleeding may be avoided during neurosurgical procedures such as petroclival meningioma surgery. To address this and improve the patient's safety, we designed a real-time blood vessel avoidance strategy that enables operation on deformable tissue during petroclival meningioma surgery using Micron, a handheld surgical robotic tool. METHODS: We integrated real-time intra-operative blood vessel segmentation of brain vasculature using deep learning, with a 3D reconstruction algorithm to obtain the vessel point cloud in real time. We then implemented a virtual-fixture-based strategy that prevented Micron's tooltip from entering a forbidden region around the vessel, thus avoiding damage to it. RESULTS: We achieved a median Dice similarity coefficient of 0.97, 0.86, 0.87 and 0.77 on datasets of phantom blood vessels, petrosal vein, internal carotid artery and superficial vessels, respectively. We conducted trials with deformable clay vessel phantoms, keeping the forbidden region 400 [Formula: see text]m outside and 400 [Formula: see text]m inside the vessel. Micron's tip entered the forbidden region with a median penetration of just 8.84 [Formula: see text]m and 9.63 [Formula: see text]m, compared to 148.74 [Formula: see text]m and 117.17 [Formula: see text]m without our strategy, for the former and latter trials, respectively. CONCLUSION: Real-time control of Micron was achieved at 33.3 fps. We achieved improvements in real-time segmentation of brain vasculature from intra-operative images and showed that our approach works even on non-stationary vessel phantoms. The results suggest that by enabling precise, real-time control, we are one step closer to using Micron in real neurosurgical procedures.

Comparative Evaluation of Handheld Robot-Aided Intraocular Laser Surgery.

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Hybrid position/force control of an active handheld micromanipulator for membrane peeling.

BACKGROUND: Peeling procedures in retinal surgery require micron-scale manipulation and control of sub-tactile forces. METHODS: Hybrid position/force control of an actuated handheld microsurgical instrument is presented as a means for simultaneously improving positioning accuracy and reducing forces to prevent avoidable trauma to tissue. The system response was evaluated, and membrane-peeling trials were performed by four test subjects in both artificial and animal models. RESULTS: Maximum force was reduced by 56% in both models compared with position control. No statistically significant effect on procedure duration was observed. CONCLUSIONS: A hybrid position/force control system has been implemented that successfully attenuates forces and minimizes unwanted excursions during microsurgical procedures such as membrane peeling. Results also suggest that improvements in safety using this technique may be attained without increasing the duration of the procedure.

Manipulator Design and Operation for a Six-Degree-of-Freedom Handheld Tremor-Canceling Microsurgical Instrument.

This paper presents the design and actuation of a six-degree-of-freedom (6-DOF) manipulator for a handheld instrument, known as "Micron," which performs active tremor compensation during microsurgery. The design incorporates a Gough-Stewart platform based on piezoelectric linear motor, with a specified minimum workspace of a cylinder 4 mm long and 4 mm in diameter at the end-effector. Given the stall force of the motors and the loading typically encountered in vitreoretinal microsurgery, the dimensions of the manipulator are optimized to tolerate a transverse load of 0.2 N on a remote center of motion near the midpoint of the tool shaft. The optimization yields a base diameter of 23 mm and a height of 37 mm. The fully handheld instrument includes a custom-built optical tracking system for control feedback, and an ergonomic housing to serve as a handle. The manipulation performance was investigated in both clamped and handheld conditions. In positioning experiments with varying side loads, the manipulator tolerates side load up to 0.25 N while tracking a sinusoidal target trajectory with less than 20 µm error. Physiological hand tremor is reduced by about 90% in a pointing task, and error less than 25 µm is achieved in handheld circle-tracing.

Toward Automated Intraocular Laser Surgery Using a Handheld Micromanipulator.

This paper presents a technique for automated intraocular laser surgery using a handheld micromanipulator known as Micron. The novel handheld manipulator enables the automated scanning of a laser probe within a cylinder of 4 mm long and 4 mm in diameter. For the automation, the surface of the retina is reconstructed using a stereomicroscope, and then preplanned targets are placed on the surface. The laser probe is precisely located on the target via visual servoing of the aiming beam, while maintaining a specific distance above the surface. In addition, the system is capable of tracking the surface of the eye in order to compensate for any eye movement introduced during the operation. We compared the performance of the automated scanning using various control thresholds, in order to find the most effective threshold in terms of accuracy and speed. Given the selected threshold, we conducted the handheld operation above a fixed target surface. The average error and execution time are reduced by 63.6% and 28.5%, respectively, compared to the unaided trials. Finally, the automated laser photocoagulation was demonstrated also in an eye phantom, including compensation for the eye movement.

Improvement of optical coherence tomography using active handheld micromanipulator in vitreoretinal surgery.

An active handheld micromanipulator has been developed to cancel hand tremor during microsurgery. The micromanipulator is also applicable in optical coherence tomography to improve the quality of scanning and minimize surgical risks during the scans. The manipulator can maneuver the tool tip with six degrees of freedom within a cylindrical workspace 4 mm in diameter and 4 mm high. The imaging system is equipped with a 25-gauge Fourier-domain common-path OCT probe. This paper introduces the handheld OCT imaging system and techniques involved and presents stabilized OCT images of A-mode and M-mode scans in air and live rabbit eyes. We show the first demonstration of OCT imaging using the active handheld micromanipulator in vivo.

Vision-Based Control of a Handheld Surgical Micromanipulator with Virtual Fixtures.

Performing micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real-time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p < 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p < 0.05).

Micron: an Actively Stabilized Handheld Tool for Microsurgery.

We describe the design and performance of a hand-held actively stabilized tool to increase accuracy in micro-surgery or other precision manipulation. It removes involuntary motion such as tremor by actuating the tip to counteract the effect of the undesired handle motion. The key components are a three-degree-of-freedom piezoelectric manipulator that has 400 µm range of motion, 1 N force capability, and bandwidth over 100 Hz, and an optical position measurement subsystem that acquires the tool pose with 4 µm resolution at 2000 samples/s. A control system using these components attenuates hand motion by at least 15 dB (a fivefold reduction). By considering the effect of the frequency response of Micron on the human visual feedback loop, we have developed a filter that reduces unintentional motion, yet preserves intuitive eye-hand coordination. We evaluated the effectiveness of Micron by measuring the accuracy of the human/machine system in three simple manipulation tasks. Handheld testing by three eye surgeons and three non-surgeons showed a reduction in position error of between 32% and 52%, depending on the error metric.

Semiautomated intraocular laser surgery using handheld instruments.

BACKGROUND AND OBJECTIVE: In laser retinal photocoagulation, hundreds of dot-like burns are applied. We introduce a robot-assisted technique to enhance the accuracy and reduce the tedium of the procedure. MATERIALS AND METHODS: Laser burn locations are overlaid on preoperative retinal images using common patterns such as grids. A stereo camera/monitor setup registers and displays the planned burn locations overlaid on real-time video. Using an active handheld micromanipulator, a 7 x 7 grid of burns spaced 650 microm apart is applied to both paper slides and porcine retina in vitro using 30 milliseconds laser pulses at 532 nm. Two scenarios were tested: unaided, in which the micromanipulator is inert and the laser fires at a fixed frequency, and aided, in which the micromanipulator actively targets burn locations and the laser fires automatically upon target acquisition. Error is defined as the distance from the center of the observed burn mark to the preoperatively selected target location. RESULTS: An experienced retinal surgeon performed trials with and without robotic assistance, on both paper slides and porcine retina in vitro. In the paper slide experiments at an unaided laser repeat rate of 0.5 Hz, error was 125+/-62 microm with robotic assistance and 149+/-76 microm without (P < 0.005), and trial duration was 70+/-8 seconds with robotic assistance and 97+/-7 seconds without (P < 0.005). At a repeat rate of 1.0 Hz, error was 129+/-69 microm with robotic assistance and 166+/-91 microm without (P < 0.005), and trial duration was 26+/-4 seconds with robotic assistance and 47+/-1 seconds without (P < 0.005). At a repeat rate of 2.0 Hz on porcine retinal tissue, error was 123+/-69 microm with robotic assistance and 203+/-104 microm without (P < 0.005). CONCLUSION: Robotic assistance can increase the accuracy of laser photocoagulation while reducing the duration of the operation.