Robotic Assistance for Intraocular Microsurgery: Challenges and Perspectives.

Intraocular surgery, one of the most challenging discipline of microsurgery, requires sensory and motor skills at the limits of human physiological capabilities combined with tremendously difficult requirements for accuracy and steadiness. Nowadays, robotics combined with advanced imaging has opened conspicuous and significant directions in advancing the field of intraocular microsurgery. Having patient treatment with greater safety and efficiency as the final goal, similar to other medical applications, robotics has a real potential to fundamentally change microsurgery by combining human strengths with computer and sensor-based technology in an information-driven environment. Still in its early stages, robotic assistance for intraocular microsurgery has been accepted with precaution in the operating room and successfully tested in a limited number of clinical trials. However, owing to its demonstrated capabilities including hand tremor reduction, haptic feedback, steadiness, enhanced dexterity, micrometer-scale accuracy, and others, microsurgery robotics has evolved as a very promising trend in advancing retinal surgery. This paper will analyze the advances in retinal robotic microsurgery, its current drawbacks and limitations, as well as the possible new directions to expand retinal microsurgery to techniques currently beyond human boundaries or infeasible without robotics.

Miniaturized Robotic End-Effector with Piezoelectric Actuation and Fiber Optic Sensing for Minimally Invasive Cardiac Procedures.

Each year 35,000 cardiac ablation procedures are performed to treat atrial fibrillation through the use of catheter systems. The success rate of this treatment is highly dependent on the force which the catheter applies on the heart wall. If the magnitude of the applied force is much higher than a certain threshold the tissue perforates, whereas if the force is lower than this threshold the lesion size may be too large and is inconsistent. Furthermore, studies have shown large variability in the applied force from trained physicians during treatment, suggesting that physicians are unable to manually regulate the levels of the force at the site of treatment. Current catheter systems do not provide the physicians with active means for contact force control and are only at most aided by visual feedback of the forces measured in situ. This paper discusses a novel design of a robotic end-effector that integrates mechanisms of sensing and actively controlling of the applied forces into a miniaturized compact form. The required specifications for design and integration were derived from the current application under investigation. An off-the-shelf miniature piezoelectric motor was chosen for actuation, and a force sensing solution was developed to meet the specifications. Experimental characterization of the actuator and the force sensor within the integrated setup show compliance with the specifications and pave the way for future experimentation where closed-loop control of the system can be implemented according to the contact force control strategies for the application.

Techniques for robot-aided intraocular surgery using monocular vision.

This paper presents techniques for robot-aided intraocular surgery using monocular vision in order to overcome erroneous stereo reconstruction in an intact eye. We propose a new retinal surface estimation method based on a structured-light approach. A handheld robot known as the Micron enables automatic scanning of a laser probe, creating projected beam patterns on the retinal surface. Geometric analysis of the patterns then allows planar reconstruction of the surface. To realize automated surgery in an intact eye, monocular hybrid visual servoing is accomplished through a scheme that incorporates surface reconstruction and partitioned visual servoing. We investigate the sensitivity of the estimation method according to relevant parameters and also evaluate its performance in both dry and wet conditions. The approach is validated through experiments for automated laser photocoagulation in a realistic eye phantom in vitro. Finally, we present the first demonstration of automated intraocular laser surgery in porcine eyes ex vivo.

Comparative Evaluation of Handheld Robot-Aided Intraocular Laser Surgery.

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Handheld-automated microsurgical instrumentation for intraocular laser surgery.

BACKGROUND AND OBJECTIVE: Laser photocoagulation is a mainstay or adjuvant treatment for a variety of common retinal diseases. Automated laser photocoagulation during intraocular surgery has not yet been established. The authors introduce an automated laser photocoagulation system for intraocular surgery, based on a novel handheld instrument. The goals of the system are to enhance accuracy and efficiency and improve safety. MATERIALS AND METHODS: Triple-ring patterns are introduced as a typical arrangement for the treatment of proliferative retinopathy and registered to a preoperative fundus image. In total, 32 target locations are specified along the circumferences of three rings having diameters of 1, 2, and 3 mm, with a burn spacing of 600 µm. Given the initial system calibration, the retinal surface is reconstructed using stereo vision, and the targets specified on the preoperative image are registered with the control system. During automated operation, the laser probe attached to the manipulator of the active handheld instrument is deflected as needed via visual servoing in order to correct the error between the aiming beam and a specified target, regardless of any erroneous handle motion by the surgeon. A constant distance of the laser probe from the retinal surface is maintained in order to yield consistent size of burns and ensure safety during operation. Real-time tracking of anatomical features enables compensation for any movement of the eye. A graphical overlay system within operating microscope provides the surgeon with guidance cues for automated operation. Two retinal surgeons performed automated and manual trials in an artificial model of the eye, with each trial repeated three times. For the automated trials, various targeting thresholds (50-200 µm) were used to automatically trigger laser firing. In manual operation, fixed repetition rates were used, with frequencies of 1.0-2.5 Hz. The power of the 532 nm laser was set at 3.0 W with a duration of 20 ms. After completion of each trial, the speed of operation and placement error of burns were measured. The performance of the automated laser photocoagulation was compared with manual operation, using interpolated data for equivalent firing rates from 1.0 to 1.75 Hz. RESULTS: In automated trials, average error increased from 45 ± 27 to 60 ± 37 µm as the targeting threshold varied from 50 to 200 µm, while average firing rate significantly increased from 0.69 to 1.71 Hz. The average error in the manual trials increased from 102 ± 67 to 174 ± 98 µm as firing rate increased from 1.0 to 2.5 Hz. Compared to the manual trials, the average error in the automated trials was reduced by 53.0-56.4%, resulting in statistically significant differences (P ≤ 10(-20) ) for all equivalent frequencies (1.0-1.75 Hz). The depth of the laser tip in the automated trials was consistently maintained within 18 ± 2 µm root-mean-square (RMS) of its initial position, whereas it significantly varied in the manual trials, yielding an error of 296 ± 30 µm RMS. At high firing rates in manual trials, such as at 2.5 Hz, laser photocoagulation is marginally attained, yielding failed burns of 30% over the entire pattern, whereas no failed burns are found in automated trials. Relatively regular burn sizes are attained in the automated trials by the depth servoing of the laser tip, while burn sizes in the manual trials vary considerably. Automated avoidance of blood vessels was also successfully demonstrated, utilizing the retina-tracking feature to identify avoidance zones. CONCLUSION: Automated intraocular laser surgery can improve the accuracy of photocoagulation while ensuring safety during operation. This paper provides an initial demonstration of the technique under reasonably realistic laboratory conditions; development of a clinically applicable system requires further work.

Manipulator Design and Operation for a Six-Degree-of-Freedom Handheld Tremor-Canceling Microsurgical Instrument.

This paper presents the design and actuation of a six-degree-of-freedom (6-DOF) manipulator for a handheld instrument, known as "Micron," which performs active tremor compensation during microsurgery. The design incorporates a Gough-Stewart platform based on piezoelectric linear motor, with a specified minimum workspace of a cylinder 4 mm long and 4 mm in diameter at the end-effector. Given the stall force of the motors and the loading typically encountered in vitreoretinal microsurgery, the dimensions of the manipulator are optimized to tolerate a transverse load of 0.2 N on a remote center of motion near the midpoint of the tool shaft. The optimization yields a base diameter of 23 mm and a height of 37 mm. The fully handheld instrument includes a custom-built optical tracking system for control feedback, and an ergonomic housing to serve as a handle. The manipulation performance was investigated in both clamped and handheld conditions. In positioning experiments with varying side loads, the manipulator tolerates side load up to 0.25 N while tracking a sinusoidal target trajectory with less than 20 µm error. Physiological hand tremor is reduced by about 90% in a pointing task, and error less than 25 µm is achieved in handheld circle-tracing.

Improving Target Acquisition for Computer Users With Athetosis.

Prior work has highlighted the challenges faced by people with athetosis when trying to acquire on-screen targets using a mouse or trackball. The difficulty of positioning the mouse cursor within a confined area has been identified as a challenging task. We have developed a target acquisition assistance algorithm that features transition assistance via directional gain variation based on target prediction, settling assistance via gain reduction in the vicinity of a predicted target, and expansion of the predicted target as the cursor approaches it. We evaluated the algorithm on improving target acquisition efficiency among seven participants with athetoid cerebral palsy. Our results showed that the algorithm significantly reduced the overall movement time by about 20%. Considering the target acquisition occurs countless times in the course of regular computer use, the accumulative effect of such improvements can be significant for improving the efficiency of computer interaction among people with athetosis.

Vision-Based Control of a Handheld Surgical Micromanipulator with Virtual Fixtures.

Performing micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real-time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p < 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p < 0.05).

Cable Tension Optimization for an Epicardial Parallel Wire Robot.

HeartPrinter is a novel under-constrained 3-cable parallel wire robot designed for minimally invasive epicardial interventions. The robot adheres to the beating heart using vacuum suction at its anchor points, with a central injector head that operates within the triangular workspace formed by the anchors, and is actuated by cables for multipoint direct gene therapy injections. Minimizing cable tensions can reduce forces on the heart at the anchor points while supporting rapid delivery of accurate injections and minimizing procedure time, risk of damage to the robot, and strain to the heart. However, cable tensions must be sufficient to hold the injector head's position as the heart moves and to prevent excessive cable slack. We pose a linear optimization problem to minimize the sum of cable tension magnitudes for HeartPrinter while ensuring the injector head is held in static equilibrium and the tensions are constrained within a feasible range. We use Karush-Kuhn-Tucker optimality conditions to derive conditional algebraic expressions for optimal cable tensions as a function of injector head position and workspace geometry, and we identify regions of injector head positions where particular combinations of cable tensions are optimally at minimum allowable tensions. The approach can rapidly solve for the minimum set of cable tensions for any robot workspace geometry and injector head position and determine whether an injection site is attainable.

A Micro Motion Sensing System for Micromanipulation Tasks.

An optical-based motion sensing system has been developed for real-time sensing of instrument motion in micromanipulation. The main components of the system consist of a pair of position sensitive detectors (PSD), lenses, an infrared (IR) diode that illuminates the workspace of the system, a non-reflective intraocular shaft, and a white reflective ball attached at the end of the shaft. The system calculates 3D displacement of the ball inside the workspace using the centroid position of the IR rays that are reflected from the ball and strike the PSDs. In order to eliminate inherent nonlinearity of the system, calibration using a feedforward neural network is proposed and presented. Handling of different ambient light and environment light conditions not to affect the system accuracy is described. Analyses of the whole optical system and effect of instrument orientation on the system accuracy are presented. Sensing resolution, dynamic accuracies at a few different frequencies, and static accuracies at a few different orientations of the instrument are reported. The system and the analyses are useful in assessing performance of hand-held microsurgical instruments and operator performance in micromanipulation tasks.

Micron: an Actively Stabilized Handheld Tool for Microsurgery.

We describe the design and performance of a hand-held actively stabilized tool to increase accuracy in micro-surgery or other precision manipulation. It removes involuntary motion such as tremor by actuating the tip to counteract the effect of the undesired handle motion. The key components are a three-degree-of-freedom piezoelectric manipulator that has 400 µm range of motion, 1 N force capability, and bandwidth over 100 Hz, and an optical position measurement subsystem that acquires the tool pose with 4 µm resolution at 2000 samples/s. A control system using these components attenuates hand motion by at least 15 dB (a fivefold reduction). By considering the effect of the frequency response of Micron on the human visual feedback loop, we have developed a filter that reduces unintentional motion, yet preserves intuitive eye-hand coordination. We evaluated the effectiveness of Micron by measuring the accuracy of the human/machine system in three simple manipulation tasks. Handheld testing by three eye surgeons and three non-surgeons showed a reduction in position error of between 32% and 52%, depending on the error metric.

Soft Miniaturized Actuation and Sensing Units for Dynamic Force Control of Cardiac Ablation Catheters.

Recently, there has been active research in finding robotized solutions for the treatment of atrial fibrillation (AF) by augmenting catheter systems through the integration of force sensors at the tip. However, limited research has been aimed at providing automatic force control by also integrating actuation of the catheter tip, which can significantly enhance safety in such procedures. This article solves the demanding challenge of miniaturizing both actuation and sensing for integration into flexible catheters. Fabrication strategies are presented for a series of novel soft thick-walled cylindrical actuators, with embedded sensing using eutectic gallium-indium. The functional catheter tips have a diameter in the range of 2.6-3.6 mm and can both generate and detect forces in the range of < 0.4 N, with a bandwidth of 1-2 Hz. The deformation modeling of thick-walled cylinders with fiber reinforcement is presented in the article. An experimental setup developed for static and dynamic characterization of these units is presented. The prototyped units were validated with respect to the design specifications. The preliminary force control results indicate that these units can be used in tracking and control of contact force, which has the potential to make AF procedures much safer and more accurate.

Introducer Design Concepts for an Epicardial Parallel Wire Robot.

BACKGROUND: Cardiac gene therapies lack effective delivery methods to the myocardium. While direct injection has demonstrated success over a small region, homogenous gene expression requires many injections over a large area. To address this need, we developed a minimally invasive flexible parallel wire robot for epicardial interventions. To accurately deploy it onto the beating heart, an introducer mechanism is required. METHODS: Two mechanisms are presented. Assessment of the robot's positioning, procedure time, and pericardium insertion forces are performed on an artificial beating heart. RESULTS: Successful positioning was demonstrated. The mean procedure time was 230 ± 7 seconds for mechanism I and 259 ± 4 seconds for mechanism II. The mean pericardium insertion force was 2.2 ± 0.4 N anteriorly and 3.1 ± 0.4 N posteriorly. CONCLUSION: Introducer mechanisms demonstrate feasibility in facilitating the robot's deployment on the epicardium. Pericardium insertion forces and procedure times are consistent and reasonable.

Coronary vessel detection methods for organ-mounted robots.

BACKGROUND: HeartLander is a tethered robot walker that utilizes suction to adhere to the beating heart. HeartLander can be used for minimally invasive administration of cardiac medications or ablation of tissue. In order to administer injections safely, HeartLander must avoid coronary vasculature. METHODS: Doppler ultrasound signals were recorded using a custom-made cardiac phantom and used to classify different coronary vessel properties. The classification was performed by two machine learning algorithms, the support vector machines and a deep convolutional neural network. These algorithms were then validated in animal trials. RESULTS: Accuracy of identifying vessels above turbulent flow reached greater than 92% in phantom trials and greater than 98% in animal trials. CONCLUSIONS: Through the use of two machine learning algorithms, HeartLander has shown the ability to identify different sized vasculature proximally above turbulent flow. These results indicate that it is feasible to use Doppler ultrasound to identify and avoid coronary vasculature during cardiac interventions using HeartLander.

Monocular Vision-Based Retinal Membrane Peeling With a Handheld Robot.

Retinal membrane peeling requires delicate manipulation. The presence of the surgeon's physiological tremor, the high variability and often low quality of the ophthalmic image, and excessive forces make the tasks more challenging. Preventing unintended movement caused by tremor and unintentional forces can reduce membrane injury. With the use of an actively stabilized handheld robot, we employ a monocular camera-based surface reconstruction method to estimate the retinal plane and we propose the use of a virtual fixture with the application of a hard stop and motion scaling to improve control of the tool tip during delaminating in a laboratory simulation of retinal membrane peeling. A hard stop helps to limit downward force exerted on the surface. Motion scaling also improves the user's control of contact force when delaminating. We demonstrate a reduction of maximum force and maximum surface-penetration distance from the estimated retinal plane using the proposed technique.

A Deep Learning Model for Automated Classification of Intraoperative Continuous EMG.

OBJECTIVE: Intraoperative neurophysiological monitoring (IONM) is the use of electrophysiological methods during certain high-risk surgeries to assess the functional integrity of nerves in real time and alert the surgeon to prevent damage. However, the efficiency of IONM in current practice is limited by latency of verbal communications, inter-rater variability, and the subjective manner in which electrophysiological signals are described. METHODS: In an attempt to address these shortcomings, we investigate automated classification of free-running electromyogram (EMG) waveforms during IONM. We propose a hybrid model with a convolutional neural network (CNN) component and a long short-term memory (LSTM) component to better capture complicated EMG patterns under conditions of both electrical noise and movement artifacts. Moreover, a preprocessing pipeline based on data normalization is used to handle classification of data from multiple subjects. To investigate model robustness, we also analyze models under different methods for processing of artifacts. RESULTS: Compared with several benchmark modeling methods, CNN-LSTM performs best in classification, achieving accuracy of 89.54% and sensitivity of 94.23% in cross-patient evaluation. CONCLUSION: The CNN-LSTM model shows promise for automated classification of continuous EMG in IONM. SIGNIFICANCE: This technique has potential to improve surgical safety by reducing cognitive load and inter-rater variability.

Semiautomated intraocular laser surgery using handheld instruments.

BACKGROUND AND OBJECTIVE: In laser retinal photocoagulation, hundreds of dot-like burns are applied. We introduce a robot-assisted technique to enhance the accuracy and reduce the tedium of the procedure. MATERIALS AND METHODS: Laser burn locations are overlaid on preoperative retinal images using common patterns such as grids. A stereo camera/monitor setup registers and displays the planned burn locations overlaid on real-time video. Using an active handheld micromanipulator, a 7 x 7 grid of burns spaced 650 microm apart is applied to both paper slides and porcine retina in vitro using 30 milliseconds laser pulses at 532 nm. Two scenarios were tested: unaided, in which the micromanipulator is inert and the laser fires at a fixed frequency, and aided, in which the micromanipulator actively targets burn locations and the laser fires automatically upon target acquisition. Error is defined as the distance from the center of the observed burn mark to the preoperatively selected target location. RESULTS: An experienced retinal surgeon performed trials with and without robotic assistance, on both paper slides and porcine retina in vitro. In the paper slide experiments at an unaided laser repeat rate of 0.5 Hz, error was 125+/-62 microm with robotic assistance and 149+/-76 microm without (P < 0.005), and trial duration was 70+/-8 seconds with robotic assistance and 97+/-7 seconds without (P < 0.005). At a repeat rate of 1.0 Hz, error was 129+/-69 microm with robotic assistance and 166+/-91 microm without (P < 0.005), and trial duration was 26+/-4 seconds with robotic assistance and 47+/-1 seconds without (P < 0.005). At a repeat rate of 2.0 Hz on porcine retinal tissue, error was 123+/-69 microm with robotic assistance and 203+/-104 microm without (P < 0.005). CONCLUSION: Robotic assistance can increase the accuracy of laser photocoagulation while reducing the duration of the operation.

Minimally invasive epicardial injections using a novel semiautonomous robotic device.

BACKGROUND: We have developed a novel miniature robotic device (HeartLander) that can navigate on the surface of the beating heart through a subxiphoid approach. This study investigates the ability of HeartLander to perform in vivo semiautonomous epicardial injections on the beating heart. METHODS AND RESULTS: The inchworm-like locomotion of HeartLander is generated using vacuum pressure for prehension of the epicardium and drive wires for actuation. The control system enables semiautonomous target acquisition by combining the joystick input with real-time 3-dimensional localization of the robot provided by an electromagnetic tracking system. In 12 porcine preparations, the device was inserted into the intrapericardial space through a subxiphoid approach. Ventricular epicardial injections of dye were performed with a custom injection system through HeartLander's working channel. HeartLander successfully navigated to designated targets located around the circumference of the ventricles (mean path length=51+/-25 mm; mean speed=38+/-26 mm/min). Injections were successfully accomplished following the precise acquisition of target patterns on the left ventricle (mean injection depth=3.0+/-0.5 mm). Semiautonomous target acquisition was achieved within 1.0+/-0.9 mm relative to the reference frame of the tracking system. No fatal arrhythmia or bleeding was noted. There were no histological injuries to the heart due to the robot prehension, locomotion, or injection. CONCLUSIONS: In this proof-of-concept study, HeartLander demonstrated semiautonomous, precise, and safe target acquisition and epicardial injection on a beating porcine heart through a subxiphoid approach. This technique may facilitate minimally invasive cardiac cell transplantation or polymer therapy in patients with heart failure.

Tracking Control of Hysteretic Piezoelectric Actuator using Adaptive Rate-Dependent Controller.

With the increasing popularity of actuators involving smart materials like piezoelectric, control of such materials becomes important. The existence of the inherent hysteretic behavior hinders the tracking accuracy of the actuators. To make matters worse, the hysteretic behavior changes with rate. One of the suggested ways is to have a feedforward controller to linearize the relationship between the input and output. Thus, the hysteretic behavior of the actuator must first be modeled by sensing the relationship between the input voltage and output displacement. Unfortunately, the hysteretic behavior is dependent on individual actuator and also environmental conditions like temperature. It is troublesome and costly to model the hysteresis regularly. In addition, the hysteretic behavior of the actuators also changes with age. Most literature model the actuator using a cascade of rate-independent hysteresis operators and a dynamical system. However, the inertial dynamics of the structure is not the only contributing factor. A complete model will be complex. Thus, based on the studies done on the phenomenological hysteretic behavior with rate, this paper proposes an adaptive rate-dependent feedforward controller with Prandtl-Ishlinskii (PI) hysteresis operators for piezoelectric actuators. This adaptive controller is achieved by adapting the coefficients to manipulate the weights of the play operators. Actual experiments are conducted to demonstrate the effectiveness of the adaptive controller. The main contribution of this paper is its ability to perform tracking control of non-periodic motion and is illustrated with the tracking control ability of a couple of different non-periodic waveforms which were created by passing random numbers through a low pass filter with a cutoff frequency of 20Hz.

Compact Sensing Design of a Handheld Active Tremor Compensation Instrument.

Active physiological tremor compensation instruments have been under research and development recently. The sensing unit of the instruments provides information on three degrees-of-freedom (DOF) motion of the instrument tip using accelerations provided by accelerometers placed inside the instruments. A complete vector of angular acceleration of the instrument needs to be known to obtain information on three DOF motions of the tip. Sensing resolution of angular acceleration about the instrument axis is directly proportional to the width of the proximal-end sensing unit. To keep the sensing resolution high enough, the width of the unit has to be made large. As a result, the proximal-end sensing unit of the instruments is bulky. In this paper, placement of accelerometers is proposed such that the angular acceleration about the instrument axis need not be known to obtain information on the three DOF motions of the tip. With the proposed placement, the instrument is no longer bulky and fewer number of accelerometers is required, thereby making the instrument compact and better in terms of ergonomics and reliability. Experiments were conducted to show that the proposed design of placement works properly.