A Micro Motion Sensing System for Micromanipulation Tasks.

An optical-based motion sensing system has been developed for real-time sensing of instrument motion in micromanipulation. The main components of the system consist of a pair of position sensitive detectors (PSD), lenses, an infrared (IR) diode that illuminates the workspace of the system, a non-reflective intraocular shaft, and a white reflective ball attached at the end of the shaft. The system calculates 3D displacement of the ball inside the workspace using the centroid position of the IR rays that are reflected from the ball and strike the PSDs. In order to eliminate inherent nonlinearity of the system, calibration using a feedforward neural network is proposed and presented. Handling of different ambient light and environment light conditions not to affect the system accuracy is described. Analyses of the whole optical system and effect of instrument orientation on the system accuracy are presented. Sensing resolution, dynamic accuracies at a few different frequencies, and static accuracies at a few different orientations of the instrument are reported. The system and the analyses are useful in assessing performance of hand-held microsurgical instruments and operator performance in micromanipulation tasks.

Drift-free position estimation of periodic or quasi-periodic motion using inertial sensors.

Position sensing with inertial sensors such as accelerometers and gyroscopes usually requires other aided sensors or prior knowledge of motion characteristics to remove position drift resulting from integration of acceleration or velocity so as to obtain accurate position estimation. A method based on analytical integration has previously been developed to obtain accurate position estimate of periodic or quasi-periodic motion from inertial sensors using prior knowledge of the motion but without using aided sensors. In this paper, a new method is proposed which employs linear filtering stage coupled with adaptive filtering stage to remove drift and attenuation. The prior knowledge of the motion the proposed method requires is only approximate band of frequencies of the motion. Existing adaptive filtering methods based on Fourier series such as weighted-frequency Fourier linear combiner (WFLC), and band-limited multiple Fourier linear combiner (BMFLC) are modified to combine with the proposed method. To validate and compare the performance of the proposed method with the method based on analytical integration, simulation study is performed using periodic signals as well as real physiological tremor data, and real-time experiments are conducted using an ADXL-203 accelerometer. Results demonstrate that the performance of the proposed method outperforms the existing analytical integration method.

Visual Servoed Robotic Mouse Oocyte Rotation.

Both injection and biopsy of a mammalian cell require positioning and orientation of a biological cell in a three-dimensional space under a microscope. Manual cell manipulation and orientation is the most commonly used method that is based on a trial-and-error and direct cell poking approach. OBJECTIVE: Solve inherent problems of existing approaches, including low efficiency, poor success rate and inconsistent output. METHODS: We present a system that is able to automatically rotate a mouse oocyte to a desired orientation based on computer vision. Experimental results demonstrate that the system's capability for intracellular structure recognition and fast oocyte orientation (11.2 s/cell). The system demonstrated overall out-of-plane and in-plane success rates of 94% and 95% respectively. CONCLUSION: Our system performs the oocyte rotation by using standard equipment yet significantly improves the efficiency and success rate. SIGNIFICANCE: Our methods improve existing techniques and provide a starting point for fast autofocusing and oocyte orientation prior to automatic ICSI or cell biopsy.

Tracking Control of Hysteretic Piezoelectric Actuator using Adaptive Rate-Dependent Controller.

With the increasing popularity of actuators involving smart materials like piezoelectric, control of such materials becomes important. The existence of the inherent hysteretic behavior hinders the tracking accuracy of the actuators. To make matters worse, the hysteretic behavior changes with rate. One of the suggested ways is to have a feedforward controller to linearize the relationship between the input and output. Thus, the hysteretic behavior of the actuator must first be modeled by sensing the relationship between the input voltage and output displacement. Unfortunately, the hysteretic behavior is dependent on individual actuator and also environmental conditions like temperature. It is troublesome and costly to model the hysteresis regularly. In addition, the hysteretic behavior of the actuators also changes with age. Most literature model the actuator using a cascade of rate-independent hysteresis operators and a dynamical system. However, the inertial dynamics of the structure is not the only contributing factor. A complete model will be complex. Thus, based on the studies done on the phenomenological hysteretic behavior with rate, this paper proposes an adaptive rate-dependent feedforward controller with Prandtl-Ishlinskii (PI) hysteresis operators for piezoelectric actuators. This adaptive controller is achieved by adapting the coefficients to manipulate the weights of the play operators. Actual experiments are conducted to demonstrate the effectiveness of the adaptive controller. The main contribution of this paper is its ability to perform tracking control of non-periodic motion and is illustrated with the tracking control ability of a couple of different non-periodic waveforms which were created by passing random numbers through a low pass filter with a cutoff frequency of 20Hz.

Compact Sensing Design of a Handheld Active Tremor Compensation Instrument.

Active physiological tremor compensation instruments have been under research and development recently. The sensing unit of the instruments provides information on three degrees-of-freedom (DOF) motion of the instrument tip using accelerations provided by accelerometers placed inside the instruments. A complete vector of angular acceleration of the instrument needs to be known to obtain information on three DOF motions of the tip. Sensing resolution of angular acceleration about the instrument axis is directly proportional to the width of the proximal-end sensing unit. To keep the sensing resolution high enough, the width of the unit has to be made large. As a result, the proximal-end sensing unit of the instruments is bulky. In this paper, placement of accelerometers is proposed such that the angular acceleration about the instrument axis need not be known to obtain information on the three DOF motions of the tip. With the proposed placement, the instrument is no longer bulky and fewer number of accelerometers is required, thereby making the instrument compact and better in terms of ergonomics and reliability. Experiments were conducted to show that the proposed design of placement works properly.

Estimating Displacement of Periodic Motion With Inertial Sensors.

Inertial sensors, like accelerometers and gyroscopes, are rarely used by themselves to measure displacement. Accuracy of inertial sensors is greatly handicapped by the notorious integration drift, which arises due to numerical integration of the sensors zero bias error. A solution is proposed in this paper to provide drift free estimation of displacement from inertial sensors.

Feedforward Controller of Ill-Conditioned Hysteresis Using Singularity-Free Prandtl-Ishlinskii Model.

Piezoelectric, magnetostrictive, and shape memory alloy actuators are gaining importance in high-frequency precision applications constrained by space. Their intrinsic hysteretic behavior makes control difficult. The Prandtl-Ishlinskii (PI) operator can model hysteresis well, albeit a major inadequacy: the inverse operator does not exist when the hysteretic curve gradient is not positive definite, i.e., ill condition occurs when slope is negative. An inevitable tradeoff between modeling accuracy and inversion stability exists. The hysteretic modeling improves with increasing number of play operators. But as the piecewise continuous interval of each operator reduces, the model tends to be ill-conditioned, especially at the turning points. Similar ill-conditioned situation arises when these actuators move heavy loads or operate at high frequency. This paper proposes an extended PI operator to map hysteresis to a domain where inversion is well behaved. The inverse weights are then evaluated to determine the inverse hysteresis model for the feedforward controller. For illustration purpose, a piezoelectric actuator is used.

Autofocusing and Polar Body Detection in Automated Cell Manipulation.

Autofocusing and feature detection are two essential processes for performing automated biological cell manipulation tasks. In this paper, we have introduced a technique capable of focusing on a holding pipette and a mammalian cell under a bright-field microscope automatically, and a technique that can detect and track the presence and orientation of the polar body of an oocyte that is rotated at the tip of a micropipette. Both algorithms were evaluated by using mouse oocytes. Experimental results show that both algorithms achieve very high success rates: 100% and 96%. As robust and accurate image processing methods, they can be widely applied to perform various automated biological cell manipulations.

Automatic segmentation of zona pellucida and its application in cleavage-stage embryo biopsy position selection.

A very important step of Pre-implantation genetic diagnosis (PGD) is embryo biopsy, in which process the zona pellucida (ZP) is cut open partially and a part of cellular material is extracted from the embryo. Recognition of the ZP is necessary not only for embryo biopsy, but also for other applications such as zona pellucida thickness variation (ZPTV), embryo dissection, etc. The ZP opening position is closely related to the cell survival rate after the biopsy. Selection of an unsuitable position may cause blastomere lysis after the ZP opening. Normal procedures of ZP recognition and biopsy position selection involve a skilled human embryologist. In order to make the process automatic, we introduce an automatic segmentation method for ZP recognition by using edge detection and ellipse fitting with a value adjustment algorithm in this paper. An application of ZP recognition in embryo biopsy position selection is also introduced. Our ZP recognition algorithm was able to correctly segment 43 out of 45 sample embryo images, achieving a success rate of 96%. Its application in embryo biopsy position selection achieved a success rate of 93%.

Visual Servoed Three-Dimensional Cell Rotation System.

Three-dimensional (3-D) positioning and orientation of embryos/oocytes is necessary to facilitate micromanipulation tasks such as cell injection and cellular structural biopsy commonly performed under a microscope. Conventional cell orientation is performed manually by using a vacuum equipped micropipette to aspirate and release the cell, which is a trial-and-error approach. The conventional method relies heavily on the skill of the operator; it also suffers from low precision, low success rate and low controllability. These drawbacks illustrate the need for a systematic 3-D cell rotational system to automate the cell orientation process. In this paper, we present a noninvasive single cell rotation system that can automatically orientate a zebrafish embryo to a desired position when both the cytoplasm and the yolk are in the focal plane. A three-point-contact model for cell rotation that involves a custom-designed rotational stage is introduced to provide precise rotational position control. A vision recognition algorithm is also proposed to enable the visual servoing function of the system. Experimental results show that the proposed system can achieve high success rates of 92.5% (x-axis rotation with 40 trails) and 97.5% (about the z-axis with 80 trails). The system can also successfully complete 3-D cell orientation at an average speed of 31 s/cell with a high in-plane rotation accuracy of 0.3 (°) . As a high precise, high controllable and deterministic cell manipulating system, it provides a starting point for automated cell manipulation for intracytoplasmic sperm injection and embryo biopsy for preimplantation genetic diagnosis.

A Hand-held Instrument for in vivo Probe-based Confocal Laser Endomicroscopy during Minimally Invasive Surgery.

Probe-based confocal laser endomicroscopy (pCLE) provides high resolution imaging of tissue in vivo. Maintaining a steady contact between target tissue and pCLE probe tip is important for image consistency. In this paper, a new prototype hand-held instrument for in vivo pCLE during Minimally Invasive Surgery (MIS) is presented. The proposed instrument incorporates adaptive force sensing and actuation, allowing improved image consistency and force control, thus minimizing tissue deformation and induced micro-structural variations. The performance and accuracy of the contact force control are evaluated in detailed laboratory settings and in vivo validation of the device during transanal microsurgery in a live porcine model further demonstrates the potential clinical value of the device.

A hand-held instrument to maintain steady tissue contact during probe-based confocal laser endomicroscopy.

Probe-based confocal laser endomicroscopy (pCLE) provides high-resolution in vivo imaging for intraoperative tissue characterization. Maintaining a desired contact force between target tissue and the pCLE probe is important for image consistency, allowing large area surveillance to be performed. A hand-held instrument that can provide a predetermined contact force to obtain consistent images has been developed. The main components of the instrument include a linear voice coil actuator, a donut load-cell, and a pCLE probe. In this paper, detailed mechanical design of the instrument is presented and system level modeling of closed-loop force control of the actuator is provided. The performance of the instrument has been evaluated in bench tests as well as in hand-held experiments. Results demonstrate that the instrument ensures a consistent predetermined contact force between pCLE probe tip and tissue. Furthermore, it compensates for both simulated physiological movement of the tissue and involuntary movements of the operator's hand. Using pCLE video feature tracking of large colonic crypts within the mucosal surface, the steadiness of the tissue images obtained using the instrument force control is demonstrated by confirming minimal crypt translation.