Correction of Rotational Eccentricity Based on Model and Microvision in the Wire-Traction Micromanipulation System.

In the realm of automatic wire-traction micromanipulation systems, the alignment of the central axis of the coil with the rotation axis of the rotary stage can be a challenge, which leads to the occurrence of eccentricity during rotation. The wire-traction is conducted at a micron-level of manipulation precision on micron electrode wires; eccentricity has a significant impact on the control accuracy of the system. To resolve the problem, a method for measuring and correcting the coil eccentricity is proposed in this paper. First, models of radial and tilt eccentricity are established respectively based on the eccentricity sources. Then, measuring eccentricity is proposed by an eccentricity model and microscopic vision; the model is used to predict eccentricity, and visual image processing algorithms are used to calibrate model parameters. In addition, a correction based on the compensation model and hardware is designed to compensate for the eccentricity. The experimental results demonstrate the accuracy of the models in predicting eccentricity and the effectiveness of correction. The results show that the models have an accurate prediction for eccentricity that relies on the evaluation of the root mean square error (RMSE); the maximal residual error after correction was within 6 µm, and the compensation was approximately 99.6%. The proposed method, which combines the eccentricity model and microvision for measuring and correcting eccentricity, offers improved wire-traction micromanipulation accuracy, enhanced efficiency, and an integrated system. It has more suitable and wider applications in the field of micromanipulation and microassembly.

Design, Analysis and Experimental Investigations of a Double-Arm Based Micro-Gripper for Thin and Flexible Metal Wires Manipulation.

A robotic system for the automatic wire pulling of coreless motor winding is designed, including the design of an opening-closing control system and a micro-gripper's tip structure with a double-armed elastic-beam structure for the support part and an enveloping clamping structure for the tip part. The micro-gripper captures the electrode wire from the root, encircles the wire after the envelope region is closed, and the thin and flexible electrode wire is pulled to the top of the electrode pad by the movement of the micro-gripper and released. The mechanical index of the micro-gripper is simulated to obtain the optimal structural parameters. The experimental results show that the electrode wire's maximum bearing force is about 0.3 N. Under this reaction force, the deformation of the tip-envelope region of the micro-gripper is about 27.5 µm, which is sufficient for electrode wire pulling micro-manipulation. By comparison with the steel micro-gripper, the silicon micro-gripper has more advantages in shape integrity, machinability and mechanical properties.

Design of thin-structured-light projection system for small-height measurement.

A thin-structured light projection system with simple setup is proposed and applied to small-height measurement. In this system, the design of structured light pattern is carried out on the computer. The structure, color, geometric dimension, motion mode, projection brightness, contrast, and other attributes of the pattern can be edited easily. The optical system with zoom function is designed, which can output thin-structured-light fringe with a minimum width of 10 µm. The camera and microscope are used to construct a vision system, which is used to capture the pattern of structured light. The three-dimensional (3D) shape reconstruction is realized by analyzing the pattern of structured light. The results show that this system can project a small width of thin-structured light pattern, which is very suitable for 3D shape reconstruction of microscopic objects with the small-height of tens of microns to hundreds of microns.

Surface Reconstruction of Microscale Objects Based on Grid-Patterned Structured-Light Measurements.

A structured-light projection system was designed for microscale objects with surface heights that ranged from tens to hundreds of microns. The system was composed of a universal projector and microscope system that supported editing the attributes of structured-light patterns in real-time and was capable of projecting microscale patterns. On this basis, reconstructing the metal surfaces of microscale objects based on grid patterns of structured light was investigated, the internal and external parameters of microscope vision and projection systems were calibrated, and an image algorithm for grid-node detection was designed. The results indicated that the proposed method successfully reconstructed the three-dimensional (3D) surface of microscale objects, and the reconstruction results were consistent with the original surfaces. With 95% confidence, the reconstruction precision in the X- and Y-directions was approximately ±4.0 µm and in the Z-direction was approximately ±7.5 µm. The designed system and the proposed method were suitable for 3D-shape measurement of microstructures in microscopic fields and can be adapted to meet a broader range of applications, as compared to current methods.

An Automatic Detection Method for Cutting Path of Chips in Wafer.

Microscopic imaging is easily affected by the strength of illumination, and the chip surface qualities of different wafers are different. Therefore, wafer images have defects such as uneven brightness distribution, obvious differences in chip region characteristics, etc., which affect the positioning accuracy of the wafer cutting path. For this reason, this thesis proposes an automatic chip-cutting path-planning method in the wafer image of the Glass Passivation Parts (GPPs) process without a mark. First, the wafer image is calibrated for brightness. Then, the template matching algorithm is used to determine the chip region and the center of gravity position of the chip region. We find the position of the geometric feature (interlayer) in the chip region, and the interlayer is used as an auxiliary location to determine the final cutting path. The experiment shows that the image quality can be improved, and chip region features can be highlighted when preprocessing the image with brightness calibration. The results show that the average deviation of the gravity coordinates of the chip region in the x direction is 2.82 pixels. We proceeded by finding the interlayer in the chip region, marking it with discrete points, and using the improved Random Sample Consensus (RANSAC) algorithm to remove the abnormal discrete points and fit the remaining discrete points. The average fitting error is 0.8 pixels, which is better than the least squares method (LSM). The cutting path location algorithm proposed in this paper can adapt to environmental brightness changes and different qualities of chips, accurately and quickly determine the cutting path, and improve the chip cutting yield.

Design and Experimental Investigations of Shape and Attitude Carding System for the Wires of Micro Coreless Motor Winding.

The shape and attitude (S&A) of the electrode wire are important characteristics of micro coreless motor winding. The purpose of this paper is to present the design of a robotic micro-manipulation system for micro wire carding with arbitrary S&A, which can be used as the pretreatment system for wire micro-gripper systems. The system is based on the principle of flexible carding, and uses nylon, bristle, nanometer-silk and wool as materials for the brushing micro-manipulator. The trajectory of the brushing micro-manipulator is designed, and the S&A of the electrode wires are straightened through the combined motion mode of horizontal and vertical brushing micro-manipulators. The experimental results show that the material of the brushing micro-manipulator has a great impact on the carding quality. Nanometer-silk material is more suitable for horizontal brushing micro-manipulators, and wool material is more suitable for vertical brushing micro-manipulators. The geometric dimension of the brushing micro-manipulator also affects the carding quality. When the diameter is in the range of 1 mm, the carding effect of the horizontal brushing micro-manipulator with a length of 4.9-8 mm is better. The system can realize the automatic carding of flexible electrode wires with arbitrary S&A, and it will not damage the structure of wires in the process.

Orthogonality Measurement of Three-Axis Motion Trajectories for Micromanipulation Robot Systems.

In robotic micromanipulation systems, the orthogonality of the three-axis motion trajectories of the motion control systems influences the accuracy of micromanipulation. A method of measuring and evaluating the orthogonality of three-axis motion trajectories is proposed in this paper. Firstly, a system for three-axis motion trajectory measurement is developed and an orthogonal reference coordinate system is designed. The influence of the assembly error of laser displacement sensors on the reference coordinate system is analyzed using simulation. An approach to estimating the orthogonality of three-axis motion trajectories and to compensating for its error is presented using spatial line fitting and vector operation. The simulation results show that when the assembly angle of the laser displacement sensors is limited within a range of 10°, the relative angle deviation of the coordinate axes of the reference coordinate frame is approximately 0.09%. The experiment results show that precision of spatial line fitting is approximately 0.02 mm and relative error of the orthogonality measurement is approximately 0.3%.

3D surface reconstruction of small height object based on thin structured light scanning.

Aiming at the problem of 3D surface reconstruction of small height objects, a method based on the scanning principle with thin structured light is proposed. The laser fringe is produced by a laser light source. It scans the surface of small height object under the control of a precise motion control system, and is integrated with a stereo light microscope and two cameras to form a complete structured light profilometry system. This method is very suitable for 3D surface reconstruction of small height objects. In order to deal with the phenomenon of "cracking" and local uneven brightness in microscope fringe image, the fringe image sequence is captured under different camera exposure parameters, and the quality of fringe image is improved by image fusion. The location of pixels on the center line of laser stripes is detected by the Loess local second-order fitting method, and the positions of these pixels are smoothed and predicted by using the Lowess local linear fitting method. The deformation of the laser stripe center curve is calculated by constructing a baseline, and the 3D surface reconstruction of the object with small height is realized in the image space. Taking the small characters on the surface of Chinese coins and human hair as test samples, their 3D surfaces are constructed by using the method proposed in this paper. The reconstructed 3D surface shapes are highly consistent with the real 3D surface shapes of objects.

Contour extraction of a laser stripe located on a microscope image from a stereo light microscope.

A stereo light microscope has a special optical structure that consists of two optical paths. With two cameras fitted on its imaging planes, a microscopic binocular vision system can be constructed, and this system can be applied in three-dimensional (3D) shape measurements of a microscopic object. A novel-shape reconstruction system is developed in this article composed of laser fringe scanning and a stereo light microscope. A laser projector emits a thin light sheet and projects it onto the microscopic object's surface. The microscopic object is placed on a micro-displacement mechanism and moved in a given direction. The entire surface of the object is scanned by the light sheet. This system captures a series of microscopic images of the laser stripe, which are used to restore the 3D shape of the object. In this article, we mainly focus on the study of the laser stripe detection method, which is derived based on the Canny rule. First, the Canny rule outputs pixels at the left and right edges of the laser stripe. Then, a subpixel edge extraction method is proposed based on polynomial fitting that outputs the center curve of the laser stripe. Finally, an edge filter is used to smooth the edge burrs, and the Hermite interpolation method is used to link the broken edges and construct continuous, smoothing edge contours. The results show that this method can effectively find the subpixel position of the laser stripe and output high-quality edge contours. The method is suitable for extracting the contours of a laser stripe located in a microscope image.

Space quantization between the object and image spaces of a microscopic stereovision system with a stereo light microscope.

Microscopic stereovision systems with a stereo light microscope have been used for quantized observations and measurements in microscopy fields. These applications depend on a quantization mathematical relationship between the object space and the image space. In this article, a novel, practical space quantization method was proposed. It was derived from two defined base cell matrices. First, it captured a series of stereo images of markers moving in various directions through the object space. The world and image coordinates of markers were acquired, stored and used to create the two base cell matrices. Next, a space quantization relationship between the object space and the image space was derived from analysis of the two base cell matrices. Finally, if two matching points located in the image space were given, the world coordinates of their object point could be calculated by the above space quantization relationship. The results show that this method of quantizing the two spaces could achieve a positioning precision of approximately ±5 µm in the horizontal direction and approximately ±10 µm in the vertical direction. It has a very simple expression, but it is more practical and efficient than other traditional projection methods. In addition, it has a better performance and can be widely used in several microscopy fields, such as microscopic shape reconstruction, micromanipulation, microassembly, and microinjection.

Disparity Surface Reconstruction Based on a Stereo Light Microscope and Laser Fringes.

Microscopic vision systems based on a stereo light microscope (SLM) are used in microscopic measuring fields. Conventional measuring methods output the disparity surface based on stereo matching methods; however, these methods require that stereo images contain sufficient distinguishing features. Moreover, matching results typically contain many mismatched points. This paper presents a novel method for disparity surface reconstruction by combining an SLM and laser measuring techniques. The surfaces of small objects are scanned by a laser fringe, and a stereo image sequence containing laser stripes is obtained. The central contours of the laser stripes are extracted, and central contours are derived for alignment. A disparity coordinate system is then defined and used to analyze the relationship between the motion direction and reference plane. Next, the method of aligning disparity contours is proposed. The results show that our method can achieve a precision of ±0.5 pixels and that the real and measured shapes described by the disparity surface are consistent based on our method. Our method is confirmed to perform much better than the conventional block-matching method. The disparity surface output obtained by our method can be used to measure the surface profiles of microscopic objects accurately.

A stereovision model applied in bio-micromanipulation system based on stereo light microscope.

A bio-micromanipulation system is designed for manipulating micro-objects with a length scale of tens or hundreds of microns based on stereo light microscope. The world coordinate reconstruction of points on the surface of micro-objects is an important goal for the micromanipulation. Traditional pinhole camera model is applied widely in macrocomputer vision. However, this model will output bad data with remarkable error if it is directly used to reconstruct three-dimensional world coordinates for stereo light microscope. Therefore, a novel and improved pinhole camera model applied in bio-micromanipulation system is proposed in this article. The new model is composed of binocular-pinhole model and error-correction model. The binocular-pinhole model is used to output the basic world coordinates. The error-correction model is used to correct the errors from the basic world coordinates and outputs the final high-precision world coordinates. The results show that the new model achieves a precision of 0.01 mm in the X direction, 0.01 mm in the Y direction, and 0.015 mm in the Z direction within a maximum reconstruction distance of 4.1 mm in the X direction, 2.9 mm in the Y direction, and 2.25 mm in the Z direction, and that traditional pinhole camera model achieves a lower and unsatisfactory precision of about 0.1 mm.

Microscopic vision modeling method by direct mapping analysis for micro-gripping system with stereo light microscope.

We present a novel and high-precision microscopic vision modeling method, which can be used for 3D data reconstruction in micro-gripping system with stereo light microscope. This method consists of four parts: image distortion correction, disparity distortion correction, initial vision model and residual compensation model. First, the method of image distortion correction is proposed. Image data required by image distortion correction comes from stereo images of calibration sample. The geometric features of image distortions can be predicted though the shape deformation of lines constructed by grid points in stereo images. Linear and polynomial fitting methods are applied to correct image distortions. Second, shape deformation features of disparity distribution are discussed. The method of disparity distortion correction is proposed. Polynomial fitting method is applied to correct disparity distortion. Third, a microscopic vision model is derived, which consists of two models, i.e., initial vision model and residual compensation model. We derive initial vision model by the analysis of direct mapping relationship between object and image points. Residual compensation model is derived based on the residual analysis of initial vision model. The results show that with maximum reconstruction distance of 4.1mm in X direction, 2.9mm in Y direction and 2.25mm in Z direction, our model achieves a precision of 0.01mm in X and Y directions and 0.015mm in Z direction. Comparison of our model with traditional pinhole camera model shows that two kinds of models have a similar reconstruction precision of X coordinates. However, traditional pinhole camera model has a lower precision of Y and Z coordinates than our model. The method proposed in this paper is very helpful for the micro-gripping system based on SLM microscopic vision.

Study on clear stereo image pair acquisition method for small objects with big vertical size in SLM vision system.

Microscopic vision system with stereo light microscope (SLM) has been applied to surface profile measurement. If the vertical size of a small object exceeds the range of depth, its images will contain clear and fuzzy image regions. Hence, in order to obtain clear stereo images, we propose a microscopic sequence image fusion method which is suitable for SLM vision system. First, a solution to capture and align image sequence is designed, which outputs an aligning stereo images. Second, we decompose stereo image sequence by wavelet analysis theory, and obtain a series of high and low frequency coefficients with different resolutions. Then fused stereo images are output based on the high and low frequency coefficient fusion rules proposed in this article. The results show that Δw1 (Δw2 ) and ΔZ of stereo images in a sequence have linear relationship. Hence, a procedure for image alignment is necessary before image fusion. In contrast with other image fusion methods, our method can output clear fused stereo images with better performance, which is suitable for SLM vision system, and very helpful for avoiding image fuzzy caused by big vertical size of small objects.

Image distortion correction for micromanipulation system based on SLM microscopic vision.

Stereo light microscope (SLM) simulates stereo imaging principle of human eyes. Microscopic vision system based on SLM has become an important visual tool for micro measurement, micromanipulation, and microinjection. We develop a micromanipulation system based on SLM and present an image distortion correction method. We mainly correct two kinds of image distortions: lateral and vertical distortion. Distortion correction consists of two steps. First, a linear fitting algorithm for each row or column of target points is developed, and the fitting errors are calculated. If the fitting errors are smaller than a given threshold, the linear fitting results are kept and used. Otherwise polynomial fitting procedure will be used. Second, the parallelism of straight lines is corrected. The results show that a line in world coordinate frame (WCF) is not necessarily a straight line in image coordinate frame (ICF), or two parallel lines in WCF may be not parallel in ICF. Distortion correction can restore the parallel and linear relationship. For distorted left and right images, the magnitude of distortion exceeds 6 pixels and 4 pixels in the horizontal direction, and 1.2 pixels and 1.7 pixels in the vertical direction, respectively. After corrected, for left and right image, distortion can be reduced to 0.8 pixels and 0.7 pixels in the horizontal direction, and 0.96 pixels and 1.3 pixels in the vertical direction, respectively. The results show that distortion parameters obtained from the proposed method can effectively correct distorted images.