CLensRimVision: A Novel Computer Vision Algorithm for Detecting Rim Defects in Contact Lenses.

Automated optical inspection (AOI) plays a pivotal role in the quality control of contact lenses, safeguarding the safety and integrity of lenses intended for both medical and cosmetic applications. As the role of computer vision in defect detection expands, our study probes its effectiveness relative to traditional methods, particularly concerning subtle and irregular defects on the lens rim. In this research study, we propose a novel algorithm designed for the precise and automated detection of rim defects in contact lenses called "CLensRimVision". This algorithm integrates a series of procedures, including image preprocessing, circle detection for identifying lens rims, polar coordinate transformation, setting defect criteria and their subsequent detection, and, finally, visualization. The method based on these criteria can be adapted either to thickness-based or area-based approaches, suiting various characteristics of the contact lens. This approach achieves an exemplary performance with a 0.937 AP score. Our results offer a richer understanding of defect detection strategies, guiding manufacturers and researchers towards optimal techniques for ensuring quality in the contact lens domain.

Autonomous Trajectory Planning for Spray Painting on Complex Surfaces Based on a Point Cloud Model.

Using teach pendants or offline programming methods can generate tool paths for robot manipulators to carry out production activities, such as spray painting on objects of different geometries. This task, in which the complexity of painting the surface is one of the main challenges, requires highly skilled operators. In addition, the time spent setting up a robot task can be justified for the mass production of the same workpiece. However, it is inconvenient for low-production and high-variation production lines. In order to overcome these challenges, this study presents an algorithm to autonomously generate robot trajectories for a spray-painting process applied to objects with complex surfaces based on input 3D point cloud data. A predefined spherical mesh wraps the object, organizing the geometrical attributes into a structured data set. Subsequently, the region of interest is extracted and isolated from the model, which serves as the basis for the automatic path-planning operation. A user-friendly graphical user interface (GUI) is developed to define input parameters, visualize the point cloud model and the generated trajectory, simulate paint quality using a color map, and ultimately generate the robot's code. A 3D sensor is used to localize the pose of the workpiece ahead of the robot and adjust the robot's trajectory. The efficacy of the proposed approach is validated first by using various workpieces within a simulated environment and second by employing a real robot to execute the motion task.

Robot Pose Estimation and Normal Trajectory Generation on Curved Surface Using an Enhanced Non-Contact Approach.

The use of robots for machining operations has become very popular in the last few decades. However, the challenge of the robotic-based machining process, such as surface finishing on curved surfaces, still persists. Prior studies (non-contact- and contact-based) have their own limitations, such as fixture error and surface friction. To cope with these challenges, this study proposes an advanced technique for path correction and normal trajectory generation while tracking a curved workpiece's surface. Initially, a key-point selection approach is used to estimate a reference workpiece's coordinates using a depth measuring tool. This approach overcomes the fixture errors and enables the robot to track the desired path, i.e., where the surface normal trajectory is needed. Subsequently, this study employs an attached RGB-D camera on the end-effector of the robot for determining the depth and angle between the robot and the contact surface, which nullifies surface friction issues. The point cloud information of the contact surface is employed by the pose correction algorithm to guarantee the robot's perpendicularity and constant contact with the surface. The efficiency of the proposed technique is analyzed by carrying out several experimental trials using a 6 DOF robot manipulator. The results reveal a better normal trajectory generation than previous state-of-the-art research, with an average angle and depth error of 1.8 degrees and 0.4 mm.

Enhancing Robotic-Based Propeller Blade Sharpening Efficiency with a Laser-Vision Sensor and a Force Compliance Mechanism.

The edge sharpness of a propeller blade plays a vital role in improving energy transmission efficiency and reducing the power required to propel the vehicle. However, producing finely sharpened edges through casting is challenging due to the risk of breakage. Additionally, the blade profile of the wax model can deform during drying, making it difficult to achieve the required edge thickness. To automate the sharpening process, we propose an intelligent system consisting of a six-DoF industrial robot and a laser-vision sensor. The system improves machining accuracy through an iterative grinding compensation strategy that eliminates material residuals based on profile data from the vision sensor. An indigenously designed compliance mechanism is employed to enhance the performance of robotic grinding which is actively controlled by an electronic proportional pressure regulator to adjust the contact force and position between the workpiece and abrasive belt. The system's reliability and functionality are validated using three different workpiece models of four-blade propellers, achieving accurate and efficient machining within the required thickness tolerances. The proposed system provides a promising solution for finely sharpened propeller blade edges, addressing challenges associated with the earlier robotic-based grinding studies.

Visual Attention and Color Cues for 6D Pose Estimation on Occluded Scenarios Using RGB-D Data.

Recently, 6D pose estimation methods have shown robust performance on highly cluttered scenes and different illumination conditions. However, occlusions are still challenging, with recognition rates decreasing to less than 10% for half-visible objects in some datasets. In this paper, we propose to use top-down visual attention and color cues to boost performance of a state-of-the-art method on occluded scenarios. More specifically, color information is employed to detect potential points in the scene, improve feature-matching, and compute more precise fitting scores. The proposed method is evaluated on the Linemod occluded (LM-O), TUD light (TUD-L), Tejani (IC-MI) and Doumanoglou (IC-BIN) datasets, as part of the SiSo BOP benchmark, which includes challenging highly occluded cases, illumination changing scenarios, and multiple instances. The method is analyzed and discussed for different parameters, color spaces and metrics. The presented results show the validity of the proposed approach and their robustness against illumination changes and multiple instance scenarios, specially boosting the performance on relatively high occluded cases. The proposed solution provides an absolute improvement of up to 30% for levels of occlusion between 40% to 50%, outperforming other approaches with a best overall recall of 71% for the LM-O, 92% for TUD-L, 99.3% for IC-MI and 97.5% for IC-BIN.

A Comprehensive Design of Six-Axis Force/Moment Sensor.

Strain gage type six-axis force/moment (F/M) sensors have been largely studied and implemented in industrial applications by using an external data acquisition board (DAQ). The use of external DAQs will ill-affect accuracy and crosstalk due to the possibility of voltage drop through the wire length. The most recent research incorporated DAQ within a relatively small F/M sensor, but only for sensors of the capacitance and optical types. This research establishes the integration of a high-efficiency DAQ on six-axis F/M sensor with a revolutionary arrangement of 32 strain gages. The updated structural design was optimized using the sequential quadratic programming method and validated using Finite Element Analysis (FEA). A new, integrated DAQ system was designed, tested, and compared to commercial DAQ systems. The proposed six-axis F/M sensor was examined with the calibrated jig. The results show that the measurement error and crosstalk have been significantly reduced to 1.15% and 0.68%, respectively, the best published combination at this moment.

A Camera-Based Position Correction System for Autonomous Production Line Inspection.

Visual inspection is an important task in manufacturing industries in order to evaluate the completeness and quality of manufactured products. An autonomous robot-guided inspection system was recently developed based on an offline programming (OLP) and RGB-D model system. This system allows a non-expert automatic optical inspection (AOI) engineer to easily perform inspections using scanned data. However, if there is a positioning error due to displacement or rotation of the object, this system cannot be used on a production line. In this study, we developed an automated position correction module to locate an object's position and correct the robot's pose and position based on the detected error values in terms of displacement or rotation. The proposed module comprised an automatic hand-eye calibration and the PnP algorithm. The automatic hand-eye calibration was performed using a calibration board to reduce manual error. After calibration, the PnP algorithm calculates the object position error using artificial marker images and compensates for the error to a new object on the production line. The position correction module then automatically maps the defined AOI target positions onto a new object, unless the target position changes. We performed experiments that showed that the robot-guided inspection system with the position correction module effectively performed the desired task. This smart innovative system provides a novel advancement by automating the AOI process on a production line to increase productivity.

Switchable Glass Enabled Contextualization for a Cyber-Physical Safe and Interactive Spatial Augmented Reality PCBA Manufacturing Inspection System.

Augmented reality (AR) has been demonstrated to improve efficiency by up to thrice the level of traditional methods. Specifically, the adoption of visual AR is performed widely using handheld and head-mount technologies. Despite spatial augmented reality (SAR) addressing several shortcomings of wearable AR, its potential is yet to be fully explored. To date, it enhances the cooperation of users with its wide field of view and supports hands-free mobile operation, yet it has remained a challenge to provide references without relying on restrictive static empty surfaces of the same object or nearby objects for projection. Towards this end, we propose a novel approach that contextualizes projected references in real-time and on demand, onto and through the surface across a wireless network. To demonstrate the effectiveness of the approach, we apply the method to the safe inspection of printed circuit board assembly (PCBA) wirelessly networked to a remote automatic optical inspection (AOI) system. A defect detected and localized by the AOI system is wirelessly remitted to the proposed remote inspection system for prompt guidance to the inspector by augmenting a rectangular bracket and a reference image. The rectangular bracket transmitted through the switchable glass aids defect localization over the PCBA, whereas the image is projected over the opaque cells of the switchable glass to provide reference to a user. The developed system is evaluated in a user study for its robustness, precision and performance. Results indicate that the resulting contextualization from variability in occlusion levels not only positively affect inspection performance but also supersedes the state of the art in user preference. Furthermore, it supports a variety of complex visualization needs including varied sizes, contrast, online or offline tracking, with a simple robust integration requiring no additional calibration for registration.

Novel Mechanically Fully Decoupled Six-Axis Force-Moment Sensor.

In this study, a novel six-axis force/moment (F/M) sensor was developed. The sensor has a novel ring structure comprising a cross-beam elastic body with sliding and rotating mechanisms to achieve complete decoupling. The unique sliding and rotating mechanisms can reduce cross-talk effects caused by minimized structural interconnection. The forces Fx, Fy, and Fz and moments Mx, My, and Mz can be measured for the six-axis F/M sensors according to the elastic deformation of strain gauges attached to the cross beam. Herein, we provide detailed descriptions of the mathematical models, model idealizations, model creation, and the mechanical decoupling principle. The paper also presents a theoretical analysis of the strain based on Timoshenko beam theory and the subsequent validation of the analysis results through a comparison of the results with those obtained from a numerical analysis conducted using finite element analysis simulations. The sensor was subjected to experimental testing to obtain the maximum cross-talk errors along the following six axes under different loadings (the errors are presented in parentheses): Fx under SMy (2.12%), Fy under SMx (1.88%), Fz under SMz (2.02%), Mx under SFz (1.15%), My under SFx (1.80%), and Mz under SFx (2.63%). The proposed sensor demonstrated a considerably improved cross-talk error performance compared with existing force sensors.

Bin-Picking for Planar Objects Based on a Deep Learning Network: A Case Study of USB Packs.

Random bin-picking is a prominent, useful, and challenging industrial robotics application. However, many industrial and real-world objects are planar and have oriented surface points that are not sufficiently compact and discriminative for those methods using geometry information, especially depth discontinuities. This study solves the above-mentioned problems by proposing a novel and robust solution for random bin-picking for planar objects in a cluttered environment. Different from other research that has mainly focused on 3D information, this study first applies an instance segmentation-based deep learning approach using 2D image data for classifying and localizing the target object while generating a mask for each instance. The presented approach, moreover, serves as a pioneering method to extract 3D point cloud data based on 2D pixel values for building the appropriate coordinate system on the planar object plane. The experimental results showed that the proposed method reached an accuracy rate of 100% for classifying two-sided objects in the unseen dataset, and 3D appropriate pose prediction was highly effective, with average translation and rotation errors less than 0.23 cm and 2.26°, respectively. Finally, the system success rate for picking up objects was over 99% at an average processing time of 0.9 s per step, fast enough for continuous robotic operation without interruption. This showed a promising higher successful pickup rate compared to previous approaches to random bin-picking problems. Successful implementation of the proposed approach for USB packs provides a solid basis for other planar objects in a cluttered environment. With remarkable precision and efficiency, this study shows significant commercialization potential.

Development of a Virtual Force Sensor for a Low-Cost Collaborative Robot and Applications to Safety Control.

To protect operators and conform to safety standards for human-machine interactions, the design of collaborative robot arms often incorporates flexible mechanisms and force sensors to detect and absorb external impact forces. However, this approach increases production costs, making the introduction of such robot arms into low-cost service applications difficult. This study proposes a low-cost, sensorless rigid robot arm design that employs a virtual force sensor and stiffness control to enable the safety collision detection and low-precision force control of robot arms. In this design, when a robot arm is subjected to an external force while in motion, the contact force observer estimates the external torques on each joint according to the motor electric current and calculation errors of the system model, which are then used to estimate the external contact force exerted on the robot arm's end-effector. Additionally, a torque saturation limiter is added to the servo drive for each axis to enable the real-time adjustment of joint torque output according to the estimated external force, regulation of system stiffness, and achievement of impedance control that can be applied in safety measures and force control. The design this study developed is a departure from the conventional multisensor flexible mechanism approach. Moreover, it is a low-cost and sensorless design that relies on model-based control for stiffness regulation, thereby improving the safety and force control in robot arm applications.

Decoupled Six-Axis Force-Moment Sensor with a Novel Strain Gauge Arrangement and Error Reduction Techniques.

In this study, a novel strain gauge arrangement and error reduction techniques were proposed to minimize crosstalk reading and simultaneously increase sensitivity on a decoupled six-axis force-moment (F/M) sensor. The calibration process that comprises the least squares method and error reduction techniques was implemented to obtain a robust decoupling matrix. A decoupling matrix is very crucial for minimizing error and crosstalk. A novel strain gauge arrangement that comprised double parallel strain gauges in the decoupled six-axis force-moment sensor was implemented to obtain high sensitivity. The experimental results revealed that the maximum calibration error, F/M sensor measurement error, and crosstalk readings were reduced to 3.91%, 1.78%, and 4.78%, respectively.

A Sensorless and Low-Gain Brushless DC Motor Controller Using a Simplified Dynamic Force Compensator for Robot Arm Application.

Robot arms used for service applications require safe human-machine interactions; therefore, the control gain of such robot arms must be minimized to limit the force output during operation, which slows the response of the control system. To improve cost efficiency, low-resolution sensors can be used to reduce cost because the robot arms do not require high precision of position sensing. However, low-resolution sensors slow the response of closed-loop control systems, leading to low accuracy. Focusing on safety and cost reduction, this study proposed a low-gain, sensorless Brushless DC motor control architecture, which performed position and torque control using only Hall-effect sensors and a current sensor. Low-pass filters were added in servo controllers to solve the sensing problems of undersampling and noise. To improve the control system's excessively slow response, we added a dynamic force compensator in the current controllers, simplified the system model, and conducted tuning experiments to expedite the calculation of dynamic force. These approaches achieved real-time current compensation, and accelerated control response and accuracy. Finally, a seven-axis robot arm was used in our experiments and analyses to verify the effectiveness of the simplified dynamic force compensators. Specifically, these experiments examined whether the sensorless drivers and compensators could achieve the required response and accuracy while reducing the control system's cost.

A Method for 6D Pose Estimation of Free-Form Rigid Objects Using Point Pair Features on Range Data.

Pose estimation of free-form objects is a crucial task towards flexible and reliable highly complex autonomous systems. Recently, methods based on range and RGB-D data have shown promising results with relatively high recognition rates and fast running times. On this line, this paper presents a feature-based method for 6D pose estimation of rigid objects based on the Point Pair Features voting approach. The presented solution combines a novel preprocessing step, which takes into consideration the discriminative value of surface information, with an improved matching method for Point Pair Features. In addition, an improved clustering step and a novel view-dependent re-scoring process are proposed alongside two scene consistency verification steps. The proposed method performance is evaluated against 15 state-of-the-art solutions on a set of extensive and variate publicly available datasets with real-world scenarios under clutter and occlusion. The presented results show that the proposed method outperforms all tested state-of-the-art methods for all datasets with an overall 6.6% relative improvement compared to the second best method.

Autonomous Robot-Guided Inspection System Based on Offline Programming and RGB-D Model.

Automatic optical inspection (AOI) is a control process for precisely evaluating the completeness and quality of manufactured products with the help of visual information. Automatic optical inspection systems include cameras, light sources, and objects; AOI requires expert operators and time-consuming setup processes. In this study, a novel autonomous industrial robot-guided inspection system was hypothesized and developed to expedite and ease inspection process development. The developed platform is an intuitive and interactive system that does not require a physical object to test or an industrial robot; this allows nonexpert operators to perform object inspection planning by only using scanned data. The proposed system comprises an offline programming (OLP) platform and three-dimensional/two-dimensional (3D/2D) vision module. A robot program generated from the OLP platform is mapped to an industrial manipulator to scan a 3D point-cloud model of an object by using a laser triangulation sensor. After a reconstructed 3D model is aligned with a computer-aided design model on a common coordinate system, the OLP platform allows users to efficiently fine-tune the required inspection positions on the basis of the rendered images. The arranged inspection positions can be directed to an industrial manipulator on a production line to capture real images by using the corresponding 2D camera/lens setup for AOI tasks. This innovative system can be implemented in smart factories, which are easily manageable from multiple locations. Workers can save scanned data when new inspection positions are included based on cloud data. The present system provides a new direction to cloud-based manufacturing industries and maximizes the flexibility and efficiency of the AOI setup process to increase productivity.

Deep Learning Based Fast Screening Approach on Ultrasound Images for Thyroid Nodules Diagnosis.

Thyroid nodules are widespread in the United States and the rest of the world, with a prevalence ranging from 19 to 68%. The problem with nodules is whether they are malignant or benign. Ultrasonography is currently recommended as the initial modality for evaluating thyroid nodules. However, obtaining a good diagnosis from ultrasound imaging depends entirely on the radiologists levels of experience and other circumstances. There is a tremendous demand for automated and more reliable methods to screen ultrasound images more efficiently. This research proposes an efficient and quick detection deep learning approach for thyroid nodules. An open and publicly available dataset, Thyroid Digital Image Database (TDID), is used to determine the robustness of the suggested method. Each image is formatted into a pyramid tile-based data structure, which the proposed VGG-16 model evaluates to provide segmentation results for nodular detection. The proposed method adopts a top-down approach to hierarchically integrate high- and low-level features to distinguish nodules of varied sizes by employing fuse features effectively. The results demonstrated that the proposed method outperformed the U-Net model, achieving an accuracy of 99%, and was two times faster than the competitive model.