Deep Learning Techniques in Intelligent Fault Diagnosis and Prognosis for Industrial Systems: A Review.

Fault diagnosis and prognosis (FDP) tries to recognize and locate the faults from the captured sensory data, and also predict their failures in advance, which can greatly help to take appropriate actions for maintenance and avoid serious consequences in industrial systems. In recent years, deep learning methods are being widely introduced into FDP due to the powerful feature representation ability, and its rapid development is bringing new opportunities to the promotion of FDP. In order to facilitate the related research, we give a summary of recent advances in deep learning techniques for industrial FDP in this paper. Related concepts and formulations of FDP are firstly given. Seven commonly used deep learning architectures, especially the emerging generative adversarial network, transformer, and graph neural network, are reviewed. Finally, we give insights into the challenges in current applications of deep learning-based methods from four different aspects of imbalanced data, compound fault types, multimodal data fusion, and edge device implementation, and provide possible solutions, respectively. This paper tries to give a comprehensive guideline for further research into the problem of intelligent industrial FDP for the community.

A Novel Ultrasound Robot with Force/torque Measurement and Control for Safe and Efficient Scanning.

Medical ultrasound is of increasing importance in medical diagnosis and intraoperative assistance and possesses great potential advantages when integrated with robotics. However, some concerns, including the operation efficiency, operation safety, image quality, and comfort of patients, remain after introducing robotics into medical ultrasound. In this paper, an ultrasound robot integrating a force control mechanism, force/torque measurement mechanism, and online adjustment method, is proposed to overcome the current limitations. The ultrasound robot can measure operating forces and torques, provide adjustable constant operating forces, eliminate great operating forces introduced by accidental operations, and achieve various scanning depths based on clinical requirements. The proposed ultrasound robot would potentially facilitate sonographers to find the targets quickly, improve operation safety and efficiency, and decrease patients' discomfort. Simulations and experiments were carried out to evaluate the performance of the ultrasound robot. Experimental results show that the proposed ultrasound robot is able to detect operating force in the z-direction and torques around the x- and y- directions with errors of 3.53% F.S., 6.68% F.S., and 6.11% F.S., respectively, maintain the constant operating force with errors of less than 0.57N, and achieve various scanning depths for target searching and imaging. This proposed ultrasound robot has good performance and would potentially be used in medical ultrasound.

SAPM: Self-Adaptive Parallel Manipulator with Pose and Force Adjustment for Robotic Ultrasonography.

Robotic ultrasonography potentially acts as an essential aid to medical diagnosis. To overcome the limitations in robotic ultrasonography, in this paper, we proposed a novel self-adaptive parallel manipulator (SAPM) that can automatically adjust the ultrasound (US) probe pose to adapt to various contours of scanned areas, provide approximate constant operating forces/torques, achieve mechanical measurement, and cushion undesired produced forces. A novel parallel adjustment mechanism is proposed to attain automatic pose adjustment with 3 degrees of freedom (DOFs). This mechanism enables the US probe to adapt to different scanned areas and to perform the scanning with approximate constant forces and torques. Besides, we present a mechanical measurement and safety protection method that can be integrated into the SAPM and used as operation status monitoring and early warning during scanning procedures by capturing operating forces and torques. Experiments were carried out to calibrate the measurement and buffer units and evaluate the performance of the SAPM. Experimental results show the ability of the SAPM to provide 3-DoFs motion and operating force/torque measurement and automatically adjust the US probe pose to capture US images of equally good quality compared to a manual sonographer scan. Moreover, it has characteristics similar to soft robots that could significantly improve operation safety, and could be extended to some other engineering or medical applications.

Fast Fault Diagnosis in Industrial Embedded Systems Based on Compressed Sensing and Deep Kernel Extreme Learning Machines.

With the complexity and refinement of industrial systems, fast fault diagnosis is crucial to ensuring the stable operation of industrial equipment. The main limitation of the current fault diagnosis methods is the lack of real-time performance in resource-constrained industrial embedded systems. Rapid online detection can help deal with equipment failures in time to prevent equipment damage. Inspired by the ideas of compressed sensing (CS) and deep extreme learning machines (DELM), a data-driven general method is proposed for fast fault diagnosis. The method contains two modules: data sampling and fast fault diagnosis. The data sampling module non-linearly projects the intensive raw monitoring data into low-dimensional sampling space, which effectively reduces the pressure of transmission, storage and calculation. The fast fault diagnosis module introduces the kernel function into DELM to accommodate sparse signals and then digs into the inner connection between the compressed sampled signal and the fault types to achieve fast fault diagnosis. This work takes full advantage of the sparsity of the signal to enable fast fault diagnosis online. It is a general method in industrial embedded systems under data-driven conditions. The results on the CWRU dataset and real platforms show that our method not only has a significant speed advantage but also maintains a high accuracy, which verifies the practical application value in industrial embedded systems.

Offset-FA: A Uniform Method to Handle Both Unbounded and Bounded Repetitions in Regular Expression Matching.

With the exponential growth of cyber-physical systems (CPSs), security challenges have emerged; attacks on critical infrastructure could result in catastrophic consequences. Intrusion detection is the foundation for CPS security protection, and deep-packet inspection is the primary method for signature-matched mechanisms. This method usually employs regular expression matching (REM) to detect possible threats in the packet payload. State explosion is the critical challenge for REM applications, which originates primarily from features of large character sets with unbounded (closures) or bounded (counting) repetitions. In this work, we propose Offset-FA to handle these repetitions in a uniform mechanism. Offset-FA eliminates state explosion by extracting the repetitions from the nonexplosive string fragments. Then, these fragments are compiled into a fragment-DFA, while a fragment relation table and a reset table are constructed to preserve their connection and offset relationship. To our knowledge, Offset-FA is the first automaton to handle these two kinds of repetitions together with a uniform mechanism. Experiments demonstrate that Offset-FA outperforms state-of-the-art solutions in both space cost and matching speed on the premise of matching correctness, and achieves a comparable matching speed with that of DFA on practical rule sets.

Compensatory force measurement and multimodal force feedback for remote-controlled vascular interventional robot.

Minimally invasive vascular interventional surgery is widely used and remote-controlled vascular interventional surgery robots (RVIRs) are being developed to reduce the occupational risk of the intervening physician in minimally invasive vascular interventional surgeries. Skilled surgeon performs surgeries mainly depending on the detection of collisions. Inaccurate force feedback will be difficult for surgeons to perform surgeries or even results in medical accidents. In addition, the surgeon cannot quickly and easily distinguish whether the proximal force exceeds the safety threshold of blood vessels or not, and thus it results in damage to the blood vessels. In this paper, we present a novel method comprising compensatory force measurement and multimodal force feedback (MFF). Calibration experiments and performance evaluation experiments were carried out. Experimental results demonstrated that the proposed method can measure the proximal force of catheter/guidewire accurately and assist surgeons to distinguish the change of proximal force more easily. This novel method is suitable for use in actual surgical operations.

A cooperation of catheters and guidewires-based novel remote-controlled vascular interventional robot.

Remote-controlled vascular interventional robots (RVIRs) are being developed to increase the overall accuracy of surgical operations and reduce the occupational risks of intervening physicians, such as radiation exposure and chronic neck/back pain. Several RVIRs have been used to operate catheters or guidewires accurately. However, a lack of cooperation between the catheters and guidewires results in the surgeon being unable to complete complex surgery by propelling the catheter/guidewire to the target position. Furthermore, it is a significant challenge to operate the catheter/guidewire accurately and detect their proximal force without damaging their surfaces. In this study, we introduce a novel method that allows catheters and guidewires to be operated simultaneously in complex surgery. Our method accurately captures force measurements and enables precisely controlled catheter and guidewire operation. A prototype is validated through various experiments. The results demonstrate the feasibility of the proposed RVIR to operate a catheter and guidewire accurately, detect the resistance forces, and complete complex surgical operations in a cooperative manner.

Operation evaluation in-human of a novel remote-controlled vascular interventional robot.

Remote-controlled vascular interventional robots (RVIRs) are being developed to increase the accuracy of surgical operations and reduce the number of occupational risks sustained by intervening physicians, such as radiation exposure and chronic neck/back pain. However, complex control of the RVIRs improves the doctor's operation difficulty and reduces the operation efficiency. Furthermore, incomplete sterilization of the RVIRs will increase the risk of infection, or even cause medical accidents. In this study, we introduced a novel method that provides higher operation efficiency than a previous prototype and allows for complete robot sterilization. A prototype was fabricated and validated through laboratory setting experiments and an in-human experiment. The results illustrated that the proposed RVIR has better performance compared with the previous prototype, and preliminarily demonstrated that the proposed RVIR has good safety and reliability and can be used in clinical surgeries.

An Isomorphic Interactive Device for the Interventional Surgical Robot after In Vivo Study.

Interventional surgical robots are widely used in neurosurgery to improve surgeons' working environment and surgical safety. Based on the actual operational needs of surgeons' feedback during preliminary in vivo experiments, this paper proposed an isomorphic interactive master controller for the master-slave interventional surgical robot. The isomorphic design of the controller allows surgeons to utilize their surgical skills during remote interventional surgeries. The controller uses the catheter and guidewire as the operating handle, the same as during actual surgeries. The collaborative operational structure design and the working methods followed the clinical operational skills. The linear force feedback and torque feedback devices were designed to improve the safety of surgeries under remote operating conditions. An eccentric force compensation was conducted to achieve accurate force feedback. Several experiments were carried out, such as calibration experiments, master-slave control performance evaluation experiments, and operation comparison experiments on the novel and previously used controllers. The experimental results show that the proposed controller can perform complex operations in remote surgery applications and has the potential for further animal experiment evaluations.

Towards Standardized Acquisition with a Dual-probe Ultrasound Robot for Fetal Imaging.

Standardized acquisitions and diagnoses using robots and AI would potentially increase the general usability and reliability of medical ultrasound. Working towards this prospect, this paper presents the recent developments of a standardized acquisition workflow using a novel dual-probe ultrasound robot, for a project known as intelligent Fetal Imaging and Diagnosis (iFIND). The workflow includes an abdominal surface mapping step to obtain a non-parametric spline surface, a rule-based end-point calculation method to position each individual joint, and a motor synchronization method to achieve a smooth motion towards a target point. The design and implementation of the robot are first presented in this paper and the proposed workflow is then explained in detail with simulation and volunteer experiments performed and analyzed. The closed-form analytical solution to the specific motion planning problem has demonstrated a reliable performance controlling the robot to move towards the expected scanning areas and the calculated proximity of the robot to the surface shows that the robot maintains a safe distance while moving around the abdomen. The volunteer study has successfully demonstrated the reliable working and controllability of the robot in terms of acquiring desired ultrasound views. Our future work will focus on improving the motion planning, and on integrating the proposed standardized acquisition workflow with newly- developed ultrasound image processing methods to obtain diagnostic results in an accurate and consistent way.

A vascular interventional surgical robot based on surgeon's operating skills.

Interventional surgery is widely used in the treatment of cardiovascular and cerebrovascular diseases, and the development of surgical robots can greatly reduce the fatigue and radiation risks brought to surgeons during surgery. In this paper, we present a novel interventional surgical robot which allows surgeons to fully use their operating skills during remote control. Fuzzy control theory is used to guarantee control precision during the master-slave operation. The safety force feedback control is designed based on the catheter and guidewire spring model, and the force-position control is designed to decrease the potential damage due to the control delay. This study first evaluates the force-position control strategy using a vascular model experiment, and then an in vivo experiment is used to evaluate the precision of the surgical robot controlling the catheter and guidewire to the designated position. The in vivo experiment results and surgeon's feedback demonstrate that the proposed surgical robot is able to perform complex remote surgery in clinical application. Graphical abstract Surgeons perform remote interventional animal surgery using interventional surgical robots.