Integrated Sensors for Soft Medical Robotics.

Minimally invasive procedures assisted by soft robots for surgery, diagnostics, and drug delivery have unprecedented benefits over traditional solutions from both patient and surgeon perspectives. However, the translation of such technology into commercialization remains challenging. The lack of perception abilities is one of the obstructive factors paramount for a safe, accurate and efficient robot-assisted intervention. Integrating different types of miniature sensors onto robotic end-effectors is a promising trend to compensate for the perceptual deficiencies in soft robots. For example, haptic feedback with force sensors helps surgeons to control the interaction force at the tool-tissue interface, impedance sensing of tissue electrical properties can be used for tumor detection. The last decade has witnessed significant progress in the development of multimodal sensors built on the advancement in engineering, material science and scalable micromachining technologies. This review article provides a snapshot on common types of integrated sensors for soft medical robots. It covers various sensing mechanisms, examples for practical and clinical applications, standard manufacturing processes, as well as insights on emerging engineering routes for the fabrication of novel and high-performing sensing devices.

Adversarial Robustness Enhancement for Deep Learning-Based Soft Sensors: An Adversarial Training Strategy Using Historical Gradients and Domain Adaptation.

Despite their high prediction accuracy, deep learning-based soft sensor (DLSS) models face challenges related to adversarial robustness against malicious adversarial attacks, which hinder their widespread deployment and safe application. Although adversarial training is the primary method for enhancing adversarial robustness, existing adversarial-training-based defense methods often struggle with accurately estimating transfer gradients and avoiding adversarial robust overfitting. To address these issues, we propose a novel adversarial training approach, namely domain-adaptive adversarial training (DAAT). DAAT comprises two stages: historical gradient-based adversarial attack (HGAA) and domain-adaptive training. In the first stage, HGAA incorporates historical gradient information into the iterative process of generating adversarial samples. It considers gradient similarity between iterative steps to stabilize the updating direction, resulting in improved transfer gradient estimation and stronger adversarial samples. In the second stage, a soft sensor domain-adaptive training model is developed to learn common features from adversarial and original samples through domain-adaptive training, thereby avoiding excessive leaning toward either side and enhancing the adversarial robustness of DLSS without robust overfitting. To demonstrate the effectiveness of DAAT, a DLSS model for crystal quality variables in silicon single-crystal growth manufacturing processes is used as a case study. Through DAAT, the DLSS achieves a balance between defense against adversarial samples and prediction accuracy on normal samples to some extent, offering an effective approach for enhancing the adversarial robustness of DLSS.

Soft Sensory-Motor System Based on Ionic Solution for Robotic Applications.

Soft robots claim the architecture of actuators, sensors, and computation demands with their soft bodies by obtaining fast responses and adapting to the environment. Sensory-motor coordination is one of the main design principles utilized for soft robots because it allows the capability to sense and actuate mutually in the environment, thereby achieving rapid response performance. This work intends to study the response for a system that presents coupled actuation and sensing functions simultaneously and is integrated in an arbitrary elastic structure with ionic conduction elements, called as soft sensory-motor system based on ionic solution (SSMS-IS). This study provides a comparative analysis of the performance of SSMS-IS prototypes with three diverse designs: toroidal, semi-toroidal, and rectangular geometries, based on a series of performance experiments, such as sensitivity, drift, and durability. The design with the best performance was the rectangular SSMS-IS using silicon rubber RPRO20 for both internal and external pressures applied in the system. Moreover, this work explores the performance of a bioinspired soft robot using rectangular SSMS-IS elements integrated in its body. Further, it investigated the feasibility of the robot to adapt its morphology online for environment variability, responding to external stimuli from the environment with different levels of stiffness and damping.

Probing with Each Step: How a Walking Crab-like Robot Classifies Buried Cylinders in Sand with Hall-Effect Sensors.

Shallow underwater environments around the world are contaminated with unexploded ordnances (UXOs). Current state-of-the-art methods for UXO detection and localization use remote sensing systems. Furthermore, human divers are often tasked with confirming UXO existence and retrieval which poses health and safety hazards. In this paper, we describe the application of a crab robot with leg-embedded Hall effect-based sensors to detect and distinguish between UXOs and non-magnetic objects partially buried in sand. The sensors consist of Hall-effect magnetometers and permanent magnets embedded in load bearing compliant segments. The magnetometers are sensitive to magnetic objects in close proximity to the legs and their movement relative to embedded magnets, allowing for both proximity and force-related feedback in dynamically obtained measurements. A dataset of three-axis measurements is collected as the robot steps near and over different UXOs and UXO-like objects, and a convolutional neural network is trained on time domain inputs and evaluated by 5-fold cross validation. Additionally, we propose a novel method for interpreting the importance of measurements in the time domain for the trained classifier. The results demonstrate the potential for accurate and efficient UXO and non-UXO discrimination in the field.

Several Small or Single Large? Quantifying the Catchment-Wide Performance of On-Site Wastewater Treatment Plants with Inaccurate Sensors.

On-site wastewater treatment plants (OSTs) often lack monitoring, resulting in unreliable treatment performance. They thus appear to be a stopgap solution despite their potential contribution to circular water management. Low-maintenance but inaccurate soft sensors are emerging that address this concern. However, how their inaccuracy impacts the catchment-wide treatment performance of a system of many OSTs has not been quantified. We develop a stochastic model to estimate catchment-wide OST performances with a Monte Carlo simulation. In our study, soft sensors with a 70% accuracy improved the treatment performance from 66% of the time functional to 98%. Soft sensors optimized for specificity, indicating the true negative rate, improve the system performance, while sensors optimized for sensitivity, indicating the true positive rate, quantify the treatment performance more accurately. This new insight leads us to suggest programming two soft sensors in practical settings with the same hardware sensor data as input: one soft sensor geared to high specificity for maintenance scheduling and one geared to high sensitivity for performance quantification. Our findings suggest that a maintenance strategy combining inaccurate sensors with appropriate alarm management can vastly improve the mean catchment-wide treatment performance of a system of OSTs.

An Interdisciplinary Tutorial: A Self-Healing Soft Finger with Embedded Sensor.

In the field of soft robotics, knowledge of material science is becoming more and more important. However, many researchers have a background in only one of both domains. To aid the understanding of the other domain, this tutorial describes the complete process from polymer synthesis over fabrication to testing of a soft finger. Enough background is provided during the tutorial such that researchers from both fields can understand and sharpen their knowledge. Self-healing polymers are used in this tutorial, showing that these polymers that were once a specialty, have become accessible for broader use. The use of self-healing polymers allows soft robots to recover from fatal damage, as shown in this tutorial, which increases their lifespan significantly.

Soft Sensing of LPG Processes Using Deep Learning.

This study investigates the integration of soft sensors and deep learning in the oil-refinery industry to improve monitoring efficiency and predictive accuracy in complex industrial processes, particularly de-ethanization and debutanization. Soft sensor models were developed to estimate critical variables such as the C2 and C5 contents in liquefied petroleum gas (LPG) after distillation and the energy consumption of distillation columns. The refinery's LPG purification process relies on periodic sampling and laboratory analysis to maintain product specifications. The models were tested using data from actual refinery operations, addressing challenges such as scalability and handling dirty data. Two deep learning models, an artificial neural network (ANN) soft sensor model and an ensemble random forest regressor (RFR) model, were developed. This study emphasizes model interpretability and the potential for real-time updating or online learning. The study also proposes a comprehensive, iterative solution for predicting and optimizing component concentrations within a dual-column distillation system, highlighting its high applicability and potential for replication in similar industrial scenarios.

An End-to-End Dynamic Posture Perception Method for Soft Actuators Based on Distributed Thin Flexible Porous Piezoresistive Sensors.

This paper proposes a method for accurate 3D posture sensing of the soft actuators, which could be applied to the closed-loop control of soft robots. To achieve this, the method employs an array of miniaturized sponge resistive materials along the soft actuator, which uses long short-term memory (LSTM) neural networks to solve the end-to-end 3D posture for the soft actuators. The method takes into account the hysteresis of the soft robot and non-linear sensing signals from the flexible bending sensors. The proposed approach uses a flexible bending sensor made from a thin layer of conductive sponge material designed for posture sensing. The LSTM network is used to model the posture of the soft actuator. The effectiveness of the method has been demonstrated on a finger-size 3 degree of freedom (DOF) pneumatic bellow-shaped actuator, with nine flexible sponge resistive sensors placed on the soft actuator's outer surface. The sensor-characterizing results show that the maximum bending torque of the sensor installed on the actuator is 4.7 Nm, which has an insignificant impact on the actuator motion based on the working space test of the actuator. Moreover, the sensors exhibit a relatively low error rate in predicting the actuator tip position, with error percentages of 0.37%, 2.38%, and 1.58% along the x-, y-, and z-axes, respectively. This work is expected to contribute to the advancement of soft robot dynamic posture perception by using thin sponge sensors and LSTM or other machine learning methods for control.

Design of a Soft Sensor Based on Long Short-Term Memory Artificial Neural Network (LSTM) for Wastewater Treatment Plants.

Assessment of wastewater effluent quality in terms of physicochemical and microbial parameters is a difficult task; therefore, an online method which combines the variables and represents a final value as the quality index could be used as a useful management tool for decision makers. However, conventional measurement methods often have limitations, such as time-consuming processes and high associated costs, which hinder efficient and practical monitoring. Therefore, this study presents an approach that underscores the importance of using both short- and long-term memory networks (LSTM) to enhance monitoring capabilities within wastewater treatment plants (WWTPs). The use of LSTM networks for soft sensor design is presented as a promising solution for accurate variable estimation to quantify effluent quality using the total chemical oxygen demand (TCOD) quality index. For the realization of this work, we first generated a dataset that describes the behavior of the activated sludge system in discrete time. Then, we developed a deep LSTM network structure as a basis for formulating the LSTM-based soft sensor model. The results demonstrate that this structure produces high-precision predictions for the concentrations of soluble X1 and solid X2 substrates in the wastewater treatment system. After hyperparameter optimization, the predictive capacity of the proposed model is optimized, with average values of performance metrics, mean square error (MSE), coefficient of determination (R2), and mean absolute percentage error (MAPE), of 23.38, 0.97, and 1.31 for X1, and 9.74, 0.93, and 1.89 for X2, respectively. According to the results, the proposed LSTM-based soft sensor can be a valuable tool for determining effluent quality index in wastewater treatment systems.

Linear and Non-Linear Soft Sensors for Predicting the Research Octane Number (RON) through Integrated Synchronization, Resolution Selection and Modelling.

The Research Octane Number (RON) is a key quality parameter for gasoline, obtained offline through complex, time-consuming, and expensive standard methods. Measurements are usually only available a few times per week and after long delays, making process control very challenging. Therefore, alternative methods have been proposed to predict RON from readily available data. In this work, we report the development of inferential models for predicting RON from process data collected in a real catalytic reforming process. Data resolution and synchronization were explicitly considered during the modelling stage, where 20 predictive linear and non-linear machine learning models were assessed and compared using a robust Monte Carlo double cross-validation approach. The workflow also handles outliers, missing data, multirate and multiresolution observations, and processes dynamics, among other features. Low RMSE were obtained under testing conditions (close to 0.5), with the best methods belonging to the class of penalized regression methods and partial least squares. The developed models allow for improved management of the operational conditions necessary to achieve the target RON, including a more effective use of the heating utilities, which improves process efficiency while reducing costs and emissions.

Test Strategy Optimization Based on Soft Sensing and Ensemble Belief Measurement.

Resulting from the short production cycle and rapid design technology development, traditional prognostic and health management (PHM) approaches become impractical and fail to match the requirement of systems with structural and functional complexity. Among all PHM designs, testability design and maintainability design face critical difficulties. First, testability design requires much labor and knowledge preparation, and wastes the sensor recording information. Second, maintainability design suffers bad influences by improper testability design. We proposed a test strategy optimization based on soft-sensing and ensemble belief measurements to overcome these problems. Instead of serial PHM design, the proposed method constructs a closed loop between testability and maintenance to generate an adaptive fault diagnostic tree with soft-sensor nodes. The diagnostic tree generated ensures high efficiency and flexibility, taking advantage of extreme learning machine (ELM) and affinity propagation (AP). The experiment results show that our method receives the highest performance with state-of-art methods. Additionally, the proposed method enlarges the diagnostic flexibility and saves much human labor on testability design.

Industrial Soft Sensor Optimized by Improved PSO: A Deep Representation-Learning Approach.

Soft sensors based on deep learning approaches are growing in popularity due to their ability to extract high-level features from training, improving soft sensors' performance. In the training process of such a deep model, the set of hyperparameters is critical to archive generalization and reliability. However, choosing the training hyperparameters is a complex task. Usually, a random approach defines the set of hyperparameters, which may not be adequate regarding the high number of sets and the soft sensing purposes. This work proposes the RB-PSOSAE, a Representation-Based Particle Swarm Optimization with a modified evaluation function to optimize the hyperparameter set of a Stacked AutoEncoder-based soft sensor. The evaluation function considers the mean square error (MSE) of validation and the representation of the features extracted through mutual information (MI) analysis in the pre-training step. By doing this, the RB-PSOSAE computes hyperparameters capable of supporting the training process to generate models with improved generalization and relevant hidden features. As a result, the proposed method can generate more than 16.4% improvement in RMSE compared to another standard PSO-based method and, in some cases, more than 50% improvement compared to traditional methods applied to the same real-world nonlinear industrial process. Thus, the results demonstrate better prediction performance than traditional and state-of-the-art methods.

An Extended Review Concerning the Relevance of Deep Learning and Privacy Techniques for Data-Driven Soft Sensors.

The continuously increasing number of mobile devices actively being used in the world amounted to approximately 6.8 billion by 2022. Consequently, this implies a substantial increase in the amount of personal data collected, transported, processed, and stored. The authors of this paper designed and implemented an integrated personal health data management system, which considers data-driven software and hardware sensors, comprehensive data privacy techniques, and machine-learning-based algorithmic models. It was determined that there are very few relevant and complete surveys concerning this specific problem. Therefore, the current scientific research was considered, and this paper comprehensively analyzes the importance of deep learning techniques that are applied to the overall management of data collected by data-driven soft sensors. This survey considers aspects that are related to demographics, health and body parameters, and human activity and behaviour pattern detection. Additionally, the relatively complex problem of designing and implementing data privacy mechanisms, while ensuring efficient data access, is also discussed, and the relevant metrics are presented. The paper concludes by presenting the most important open research questions and challenges. The paper provides a comprehensive and thorough scientific literature survey, which is useful for any researcher or practitioner in the scope of data-driven soft sensors and privacy techniques, in relation to the relevant machine-learning-based models.

A Bionic Testbed for Cardiac Ablation Tools.

Bionic-engineered tissues have been proposed for testing the performance of cardiovascular medical devices and predicting clinical outcomes ex vivo. Progress has been made in the development of compliant electronics that are capable of monitoring treatment parameters and being coupled to engineered tissues; however, the scale of most engineered tissues is too small to accommodate the size of clinical-grade medical devices. Here, we show substantial progress toward bionic tissues for evaluating cardiac ablation tools by generating a centimeter-scale human cardiac disk and coupling it to a hydrogel-based soft-pressure sensor. The cardiac tissue with contiguous electromechanical function was made possible by our recently established method to 3D bioprint human pluripotent stem cells in an extracellular matrix-based bioink that allows for in situ cell expansion prior to cardiac differentiation. The pressure sensor described here utilized electrical impedance tomography to enable the real-time spatiotemporal mapping of pressure distribution. A cryoablation tip catheter was applied to the composite bionic tissues with varied pressure. We found a close correlation between the cell response to ablation and the applied pressure. Under some conditions, cardiomyocytes could survive in the ablated region with more rounded morphology compared to the unablated controls, and connectivity was disrupted. This is the first known functional characterization of living human cardiomyocytes following an ablation procedure that suggests several mechanisms by which arrhythmia might redevelop following an ablation. Thus, bionic-engineered testbeds of this type can be indicators of tissue health and function and provide unique insight into human cell responses to ablative interventions.

Integration of 3D Printed Flexible Pressure Sensors into Physical Interfaces for Wearable Robots.

Sensing pressure at the physical interface between the robot and the human has important implications for wearable robots. On the one hand, monitoring pressure distribution can give valuable benefits on the aspects of comfortability and safety of such devices. Additionally, on the other hand, they can be used as a rich sensory input to high level interaction controllers. However, a problem is that the commercial availability of this technology is mostly limited to either low-cost solutions with poor performance or expensive options, limiting the possibilities for iterative designs. As an alternative, in this manuscript we present a three-dimensional (3D) printed flexible capacitive pressure sensor that allows seamless integration for wearable robotic applications. The sensors are manufactured using additive manufacturing techniques, which provides benefits in terms of versatility of design and implementation. In this study, a characterization of the 3D printed sensors in a test-bench is presented after which the sensors are integrated in an upper arm interface. A human-in-the-loop calibration of the sensors is then shown, allowing to estimate the external force and pressure distribution that is acting on the upper arm of seven human subjects while performing a dynamic task. The validation of the method is achieved by means of a collaborative robot for precise force interaction measurements. The results indicate that the proposed sensors are a potential solution for further implementation in human-robot interfaces.

Dual Properties of Polyvinyl Alcohol-Based Magnetorheological Plastomer with Different Ratio of DMSO/Water.

Polyvinyl alcohol (PVA)-based magnetorheological plastomer (MRP) possesses excellent magnetically dependent mechanical properties such as the magnetorheological effect (MR effect) when exposed to an external magnetic field. PVA-based MRP also shows a shear stiffening (ST) effect, which is very beneficial in fabricating pressure sensor. Thus, it can automatically respond to external stimuli such as shear force without the magnetic field. The dual properties of PVA-based MRP mainly on the ST and MR effect are rarely reported. Therefore, this work empirically investigates the dual properties of this smart material under the influence of different solvent compositions (20:80, 40:60, 60:40, and 80:20) by varying the ratios of binary solvent mixture (dimethyl sulfoxide (DMSO) to water). Upon applying a shear stress with excitation frequencies from 0.01 to 10 Hz, the storage modulus (G') for PVA-based MRP with DMSO to water ratio of 20:40 increases from 6.62 × 10-5 to 0.035 MPa. This result demonstrates an excellent ST effect with the relative shear stiffening effect (RSTE) up to 52,827%. In addition, both the ST and MR effect show a downward trend with increasing DMSO content to water. Notably, the physical state of hydrogel MRP could be changed with different solvent ratios either in the liquid-like or solid-like state. On the other hand, a transient stepwise experiment showed that the solvent's composition had a positive effect on the arrangement of CIPs within the matrix as a function of the external magnetic field. Therefore, the solvent ratio (DMSO/water) can influence both ST and MR effects of hydrogel MRP, which need to be emphasized in the fabrication of hydrogel MRP for appropriate applications primarily with soft sensors and actuators for dynamic motion control.

Industrial Semi-Supervised Dynamic Soft-Sensor Modeling Approach Based on Deep Relevant Representation Learning.

Soft sensors based on deep learning have been growing in industrial process applications, inferring hard-to-measure but crucial quality-related variables. However, applications may present strong non-linearity, dynamicity, and a lack of labeled data. To deal with the above-cited problems, the extraction of relevant features is becoming a field of interest in soft-sensing. A novel deep representative learning soft-sensor modeling approach is proposed based on stacked autoencoder (SAE), mutual information (MI), and long-short term memory (LSTM). SAE is trained layer by layer with MI evaluation performed between extracted features and targeted output to evaluate the relevance of learned representation in each layer. This approach highlights relevant information and eliminates irrelevant information from the current layer. Thus, deep output-related representative features are retrieved. In the supervised fine-tuning stage, an LSTM is coupled to the tail of the SAE to address system inherent dynamic behavior. Also, a k-fold cross-validation ensemble strategy is applied to enhance the soft-sensor reliability. Two real-world industrial non-linear processes are employed to evaluate the proposed method performance. The obtained results show improved prediction performance in comparison to other traditional and state-of-art methods. Compared to the other methods, the proposed model can generate more than 38.6% and 39.4% improvement of RMSE for the two analyzed industrial cases.

Soft System Based on Fiber Bragg Grating Sensor for Loss of Resistance Detection during Epidural Procedures: In Silico and In Vivo Assessment.

Epidural analgesia represents a clinical common practice aiming at pain mitigation. This loco-regional technique is widely used in several applications such as labor, surgery and lower back pain. It involves the injections of anesthetics or analgesics into the epidural space (ES). The ES detection is still demanding and is usually performed by the techniques named loss of resistance (LOR). In this study, we propose a novel soft system (SS) based on one fiber Bragg grating sensor (FBG) embedded in a soft polymeric matrix for LOR detection during the epidural puncture. The SS was designed to allow instrumenting the syringe's plunger without relevant modifications of the anesthetist's sensations during the procedure. After the metrological characterization of the SS, we assessed the capability of this solution in detecting LOR by carrying it out in silico and in clinical settings. For both trials, results revealed the capability of the proposed solutions in detecting the LOR and then in recording the force exerted on the plunger.

RNN- and LSTM-Based Soft Sensors Transferability for an Industrial Process.

The design and application of Soft Sensors (SSs) in the process industry is a growing research field, which needs to mediate problems of model accuracy with data availability and computational complexity. Black-box machine learning (ML) methods are often used as an efficient tool to implement SSs. Many efforts are, however, required to properly select input variables, model class, model order and the needed hyperparameters. The aim of this work was to investigate the possibility to transfer the knowledge acquired in the design of a SS for a given process to a similar one. This has been approached as a transfer learning problem from a source to a target domain. The implementation of a transfer learning procedure allows to considerably reduce the computational time dedicated to the SS design procedure, leaving out many of the required phases. Two transfer learning methods have been proposed, evaluating their suitability to design SSs based on nonlinear dynamical models. Recurrent neural structures have been used to implement the SSs. In detail, recurrent neural networks and long short-term memory architectures have been compared in regard to their transferability. An industrial case of study has been considered, to evaluate the performance of the proposed procedures and the best compromise between SS performance and computational effort in transferring the model. The problem of labeled data scarcity in the target domain has been also discussed. The obtained results demonstrate the suitability of the proposed transfer learning methods in the design of nonlinear dynamical models for industrial systems.

Control of Microalgae Growth in Artificially Lighted Photobioreactors Using Metaheuristic-Based Predictions.

A metaheuristic algorithm can be a realistic solution when optimal control problems require a significant computational effort. The problem stated in this work concerns the optimal control of microalgae growth in an artificially lighted photobioreactor working in batch mode. The process and the dynamic model are very well known and have been validated in previous papers. The control solution is a closed-loop structure whose controller generates predicted control sequences. An efficient way to make optimal predictions is to use a metaheuristic algorithm, the particle swarm optimization algorithm. Even if this metaheuristic is efficient in treating predictions with a very large prediction horizon, the main objective of this paper is to find a tool to reduce the controller's computational complexity. We propose a soft sensor that gives information used to reduce the interval where the control input's values are placed in each sampling period. The sensor is based on measurement of the biomass concentration and numerical integration of the process model. The returned information concerns the specific growth rate of microalgae and the biomass yield on light energy. Algorithms, which can be used in real-time implementation, are proposed for all modules involved in the simulation series. Details concerning the implementation of the closed loop, controller, and soft sensor are presented. The simulation results prove that the soft sensor leads to a significant decrease in computational complexity.