Underwater images usually exhibit severe color cast, hazy appearance, and/or dark regions because of the complex lighting absorption and scattering in water. How to increase the quality of these degraded underwater images has emerged as a key issue for various underwater application tasks. Recent efforts have been made to deal with single type degradation, however, it is still challenging to deal with multiple degradations that usually coexist in an underwater image with a general network. The degradations in underwater images can be divided into medium-agnostic (hazy or low-light which also encountered in in-air images) and medium-specific (color distortion caused by the specific light attenuation property in water) ones. According to this observation, this article proposes a cascaded multimodule underwater image enhancement (UIE) framework to address the coexisted multiple degradations. In the proposed framework, an in-air image enhancement module and a novel proposed adaptive color channel compensation network (AC 3 Net) are cascaded, in which the former focuses primarily on solving medium-agnostic degradations and the latter is for handling the medium-specific degradation. This framework has good flexibility by cascading different types of in-air image enhancement networks with AC 3 Net to achieve various UIE. The effectiveness of the proposed framework has been extensively validated on various degraded underwater images as well as different underwater visual perception tasks.
Flexible actuation materials play a crucial role in biomimetic robots. Seeking methods to enhance actuation and functionality is one of the directions in which actuators strive to meet the high-performance and diverse requirements of environmental conditions. Herein, by utilizing the method of adsorbing N-doped carbon dots (NCDs) onto SiO2 to form clusters of functional particles, a NCDs@SiO2/PDMS elastomer was prepared and its combined optical and electrical co-stimulation properties were effectively harnessed to develop a biomimetic crawling robot resembling Rhagophthalmus (firefly). The introduction of NCDs@SiO2 cluster particles not only effectively improves the mechanical and dielectric properties of the elastomer but also exhibits fluorescence response and actuation response under the co-stimulation of UV and electricity, respectively. Additionally, a hybrid dielectric elastomer actuator (DEA) with a transparent SWCNT mesh electrode exhibits two notable advancements: an 826% increase in out-of-plane displacement under low electric field stimulation compared to the pure matrix and the ability of NCDs to maintain a stable excited state within the polymer for an extended duration under UV-excitation. Simultaneously, the transparent biomimetic crawling robot can stealthily move in specific environments and fluoresce under UV light.
Magnetic soft robots have shown great potential for biomedical applications due to their high shape reconfigurability, motion agility, and multi-functionality in physiological environments. Magnetic soft robots with multi-layer structures can enhance the loading capacity and function complexity for targeted delivery. However, the interactions between soft entities have yet to be fully investigated, and thus the assembly of magnetic soft robots with on-demand motion modes from multiple film-like layers is still challenging. Herein, we model and tailor the magnetic interaction between soft film-like layers with distinct in-plane structures, and then realize multi-layer soft robots that are capable of performing agile motions and targeted adhesion. Each layer of the robot consists of a soft magnetic substrate and an adhesive film. The mechanical properties and adhesion performance of the adhesive films are systematically characterized. The robot is capable of performing two locomotion modes, i.e., translational motion and tumbling motion, and also the on-demand separation with one side layer adhered to tissues. Simulation results are presented, which have a good qualitative agreement with the experimental results. The feasibility of using the robot to perform multi-target adhesion in a stomach is validated in both ex-vivo and in-vivo experiments.
Robotics , Humans , Physical Phenomena , Motion , Computer Simulation , Tissue Adhesions , Magnetic Phenomena
Optical detection equipment (ODE) is subjected to vibrations that hamper the quality of imaging. In this paper, an active vibration isolation and compensation system (VICS) for the ODE is developed and systematically studied to improve the optical imaging quality. An active vibration isolator for cameras is designed, employing a dual-loop control strategy with position compensation and integral force feedback (IFF) control, and establishing the mapping relationship between vibration and image quality. A performance metric for evaluating images is also proposed. Finally, an experimental platform is constructed to verify its effectiveness. Based on the experimental results, it can be concluded that the proposed VICS effectively isolates vibrations, resulting in a reduction of 13.95 dB in the peak at the natural frequency and an 11.76 Hz widening of the isolation bandwidth compared with the system without it. At the same time, the experiments demonstrate that the image performance metric value increases by 46.03% near the natural frequency.
Object detection accuracy degrades seriously in visually degraded scenes. A natural solution is to first enhance the degraded image and then perform object detection. However, it is suboptimal and does not necessarily lead to the improvement of object detection due to the separation of the image enhancement and object detection tasks. To solve this problem, we propose an image enhancement guided object detection method, which refines the detection network with an additional enhancement branch in an end-to-end way. Specifically, the enhancement branch and detection branch are organized in a parallel way, and a feature guided module is designed to connect the two branches, which optimizes the shallow feature of the input image in the detection branch to be as consistent as possible with that of the enhanced image. As the enhancement branch is frozen during training, such a design plays a role in using the features of enhanced images to guide the learning of object detection branch, so as to make the learned detection branch being aware of both image quality and object detection. When testing, the enhancement branch and feature guided module are removed, and so no additional computation cost is introduced for detection. Extensive experimental results, on underwater, hazy, and low-light object detection datasets, demonstrate that the proposed method can improve the detection performance of popular detection networks (YOLO v3, Faster R-CNN, DetectoRS) significantly in visually degraded scenes.
The modern engineering systems often operate under varying environments and only partial information can be observed at discrete monitoring epochs. For such systems, few works have been done for the prognostics of health status using the available environment and monitoring information. Therefore, the aim of this article is to present a new health prediction method for modern engineering systems whose condition is partially observable under varying environments. A dynamic Gamma process is proposed to model the system degradation observations under changing environments. To describe the relation of system actual status to the observed information, a proportional hazard (PH) model integrating internal aging and external observations is presented for modeling the system hazard rate. To realize prediction of residual life of such systems, a matrix operation-based prognostic method is presented to calculate the closed-form solutions of health characteristics for the system. A case study of partially observable failing systems is demonstrated, and comparisons with other recent developed approaches are also given to show the effectiveness of the model.
The dynamical change of cellular mechanical properties plays an important role in cell metastasis process, while how the cancer cells modulate their mechanical properties during metastasis are still not fully understood. In this report, the cellular detaching and seeding processes, two vital steps of cell metastasis, were simulated in vitro using a self-developed protocol and characterized by the dynamical mechanical properties using AFM. The measured results show that cells decrease their stiffness and increase their surface adhesion force as they are detaching from substrate, while cells present an opposite change in mechanical properties as they seeding. Additionally, the effect of anti-cancer drug (docetaxel) on the detaching and attaching process of cancer cells (PC-3) was also investigated from the aspect of mechanical properties. The results shows that the docetaxel can increase stiffness, decrease surface adhesion force of PC-3 cell, and slow down the change speed of these mechanical properties during PC-3 cell detaching and seeding process. These discoveries demonstrated that a dynamical change of cell mechanical properties is required for cancer cell metastasis, which provide a new drug development strategy for cancer treatment.
Mechanical Phenomena , Docetaxel/pharmacology , Cell Line, Tumor , Microscopy, Atomic Force , Cell Adhesion
Various domain adaptation (DA) methods have been proposed to address distribution discrepancy and knowledge transfer between the source and target domains. However, many DA models focus on matching the marginal distributions of two domains and cannot satisfy fault-diagnosed-task requirements. To enhance the ability of DA, a new DA mechanism, called deep joint distribution alignment (DJDA), is proposed to simultaneously reduce the discrepancy in marginal and conditional distributions between two domains. A new statistical metric that can align the means and covariances of two domains is designed to match the marginal distributions of the source and target domains. To align the class conditional distributions, a Gaussian mixture model is used to obtain the distribution of each category in the target domain. Then, the conditional distributions of the source domain are computed via maximum-likelihood estimation, and information entropy and Wasserstein distance are employed to reduce class conditional distribution discrepancy between the two domains. With joint distribution alignment, DJDA can achieve domain confusion to the highest degree. DJDA is applied to the fault transfer diagnosis of a wind turbine gearbox and cross-bearing with unlabeled target-domain samples. Experimental results verify that DJDA outperforms other typical DA models.
Extracting robust and discriminative local features from images plays a vital role for long term visual localization, whose challenges are mainly caused by the severe appearance differences between matching images due to the day-night illuminations, seasonal changes, and human activities. Existing solutions resort to jointly learning both keypoints and their descriptors in an end-to-end manner, leveraged on large number of annotations of point correspondence which are harvested from the structure from motion and depth estimation algorithms. While these methods show improved performance over non-deep methods or those two-stage deep methods, i.e., detection and then description, they are still struggled to conquer the problems encountered in long term visual localization. Since the intrinsic semantics are invariant to the local appearance changes, this paper proposes to learn semantic-aware local features in order to improve robustness of local feature matching for long term localization. Based on a state of the art CNN architecture for local feature learning, i.e., ASLFeat, this paper leverages on the semantic information from an off-the-shelf semantic segmentation network to learn semantic-aware feature maps. The learned correspondence-aware feature descriptors and semantic features are then merged to form the final feature descriptors, for which the improved feature matching ability has been observed in experiments. In addition, the learned semantics embedded in the features can be further used to filter out noisy keypoints, leading to additional accuracy improvement and faster matching speed. Experiments on two popular long term visual localization benchmarks (Aachen Day and Night v1.1, Robotcar Seasons) and one challenging indoor benchmark (InLoc) demonstrate encouraging improvements of the localization accuracy over its counterpart and other competitive methods.
Algorithms , Semantics , Humans , Motion
Jumping is an important locomotion function to extend navigation range, overcome obstacles, and adapt to unstructured environments. In that sense, continuous jumping and direction adjustability can be essential properties for terrestrial robots with multimodal locomotion. However, only few soft jumping robots can achieve rapid continuous jumping and controlled turning locomotion for obstacle crossing. Here, we present an electrohydrostatically driven tethered legless soft jumping robot capable of rapid, continuous, and steered jumping based on a soft electrohydrostatic bending actuator. This 1.1 g and 6.5 cm tethered soft jumping robot is able to achieve a jumping height of 7.68 body heights and a continuous forward jumping speed of 6.01 body lengths per second. Combining two actuator units, it can achieve rapid turning with a speed of 138.4° per second. The robots are also demonstrated to be capable of skipping across a multitude of obstacles. This work provides a foundation for the application of electrohydrostatic actuation in soft robots for agile and fast multimodal locomotion.
A novel hybridization scheme is proposed with electromagnetic transduction to improve the power density of piezoelectric energy harvester (PEH) in this paper. Based on the basic cantilever piezoelectric energy harvester (BC-PEH) composed of a mass block, a piezoelectric patch, and a cantilever beam, we replaced the mass block by a magnet array and added a coil array to form the hybrid energy harvester. To enhance the output power of the electromagnetic energy harvester (EMEH), we utilized an alternating magnet array. Then, to compare the power density of the hybrid harvester and BC-PEH, the experiments of output power were conducted. According to the experimental results, the power densities of the hybrid harvester and BC-PEH are, respectively, 3.53 mW/cm3 and 5.14 µW/cm3 under the conditions of 18.6 Hz and 0.3 g. Therefore, the power density of the hybrid harvester is 686 times as high as that of the BC-PEH, which verified the power density improvement of PEH via a hybridization scheme with EMEH. Additionally, the hybrid harvester exhibits better performance for charging capacitors, such as charging a 2.2 mF capacitor to 8 V within 17 s. It is of great significance to further develop self-powered devices.
To ensure the mission implementation of Autonomous Underwater Vehicles (AUVs), faults occurring on actuators should be detected and located promptly; therefore, reliable control strategies and inputs can be effectively provided. In this paper, faults occurring on the propulsion and attitude control systems of a torpedo-shaped AUV are analyzed and located while fault features may induce confusions for conventional fault localization (FL). Selective features of defined fault parameters are assorted as necessary conditions against different faulty actuators and synthesized in a fault tree subsequently to state the sufficiency towards possible abnormal parts. By matching fault features with those of estimated fault parameters, suspected faulty sections are located. Thereafter, active FL strategies that analyze the related fault parameters after executing purposive actuator control are proposed to provide precise fault location. Moreover, the generality of the proposed methods is analyzed to support extensive implementations. Simulations based on finite element analysis against a torpedo-shaped AUV with actuator faults are carried out to illustrate the effectiveness of the proposed methods.
In this paper, we propose a generator for piezoelectric energy harvesting from suspension structures. This device consists of a leaf spring and eight pairs of piezoelectric layers attached to inner and outer surfaces. We present a special type of leaf spring, which can magnify the force from the workload to allow the piezoelectric layers to achieve larger deformation. The generator is to solve the problem of vibration energy reutilization in a low-frequency vibration system. To verify the efficiency of the proposed configuration, a series of experiments are operated. The results indicate that the resonance frequency (25.2 Hz) obtained from the sweep experiment is close to the simulation result (26.1 Hz). Impedance-matching experiments show that the sum of the output power attains 1.7 mW, and the maximum single layer reaches 0.6 mW with an impedance matching of 610 KΩ, and the instantaneous peak-peak power density is 3.82 mW/cm3. The capacitor-charging performance of the generator is also excellent under the series condition. For a 4.7 µF capacitor, the voltage is charged to 25 V in 30 s and limited at 32 V in 80 s. These results demonstrate the exploitable potential of piezoelectric energy harvesting from suspension structures.
Both Young's modulus and specific membrane capacitance (SMC) are two important physical parameters for characterizing cell status. In this paper, we utilized a thin-neck-micropipette aspiration system to simultaneously quantify Young's modulus and SMC value of six types of cell lines in different progression grades, which include four grades from the lowest metastatic potential G1 to the highest potential G4. We investigated how these two physical properties possess heterogeneities in bladder cancer cells with different grades and what roles they might play in grading bladder cancer. The characterization results of these cells of different cancer grades is linearly correlated with the cancer grades, showing that the Young's modulus is negatively linearly correlated with bladder cancer grades, while SMC shows a positive linear correlation. Furthermore, the combination of these two physical properties on a scatter diagram clearly shows the cell groups with different cancer grades, which means that this combination could be a potential tumor grading marker to identify cancer cells with different metastatic potential.
Atomic force microscope (AFM) has been widely used in the biological field owing to its high sensitivity (subnanonewton), high spatial resolution (nanometer), and adaptability to physiological environments. Nowadays, force volume (FV) and peakforce quantitative nanomechanical (QNM) are two distinct modes of AFM used in biomechanical research. However, numerous studies have revealed an extremely confusing phenomenon that FV mode has a significant difference with QNM in determining the mechanical properties of the same samples. In this article, for the case of human benign prostatic hyperplasia cells (BPH) and two cancerous prostate cells with different grades of malignancy (PC3 and DU145), the differences were compared between FV and QNM modes in detecting mechanical properties. The results show measured Young's modulus of the same cells in FV mode was much lower than that obtained by QNM mode. Combining experimental results with working principles of two modes, it is indicated that surface adhesion is highly suspected to be a critical factor resulting in the measurement difference between two modes. To further confirm this conjecture, various weight ratios of polydimethylsiloxane (PDMS) were assessed by two modes, respectively. The results show that the difference of Young's modulus measured by two modes increases with the surface adhesion of PDMS, confirming that adhesion is one of the significant elements that lead to the measurement difference between FV and QNM modes.
Cell Adhesion/physiology , Elastic Modulus/physiology , Mechanical Phenomena , Microscopy, Atomic Force/methods , Cell Line, Tumor , Dimethylpolysiloxanes/chemistry , Humans , PC-3 Cells , Surface Properties
Constructing tissue engineered vascular grafts (TEVG) is of great significance for cardiovascular research. However, most of the fabrication techniques are unable to construct TEVG with a bifurcated and curved structure. This paper presents multilayered biodegradable TEVGs with a curved structure and multi-branches. The technique combined 3D printed molds and casting hydrogel and sacrificial material to create vessel-mimicking constructs with customizable structural parameters. Compared with other fabrication methods, the proposed technique can create more native-like 3D geometries. The diameter and wall thickness of the fabricated constructs can be independently controlled, providing a feasible approach for TEVG construction. Enzymatically-crosslinked gelatin was used as the material of the constructs. The mechanical properties and thermostability of the constructs were evaluated. Fluid-structure interaction simulations were conducted to examine the displacement of the construct's wall when blood flows through it. Human umbilical vein endothelial cells (HUVECs) were seeded on the inner channel of the constructs and cultured for 72 h. The cell morphology was assessed. The results showed that the proposed technique had good application potentials, and will hopefully provide a novel technological approach for constructing integrated vasculature for tissue engineering.
Cellular physical properties have been identified to reflect cell states. Existing techniques are able to characterize either mechanical or electrical properties of a cell. This paper presents a micropipette aspiration technique that enables the characterization of both mechanical (instantaneous elastic modulus, equilibrium elastic modulus, and viscosity), and electrical (specific membrane capacitance) properties of the same single cell. Two bladder cancer cell lines (RT4 and T24) with different metastatic potential were used to evaluate the technique. The results showed that high-grade bladder cancer cells (T24, grade III) possess lower viscosity, lower elastic modulus, and larger SMC than the low-grade cancer cells (RT4, grade I). The Naive Bayes classifier was utilized to assess the classification accuracy using single-physical and multi-physical parameters. The classification results confirmed that the use of multi-biophysical parameters resulted in higher accuracy (97.5%), sensitivity (100%), and specificity (95.2%) than the use of a single-physical parameter for distinguishing T24 and RT4 cells.
Biomechanical Phenomena/physiology , Cell Physiological Phenomena/physiology , Micromanipulation/methods , Single-Cell Analysis/methods , Bayes Theorem , Cell Line, Tumor , Elasticity/physiology , Equipment Design , Humans , Image Processing, Computer-Assisted/methods , Micromanipulation/instrumentation , Single-Cell Analysis/instrumentation , Viscosity
OBJECTIVE: In clinical intracytoplasmic sperm injection (ICSI), a motile sperm must be immobilized before insertion into an oocyte. This paper aims to develop a robotic system for automated tracking, orientation control, and immobilization of motile sperms for clinical ICSI applications. METHODS: We adapt the probabilistic data association filter by adding sperm head orientation into state variables for robustly tracking the sperm head and estimating sperm tail positions under interfering conditions. The robotic system also utilizes a motorized rotational microscopy stage and a new visual servo control strategy that predicts and compensates for sperm movements to actively adjust sperm orientation for immobilizing a sperm swimming in any direction. RESULTS: The system robustly tracked sperm head with a tracking success rate of 96.0% and estimated sperm tail position with an accuracy of 1.08 µm under clinical conditions where the occlusion of the target sperm and interference from other sperms occur. Experimental results from robotic immobilization of 400 sperms confirmed that the system achieved a consistent immobilization success rate of 94.5%, independent of sperm velocity or swimming direction. CONCLUSION: Our adapted tracking algorithm effectively distinguishes the target sperm from interfering sperms. Predicting and compensating for sperm movements significantly reduce the positioning error during sperm orientation control. These features make the robotic system suitable for automated sperm immobilization. SIGNIFICANCE: The robotic system eliminates stringent skill requirements in manual sperm immobilization. It is capable of manipulating sperms swimming in an arbitrary direction with a high success rate.
Robotics , Sperm Injections, Intracytoplasmic , Spermatozoa/cytology , Equipment Design , Female , Humans , Male , Micromanipulation , Nanomedicine , Oocytes/cytology , Robotics/instrumentation , Robotics/methods , Sperm Injections, Intracytoplasmic/instrumentation , Sperm Injections, Intracytoplasmic/methods
This paper studies the compound learning control of disturbed uncertain strict-feedback systems. The design is using the dynamic surface control equipped with a novel learning scheme. This paper integrates the recently developed online recorded data-based neural learning with the nonlinear disturbance observer (DOB) to achieve good "understanding" of the system uncertainty including unknown dynamics and time-varying disturbance. With the proposed method to show how the neural networks and DOB are cooperating with each other, one indicator is constructed and included into the update law. The closed-loop system stability analysis is rigorously presented. Different kinds of disturbances are considered in a third-order system as simulation examples and the results confirm that the proposed method achieves higher tracking accuracy while the compound estimation is much more precise. The design is applied to the flexible hypersonic flight dynamics and a better tracking performance is obtained.
Microinjection is a widely used technique for introducing exogenous materials into cells. Many applications of microinjection, such as gene editing and drug testing, rely on the accurate control of the deposition volume. However, the deposition volume in microinjection is presently calibrated in an open medium without considering the cell inner pressure effect, which we experimentally show in this paper that it can induce an error as large as 30% between the actual deposition volume and the set volume. In this work, the relationship between the cell inner pressure and the deposition volume was analytically modeled and experimentally validated. On the basis of the developed model, the cell inner pressure of a given cell type can be well estimated from the injection pressure and the resulting deposition volume. The quantitated cell inner pressure is then used to reduce the error between the set volume and the actual deposition volume. Experiments conducted on human bladder cancer cells (T24 and RT4) showed that T24 cells have a higher inner pressure than RT4 cells (405 ± 45 Pa vs 341 ± 34 Pa), and after compensating for the cell inner pressure, the error between the intended set volume and the actual deposition volume into a cell became less than 3%.
