Remaining Useful Life Prediction Based on Deep Learning: A Survey.

Remaining useful life (RUL) is a metric of health state for essential equipment. It plays a significant role in health management. However, RUL is often random and unknown. One type of physics-based method builds a mathematical model for RUL using prior principles, but this is a tough task in real-world applications. Another type of method estimates RUL from available information through condition and health monitoring; this is known as the data-driven method. Traditional data-driven methods require significant human effort in designing health features to represent performance degradation, yet the prediction accuracy is limited. With breakthroughs in various application scenarios in recent years, deep learning techniques provide new insights into this problem. Over the past few years, deep-learning-based RUL prediction has attracted increasing attention from the academic community. Therefore, it is necessary to conduct a survey on deep-learning-based RUL prediction. To ensure a comprehensive survey, the literature is reviewed from three dimensions. Firstly, a unified framework is proposed for deep-learning-based RUL prediction and the models and approaches in the literature are reviewed under this framework. Secondly, detailed estimation processes are compared from the perspective of different deep learning models. Thirdly, the literature is examined from the perspective of specific problems, such as scenarios where the collected data consist of limited labeled data. Finally, the main challenges and future directions are summarized.

Research on Chinese medical named entity recognition based on collaborative cooperation of multiple neural network models.

Medical named entity recognition (NER) in Chinese electronic medical records (CEMRs) has drawn much research attention, and plays a vital prerequisite role for extracting high-value medical information. In 2018, China Health Information Processing Conference (CHIP2018) organized a medical NER academic competition aiming to extract three types of malignant tumor entity from CEMRs. Since the three types of entity are highly domain-specific and interdependency, extraction of them cannot be achieved with a single neural network model. Based on comprehensive study of the three types of entity and the entity interdependencies, we propose a collaborative cooperation of multiple neural network models based approach, which consists of two BiLSTM-CRF models and a CNN model. In order to tackle the problem that target scene dataset is small and entity distributions are sparse, we introduce non-target scene datasets and propose sentence-level neural network model transfer learning. Based on 30,000 real-world CEMRs, we pre-train medical domain-specific Chinese character embeddings with word2vec, GloVe and ELMo, and apply them to our approach respectively to validate effects of pre-trained language models in Chinese medical NER. Also, as control experiments, we apply Gated Recurrent Unit to our approach. Finally, our approach achieves an overall F1-score of 87.60%, which is the state-of-the-art performance to the best of our knowledge. In addition, our approach has won the champion of the medical NER academic competition organized by 2019 China Conference on Knowledge Graph and Semantic Computing, which proves the outstanding generalization ability of our approach.

Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image.

Removing raindrops from a single image is a challenging problem due to the complex changes in shape, scale, and transparency among raindrops. Previous explorations have mainly been limited in two ways. First, publicly available raindrop image datasets have limited capacity in terms of modeling raindrop characteristics (e.g., raindrop collision and fusion) in real-world scenes. Second, recent deraining methods tend to apply shape-invariant filters to cope with diverse rainy images and fail to remove raindrops that are especially varied in shape and scale. In this paper, we address these raindrop removal problems from two perspectives. First, we establish a large-scale dataset named RaindropCityscapes, which includes 11,583 pairs of raindrop and raindrop-free images, covering a wide variety of raindrops and background scenarios. Second, a two-branch Multi-scale Shape Adaptive Network (MSANet) is proposed to detect and remove diverse raindrops, effectively filtering the occluded raindrop regions and keeping the clean background well-preserved. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art raindrop removal methods. Moreover, the extension of our method towards the rainy image segmentation and detection tasks validates the practicality of the proposed method in outdoor applications.

A hybrid approach for named entity recognition in Chinese electronic medical record.

BACKGROUND: With the rapid spread of electronic medical records and the arrival of medical big data era, the application of natural language processing technology in biomedicine has become a hot research topic. METHODS: In this paper, firstly, BiLSTM-CRF model is applied to medical named entity recognition on Chinese electronic medical record. According to the characteristics of Chinese electronic medical records, obtain the low-dimensional word vector of each word in units of sentences. And then input the word vector to BiLSTM to realize automatic extraction of sentence features. And then CRF performs sentence-level word tagging. Secondly, attention mechanism is added between the BiLSTM and the CRF to construct Attention-BiLSTM-CRF model, which can leverage document-level information to alleviate tagging inconsistency. In addition, this paper proposes an entity auto-correct algorithm to rectify entities according to historical entity information. At last, a drug dictionary and post-processing rules are well-built to rectify entities, to further improve performance. RESULTS: The final F1 scores of the BiLSTM-CRF and Attention-BiLSTM-CRF model on given test dataset are 90.15 and 90.82% respectively, both of which are higher than 89.26%, which is the best F1 score on the test dataset except ours. CONCLUSION: Our approach can be used to recognize medical named entity on Chinese electronic medical records and achieves the state-of-the-art performance on the given test dataset.

Runt-Related Transcription Factor 1 Regulates LPS-Induced Acute Lung Injury via NF-κB Signaling.

Runt-related transcription factor 1 (RUNX1), a transcription factor expressed in multiple organs, plays important roles in embryonic development and hematopoiesis. Although RUNX1 is highly expressed in pulmonary tissues, its roles in lung function and homeostasis are unknown. We sought to assess the role of RUNX1 in lung development and inflammation after LPS challenge. Expression of RUNX1 was assessed in the developing and postnatal lung. RUNX1 was conditionally deleted in pulmonary epithelial cells. Pulmonary maturation was evaluated in the developing and postnatal lung, and lung inflammation was investigated in adult mice after LPS challenge. Interactions between RUNX1 and inflammatory signaling via NF-κB-IkB kinase ß were assessed in vitro. RUNX1 was expressed in both mesenchymal and epithelial compartments of the developing and postnatal lung. The RUNX1 gene was efficiently deleted from respiratory epithelial cells producing Runx1∆/∆ mice. Although lung maturation was delayed, Runx1∆/∆ mice survived postnatally and subsequent growth and maturation of the lung proceeded normally. Increased respiratory distress, inflammation, and proinflammatory cytokines were observed in the Runx1-deleted mice after pulmonary LPS exposure. RUNX1 deletion was associated with the activation of NF-κB in respiratory epithelial cells. RUNX1 was required for the suppression of NF-κB signaling pathway via inhibition of IkB kinase ß in in vitro studies. RUNX1 plays a critical role in the lung inflammation after LPS-induced injury.

Assessing atrazine-induced toxicities in Bufo bufo gargarizans Cantor.

Atrazine (AZ), a widely used herbicide has drawn attentions for its potential impacts on amphibians. This study aims to investigate the toxicity of AZ in Bufo bufo gargarizans Cantor (B. bufo gargarizans), a species of toad commonly found in China and countries in East Asia. We treated tadpoles with 0.1, 1, 10 and 100 µg/L AZ for 85 days and examined related parameters. The results showed that the mortality of the toads in the treatment group increased dramatically in a U-shaped dose-response relationship. The hindlimb extension and metamorphosis rate of the toads were significantly inhibited by AZ at 10 and 100 µg/L. Under the same condition, there were significant progressive changes in the testicular structures. Moreover, we found that AZ has no significant effects on growth, sex ratios, gonadal morphology, forelimb emergence and histology in the ovaries. Our results support the idea that environmental contaminants including AZ may be relevant to global amphibian decline.

Foxa2 regulates leukotrienes to inhibit Th2-mediated pulmonary inflammation.

Foxa2 is a member of the Forkhead family of nuclear transcription factors that is highly expressed in respiratory epithelial cells of the developing and mature lung. Foxa2 is required for normal airway epithelial differentiation, and its deletion causes goblet-cell metaplasia and Th2-mediated pulmonary inflammation during postnatal development. Foxa2 expression is inhibited during aeroallergen sensitization and after stimulation with Th2 cytokines, when its loss is associated with goblet-cell metaplasia. Mechanisms by which Foxa2 controls airway epithelial differentiation and Th2 immunity are incompletely known. During the first 2 weeks after birth, the loss of Foxa2 increases the production of leukotrienes (LTs) and Th2 cytokines in the lungs of Foxa2 gene-targeted mice. Foxa2 expression inhibited 15-lipoxygenase (Alox15) and increased Alox5 transcription, each encoding key lipoxygenases associated with asthma. The inhibition of the cysteinyl LT (CysLT) signaling pathway by montelukast inhibited IL-4, IL-5, eotaxin-2, and regulated upon activation normal T cell expressed and presumably secreted expression in the developing lungs of Foxa2 gene-targeted mice. Montelukast inhibited the expression of genes regulating mucus metaplasia, including Spdef, Muc5ac, Foxa3, and Arg2. Foxa2 plays a cell-autonomous role in the respiratory epithelium, and is required for the suppression of Th2 immunity and mucus metaplasia in the developing lung in a process determined in part by its regulation of the CysLT pathway.

[Construction of Runx1 transgenic mice].

This study was aimed to construct transgenic mouse model with target for Runxl gene. Runxl cDNA of mice was amplified by PCR from pcDNA3. 1 Flag Runx1 FL vector and inserted into ptetO7-Asc-IRES-EGFP vector to form a recombinant vector, and then the recombinant vector was injected into fertilized egg by microinjection technology to get a transgenic mouse. The results of PCR and Southern blot indicated that the Runx1 transgenic mouse was constructed successfully, and this could provide an important tool for studying the function of Runxl gene in vivo.

Unsupervised Visual Representation Learning via Multi-Dimensional Relationship Alignment.

Recently, contrastive learning based on augmentation invariance and instance discrimination has made great achievements, owing to its excellent ability to learn beneficial representations without any manual annotations. However, the natural similarity among instances conflicts with instance discrimination which treats each instance as a unique individual. In order to explore the natural relationship among instances and integrate it into contrastive learning, we propose a novel approach in this paper, Relationship Alignment (RA for abbreviation), which forces different augmented views of current batch instances to main a consistent relationship with other instances. In order to perform RA effectively in existing contrastive learning framework, we design an alternating optimization algorithm where the relationship exploration step and alignment step are optimized respectively. In addition, we add an equilibrium constraint for RA to avoid the degenerate solution, and introduce the expansion handler to make it approximately satisfied in practice. In order to better capture the complex relationship among instances, we additionally propose Multi-Dimensional Relationship Alignment (MDRA for abbreviation), which aims to explore the relationship from multiple dimensions. In practice, we decompose the final high-dimensional feature space into a cartesian product of several low-dimensional subspaces and perform RA in each subspace respectively. We validate the effectiveness of our approach on multiple self-supervised learning benchmarks and get consistent improvements compared with current popular contrastive learning methods. On the most commonly used ImageNet linear evaluation protocol, our RA obtains significant improvements over other methods, our MDRA gets further improvements based on RA to achieve the best performance. The source code of our approach will be released soon.

A food poisoning caused by ST7 Staphylococcal aureus harboring sea gene in Hainan province, China.

ST7 Staphylococcus aureus is highly prevalent in humans, pigs, as well as food in China; however, staphylococcal food poisoning (SFP) caused by this ST type has rarely been reported. On May 13, 2017, an SFP outbreak caused by ST7 S. aureus strains occurred in two campuses of a kindergarten in Hainan Province, China. We investigated the genomic characteristics and phylogenetic analysis of ST7 SFP strains combined with the 91 ST7 food-borne strains from 12 provinces in China by performing whole-genome sequencing (WGS). There was clear phylogenetic clustering of seven SFP isolates. Six antibiotic genes including blaZ, ANT (4')-Ib, tetK, lnuA, norA, and lmrS were present in all SFP strains and also showed a higher prevalence rate in 91 food-borne strains. A multiple resistance plasmid pDC53285 was present in SFP strain DC53285. Among 27 enterotoxin genes, only sea and selx were found in all SFP strains. A ФSa3int prophage containing type A immune evasion cluster (sea, scn, sak, and chp) was identified in SFP strain. In conclusion, we concluded that this SFP event was caused by the contamination of cakes with ST7 S. aureus. This study indicated the potential risk of new emergencing ST7 clone for SFP.

Non-Homogeneous Haze Removal via Artificial Scene Prior and Bidimensional Graph Reasoning.

Due to the lack of natural scene and haze prior information, it is greatly challenging to completely remove the haze from a single image without distorting its visual content. Fortunately, the real-world haze usually presents non-homogeneous distribution, which provides us with many valuable clues in partial well-preserved regions. In this paper, we propose a Non-Homogeneous Haze Removal Network (NHRN) via artificial scene prior and bidimensional graph reasoning. Firstly, we employ the gamma correction iteratively to simulate artificial multiple shots under different exposure conditions, whose haze degrees are different and enrich the underlying scene prior. Secondly, beyond utilizing the local neighboring relationship, we build a bidimensional graph reasoning module to conduct non-local filtering in the spatial and channel dimensions of feature maps, which models their long-range dependency and propagates the natural scene prior between the well-preserved nodes and the nodes contaminated by haze. To the best of our knowledge, this is the first exploration to remove non-homogeneous haze via the graph reasoning based framework. We evaluate our method on different benchmark datasets. The results demonstrate that our method achieves superior performance over many state-of-the-art algorithms for both the single image dehazing and hazy image understanding tasks. The source code of the proposed NHRN is available on https://github.com/whrws/NHRNet.

Parametric Deformable Exponential Linear Units for deep neural networks.

Rectified activation units make an important contribution to the success of deep neural networks in many computer vision tasks. In this paper, we propose a Parametric Deformable Exponential Linear Unit (PDELU) and theoretically verify its effectiveness for improving the convergence speed of learning procedure. By means of flexible map shape, the proposed PDELU could push the mean value of activation responses closer to zero, which ensures the steepest descent in training a deep neural network. We verify the effectiveness of the proposed method in the image classification task. Extensive experiments on three classical databases (i.e., CIFAR-10, CIFAR-100, and ImageNet-2015) indicate that the proposed method leads to higher convergence speed and better accuracy when it is embedded into different CNN architectures (i.e., NIN, ResNet, WRN, and DenseNet). Meanwhile, the proposed PDELU outperforms many existing shape-specific activation functions (i.e., Maxout, ReLU, LeakyReLU, ELU, SELU, SoftPlus, Swish) and the shape-adaptive activation functions (i.e., APL, PReLU, MPELU, FReLU).

Group Maximum Differentiation Competition: Model Comparison with Few Samples.

In many science and engineering fields that require computational models to predict certain physical quantities, we are often faced with the selection of the best model under the constraint that only a small sample set can be physically measured. One such example is the prediction of human perception of visual quality, where sample images live in a high dimensional space with enormous content variations. We propose a new methodology for model comparison named group maximum differentiation (gMAD) competition. Given multiple computational models, gMAD maximizes the chances of falsifying a "defender" model using the rest models as "attackers". It exploits the sample space to find sample pairs that maximally differentiate the attackers while holding the defender fixed. Based on the results of the attacking-defending game, we introduce two measures, aggressiveness and resistance, to summarize the performance of each model at attacking other models and defending attacks from other models, respectively. We demonstrate the gMAD competition using three examples-image quality, image aesthetics, and streaming video quality-of-experience. Although these examples focus on visually discriminable quantities, the gMAD methodology can be extended to many other fields, and is especially useful when the sample space is large, the physical measurement is expensive and the cost of computational prediction is low.

Learning Interleaved Cascade of Shrinkage Fields for Joint Image Dehazing and Denoising.

Most existing image dehazing methods deteriorate to different extents when processing hazy inputs with noise. The main reason is that the commonly adopted two-step strategy tends to amplify noise in the inverse operation of division by the transmission. To address this problem, we learn an interleaved Cascade of Shrinkage Fields (CSF) to reduce noise in jointly recovering the transmission map and the scene radiance from a single hazy image. Specifically, an auxiliary shrinkage field (SF) model is integrated into each cascade of the proposed scheme to reduce undesirable artifacts during the transmission estimation. Different from conventional CSF, our learned SF models have special visual patterns, which facilitate the specific task of noise reduction in haze removal. Furthermore, a numerical algorithm is proposed to efficiently update the scene radiance and the transmission map in each cascade. Extensive experiments on synthetic and real-world data demonstrate that the proposed algorithm performs favorably against state-of-the-art dehazing methods on hazy and noisy images.

Accurate Transmission Estimation for Removing Haze and Noise from a Single Image.

Image noise usually causes depth-dependent visual artifacts in single image dehazing. Most existing dehazing methods exploit a two-step strategy in the restoration, which inevitably leads to inaccurate transmission maps and low-quality scene radiance for noisy and hazy inputs. To address these problems, we present a novel variational model for joint recovery of the transmission map and the scene radiance from a single image. In the model, we propose a transmission-aware non-local regularization to avoid noise amplification by adaptively suppressing noise and preserving fine details in the recovered image. Meanwhile, to improve the accuracy of transmission estimation, we introduce a semantic-guided regularization to smooth out the transmission map while keeping depth inconsistency at the boundaries of different objects. Furthermore, we design an alternating scheme to jointly optimize the transmission map and the scene radiance as well as the segmentation map. Extensive experiments on synthetic and real-world data demonstrate that the proposed algorithm performs favorably against state-of-the-art dehazing methods on noisy and hazy images.

A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment.

In the field of objective image quality assessment (IQA), Spearman's ρ and Kendall's τ, which straightforwardly assign uniform weights to all quality levels and assume that each pair of images is sortable, are the two most popular rank correlation indicators. These indicators can successfully measure the average accuracy of an IQA metric for ranking multiple processed images. However, two important perceptual properties are ignored. First, the sorting accuracy (SA) of high-quality images is usually more important than that of poor-quality images in many real-world applications, where only top-ranked images are pushed to the users. Second, due to the subjective uncertainty in making judgments, two perceptually similar images are usually barely sortable, and their ranks do not contribute to the evaluation of an IQA metric. To more accurately compare different IQA algorithms, in this paper, we explore a perceptually weighted rank correlation indicator, which rewards the capability of correctly ranking high-quality images and suppresses the attention towards insensitive rank mistakes. Specifically, we focus on activating a 'valid' pairwise comparison of images whose quality difference exceeds a given sensory threshold (ST). Meanwhile, each image pair is assigned a unique weight that is determined by both the quality level and rank deviation. By modifying the perception threshold, we can illustrate the sorting accuracy with a sophisticated SA-ST curve rather than a single rank correlation coefficient. The proposed indicator offers new insight into interpreting visual perception behavior. Furthermore, the applicability of our indicator is validated for recommending robust IQA metrics for both degraded and enhanced image data.

Feature adaptive co-segmentation by complexity awareness.

In this paper, we propose a novel feature adaptive co-segmentation method that can learn adaptive features of different image groups for accurate common objects segmentation. We also propose image complexity awareness for adaptive feature learning. In the proposed method, the original images are first ranked according to the image complexities that are measured by superpixel changing cue and object detection cue. Then, the unsupervised segments of the simple images are used to learn the adaptive features, which are achieved using an expectation-minimization algorithm combining l 1-regularized least squares optimization with the consideration of the confidence of the simple image segmentation accuracies and the fitness of the learned model. The error rate of the final co-segmentation is tested by the experiments on different image groups and verified to be lower than the existing state-of-the-art co-segmentation methods.

Effect of antler extract on corticosteroid-induced avascular necrosis of the femoral head in rats.

ETHNOPHARMACOLOGICAL RELEVANCE: Deer antler, traditionally used as a tonic and valuable drug in oriental medicine, has been considered to possess bone-strengthening activity and effectively used in bone diseases therapy. AIM OF THE STUDY: The present study was designed to investigate therapeutic effect of antler extract on avascular necrosis of the femoral head (ANFH) induced by corticosteroids in rats. MATERIALS AND METHODS: Rats were intragluteally injected with dexamethasone at 50mg/kg twice per week for 6 weeks to induce ANFH. Then the rats were treated with antler extract by oral gavage at 200mg/kg, 400mg/kg and 800 mg/kg once per day for 60 days. The concentration of hydroxyproline and hexosamine in serum was measured and the ultrastructure of femoral head was examined. In vitro, effect of the drug-containing serum of antler extract on proliferation and differentiation of primary osteoblasts were investigated by MIT assay, ALP activity assay and cell cycle analysis. RESULTS: After treatment with antler extract, the degree of necrosis induced by dexamethasone was significantly reduced, hydroxyproline was significantly decreased, and hexosamine and the ratio of hexosamine/hydroxyproline were significantly increased. The drug-containing serum of antler extract promotes osteoblastic proliferation through regulation of cell cycle progression. CONCLUSIONS: The results suggest that antler extract has a positive curative effect on ANFH by promoting osteoblastic proliferation.

[Effects of human engineering activities on permafrost active layer and its environment in northern Qinghai-Tibetan plateau].

With disturbed and undisturbed belts during the construction of Qinghai-Tibet highway as test objectives, this paper studied the effects of human engineering activities on the permafrost ecosystem in northern Qinghai-Tibetan plateau. The results showed that the thickness of permafrost active layer was smaller in disturbed than in undisturbed belt, and decreased with increasing altitude in undisturbed belt while no definite pattern was observed in disturbed belt. Different vegetation types had different effects on the thickness of permafrost active layer, being decreased in the order of steppe > shrub > meadow. In the two belts, altitude was the main factor affecting the vertical distribution of soil moisture, but vegetation type was also an important affecting factor if the altitude was similar. Due to the human engineering activities, soil temperature in summer was lower in disturbed than in undisturbed belt.