Rolling bearings fault diagnosis based on two-stage signal fusion and deep multi-scale multi-sensor network.

In order to realize high-precision diagnosis of bearings faults in a multi-sensor detection environment, a fault diagnosis method based on two-stage signal fusion and deep multi-scale multi-sensor networks is proposed. Firstly, the signals are decomposed and fused using weighted empirical wavelet transform to enhance weak features and reduce noise. Secondly, an improved random weighting algorithm is proposed to perform a second weighted fusion of the signals to reduce the total mean square error. The fused signals are input into the deep multi-scale residual network, the feature information of different convolutional layers is extracted through dilated convolution, and the features are fused using pyramid theory. Finally, the bearings states are classified according to the fusion features. Experiment results show the effectiveness and superiority of this method.

GC-WIR : 3D global coordinate attention wide inverted ResNet network for pulmonary nodules classification.

PURPOSE: Currently, deep learning methods for the classification of benign and malignant lung nodules encounter challenges encompassing intricate and unstable algorithmic models, limited data adaptability, and an abundance of model parameters.To tackle these concerns, this investigation introduces a novel approach: the 3D Global Coordinated Attention Wide Inverted ResNet Network (GC-WIR). This network aims to achieve precise classification of benign and malignant pulmonary nodules, leveraging its merits of heightened efficiency, parsimonious parameterization, and robust stability. METHODS: Within this framework, a 3D Global Coordinate Attention Mechanism (3D GCA) is designed to compute the features of the input images by converting 3D channel information and multi-dimensional positional cues. By encompassing both global channel details and spatial positional cues, this approach maintains a judicious balance between flexibility and computational efficiency. Furthermore, the GC-WIR architecture incorporates a 3D Wide Inverted Residual Network (3D WIRN), which augments feature computation by expanding input channels. This augmentation mitigates information loss during feature extraction, expedites model convergence, and concurrently enhances performance. The utilization of the inverted residual structure imbues the model with heightened stability. RESULTS: Empirical validation of the GC-WIR method is performed on the LUNA 16 dataset, yielding predictions that surpass those generated by previous models. This novel approach achieves an impressive accuracy rate of 94.32%, coupled with a specificity of 93.69%. Notably, the model's parameter count remains modest at 5.76M, affording optimal classification accuracy. CONCLUSION: Furthermore, experimental results unequivocally demonstrate that, even under stringent computational constraints, GC-WIR outperforms alternative deep learning methodologies, establishing a new benchmark in performance.

Compressor fault diagnosis and result visualization based on fusion of vision transformer and improved residual network.

Compressors are important production equipment in the petrochemical industry, and the accuracy of their fault diagnosis is critical. In order to detect and diagnose compressor equipment faults in a timely manner, this paper constructs a deep residual shrinkage visual network (DRS-ViT). The network comprises a modified residual network (ResNet) and a vision transformer (ViT). The obtained compressor vibration signals were transformed into gram angle sum field (GASF) plots using gram angle field (GAF). The resulting image is the passed through a modified ResNet network to extract initial features. The extracted feature images are subsequently input into the ViT model for fault classification. The experimental results demonstrate that the fault diagnosis accuracy achieved by the DRS-ViT model is 99.5 %. The visualization of the model indicates that it can effectively identify the fault points. The validity and robustness of the DRS-ViT model are confirmed through comparison and analysis with various models.

Hyperspectral Anomaly Detection Based on Spectral Similarity Variability Feature.

In the traditional method for hyperspectral anomaly detection, spectral feature mapping is used to map hyperspectral data to a high-level feature space to make features more easily distinguishable between different features. However, the uncertainty in the mapping direction makes the mapped features ineffective in distinguishing anomalous targets from the background. To address this problem, a hyperspectral anomaly detection algorithm based on the spectral similarity variability feature (SSVF) is proposed. First, the high-dimensional similar neighborhoods are fused into similar features using AE networks, and then the SSVF are obtained using residual autoencoder. Finally, the final detection of SSVF was obtained using Reed and Xiaoli (RX) detectors. Compared with other comparison algorithms with the highest accuracy, the overall detection accuracy (AUCODP) of the SSVFRX algorithm is increased by 0.2106. The experimental results show that SSVF has great advantages in both highlighting anomalous targets and improving separability between different ground objects.

[A deep transfer learning approach for cross-subject recognition of mental tasks based on functional near-infrared spectroscopy].

In the field of brain-computer interfaces (BCIs) based on functional near-infrared spectroscopy (fNIRS), traditional subject-specific decoding methods suffer from the limitations of long calibration time and low cross-subject generalizability, which restricts the promotion and application of BCI systems in daily life and clinic. To address the above dilemma, this study proposes a novel deep transfer learning approach that combines the revised inception-residual network (rIRN) model and the model-based transfer learning (TL) strategy, referred to as TL-rIRN. This study performed cross-subject recognition experiments on mental arithmetic (MA) and mental singing (MS) tasks to validate the effectiveness and superiority of the TL-rIRN approach. The results show that the TL-rIRN significantly shortens the calibration time, reduces the training time of the target model and the consumption of computational resources, and dramatically enhances the cross-subject decoding performance compared to subject-specific decoding methods and other deep transfer learning methods. To sum up, this study provides a basis for the selection of cross-subject, cross-task, and real-time decoding algorithms for fNIRS-BCI systems, which has potential applications in constructing a convenient and universal BCI system.

Res-GCN: Identification of protein phosphorylation sites using graph convolutional network and residual network.

An essential post-translational modification, phosphorylation is intimately related with a wide range of biological activities. The advancement of effective computational methods for correctly recognizing phosphorylation sites is important for in-depth understanding of various physiological phenomena. However, the traditional method of identifying phosphorylation sites experimentally is time-consuming and laborious, which makes it difficult to meet the processing demands of today's big data. This research proposes the use of a novel model, Res-GCN, to recognize the phosphorylation sites of SARS-CoV-2. Firstly, eight feature extraction strategies are utilized to digitize the protein sequence from multiple viewpoints, including amino acid property encodings (AAindex), pseudo-amino acid composition (PseAAC), adapted normal distribution bi-profile Bayes (ANBPB), dipeptide composition (DC), binary encoding (BE), enhanced amino acid composition (EAAC), Word2Vec, and BLOSUM62 matrices. Secondly, elastic net is utilized to eliminate redundant data in the fused matrix. Finally, a combination of graph convolutional network (GCN) and residual network (ResNet) is used to classify the phosphorylated sites and output predictions using a fully connected layer (FC). The performance of Res-GCN is tested by 5-fold cross-validation and independent testing, and excellent results are obtained on S/T and Y datasets. This demonstrates that the Res-GCN model exhibits exceptional predictive performance and generalizability.

A novel diagnosis method combined dual-channel SE-ResNet with expert features for inter-patient heartbeat classification.

As the number of patients with cardiovascular diseases (CVDs) increases annually, a reliable and automated system for detecting electrocardiogram (ECG) abnormalities is becoming increasingly essential. Scholars have developed numerous methods of arrhythmia classification using machine learning or deep learning. However, the issue of low classification rates of individual classes in inter-patient heartbeat classification remains a challenge. This study proposes a method for inter-patient heartbeat classification by fusing dual-channel squeeze-and-excitation residual neural networks (SE-ResNet) and expert features. In the preprocessing stage, ECG heartbeats extracted from both leads of ECG signals are filtered and normalized. Additionally, nine features representing waveform morphology and heartbeat contextual information are selected to be fused with the deep neural networks. Using different filter and kernel sizes for each block, the SE-residual block-based model can effectively learn long-term features between heartbeats. The divided ECG heartbeats and extracted features are then input to the improved SE-ResNet for training and testing according to the inter-patient scheme. The focal loss is utilized to handle the heartbeat of the imbalance category. The proposed arrhythmia classification method is evaluated on three open-source databases, and it achieved an overall F1-score of 83.39 % in the MIT-BIH database. This system can be applied in the scenario of daily monitoring of ECG and plays a significant role in diagnosing arrhythmias.

Drug repositioning based on residual attention network and free multiscale adversarial training.

BACKGROUND: Conducting traditional wet experiments to guide drug development is an expensive, time-consuming and risky process. Analyzing drug function and repositioning plays a key role in identifying new therapeutic potential of approved drugs and discovering therapeutic approaches for untreated diseases. Exploring drug-disease associations has far-reaching implications for identifying disease pathogenesis and treatment. However, reliable detection of drug-disease relationships via traditional methods is costly and slow. Therefore, investigations into computational methods for predicting drug-disease associations are currently needed. RESULTS: This paper presents a novel drug-disease association prediction method, RAFGAE. First, RAFGAE integrates known associations between diseases and drugs into a bipartite network. Second, RAFGAE designs the Re_GAT framework, which includes multilayer graph attention networks (GATs) and two residual networks. The multilayer GATs are utilized for learning the node embeddings, which is achieved by aggregating information from multihop neighbors. The two residual networks are used to alleviate the deep network oversmoothing problem, and an attention mechanism is introduced to combine the node embeddings from different attention layers. Third, two graph autoencoders (GAEs) with collaborative training are constructed to simulate label propagation to predict potential associations. On this basis, free multiscale adversarial training (FMAT) is introduced. FMAT enhances node feature quality through small gradient adversarial perturbation iterations, improving the prediction performance. Finally, tenfold cross-validations on two benchmark datasets show that RAFGAE outperforms current methods. In addition, case studies have confirmed that RAFGAE can detect novel drug-disease associations. CONCLUSIONS: The comprehensive experimental results validate the utility and accuracy of RAFGAE. We believe that this method may serve as an excellent predictor for identifying unobserved disease-drug associations.

Evaluation of Denoising Performance of ResNet Deep Learning Model for Ultrasound Images Corresponding to Two Frequency Parameters.

Ultrasound imaging is widely used for accurate diagnosis due to its noninvasive nature and the absence of radiation exposure, which is achieved by controlling the scan frequency. In addition, Gaussian and speckle noises degrade image quality. To address this issue, filtering techniques are typically used in the spatial domain. Recently, deep learning models have been increasingly applied in the field of medical imaging. In this study, we evaluated the effectiveness of a convolutional neural network-based residual network (ResNet) deep learning model for noise reduction when Gaussian and speckle noises were present. We compared the results with those obtained from conventional filtering techniques. A dataset of 500 images was prepared, and Gaussian and speckle noises were added to create noisy input images. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. The ResNet deep learning model, comprising 16 residual blocks, was trained using optimized hyperparameters, including the learning rate, optimization function, and loss function. For quantitative analysis, we calculated the normalized noise power spectrum, peak signal-to-noise ratio, and root mean square error. Our findings showed that the ResNet deep learning model exhibited superior noise reduction performance to median, Wiener, and median-modified Wiener filter algorithms.

A Multi-Scale-Enhanced YOLO-V5 Model for Detecting Small Objects in Remote Sensing Image Information.

As a typical component of remote sensing signals, remote sensing image (RSI) information plays a strong role in showing macro, dynamic and accurate information on the earth's surface and environment, which is critical to many application fields. One of the core technologies is the object detection (OD) of RSI signals (RSISs). The majority of existing OD algorithms only consider medium and large objects, regardless of small-object detection, resulting in an unsatisfactory performance in detection precision and the miss rate of small objects. To boost the overall OD performance of RSISs, an improved detection framework, I-YOLO-V5, was proposed for OD in high-altitude RSISs. Firstly, the idea of a residual network is employed to construct a new residual unit to achieve the purpose of improving the network feature extraction. Then, to avoid the gradient fading of the network, densely connected networks are integrated into the structure of the algorithm. Meanwhile, a fourth detection layer is employed in the algorithm structure in order to reduce the deficiency of small-object detection in RSISs in complex environments, and its effectiveness is verified. The experimental results confirm that, compared with existing advanced OD algorithms, the average accuracy of the proposed I-YOLO-V5 is improved by 15.4%, and the miss rate is reduced by 46.8% on the RSOD dataset.

Residual-Based Multi-Stage Deep Learning Framework for Computer-Aided Alzheimer's Disease Detection.

Alzheimer's Disease (AD) poses a significant health risk globally, particularly among the elderly population. Recent studies underscore its prevalence, with over 50% of elderly Japanese facing a lifetime risk of dementia, primarily attributed to AD. As the most prevalent form of dementia, AD gradually erodes brain cells, leading to severe neurological decline. In this scenario, it is important to develop an automatic AD-detection system, and many researchers have been working to develop an AD-detection system by taking advantage of the advancement of deep learning (DL) techniques, which have shown promising results in various domains, including medical image analysis. However, existing approaches for AD detection often suffer from limited performance due to the complexities associated with training hierarchical convolutional neural networks (CNNs). In this paper, we introduce a novel multi-stage deep neural network architecture based on residual functions to address the limitations of existing AD-detection approaches. Inspired by the success of residual networks (ResNets) in image-classification tasks, our proposed system comprises five stages, each explicitly formulated to enhance feature effectiveness while maintaining model depth. Following feature extraction, a deep learning-based feature-selection module is applied to mitigate overfitting, incorporating batch normalization, dropout and fully connected layers. Subsequently, machine learning (ML)-based classification algorithms, including Support Vector Machines (SVM), Random Forest (RF) and SoftMax, are employed for classification tasks. Comprehensive evaluations conducted on three benchmark datasets, namely ADNI1: Complete 1Yr 1.5T, MIRAID and OASIS Kaggle, demonstrate the efficacy of our proposed model. Impressively, our model achieves accuracy rates of 99.47%, 99.10% and 99.70% for ADNI1: Complete 1Yr 1.5T, MIRAID and OASIS datasets, respectively, outperforming existing systems in binary class problems. Our proposed model represents a significant advancement in the AD-analysis domain.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals.

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Surface Defect Detection of Aluminum Profiles Based on Multiscale and Self-Attention Mechanisms.

To address the various challenges in aluminum surface defect detection, such as multiscale intricacies, sensitivity to lighting variations, occlusion, and noise, this study proposes the AluDef-ClassNet model. Firstly, a Gaussian difference pyramid is utilized to capture multiscale image features. Secondly, a self-attention mechanism is introduced to enhance feature representation. Additionally, an improved residual network structure incorporating dilated convolutions is adopted to increase the receptive field, thereby enhancing the network's ability to learn from extensive information. A small-scale dataset of high-quality aluminum surface defect images is acquired using a CCD camera. To better tackle the challenges in surface defect detection, advanced deep learning techniques and data augmentation strategies are employed. To address the difficulty of data labeling, a transfer learning approach based on fine-tuning is utilized, leveraging prior knowledge to enhance the efficiency and accuracy of model training. In dataset testing, our model achieved a classification accuracy of 97.6%, demonstrating significant advantages over other classification models.

DeepOCR: A multi-species deep-learning framework for accurate identification of open chromatin regions in livestock.

A wealth of experimental evidence has suggested that open chromatin regions (OCRs) are involved in many critical biological activities, such as DNA replication, enhancer activity, and gene transcription. Accurately identifying OCRs in livestock species can provide critical insights into the distribution and characteristics of OCRs for disease treatment in livestock, thereby improving animal welfare. However, most current machine-learning methods for OCR prediction were originally designed for a limited number of model organisms, such as humans and some model organisms, and thus their performance on non-model organisms, specifically livestock, is often unsatisfactory. To bridge this gap, we propose DeepOCR, a lightweight depth-separable residual network model for predicting OCRs in livestock, including chicken, cattle, and sheep. DeepOCR integrates a single convolution layer and two improved residue structure blocks to extract and learn important features from the input DNA sequences. A fully connected layer was also employed to further process the extracted features and improve the robustness of the entire network. Our benchmarking experiments demonstrated superior prediction performance of DeepOCR compared to state-of-the-art approaches on testing datasets of the three species. The source code of DeepOCR is freely available for academic purposes at https://github.com/jasonzhao371/DeepOCR/. We anticipate DeepOCR servers as a practical and reliable computational tool for OCR-related studies in livestock species.

Meta learning based residual network for industrial production quality prediction with limited data.

Due to the challenge of collecting a substantial amount of production-quality data in real-world industrial settings, the implementation of production quality prediction models based on deep learning is not effective. To achieve the goal of predicting production quality with limited data and address the issue of model degradation in the training process of deep learning networks, we propose Meta-Learning based on Residual Network (MLRN) models for production quality prediction with limited data. Firstly, the MLRN model is trained on a variety of learning tasks to acquire knowledge for predicting production quality. Furthermore, to obtain more features with limited data and avoid the issues of gradient disappearing or exploding in deep network training, the enhanced residual network with the effective channel attention (ECA) mechanism is chosen as the basic network structure of MLRN. Additionally, a multi-batch and multi-task data input approach is implemented to prevent overfitting. Finally, the availability of the MLRN model is demonstrated by comparing it with other models using both numerical and graphical datasets.

Optical frequency multiplication using residual network with random forest regression.

In this work, we present a method for optical frequency multiplication utilizing a hybrid deep learning approach that integrates the Residual Network (ResNet) with the Random Forest Regression (RFR) algorithm. Three different frequency multiplication modulation schemes are adopted to illustrate the method, which can obtain suitable parameters for these schemes. Based on the parameters predicted by the algorithm, the 8-tupling, 12-tupling, and 16-tupling mm-wave signals are generated by numerical simulation. The simulation results show that for 8-tupling frequency multiplication, an OSSR (optical sideband suppression ratio) is 30.73 dB and an RFSSR (radio frequency spurious suppression ratio) of 80 GHz is 42.29 dB. For 12-tupling frequency multiplication, the OSSR is 30.09 dB, and the RFSSR of the 120 GHz mm wave is 36.21 dB. For generating 16-tupling frequency mm-wave, an OSSR of 29.86 dB and an RFSSR of 34.52 dB are obtained. In addition, the impact of amplitude fluctuation and bias voltage drift on the quality of mm-wave signals is also studied.

Residual network improves the prediction accuracy of genomic selection.

Genetic improvement of complex traits in animal and plant breeding depends on the efficient and accurate estimation of breeding values. Deep learning methods have been shown to be not superior over traditional genomic selection (GS) methods, partially due to the degradation problem (i.e. with the increase of the model depth, the performance of the deeper model deteriorates). Since the deep learning method residual network (ResNet) is designed to solve gradient degradation, we examined its performance and factors related to its prediction accuracy in GS. Here we compared the prediction accuracy of conventional genomic best linear unbiased prediction, Bayesian methods (BayesA, BayesB, BayesC, and Bayesian Lasso), and two deep learning methods, convolutional neural network and ResNet, on three datasets (wheat, simulated and real pig data). ResNet outperformed other methods in both Pearson's correlation coefficient (PCC) and mean squared error (MSE) on the wheat and simulated data. For the pig backfat depth trait, ResNet still had the lowest MSE, whereas Bayesian Lasso had the highest PCC. We further clustered the pig data into four groups and, on one separated group, ResNet had the highest prediction accuracy (both PCC and MSE). Transfer learning was adopted and capable of enhancing the performance of both convolutional neural network and ResNet. Taken together, our findings indicate that ResNet could improve GS prediction accuracy, affected potentially by factors such as the genetic architecture of complex traits, data volume, and heterogeneity.

A multi-scale feature fusion neural network for multi-class disease classification on the maize leaf images.

Maize is a globally important cereal crop, however, maize leaf disease is one of the most common and devastating diseases that afflict it. Artificial intelligence methods face challenges in identifying and classifying maize leaf diseases due to variations in image quality, similarity among diseases, disease severity, limited dataset availability, and limited interpretability. To address these challenges, we propose a residual-based multi-scale network (MResNet) for classifying multi-type maize leaf diseases from maize images. MResNet consists of two residual subnets with different scales, enabling the model to detect diseases in maize leaf images at different scales. We further utilize a hybrid feature weight optimization method to optimize and fuse the feature mapping weights of two subnets. We validate MResNet on a maize leaf diseases dataset. MResNet achieves 97.45% accuracy. The performance of MResNet surpasses other state-of-the-art methods. Various experiments and two additional datasets confirm the generalization performance of our model. Furthermore, thermodynamic diagram analysis increases the interpretability of the model. This study provides technical support for the disease classification of agricultural plants.

Intelligent Fault Diagnosis of Rolling Bearing Based on Gramian Angular Difference Field and Improved Dual Attention Residual Network.

With the rapid development of smart manufacturing, data-driven deep learning (DL) methods are widely used for bearing fault diagnosis. Aiming at the problem of model training crashes when data are imbalanced and the difficulty of traditional signal analysis methods in effectively extracting fault features, this paper proposes an intelligent fault diagnosis method of rolling bearings based on Gramian Angular Difference Field (GADF) and Improved Dual Attention Residual Network (IDARN). The original vibration signals are encoded as 2D-GADF feature images for network input; the residual structures will incorporate dual attention mechanism to enhance the integration ability of the features, while the group normalization (GN) method is introduced to overcome the bias caused by data discrepancies; and then the model is trained to complete the classification of faults. In order to verify the superiority of the proposed method, the data obtained from Case Western Reserve University (CWRU) bearing data and bearing fault experimental equipment were compared with other popular DL methods, and the proposed model performed optimally. The method eventually achieved an average identification accuracy of 99.2% and 97.9% on two different types of datasets, respectively.

Classification Method of ECG Signals Based on RANet.

BACKGROUND: Electrocardiograms (ECG) are an important source of information on human heart health and are widely used to detect different types of arrhythmias. OBJECTIVE: With the advancement of deep learning, end-to-end ECG classification models based on neural networks have been developed. However, deeper network layers lead to gradient vanishing. Moreover, different channels and periods of an ECG signal hold varying significance for identifying different types of ECG abnormalities. METHODS: To solve these two problems, an ECG classification method based on a residual attention neural network is proposed in this paper. The residual network (ResNet) is used to solve the gradient vanishing problem. Moreover, it has fewer model parameters, and its structure is simpler. An attention mechanism is added to focus on key information, integrate channel features, and improve voting methods to alleviate the problem of data imbalance. RESULTS: Experiments and verifications are conducted using the PhysioNet/CinC Challenge 2017 dataset. The average F1 value is 0.817, which is 0.064 higher than that for the ResNet model. Compared with the mainstream methods, the performance is excellent.