DeepGpgs: a novel deep learning framework for predicting arginine methylation sites combined with Gaussian prior and gated self-attention mechanism.

Protein arginine methylation is an important posttranslational modification (PTM) associated with protein functional diversity and pathological conditions including cancer. Identification of methylation binding sites facilitates a better understanding of the molecular function of proteins. Recent developments in the field of deep neural networks have led to a proliferation of deep learning-based methylation identification studies because of their fast and accurate prediction. In this paper, we propose DeepGpgs, an advanced deep learning model incorporating Gaussian prior and gated attention mechanism. We introduce a residual network channel to extract the evolutionary information of proteins. Then we combine the adaptive embedding with bidirectional long short-term memory networks to form a context-shared encoder layer. A gated multi-head attention mechanism is followed to obtain the global information about the sequence. A Gaussian prior is injected into the sequence to assist in predicting PTMs. We also propose a weighted joint loss function to alleviate the false negative problem. We empirically show that DeepGpgs improves Matthews correlation coefficient by 6.3% on the arginine methylation independent test set compared with the existing state-of-the-art methylation site prediction methods. Furthermore, DeepGpgs has good robustness in phosphorylation site prediction of SARS-CoV-2, which indicates that DeepGpgs has good transferability and the potential to be extended to other modification sites prediction. The open-source code and data of the DeepGpgs can be obtained from https://github.com/saizhou1/DeepGpgs.

Drug repositioning based on residual attention network and free multiscale adversarial training.

BACKGROUND: Conducting traditional wet experiments to guide drug development is an expensive, time-consuming and risky process. Analyzing drug function and repositioning plays a key role in identifying new therapeutic potential of approved drugs and discovering therapeutic approaches for untreated diseases. Exploring drug-disease associations has far-reaching implications for identifying disease pathogenesis and treatment. However, reliable detection of drug-disease relationships via traditional methods is costly and slow. Therefore, investigations into computational methods for predicting drug-disease associations are currently needed. RESULTS: This paper presents a novel drug-disease association prediction method, RAFGAE. First, RAFGAE integrates known associations between diseases and drugs into a bipartite network. Second, RAFGAE designs the Re_GAT framework, which includes multilayer graph attention networks (GATs) and two residual networks. The multilayer GATs are utilized for learning the node embeddings, which is achieved by aggregating information from multihop neighbors. The two residual networks are used to alleviate the deep network oversmoothing problem, and an attention mechanism is introduced to combine the node embeddings from different attention layers. Third, two graph autoencoders (GAEs) with collaborative training are constructed to simulate label propagation to predict potential associations. On this basis, free multiscale adversarial training (FMAT) is introduced. FMAT enhances node feature quality through small gradient adversarial perturbation iterations, improving the prediction performance. Finally, tenfold cross-validations on two benchmark datasets show that RAFGAE outperforms current methods. In addition, case studies have confirmed that RAFGAE can detect novel drug-disease associations. CONCLUSIONS: The comprehensive experimental results validate the utility and accuracy of RAFGAE. We believe that this method may serve as an excellent predictor for identifying unobserved disease-drug associations.

ICSDA: a multi-modal deep learning model to predict breast cancer recurrence and metastasis risk by integrating pathological, clinical and gene expression data.

Breast cancer patients often have recurrence and metastasis after surgery. Predicting the risk of recurrence and metastasis for a breast cancer patient is essential for the development of precision treatment. In this study, we proposed a novel multi-modal deep learning prediction model by integrating hematoxylin & eosin (H&E)-stained histopathological images, clinical information and gene expression data. Specifically, we segmented tumor regions in H&E into image blocks (256 × 256 pixels) and encoded each image block into a 1D feature vector using a deep neural network. Then, the attention module scored each area of the H&E-stained images and combined image features with clinical and gene expression data to predict the risk of recurrence and metastasis for each patient. To test the model, we downloaded all 196 breast cancer samples from the Cancer Genome Atlas with clinical, gene expression and H&E information simultaneously available. The samples were then divided into the training and testing sets with a ratio of 7: 3, in which the distributions of the samples were kept between the two datasets by hierarchical sampling. The multi-modal model achieved an area-under-the-curve value of 0.75 on the testing set better than those based solely on H&E image, sequencing data and clinical data, respectively. This study might have clinical significance in identifying high-risk breast cancer patients, who may benefit from postoperative adjuvant treatment.

Multi-model predictive analysis of RNA solvent accessibility based on modified residual attention mechanism.

Predicting RNA solvent accessibility using only primary sequence data can be regarded as sequence-based prediction work. Currently, the established studies for sequence-based RNA solvent accessibility prediction are limited due to the available number of datasets and black box prediction. To improve these issues, we first expanded the available RNA structures and then developed a sequence-based model using modified attention layers with different receptive fields to conform to the stem-loop structure of RNA chains. We measured the improvement with an extended dataset and further explored the model's interpretability by analysing the model structures, attention values and hyperparameters. Finally, we found that the developed model regarded the pieces of a sequence as templates during the training process. This work will be helpful for researchers who would like to build RNA attribute prediction models using deep learning in the future.

MAResNet: predicting transcription factor binding sites by combining multi-scale bottom-up and top-down attention and residual network.

Accurate identification of transcription factor binding sites is of great significance in understanding gene expression, biological development and drug design. Although a variety of methods based on deep-learning models and large-scale data have been developed to predict transcription factor binding sites in DNA sequences, there is room for further improvement in prediction performance. In addition, effective interpretation of deep-learning models is greatly desirable. Here we present MAResNet, a new deep-learning method, for predicting transcription factor binding sites on 690 ChIP-seq datasets. More specifically, MAResNet combines the bottom-up and top-down attention mechanisms and a state-of-the-art feed-forward network (ResNet), which is constructed by stacking attention modules that generate attention-aware features. In particular, the multi-scale attention mechanism is utilized at the first stage to extract rich and representative sequence features. We further discuss the attention-aware features learned from different attention modules in accordance with the changes as the layers go deeper. The features learned by MAResNet are also visualized through the TMAP tool to illustrate that the method can extract the unique characteristics of transcription factor binding sites. The performance of MAResNet is extensively tested on 690 test subsets with an average AUC of 0.927, which is higher than that of the current state-of-the-art methods. Overall, this study provides a new and useful framework for the prediction of transcription factor binding sites by combining the funnel attention modules with the residual network.

Fusion of laser-induced breakdown spectroscopy technology and deep learning: a new method to identify malignant and benign lung tumors with high accuracy.

Precisely distinguishing between malignant and benign lung tumors is pivotal for suggesting therapeutic strategies and enhancing prognosis, yet this differentiation remains a daunting task. The growth rates, metastatic potentials, and prognoses of benign and malignant tumors differ significantly. Developing specialized treatment protocols tailored to various tumor types is essential for enhancing patient survival outcomes. Employing laser-induced breakdown spectroscopy (LIBS) in conjunction with a deep learning methodology, we attained a high-precision differential diagnosis of malignant and benign lung tumors. First, LIBS spectra of malignant tumors, benign tumors, and normal tissues were collected. The spectra were preprocessed and Z score normalized. Then, the intensities of the Mg II 279.6, Mg I 285.2, Ca II 393.4, Cu II 518.3, and Na I 589.6 nm lines were analyzed in the spectra of the three tissues. The analytical results show that the elemental lines have different contents in the three tissues and can be used as a basis for distinguishing between the three tissues. Finally, the RF-1D ResNet model was constructed by combining the feature importance assessment method of random forest (RF) and one-dimensional residual network (1D ResNet). The classification accuracy, precision, sensitivity, and specificity of the RF-1D ResNet model were 91.1%, 91.6%, 91.3%, and 91.3%, respectively. And the model demonstrates superior performance with an area under the curve (AUC) value of 0.99. The above results show that combining LIBS with deep learning is an effective way to diagnose malignant and benign tumors.

Residual network improves the prediction accuracy of genomic selection.

Genetic improvement of complex traits in animal and plant breeding depends on the efficient and accurate estimation of breeding values. Deep learning methods have been shown to be not superior over traditional genomic selection (GS) methods, partially due to the degradation problem (i.e. with the increase of the model depth, the performance of the deeper model deteriorates). Since the deep learning method residual network (ResNet) is designed to solve gradient degradation, we examined its performance and factors related to its prediction accuracy in GS. Here we compared the prediction accuracy of conventional genomic best linear unbiased prediction, Bayesian methods (BayesA, BayesB, BayesC, and Bayesian Lasso), and two deep learning methods, convolutional neural network and ResNet, on three datasets (wheat, simulated and real pig data). ResNet outperformed other methods in both Pearson's correlation coefficient (PCC) and mean squared error (MSE) on the wheat and simulated data. For the pig backfat depth trait, ResNet still had the lowest MSE, whereas Bayesian Lasso had the highest PCC. We further clustered the pig data into four groups and, on one separated group, ResNet had the highest prediction accuracy (both PCC and MSE). Transfer learning was adopted and capable of enhancing the performance of both convolutional neural network and ResNet. Taken together, our findings indicate that ResNet could improve GS prediction accuracy, affected potentially by factors such as the genetic architecture of complex traits, data volume, and heterogeneity.

GC-WIR : 3D global coordinate attention wide inverted ResNet network for pulmonary nodules classification.

PURPOSE: Currently, deep learning methods for the classification of benign and malignant lung nodules encounter challenges encompassing intricate and unstable algorithmic models, limited data adaptability, and an abundance of model parameters.To tackle these concerns, this investigation introduces a novel approach: the 3D Global Coordinated Attention Wide Inverted ResNet Network (GC-WIR). This network aims to achieve precise classification of benign and malignant pulmonary nodules, leveraging its merits of heightened efficiency, parsimonious parameterization, and robust stability. METHODS: Within this framework, a 3D Global Coordinate Attention Mechanism (3D GCA) is designed to compute the features of the input images by converting 3D channel information and multi-dimensional positional cues. By encompassing both global channel details and spatial positional cues, this approach maintains a judicious balance between flexibility and computational efficiency. Furthermore, the GC-WIR architecture incorporates a 3D Wide Inverted Residual Network (3D WIRN), which augments feature computation by expanding input channels. This augmentation mitigates information loss during feature extraction, expedites model convergence, and concurrently enhances performance. The utilization of the inverted residual structure imbues the model with heightened stability. RESULTS: Empirical validation of the GC-WIR method is performed on the LUNA 16 dataset, yielding predictions that surpass those generated by previous models. This novel approach achieves an impressive accuracy rate of 94.32%, coupled with a specificity of 93.69%. Notably, the model's parameter count remains modest at 5.76M, affording optimal classification accuracy. CONCLUSION: Furthermore, experimental results unequivocally demonstrate that, even under stringent computational constraints, GC-WIR outperforms alternative deep learning methodologies, establishing a new benchmark in performance.

A Novel Fault Diagnosis Strategy for Diaphragm Pumps Based on Signal Demodulation and PCA-ResNet.

The efficient and accurate identification of diaphragm pump faults is crucial for ensuring smooth system operation and reducing energy consumption. The structure of diaphragm pumps is complex and using traditional fault diagnosis strategies to extract typical fault characteristics is difficult, facing the risk of model overfitting and high diagnostic costs. In response to the shortcomings of traditional methods, this study innovatively combines signal demodulation methods with residual networks (ResNet) to propose an efficient fault diagnosis strategy for diaphragm pumps. By using a demodulation method based on principal component analysis (PCA), the vibration signal demodulation spectrum of the fault condition is obtained, the typical fault characteristics of the diaphragm pump are accurately extracted, and the sample features are enhanced, reducing the cost of fault diagnosis. Afterward, the PCA-ResNet model is applied to the fault diagnosis of diaphragm pumps. A reasonable model structure and advanced residual block design can effectively reduce the risk of model overfitting and improve the accuracy of fault diagnosis. Compared with the visual geometry group (VGG) 16, VGG19, ResNet50, and autoencoder models, the proposed model has improved accuracy by 35.89%, 80.27%, 2.72%, and 6.12%. Simultaneously, it has higher operational efficiency and lower loss rate, solving the problem of diagnostic lag in practical engineering. Finally, a model optimization strategy is proposed through model evaluation metrics and testing. The reasonable parameter range of the model is obtained, providing a reference and guarantee for further optimization of the model.

Atrial Fibrillation Detection with Single-Lead Electrocardiogram Based on Temporal Convolutional Network-ResNet.

Atrial fibrillation, one of the most common persistent cardiac arrhythmias globally, is known for its rapid and irregular atrial rhythms. This study integrates the temporal convolutional network (TCN) and residual network (ResNet) frameworks to effectively classify atrial fibrillation in single-lead ECGs, thereby enhancing the application of neural networks in this field. Our model demonstrated significant success in detecting atrial fibrillation, with experimental results showing an accuracy rate of 97% and an F1 score of 87%. These figures indicate the model's exceptional performance in identifying both majority and minority classes, reflecting its balanced and accurate classification capability. This research offers new perspectives and tools for diagnosis and treatment in cardiology, grounded in advanced neural network technology.

Machine Tool Wear Prediction Technology Based on Multi-Sensor Information Fusion.

The intelligent monitoring of cutting tools used in the manufacturing industry is steadily becoming more convenient. To accurately predict the state of tools and tool breakages, this study proposes a tool wear prediction technique based on multi-sensor information fusion. First, the vibrational, current, and cutting force signals transmitted during the machining process were collected, and the features were extracted. Next, the Kalman filtering algorithm was used for feature fusion, and a predictive model for tool wear was constructed by combining the ResNet and long short-term memory (LSTM) models (called ResNet-LSTM). Experimental data for thin-walled parts obtained under various machining conditions were utilized to monitor the changes in tool conditions. A comparison between the ResNet and LSTM tool wear prediction models indicated that the proposed ResNet-LSTM model significantly improved the prediction accuracy compared to the individual LSTM and ResNet models. Moreover, ResNet-LSTM exhibited adaptive noise reduction capabilities at the front end of the network for signal feature extraction, thereby enhancing the signal feature extraction capability. The ResNet-LSTM model yielded an average prediction error of 0.0085 mm and a tool wear prediction accuracy of 98.25%. These results validate the feasibility of the tool wear prediction method proposed in this study.

An End-to-End Inclination State Monitoring Method for Collaborative Robotic Drilling Based on Resnet Neural Network.

The collaborative robot can complete various drilling tasks in complex processing environments thanks to the high flexibility, small size and high load ratio. However, the inherent weaknesses of low rigidity and variable rigidity in robots bring detrimental effects to surface quality and drilling efficiency. Effective online monitoring of the drilling quality is critical to achieve high performance robotic drilling. To this end, an end-to-end drilling-state monitoring framework is developed in this paper, where the drilling quality can be monitored through online-measured vibration signals. To evaluate the drilling effect, a Canny operator-based edge detection method is used to quantify the inclination state of robotic drilling, which provides the data labeling information. Then, a robotic drilling inclination state monitoring model is constructed based on the Resnet network to classify the drilling inclination states. With the aid of the training dataset labeled by different inclination states and the end-to-end training process, the relationship between the inclination states and vibration signals can be established. Finally, the proposed method is verified by collaborative robotic drilling experiments with different workpiece materials. The results show that the proposed method can effectively recognize the drilling inclination state with high accuracy for different workpiece materials, which demonstrates the effectiveness and applicability of this method.

A Multi-Scale-Enhanced YOLO-V5 Model for Detecting Small Objects in Remote Sensing Image Information.

As a typical component of remote sensing signals, remote sensing image (RSI) information plays a strong role in showing macro, dynamic and accurate information on the earth's surface and environment, which is critical to many application fields. One of the core technologies is the object detection (OD) of RSI signals (RSISs). The majority of existing OD algorithms only consider medium and large objects, regardless of small-object detection, resulting in an unsatisfactory performance in detection precision and the miss rate of small objects. To boost the overall OD performance of RSISs, an improved detection framework, I-YOLO-V5, was proposed for OD in high-altitude RSISs. Firstly, the idea of a residual network is employed to construct a new residual unit to achieve the purpose of improving the network feature extraction. Then, to avoid the gradient fading of the network, densely connected networks are integrated into the structure of the algorithm. Meanwhile, a fourth detection layer is employed in the algorithm structure in order to reduce the deficiency of small-object detection in RSISs in complex environments, and its effectiveness is verified. The experimental results confirm that, compared with existing advanced OD algorithms, the average accuracy of the proposed I-YOLO-V5 is improved by 15.4%, and the miss rate is reduced by 46.8% on the RSOD dataset.

Multi-Source Information-Based Bearing Fault Diagnosis Using Multi-Branch Selective Fusion Deep Residual Network.

Rolling bearing is the core component of industrial machines, but it is difficult for common single signal source-based fault diagnosis methods to ensure reliable results since sensor signals are vulnerable to the pollution of background noises and the attenuation of transmitted information. Recently, multi-source information-based fault diagnosis methods have become popular, but the information redundancy between multiple signals is a tough problem that will negatively impact the representational capacity of deep learning algorithms and the precision of fault diagnosis methods. Besides that, the characteristics of various signals are actually different, but this problem was usually omitted by researchers, and it has potential to further improve the diagnosing performance by adaptively adjusting the feature extraction process for every input signal source. Aimed at solving the above problems, a novel model for bearing fault diagnosis called multi-branch selective fusion deep residual network is proposed in this paper. The model adopts a multi-branch structure design to enable every input signal source to have a unique feature processing channel, avoiding the information of multiple signal sources blindly coupled by convolution kernels. And in each branch, different convolution kernel sizes are assigned according to the characteristics of every input signal, fully digging the precious fault components on respective information sources. Lastly, the dropout technique is used to randomly throw out some activated neurons, alleviating the redundancy and enhancing the quality of the multiscale features extracted from different signals. The proposed method was experimentally compared with other intelligent methods on two authoritative public bearing datasets, and the experimental results prove the feasibility and superiority of the proposed model.

Hyperspectral Anomaly Detection Based on Spectral Similarity Variability Feature.

In the traditional method for hyperspectral anomaly detection, spectral feature mapping is used to map hyperspectral data to a high-level feature space to make features more easily distinguishable between different features. However, the uncertainty in the mapping direction makes the mapped features ineffective in distinguishing anomalous targets from the background. To address this problem, a hyperspectral anomaly detection algorithm based on the spectral similarity variability feature (SSVF) is proposed. First, the high-dimensional similar neighborhoods are fused into similar features using AE networks, and then the SSVF are obtained using residual autoencoder. Finally, the final detection of SSVF was obtained using Reed and Xiaoli (RX) detectors. Compared with other comparison algorithms with the highest accuracy, the overall detection accuracy (AUCODP) of the SSVFRX algorithm is increased by 0.2106. The experimental results show that SSVF has great advantages in both highlighting anomalous targets and improving separability between different ground objects.

Research on Fault Diagnosis of Rolling Bearing Based on Gramian Angular Field and Lightweight Model.

Due to the limitations of deep learning models in processing one-dimensional signal feature extraction, and high model complexity leading to low training accuracy and large consumption of computing resources, this paper innovatively proposes a rolling bearing fault diagnosis method based on Gramian Angular Field (GAF) and enhanced lightweight residual network. Firstly, the one-dimensional signal is transformed into a two-dimensional GAF image, fully preserving the signal's temporal dependency. Secondly, to address the parameter redundancy and high computational complexity of the ResNet-18 model, its residual blocks are improved. The second convolutional layer in the downsampling residual blocks is removed, traditional convolutional layers are replaced with depthwise separable convolutions, and the lightweight Efficient Channel Attention (ECA) module is embedded after each residual block. This further enhances the model's ability to capture key features while maintaining low computational cost, resulting in a lightweight model referred to as E-ResNet13. Finally, the generated GAF feature maps are fed into the E-ResNet13 model for training, and through a global average pooling layer, they are mapped to a fully connected layer for classifying the faults of rolling bearings. Verifying the superiority of the proposed GAF-E-ResNet13 model, experimental results show that the GAF image encoding method achieves higher fault recognition accuracy compared to other encoding methods. Compared with other intelligent diagnosis methods, the E-ResNet13 model demonstrates strong diagnostic performance and generalization capability under both a single condition and complex varying conditions, fully proving the innovation and practicality of this method.

Intelligent Fault Diagnosis of Rolling Bearing Based on Gramian Angular Difference Field and Improved Dual Attention Residual Network.

With the rapid development of smart manufacturing, data-driven deep learning (DL) methods are widely used for bearing fault diagnosis. Aiming at the problem of model training crashes when data are imbalanced and the difficulty of traditional signal analysis methods in effectively extracting fault features, this paper proposes an intelligent fault diagnosis method of rolling bearings based on Gramian Angular Difference Field (GADF) and Improved Dual Attention Residual Network (IDARN). The original vibration signals are encoded as 2D-GADF feature images for network input; the residual structures will incorporate dual attention mechanism to enhance the integration ability of the features, while the group normalization (GN) method is introduced to overcome the bias caused by data discrepancies; and then the model is trained to complete the classification of faults. In order to verify the superiority of the proposed method, the data obtained from Case Western Reserve University (CWRU) bearing data and bearing fault experimental equipment were compared with other popular DL methods, and the proposed model performed optimally. The method eventually achieved an average identification accuracy of 99.2% and 97.9% on two different types of datasets, respectively.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals.

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Surface Defect Detection of Aluminum Profiles Based on Multiscale and Self-Attention Mechanisms.

To address the various challenges in aluminum surface defect detection, such as multiscale intricacies, sensitivity to lighting variations, occlusion, and noise, this study proposes the AluDef-ClassNet model. Firstly, a Gaussian difference pyramid is utilized to capture multiscale image features. Secondly, a self-attention mechanism is introduced to enhance feature representation. Additionally, an improved residual network structure incorporating dilated convolutions is adopted to increase the receptive field, thereby enhancing the network's ability to learn from extensive information. A small-scale dataset of high-quality aluminum surface defect images is acquired using a CCD camera. To better tackle the challenges in surface defect detection, advanced deep learning techniques and data augmentation strategies are employed. To address the difficulty of data labeling, a transfer learning approach based on fine-tuning is utilized, leveraging prior knowledge to enhance the efficiency and accuracy of model training. In dataset testing, our model achieved a classification accuracy of 97.6%, demonstrating significant advantages over other classification models.

[A deep transfer learning approach for cross-subject recognition of mental tasks based on functional near-infrared spectroscopy].

In the field of brain-computer interfaces (BCIs) based on functional near-infrared spectroscopy (fNIRS), traditional subject-specific decoding methods suffer from the limitations of long calibration time and low cross-subject generalizability, which restricts the promotion and application of BCI systems in daily life and clinic. To address the above dilemma, this study proposes a novel deep transfer learning approach that combines the revised inception-residual network (rIRN) model and the model-based transfer learning (TL) strategy, referred to as TL-rIRN. This study performed cross-subject recognition experiments on mental arithmetic (MA) and mental singing (MS) tasks to validate the effectiveness and superiority of the TL-rIRN approach. The results show that the TL-rIRN significantly shortens the calibration time, reduces the training time of the target model and the consumption of computational resources, and dramatically enhances the cross-subject decoding performance compared to subject-specific decoding methods and other deep transfer learning methods. To sum up, this study provides a basis for the selection of cross-subject, cross-task, and real-time decoding algorithms for fNIRS-BCI systems, which has potential applications in constructing a convenient and universal BCI system.