A Novel Hierarchical Vision Transformer and Wavelet Time-Frequency Based on Multi-Source Information Fusion for Intelligent Fault Diagnosis.

Deep learning (DL) has been widely used to promote the development of intelligent fault diagnosis, bringing significant performance improvement. However, most of the existing methods cannot capture the temporal information and global features of mechanical equipment to collect sufficient fault information, resulting in performance collapse. Meanwhile, due to the complex and harsh operating environment, it is difficult to extract fault features stably and extensively using single-source fault diagnosis methods. Therefore, a novel hierarchical vision transformer (NHVT) and wavelet time-frequency architecture combined with a multi-source information fusion (MSIF) strategy has been suggested in this paper to boost stable performance by extracting and integrating rich features. The goal is to improve the end-to-end fault diagnosis performance of mechanical components. First, multi-source signals are transformed into two-dimensional time and frequency diagrams. Then, a novel hierarchical vision transformer is introduced to improve the nonlinear representation of feature maps to enrich fault features. Next, multi-source information diagrams are fused into the proposed NHVT to produce more comprehensive presentations. Finally, we employed two different multi-source datasets to verify the superiority of the proposed NHVT. Then, NHVT outperformed the state-of-the-art approach (SOTA) on the multi-source dataset of mechanical components, and the experimental results show that it is able to extract useful features from multi-source information.

Intelligent Sensing of Thermal Error of CNC Machine Tool Spindle Based on Multi-Source Information Fusion.

Aiming at the shortcomings of single-sensor sensing information characterization ability, which is easily interfered with by external environmental factors, a method of intelligent perception is proposed in this paper. This method integrates multi-source and multi-level information, including spindle temperature field, spindle thermal deformation, operating parameters, and motor current. Firstly, the internal and external thermal-error-related signals of the spindle system are collected by sensors, and the feature parameters are extracted; then, the radial basis function (RBF) neural network is utilized to realize the preliminary integration of the feature parameters because of the advantages of the RBF neural network, which offers strong multi-dimensional solid nonlinear mapping ability and generalization ability. Thermal-error decision values are then generated by a weighted fusion of different pieces of evidence by considering uncertain information from multiple sources. The spindle thermal-error sensing experiment was based on the spindle system of the VMC850 (Yunnan Machine Tool Group Co., LTD, Yunnan, China) vertical machining center of the Yunnan Machine Tool Factory. Experiments were designed for thermal-error sensing of the spindle under constant speed (2000 r/min and 4000 r/min), standard variable speed, and stepped variable speed conditions. The experiment's results show that the prediction accuracy of the intelligent-sensing model with multi-source information fusion can reach 98.1%, 99.3%, 98.6%, and 98.8% under the above working conditions, respectively. The intelligent-perception model proposed in this paper has higher accuracy and lower residual error than the traditional BP neural network perception and wavelet neural network models. The research in this paper provides a theoretical basis for the operation, maintenance management, and performance optimization of machine tool spindle systems.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals.

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Multi-Source Feature-Fusion Method for the Seismic Data of Cultural Relics Based on Deep Learning.

The museum system is exposed to a high risk of seismic hazards. However, it is difficult to carry out seismic hazard prevention to protect cultural relics in collections due to the lack of real data and diverse types of seismic hazards. To address this problem, we developed a deep-learning-based multi-source feature-fusion method to assess the data on seismic damage caused by collected cultural relics. Firstly, a multi-source data-processing strategy was developed according to the needs of seismic impact analysis of the cultural relics in the collection, and a seismic event-ontology model of cultural relics was constructed. Additionally, a seismic damage data-classification acquisition method and empirical calculation model were designed. Secondly, we proposed a deep learning-based multi-source feature-fusion matching method for cultural relics. By constructing a damage state assessment model of cultural relics using superpixel map convolutional fusion and an automatic data-matching model, the quality and processing efficiency of seismic damage data of the cultural relics in the collection were improved. Finally, we formed a dataset oriented to the seismic damage risk analysis of the cultural relics in the collection. The experimental results show that the accuracy of this method reaches 93.6%, and the accuracy of cultural relics label matching is as high as 82.6% compared with many kinds of earthquake damage state assessment models. This method can provide more accurate and efficient data support, along with a scientific basis for subsequent research on the impact analysis of seismic damage to cultural relics in collections.

Uncertainty Management in Assessment of FMEA Expert Based on Negation Information and Belief Entropy.

The failure mode and effects analysis (FMEA) is a commonly adopted approach in engineering failure analysis, wherein the risk priority number (RPN) is utilized to rank failure modes. However, assessments made by FMEA experts are full of uncertainty. To deal with this issue, we propose a new uncertainty management approach for the assessments given by experts based on negation information and belief entropy in the Dempster-Shafer evidence theory framework. First, the assessments of FMEA experts are modeled as basic probability assignments (BPA) in evidence theory. Next, the negation of BPA is calculated to extract more valuable information from a new perspective of uncertain information. Then, by utilizing the belief entropy, the degree of uncertainty of the negation information is measured to represent the uncertainty of different risk factors in the RPN. Finally, the new RPN value of each failure mode is calculated for the ranking of each FMEA item in risk analysis. The rationality and effectiveness of the proposed method is verified through its application in a risk analysis conducted for an aircraft turbine rotor blade.

Health State Estimation of On-Board Lithium-Ion Batteries Based on GMM-BID Model.

As a single feature parameter cannot comprehensively evaluate the health status of a battery, a multi-source information fusion method based on the Gaussian mixture model and Bayesian inference distance is proposed for the health assessment of vehicle batteries. The missing and abnormal data from real-life vehicle operations are preprocessed to extract the sensitive characteristic parameters which determine the battery performance. The normal state Gaussian mixture model is established using the fault-free state data, whereas the Bayesian inference distance is constructed as an index to quantitatively evaluate the battery performance state. In order to solve the problem that abnormal data may exist in the measured data and introduce errors into evaluation results, the determination rules of abnormal data are formulated. The verification of real-life vehicle operation data reveals that the proposed method can accurately evaluate the onboard battery state and reduce safety hazards of electric vehicles during the normal operation process.

Research on the Application of Multi-Source Information Fusion in Multiple Gait Pattern Transition Recognition.

Multi-source information fusion technology is a kind of information processing technology which comprehensively processes and utilizes multi-source uncertain information. It is an effective scheme to solve complex pattern recognition and improve classification performance. This study aims to improve the accuracy and robustness of exoskeleton gait pattern transition recognition in complex environments. Based on the theory of multi-source information fusion, this paper explored a multi-source information fusion model for exoskeleton gait pattern transition recognition in terms of two aspects of multi-source information fusion strategy and multi-classifier fusion. For eight common gait pattern transitions (between level and stair walking and between level and ramp walking), we proposed a hybrid fusion strategy of multi-source information at the feature level and decision level. We first selected an optimal feature subset through correlation feature extraction and feature selection algorithm, followed by the feature fusion through the classifier. We then studied the construction of a multi-classifier fusion model with a focus on the selection of base classifier and multi-classifier fusion algorithm. By analyzing the classification performance and robustness of the multi-classifier fusion model integrating multiple classifier combinations with a number of multi-classifier fusion algorithms, we finally constructed a multi-classifier fusion model based on D-S evidence theory and the combination of three SVM classifiers with different kernel functions (linear, RBF, polynomial). Such multi-source information fusion model improved the anti-interference and fault tolerance of the model through the hybrid fusion strategy of feature level and decision level and had higher accuracy and robustness in the gait pattern transition recognition, whose average recognition accuracy for eight gait pattern transitions reached 99.70%, which increased by 0.15% compared with the highest average recognition accuracy of the single classifier. Moreover, the average recognition accuracy in the absence of different feature data reached 97.47% with good robustness.

Application of a Bayesian Network Based on Multi-Source Information Fusion in the Fault Diagnosis of a Radar Receiver.

A radar is an important part of an air defense and combat system. It is of great significance to military defense to improve the effectiveness of radar state monitoring and the accuracy of fault diagnosis during operation. However, the complexity of radar equipment's structure and the uncertainty of the operating environment greatly increase the difficulty of fault diagnosis in real life situations. Therefore, a Bayesian network diagnosis method based on multi-source information fusion technology is proposed to solve the fault diagnosis problems caused by uncertain factors such as the high integration and complexity of the system during the process of fault diagnosis. Taking a fault of a radar receiver as an example, we study 2 typical fault phenomena and 21 fault points. After acquiring and processing multi-source information, establishing a Bayesian network model, determining conditional probability tables (CPTs), and finally outputting the diagnosis results. The results are convincing and consistent with reality, which verifies the effectiveness of this method for fault diagnosis in radar receivers. It realizes device-level fault diagnosis, which shortens the maintenance time for radars and improves the reliability and maintainability of radars. Our results have significance as a guide for judging the fault location of radars and predicting the vulnerable components of radars.

A Detection and Tracking Method Based on Heterogeneous Multi-Sensor Fusion for Unmanned Mining Trucks.

There exist many difficulties in environmental perception in transportation at open-pit mines, such as unpaved roads, dusty environments, and high requirements for the detection and tracking stability of small irregular obstacles. In order to solve the above problems, a new multi-target detection and tracking method is proposed based on the fusion of Lidar and millimeter-wave radar. It advances a secondary segmentation algorithm suitable for open-pit mine production scenarios to improve the detection distance and accuracy of small irregular obstacles on unpaved roads. In addition, the paper also proposes an adaptive heterogeneous multi-source fusion strategy of filtering dust, which can significantly improve the detection and tracking ability of the perception system for various targets in the dust environment by adaptively adjusting the confidence of the output target. Finally, the test results in the open-pit mine show that the method can stably detect obstacles with a size of 30-40 cm at 60 m in front of the mining truck, and effectively filter out false alarms of concentration dust, which proves the reliability of the method.

Multi-Source Information Fusion Based on Negation of Reconstructed Basic Probability Assignment with Padded Gaussian Distribution and Belief Entropy.

Multi-source information fusion is widely used because of its similarity to practical engineering situations. With the development of science and technology, the sources of information collected under engineering projects and scientific research are more diverse. To extract helpful information from multi-source information, in this paper, we propose a multi-source information fusion method based on the Dempster-Shafer (DS) evidence theory with the negation of reconstructed basic probability assignments (nrBPA). To determine the initial basic probability assignment (BPA), the Gaussian distribution BPA functions with padding terms are used. After that, nrBPAs are determined by two processes, reassigning the high blur degree BPA and transforming them into the form of negation. In addition, evidence of preliminary fusion is obtained using the entropy weight method based on the improved belief entropy of nrBPAs. The final fusion results are calculated from the preliminary fused evidence through the Dempster's combination rule. In the experimental section, the UCI iris data set and the wine data set are used for validating the arithmetic processes of the proposed method. In the comparative analysis, the effectiveness of the BPA determination using a padded Gaussian function is verified by discussing the classification task with the iris data set. Subsequently, the comparison with other methods using the cross-validation method proves that the proposed method is robust. Notably, the classification accuracy of the iris data set using the proposed method can reach an accuracy of 97.04%, which is higher than many other methods.

Measuring Uncertainty in the Negation Evidence for Multi-Source Information Fusion.

Dempster-Shafer evidence theory is widely used in modeling and reasoning uncertain information in real applications. Recently, a new perspective of modeling uncertain information with the negation of evidence was proposed and has attracted a lot of attention. Both the basic probability assignment (BPA) and the negation of BPA in the evidence theory framework can model and reason uncertain information. However, how to address the uncertainty in the negation information modeled as the negation of BPA is still an open issue. Inspired by the uncertainty measures in Dempster-Shafer evidence theory, a method of measuring the uncertainty in the negation evidence is proposed. The belief entropy named Deng entropy, which has attracted a lot of attention among researchers, is adopted and improved for measuring the uncertainty of negation evidence. The proposed measure is defined based on the negation function of BPA and can quantify the uncertainty of the negation evidence. In addition, an improved method of multi-source information fusion considering uncertainty quantification in the negation evidence with the new measure is proposed. Experimental results on a numerical example and a fault diagnosis problem verify the rationality and effectiveness of the proposed method in measuring and fusing uncertain information.

Dam Safety Evaluation Based on Interval-Valued Intuitionistic Fuzzy Sets and Evidence Theory.

Considering the multi-sources, heterogeneity and complexity of dam safety assessment, a dam safety assessment model based on interval-valued intuitionistic fuzzy set and evidence theory is proposed to perform dam safety reliability evaluations. In the proposed model, the dynamic reliability based on the supporting degree is applied to modify the data from homologous information. The interval-valued intuitionistic fuzzy set is used to describing the uncertainty and fuzziness between heterogeneous information. Evidence theory is employed to integrate the data from heterogeneous information. Finally, a multiple-arch dam undergoing structural reinforcement is taken as an example. The evaluation result before reinforcement shows that the safety degree of the dam is low and the potential risk is more likely to be located at the dam section #13. From the geological survey before reinforcement, there exist weak fracture zone and broken mud belt in the foundation of the dam section #13. The comparison between the evaluation results before and after reinforcement indicates that the dam become safer and more stable after reinforcement.

Research Progress on Quality Detection of Livestock and Poultry Meat Based on Machine Vision, Hyperspectral and Multi-Source Information Fusion Technologies.

Presently, the traditional methods employed for detecting livestock and poultry meat predominantly involve sensory evaluation conducted by humans, chemical index detection, and microbial detection. While these methods demonstrate commendable accuracy in detection, their application becomes more challenging when applied to large-scale production by enterprises. Compared with traditional detection methods, machine vision and hyperspectral technology can realize real-time online detection of large throughput because of their advantages of high efficiency, accuracy, and non-contact measurement, so they have been widely concerned by researchers. Based on this, in order to further enhance the accuracy of online quality detection for livestock and poultry meat, this article presents a comprehensive overview of methods based on machine vision, hyperspectral, and multi-sensor information fusion technologies. This review encompasses an examination of the current research status and the latest advancements in these methodologies while also deliberating on potential future development trends. The ultimate objective is to provide pertinent information and serve as a valuable research resource for the non-destructive online quality detection of livestock and poultry meat.

MIFAM-DTI: a drug-target interactions predicting model based on multi-source information fusion and attention mechanism.

Accurate identification of potential drug-target pairs is a crucial step in drug development and drug repositioning, which is characterized by the ability of the drug to bind to and modulate the activity of the target molecule, resulting in the desired therapeutic effect. As machine learning and deep learning technologies advance, an increasing number of models are being engaged for the prediction of drug-target interactions. However, there is still a great challenge to improve the accuracy and efficiency of predicting. In this study, we proposed a deep learning method called Multi-source Information Fusion and Attention Mechanism for Drug-Target Interaction (MIFAM-DTI) to predict drug-target interactions. Firstly, the physicochemical property feature vector and the Molecular ACCess System molecular fingerprint feature vector of a drug were extracted based on its SMILES sequence. The dipeptide composition feature vector and the Evolutionary Scale Modeling -1b feature vector of a target were constructed based on its amino acid sequence information. Secondly, the PCA method was employed to reduce the dimensionality of the four feature vectors, and the adjacency matrices were constructed by calculating the cosine similarity. Thirdly, the two feature vectors of each drug were concatenated and the two adjacency matrices were subjected to a logical OR operation. And then they were fed into a model composed of graph attention network and multi-head self-attention to obtain the final drug feature vectors. With the same method, the final target feature vectors were obtained. Finally, these final feature vectors were concatenated, which served as the input to a fully connected layer, resulting in the prediction output. MIFAM-DTI not only integrated multi-source information to capture the drug and target features more comprehensively, but also utilized the graph attention network and multi-head self-attention to autonomously learn attention weights and more comprehensively capture information in sequence data. Experimental results demonstrated that MIFAM-DTI outperformed state-of-the-art methods in terms of AUC and AUPR. Case study results of coenzymes involved in cellular energy metabolism also demonstrated the effectiveness and practicality of MIFAM-DTI. The source code and experimental data for MIFAM-DTI are available at https://github.com/Search-AB/MIFAM-DTI.

Research on rapid quality identification method of Panax notoginseng powder based on artificial intelligence sensory technology and multi-source information fusion technology.

Panax notoginseng powder (PNP) has high medicinal value and is widely used in the medical and health food industries. However, the adulteration of PNP in the market has dramatically reduced its efficacy. Therefore, this study intends to use artificial intelligence sensory (AIS) and multi-source information fusion (MIF) technology to try to establish a quality evaluation system for different grades of PNP and adulterated Panax notoginseng powder (AD-PNP). The highest accuracy rate reached 100% in identifying PNP grade and adulteration. In the prediction of adulteration ratio and total saponin content, the optimal determination coefficients of the test set were 0.9965 and 0.9948, respectively, and the root mean square errors were 0.0109 and 0.0123, respectively. Therefore, the grade identification method of PNP and the evaluation system of AD-PNP based on AIS and MIF technology can rapidly and accurately evaluate the quality of PNP.

Multi-source information fusion-driven corn yield prediction using the Random Forest from the perspective of Agricultural and Forestry Economic Management.

The objective of this study is to promptly and accurately allocate resources, scientifically guide grain distribution, and enhance the precision of crop yield prediction (CYP), particularly for corn, along with ensuring application stability. The digital camera is selected to capture the digital image of a 60 m × 10 m experimental cornfield. Subsequently, the obtained data on corn yield and statistical growth serve as inputs for the multi-source information fusion (MSIF). The study proposes an MSIF-based CYP Random Forest model by amalgamating the fluctuating corn yield dataset. In relation to the spatial variability of the experimental cornfield, the fitting degree and prediction ability of the proposed MSIF-based CYP Random Forest are analyzed, with statistics collected from 1-hectare, 10-hectare, 20-hectare, 30-hectare, and 50-hectare experimental cornfields. Results indicate that the proposed MSIF-based CYP Random Forest model outperforms control models such as support vector machine (SVM) and Long Short-Term Memory (LSTM), achieving the highest prediction accuracy of 89.30%, surpassing SVM and LSTM by approximately 13.44%. Meanwhile, as the experimental field size increases, the proposed model demonstrates higher prediction accuracy, reaching a maximum of 98.71%. This study is anticipated to offer early warnings of potential factors affecting crop yields and to further advocate for the adoption of MSIF-based CYP. These findings hold significant research implications for personnel involved in Agricultural and Forestry Economic Management within the context of developing agricultural economy.

DeepCMI: a graph-based model for accurate prediction of circRNA-miRNA interactions with multiple information.

Recently, the role of competing endogenous RNAs in regulating gene expression through the interaction of microRNAs has been closely associated with the expression of circular RNAs (circRNAs) in various biological processes such as reproduction and apoptosis. While the number of confirmed circRNA-miRNA interactions (CMIs) continues to increase, the conventional in vitro approaches for discovery are expensive, labor intensive, and time consuming. Therefore, there is an urgent need for effective prediction of potential CMIs through appropriate data modeling and prediction based on known information. In this study, we proposed a novel model, called DeepCMI, that utilizes multi-source information on circRNA/miRNA to predict potential CMIs. Comprehensive evaluations on the CMI-9905 and CMI-9589 datasets demonstrated that DeepCMI successfully infers potential CMIs. Specifically, DeepCMI achieved AUC values of 90.54% and 94.8% on the CMI-9905 and CMI-9589 datasets, respectively. These results suggest that DeepCMI is an effective model for predicting potential CMIs and has the potential to significantly reduce the need for downstream in vitro studies. To facilitate the use of our trained model and data, we have constructed a computational platform, which is available at http://120.77.11.78/DeepCMI/. The source code and datasets used in this work are available at https://github.com/LiYuechao1998/DeepCMI.

Research progress and visualization of underground coal fire detection methods.

Underground coal fires are a widespread disaster prevailing in major coal-producing countries globally, posing serious threats to the ecological environment and restricting the safe exploitation of coal mines. The accuracy of underground coal fire detection directly affects the effectiveness of fire control engineering. In this study, we searched 426 articles from the Web of Science database within 2002-2022 as the data foundation and visualized the research contents of the underground coal fire field using VOSviewer and CiteSpace. The results reveal that the investigation of "underground coal fire detection techniques" is currently the focal area of research in this field. Additionally, the "underground coal fire multi-information fusion inversion detection methods" are considered to be the future research trend. Moreover, we reviewed the strengths and weaknesses of various single-indicator inversion detection methods, including the temperature method, gas and radon method, natural potential method, magnetic method, electric method, remote sensing, and geological radar method. Furthermore, we conducted an analysis of the advantages of the multi-information fusion inversion detection methods, which possesses high precision and wide applicability for detecting coal fires, while highlighting the challenges in handling diverse data sources. It is our hope that the research results presented in this paper will provide valuable insights and ideas for researchers involved in the detection and practical research of underground coal fires.

Rapid quantification of royal jelly quality by mid-infrared spectroscopy coupled with backpropagation neural network.

Royal jelly is rich in nutrients but its quality is greatly affected by storage conditions. To determine the quality of royal jelly accurately and quickly, a qualitative discrimination model was established based on the fusion of conventional parameters and mid-infrared spectrum, using support vector machine. The prediction models for three representative quality parameters were developed by the backpropagation neural network with various algorithms. The results demonstrated that the recognition rate of the multi-source information fusion model was increased to 100% when compared with that of the spectral data preprocessed by Savitzky-golay smoothing (95.83%). The mean square errors of the constructed model for moisture, water-soluble protein, and total sugar were 0.0032, 0.0058, and 0.0069, respectively. The constructed model had an ensured accuracy for the calibration set, with the correlation coefficient of prediction maintained at 0.9353, 0.9533, and 0.9563, which could meet the requirement of non-destructive rapid detection of royal jelly quality.

Detection Method for Tomato Leaf Mildew Based on Hyperspectral Fusion Terahertz Technology.

Leaf mildew is a common disease of tomato leaves. Its detection is an important means to reduce yield loss from the disease and improve tomato quality. In this study, a new method was developed for the multi-source detection of tomato leaf mildew by THz hyperspectral imaging through combining internal and external leaf features. First, multi-source information obtained from tomato leaves of different disease grades was extracted by near-infrared hyperspectral imaging and THz time-domain spectroscopy, while the influence of low-frequency noise was removed by the Savitzky Golay (SG) smoothing algorithm. A genetic algorithm (GA) was used to optimize the selection of the characteristic near-infrared hyperspectral band. Principal component analysis (PCA) was employed to optimize the THz characteristic absorption spectra and power spectrum dimensions. Recognition models were developed for different grades of tomato leaf mildew infestation by incorporating near-infrared hyperspectral imaging, THz absorbance, and power spectra using the backpropagation neural network (BPNN), and the models had recognition rates of 95%, 96.67%, and 95%, respectively. Based on the near-infrared hyperspectral features, THz time-domain spectrum features, and classification model, the probability density of the posterior distribution of tomato leaf health parameter variables was recalculated by a Bayesian network model. Finally, a fusion diagnosis and health evaluation model of tomato leaf mildew with hyperspectral fusion THz was established, and the recognition rate of tomato leaf mildew samples reached 97.12%, which improved the recognition accuracy by 0.45% when compared with the single detection method, thereby achieving the accurate detection of facility diseases.