In this paper, a novel study on the way inter-individual information interacts in meta-heuristic algorithms (MHAs) is carried out using a scheme known as population interaction networks (PIN). Specifically, three representative MHAs, including the differential evolutionary algorithm (DE), the particle swarm optimization algorithm (PSO), the gravitational search algorithm (GSA), and four classical variations of the gravitational search algorithm, are analyzed in terms of inter-individual information interactions and the differences in the performance of each of the algorithms on IEEE Congress on Evolutionary Computation 2017 benchmark functions. The cumulative distribution function (CDF) of the node degree obtained by the algorithm on the benchmark function is fitted to the seven distribution models by using PIN. The results show that among the seven compared algorithms, the more powerful DE is more skewed towards the Poisson distribution, and the weaker PSO, GSA, and GSA variants are more skewed towards the Logistic distribution. The more deviation from Logistic distribution GSA variants conform, the stronger their performance. From the point of view of the CDF, deviating from the Logistic distribution facilitates the improvement of the GSA. Our findings suggest that the population interaction network is a powerful tool for characterizing and comparing the performance of different MHAs in a more comprehensive and meaningful way.
The adversarial robustness is critical to deep neural networks (DNNs) in deployment. However, the improvement of adversarial robustness often requires compromising with the network size. Existing approaches to addressing this problem mainly focus on the combination of model compression and adversarial training. However, their performance heavily relies on neural architectures, which are typically manual designs with extensive expertise. In this article, we propose a lightweight and robust neural architecture search (LRNAS) method to automatically search for adversarially robust lightweight neural architectures. Specifically, we propose a novel search strategy to quantify contributions of the components in the search space, based on which the beneficial components can be determined. In addition, we further propose an architecture selection method based on a greedy strategy, which can keep the model size while deriving sufficient beneficial components. Owing to these designs in LRNAS, the lightness, the natural accuracy, and the adversarial robustness can be collectively guaranteed to the searched architectures. We conduct extensive experiments on various benchmark datasets against the state of the arts. The experimental results demonstrate that the proposed LRNAS method is superior at finding lightweight neural architectures that are both accurate and adversarially robust under popular adversarial attacks. Moreover, ablation studies are also performed, which reveals the validity of the individual components designed in LRNAS and the component effects in positively deciding the overall performance.
Ultrasound detection is a potent tool for the clinical diagnosis of various diseases due to its real-time, convenient, and noninvasive qualities. Yet, existing ultrasound beamforming and related methods face a big challenge to improve both the quality and speed of imaging for the required clinical applications. The most notable characteristic of ultrasound signal data is its spatial and temporal features. Because most signals are complex-valued, directly processing them by using real-valued networks leads to phase distortion and inaccurate output. In this study, for the first time, we propose a complex-valued convolutional gated recurrent (CCGR) neural network to handle ultrasound analytic signals with the aforementioned properties. The complex-valued network operations proposed in this study improve the beamforming accuracy of complex-valued ultrasound signals over traditional real-valued methods. Further, the proposed deep integration of convolution and recurrent neural networks makes a great contribution to extracting rich and informative ultrasound signal features. Our experimental results reveal its outstanding imaging quality over existing state-of-the-art methods. More significantly, its ultrafast processing speed of only 0.07 s per image promises considerable clinical application potential. The code is available at https://github.com/zhangzm0128/CCGR.
Human activity analysis in the legal monitoring environment plays an important role in the physical rehabilitation field, as it helps patients with physical injuries improve their postoperative conditions and reduce their medical costs. Recently, several deep learning-based action quality assessment (AQA) frameworks have been proposed to evaluate physical rehabilitation exercises. However, most of them treat this problem as a simple regression task, which requires both the action instance and its score label as input. This approach is limited by the fact that the annotations in this field usually consist of healthy or unhealthy labels rather than quality scores provided by professional physicians. Additionally, most of these methods cannot provide informative feedback on a patient's motion defects, which weakens their practical application. To address these problems, we propose a multi-task contrastive learning framework to learn subtle and critical differences from skeleton sequences to deal with the performance metric and AQA problems of physical rehabilitation exercises. Specifically, we propose a performance metric network that takes triplets of training samples as input for score generation. For the AQA task, the same contrast learning strategy is used, but pairwise training samples are fed into the action quality assessment network for score prediction. Notably, we propose quantifying the deviation of the joint attention matrix between different skeleton sequences and introducing it into the loss function of our learning network. It is proven that considering both score prediction loss and joint attention deviation loss improves physical exercises AQA performance. Furthermore, it helps to obtain informative feedback for patients to improve their motion defects by visualizing the joint attention matrix's difference. The proposed method is verified on the UI-PRMD and KIMORE datasets. Experimental results show that the proposed method achieves state-of-the-art performance.
Exercise Therapy , Exercise , Humans , Motion
Slime mold algorithm (SMA) is a nature-inspired algorithm that simulates the biological optimization mechanisms and has achieved great results in various complex stochastic optimization problems. Owing to the simulated biological search principle of slime mold, SMA has a unique advantage in global optimization problem. However, it still suffers from issues of missing the optimal solution or collapsing to local optimum when facing complicated problems. To conquer these drawbacks, we consider adding a novel multi-chaotic local operator to the bio-shock feedback mechanism of SMA to compensate for the lack of exploration of the local solution space with the help of the perturbation nature of the chaotic operator. Based on this, we propose an improved algorithm, namely MCSMA, by investigating how to improve the probabilistic selection of chaotic operators based on the maximum Lyapunov exponent (MLE), an inherent property of chaotic maps. We implement the comparison between MCSMA with other state-of-the-art methods on IEEE Congress on Evolution Computation (CEC) i.e., CEC2017 benchmark test suits and CEC2011 practical problems to demonstrate its potency and perform dendritic neuron model training to test the robustness of MCSMA on classification problems. Finally, the parameters' sensitivities of MCSMA, the utilization of the solution space, and the effectiveness of the MLE are adequately discussed.
Ultrasound imaging is widely used in medical diagnosis. It has the advantages of being performed in real time, cost-efficient, noninvasive, and nonionizing. The traditional delay-and-sum (DAS) beamformer has low resolution and contrast. Several adaptive beamformers (ABFs) have been proposed to improve them. Although they improve image quality, they incur high computation cost because of the dependence on data at the expense of real-time performance. Deep-learning methods have been successful in many areas. They train an ultrasound imaging model that can be used to quickly handle ultrasound signals and construct images. Real-valued radio-frequency signals are typically used to train a model, whereas complex-valued ultrasound signals with complex weights enable the fine-tuning of time delay for enhancing image quality. This work, for the first time, proposes a fully complex-valued gated recurrent neural network to train an ultrasound imaging model for improving ultrasound image quality. The model considers the time attributes of ultrasound signals and uses complete complex-number calculation. The model parameter and architecture are analyzed to select the best setup. The effectiveness of complex batch normalization is evaluated in training the model. The effect of analytic signals and complex weights is analyzed, and the results verify that analytic signals with complex weights enhance the model performance to reconstruct high-quality ultrasound images. The proposed model is finally compared with seven state-of-the-art methods. Experimental results reveal its great performance.
The rapid industrial development in the human society has brought about the air pollution, which seriously affects human health. PM2.5 concentration is one of the main factors causing the air pollution. To accurately predict PM2.5 microns, we propose a dendritic neuron model (DNM) trained by an improved state-of-matter heuristic algorithm (DSMS) based on STL-LOESS, namely DS-DNM. Firstly, DS-DNM adopts STL-LOESS for the data preprocessing to obtain three characteristic quantities from original data: seasonal, trend, and residual components. Then, DNM trained by DSMS predicts the residual values. Finally, three sets of feature quantities are summed to obtain the predicted values. In the performance test experiments, five real-world PM2.5 concentration data are used to test DS-DNM. On the other hand, four training algorithms and seven prediction models were selected for comparison to verify the rationality of the training algorithms and the accuracy of the prediction models, respectively. The experimental results show that DS-DNM has the more competitive performance in PM2.5 concentration prediction problem.
Protein structure prediction (PSP) is predicting the three-dimensional of protein from its amino acid sequence only based on the information hidden in the protein sequence. One of the efficient tools to describe this information is protein energy functions. Despite the advancements in biology and computer science, PSP is still a challenging problem due to its large protein conformation space and inaccurate energy functions. In this study, PSP is treated as a many-objective optimization problem and four conflicting energy functions are used as different objectives to be optimized. A novel Pareto-dominance-archive and Coordinated-selection-strategy-based Many-objective-optimizer (PCM) is proposed to perform the conformation search. In it, convergence and diversity-based selection metrics are used to enable PCM to find near-native proteins with well-distributed energy values, while a Pareto-dominance-based archive is proposed to save more potential conformations that can guide the search to more promising conformation areas. The experimental results on thirty-four benchmark proteins demonstrate the significant superiority of PCM in comparison with other single, multiple, and many-objective evolutionary algorithms. Additionally, the inherent characteristics of iterative search of PCM can also give more insights into the dynamic progress of protein folding besides the final predicted static tertiary structure. All these confirm that PCM is a fast, easy-to-use, and fruitful solution generation method for PSP.
Algorithms , Proteins , Proteins/genetics , Proteins/chemistry , Protein Conformation , Amino Acid Sequence , Protein Folding
Deep learning (DL) has achieved breakthrough successes in various tasks, owing to its layer-by-layer information processing and sufficient model complexity. However, DL suffers from the issues of both redundant model complexity and low interpretability, which are mainly because of its oversimplified basic McCulloch-Pitts neuron unit. A widely recognized biologically plausible dendritic neuron model (DNM) has demonstrated its effectiveness in alleviating the aforementioned issues, but it can only solve binary classification tasks, which significantly limits its applicability. In this study, a novel extended network based on the dendritic structure is innovatively proposed, thereby enabling it to solve multiple-class classification problems. Also, for the first time, an efficient error-back-propagation learning algorithm is derived. In the extensive experimental results, the effectiveness and superiority of the proposed method in comparison with other nine state-of-the-art classifiers on ten datasets are demonstrated, including a real-world quality of web service application. The experimental results suggest that the proposed learning algorithm is competent and reliable in terms of classification performance and stability and has a notable advantage in small-scale disequilibrium data. Additionally, aspects of network structure constrained by scale are examined.
Algorithms , Neurons , Neurons/physiology , Software
A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.
Neural Networks, Computer , Neurons , Signal Processing, Computer-Assisted , Algorithms
Feature selection (FS) has received significant attention since the use of a well-selected subset of features may achieve better classification performance than that of full features in many real-world applications. It can be considered as a multiobjective optimization consisting of two objectives: 1) minimizing the number of selected features and 2) maximizing classification performance. Ant colony optimization (ACO) has shown its effectiveness in FS due to its problem-guided search operator and flexible graph representation. However, there lacks an effective ACO-based approach for multiobjective FS to handle the problematic characteristics originated from the feature interactions and highly discontinuous Pareto fronts. This article presents an Information-theory-based Nondominated Sorting ACO (called INSA) to solve the aforementioned difficulties. First, the probabilistic function in ACO is modified based on the information theory to identify the importance of features; second, a new ACO strategy is designed to construct solutions; and third, a novel pheromone updating strategy is devised to ensure the high diversity of tradeoff solutions. INSA's performance is compared with four machine-learning-based methods, four representative single-objective evolutionary algorithms, and six state-of-the-art multiobjective ones on 13 benchmark classification datasets, which consist of both low and high-dimensional samples. The empirical results verify that INSA is able to obtain solutions with better classification performance using features whose count is similar to or less than those obtained by its peers.
INTRODUCTION: According to the competing failure theorem, the fractional-order Resistor Capacitance (RC) circuit system suffers not only from internal degradation but also from external shocks. However, due to the general differences of each failure type in the data availability and cognitive uncertainty, a better model is needed to describe the degradation process within the system. Also, a new reliability analysis method is needed for the circuit system under internal degradation and external shocks. OBJECTIVES: To demonstrate this problem, this paper proposes a novel class of Caputo-type uncertain random fractional-order model that focuses on the reliability analysis of a fractional-order RC circuit system. METHODS: First, an uncertain Liu process is used to describe the internal degradation of soft faults and a stochastic process is used to describe the external random shocks of hard faults. Secondly, taking into account the correlation and competition among the fault types, an extreme shock model and a cumulative shock model are constructed, and chance theory is introduced to further explore the fault mechanisms, from which the corresponding reliability indices are derived. Finally, the predictor-corrector method is applied and numerical examples are given. RESULTS: This paper presents a reliability analysis of a fractional-order RC circuit system with internal failure obeying an uncertain process and external failure obeying a stochastic process, and gives the calculation of the reliability indexes for different cases and the corresponding numerical simulations. CONCLUSION: A new competing failure model for a fractional-order RC circuit system is presented and analyzed for reliability, which is proved to be of practical importance by numerical simulations.
A dendritic neuron model with adaptive synapses (DMASs) based on differential evolution (DE) algorithm training is proposed. According to the signal transmission order, a DNM can be divided into four parts: the synaptic layer, dendritic layer, membrane layer, and somatic cell layer. It can be converted to a logic circuit that is easily implemented on hardware by removing useless synapses and dendrites after training. This logic circuit can be designed to solve complex nonlinear problems using only four basic logical devices: comparators, AND (conjunction), OR (disjunction), and NOT (negation). To obtain a faster and better solution, we adopt the most popular DE for DMAS training. We have chosen five classification datasets from the UCI Machine Learning Repository for an experiment. We analyze and discuss the experimental results in terms of the correct rate, convergence rate, ROC curve, and the cross-validation and then compare the results with a dendritic neuron model trained by the backpropagation algorithm (BP-DNM) and a neural network trained by the backpropagation algorithm (BPNN). The analysis results show that the DE-DMAS shows better performance in all aspects.
Algorithms , Dendrites , Models, Neurological , Neural Networks, Computer , Synapses , Animals , Humans
In this paper, a novel algorithm called bi-objective elite differential evolution (BOEDE) is proposed to optimize multivalued logic (MVL) networks. It is a multiobjective algorithm completely different from all previous single-objective optimization ones. The two objective functions, error and optimality, are put into evaluating the fitness of individuals in evolution simultaneously. BOEDE innovatively uses an archive population with different ranks to store elite individuals and offsprings. Moreover, a characteristic updating method based on this archive structure is designed to produce the parent population. Because of the particularity of MVL network problems, the performance of BOEDE to solve them is further improved by strictly distinguishing elite solutions and Pareto optimal solutions, and by modifying the method of dealing with illegal variables. The simulations show that BOEDE can collect a great number of solutions to provide decision support for a variety of applications. The comparison results also indicate that BOEDE is significantly better than the existing algorithms.
By employing a neuron plasticity mechanism, the original dendritic neuron model (DNM) has been succeeded in the classification tasks with not only an encouraging accuracy but also a simple learning rule. However, the data collected in real world contain a lot of redundancy, which causes the process of analyzing data by DNM become complicated and time-consuming. This paper proposes a reliable hybrid model which combines a maximum relevance minimum redundancy (Mr2) feature selection technique with DNM (namely, Mr2DNM) for classifying the practical classification problems. The mutual information-based Mr2 is applied to evaluate and rank the most informative and discriminative features for the given dataset. The obtained optimal feature subset is used to train and test the DNM for classifying five different problems arisen from medical, physical, and social scenarios. Experimental results suggest that the proposed Mr2DNM outperforms DNM and other six classification algorithms in terms of accuracy and computational efficiency.
Algorithms , Neuronal Plasticity/physiology , Neurons/physiology , Dendritic Cells/physiology , Models, Biological , Support Vector Machine
An artificial neural network (ANN) that mimics the information processing mechanisms and procedures of neurons in human brains has achieved a great success in many fields, e.g., classification, prediction, and control. However, traditional ANNs suffer from many problems, such as the hard understanding problem, the slow and difficult training problems, and the difficulty to scale them up. These problems motivate us to develop a new dendritic neuron model (DNM) by considering the nonlinearity of synapses, not only for a better understanding of a biological neuronal system, but also for providing a more useful method for solving practical problems. To achieve its better performance for solving problems, six learning algorithms including biogeography-based optimization, particle swarm optimization, genetic algorithm, ant colony optimization, evolutionary strategy, and population-based incremental learning are for the first time used to train it. The best combination of its user-defined parameters has been systemically investigated by using the Taguchi's experimental design method. The experiments on 14 different problems involving classification, approximation, and prediction are conducted by using a multilayer perceptron and the proposed DNM. The results suggest that the proposed learning algorithms are effective and promising for training DNM and thus make DNM more powerful in solving classification, approximation, and prediction problems.
AIMS AND OBJECTIVE: Redundant information of microarray gene expression data makes it difficult for cancer classification. Hence, it is very important for researchers to find appropriate ways to select informative genes for better identification of cancer. This study was undertaken to present a hybrid feature selection method mRMR-ICA which combines minimum redundancy maximum relevance (mRMR) with imperialist competition algorithm (ICA) for cancer classification in this paper. MATERIALS AND METHODS: The presented algorithm mRMR-ICA utilizes mRMR to delete redundant genes as preprocessing and provide the small datasets for ICA for feature selection. It will use support vector machine (SVM) to evaluate the classification accuracy for feature genes. The fitness function includes classification accuracy and the number of selected genes. RESULTS: Ten benchmark microarray gene expression datasets are used to test the performance of mRMR-ICA. Experimental results including the accuracy of cancer classification and the number of informative genes are improved for mRMR-ICA compared with the original ICA and other evolutionary algorithms. CONCLUSION: The comparison results demonstrate that mRMR-ICA can effectively delete redundant genes to ensure that the algorithm selects fewer informative genes to get better classification results. It also can shorten calculation time and improve efficiency.
Gene Expression/genetics , Microarray Analysis/statistics & numerical data , Neoplasms/classification , Algorithms , Computational Biology/statistics & numerical data , Gene Expression Profiling/methods , Humans , Models, Theoretical , Support Vector Machine
Nowadays, credit classification models are widely applied because they can help financial decision-makers to handle credit classification issues. Among them, artificial neural networks (ANNs) have been widely accepted as the convincing methods in the credit industry. In this paper, we propose a pruning neural network (PNN) and apply it to solve credit classification problem by adopting the well-known Australian and Japanese credit datasets. The model is inspired by synaptic nonlinearity of a dendritic tree in a biological neural model. And it is trained by an error back-propagation algorithm. The model is capable of realizing a neuronal pruning function by removing the superfluous synapses and useless dendrites and forms a tidy dendritic morphology at the end of learning. Furthermore, we utilize logic circuits (LCs) to simulate the dendritic structures successfully which makes PNN be implemented on the hardware effectively. The statistical results of our experiments have verified that PNN obtains superior performance in comparison with other classical algorithms in terms of accuracy and computational efficiency.
Algorithms , Neural Networks, Computer , Humans
The problem of predicting the three-dimensional (3-D) structure of a protein from its one-dimensional sequence has been called the "holy grail of molecular biology", and it has become an important part of structural genomics projects. Despite the rapid developments in computer technology and computational intelligence, it remains challenging and fascinating. In this paper, to solve it we propose a multi-objective evolutionary algorithm. We decompose the protein energy function Chemistry at HARvard Macromolecular Mechanics force fields into bond and non-bond energies as the first and second objectives. Considering the effect of solvent, we innovatively adopt a solvent-accessible surface area as the third objective. We use 66 benchmark proteins to verify the proposed method and obtain better or competitive results in comparison with the existing methods. The results suggest the necessity to incorporate the effect of solvent into a multi-objective evolutionary algorithm to improve protein structure prediction in terms of accuracy and efficiency.
Algorithms , Computational Biology/methods , Protein Conformation , Proteins , Databases, Protein , Hydrophobic and Hydrophilic Interactions , Models, Molecular , Proteins/chemistry , Proteins/genetics , Solvents , Water
With the impact of global internationalization, tourism economy has also been a rapid development. The increasing interest aroused by more advanced forecasting methods leads us to innovate forecasting methods. In this paper, the seasonal trend autoregressive integrated moving averages with dendritic neural network model (SA-D model) is proposed to perform the tourism demand forecasting. First, we use the seasonal trend autoregressive integrated moving averages model (SARIMA model) to exclude the long-term linear trend and then train the residual data by the dendritic neural network model and make a short-term prediction. As the result showed in this paper, the SA-D model can achieve considerably better predictive performances. In order to demonstrate the effectiveness of the SA-D model, we also use the data that other authors used in the other models and compare the results. It also proved that the SA-D model achieved good predictive performances in terms of the normalized mean square error, absolute percentage of error, and correlation coefficient.
Forecasting/methods , Models, Statistical , Neural Networks, Computer , Humans
