Bi-Level Spectral Feature Selection.

Unsupervised feature selection (UFS) aims to learn an indicator matrix relying on some characteristics of the high-dimensional data to identify the features to be selected. However, traditional unsupervised methods perform only at the feature level, i.e., they directly select useful features by feature ranking. Such methods do not pay any attention to the interaction information with other tasks such as classification, which severely degrades their feature selection performance. In this article, we propose an UFS method which also takes into account the classification level, and selects features that perform well both in clustering and classification. To achieve this, we design a bi-level spectral feature selection (BLSFS) method, which combines classification level and feature level. More concretely, at the classification level, we first apply the spectral clustering to generate pseudolabels, and then train a linear classifier to obtain the optimal regression matrix. At the feature level, we select useful features via maintaining the intrinsic structure of data in the embedding space with the learned regression matrix from the classification level, which in turn guides classifier training. We utilize a balancing parameter to seamlessly bridge the classification and feature levels together to construct a unified framework. A series of experiments on 12 benchmark datasets are carried out to demonstrate the superiority of BLSFS in both clustering and classification performance.

Fuzzy Adaptive Knowledge-Based Inference Neural Networks: Design and Analysis.

A novel fuzzy adaptive knowledge-based inference neural network (FAKINN) is proposed in this study. Conventional fuzzy cluster-based neural networks (FCBNNs) suffer from the challenge of a direct extraction of fuzzy rules that can capture and represent the interclass heterogeneity and intraclass homogeneity when the data possess complex structures. Moreover, the capability of the cluster-based rule generator in FCBNNs may decrease with the increase of data dimensionality. These drawbacks impede the generation of desired fuzzy rules, and affect the inference results depending on the fuzzy rules, thereby limiting their generalization ability. To address these drawbacks, an adaptive knowledge generator (AKG), consisting of the observation paradigm (OP) and clustering strategy (CS), is effectively designed to improve the generalization ability in FAKINN. The OP distills the characteristic information (CI) from data to highlight the homogeneity and heterogeneity of objects, and the CS, viz., the weighted condition-driven fuzzy clustering method (WCFCM), is proposed to summarize the CI to construct fuzzy rules. Moreover, the feedback between the OP and CS can control the dimensionality of CI, which endows FAKINN with the potential to tackle high-dimensional data. The main originality of the study focuses on the AKG and WCFCM that are proposed to develop the structural design methodology of FNNs. The performance of FAKINN is evaluated on various benchmarks with 27 comparative methods, and two real-world problems are adopted to validate its effectiveness. Experimental results show that FAKINN outperforms the comparison methods.

Local Boundary Fuzzified Rough K-Means-Based Information Granulation Algorithm Under the Principle of Justifiable Granularity.

Information granularity and information granules are fundamental concepts that permeate the entire area of granular computing. With this regard, the principle of justifiable granularity was proposed by Pedrycz, and subsequently a general two-phase framework of designing information granules based on Fuzzy C-means clustering was successfully developed. This design process leads to information granules that are likely to intersect each other in substantially overlapping clusters, which inevitably leads to some ambiguity and misperception as well as loss of semantic clarity of information granules. This limitation is largely due to imprecise description of boundary-overlapping data in the existing algorithms. To address this issue, the rough k -means clustering is introduced in an innovative way into Pedrycz's two-phase information granulation framework, together with the proposed local boundary fuzzy metric. To further strengthen the characteristics of support and inhibition of boundary-overlapping data, an augmented parametric version of the principle is refined. On this basis, a local boundary fuzzified rough k -means-based information granulation algorithm is developed. In this manner, the generated granules are unique and representative whilst ensuring clearer boundaries. The validity and performance of this algorithm are demonstrated through the results of comparative experiments.

Asymmetric Transfer Hashing With Adaptive Bipartite Graph Learning.

Thanks to the efficient retrieval speed and low storage consumption, learning to hash has been widely used in visual retrieval tasks. However, the known hashing methods assume that the query and retrieval samples lie in homogeneous feature space within the same domain. As a result, they cannot be directly applied to heterogeneous cross-domain retrieval. In this article, we propose a generalized image transfer retrieval (GITR) problem, which encounters two crucial bottlenecks: 1) the query and retrieval samples may come from different domains, leading to an inevitable domain distribution gap and 2) the features of the two domains may be heterogeneous or misaligned, bringing up an additional feature gap. To address the GITR problem, we propose an asymmetric transfer hashing (ATH) framework with its unsupervised/semisupervised/supervised realizations. Specifically, ATH characterizes the domain distribution gap by the discrepancy between two asymmetric hash functions, and minimizes the feature gap with the help of a novel adaptive bipartite graph constructed on cross-domain data. By jointly optimizing asymmetric hash functions and the bipartite graph, not only can knowledge transfer be achieved but information loss caused by feature alignment can also be avoided. Meanwhile, to alleviate negative transfer, the intrinsic geometrical structure of single-domain data is preserved by involving a domain affinity graph. Extensive experiments on both single-domain and cross-domain benchmarks under different GITR subtasks indicate the superiority of our ATH method in comparison with the state-of-the-art hashing methods.

Frequency Domain Channel-Wise Attack to CNN Classifiers in Motor Imagery Brain-Computer Interfaces.

OBJECTIVE: Convolutional neural network (CNN), a classical structure in deep learning, has been commonly deployed in the motor imagery brain-computer interface (MIBCI). Many methods have been proposed to evaluate the vulnerability of such CNN models, primarily by attacking them using direct temporal perturbations. In this work, we propose a novel attacking approach based on perturbations in the frequency domain instead. METHODS: For a given natural MI trial in the frequency domain, the proposed approach, called frequency domain channel-wise attack (FDCA), generates perturbations at each channel one after another to fool the CNN classifiers. The advances of this strategy are two-fold. First, instead of focusing on the temporal domain, perturbations are generated in the frequency domain where discriminative patterns can be extracted for motor imagery (MI) classification tasks. Second, the perturbing optimization is performed based on differential evolution algorithm in a black-box scenario where detailed model knowledge is not required. RESULTS: Experimental results demonstrate the effectiveness of the proposed FDCA which achieves a significantly higher success rate than the baselines and existing methods in attacking three major CNN classifiers on four public MI benchmarks. CONCLUSION: Perturbations generated in the frequency domain yield highly competitive results in attacking MIBCI deployed by CNN models even in a black-box setting, where the model information is well-protected. SIGNIFICANCE: To our best knowledge, existing MIBCI attack approaches are all gradient-based methods and require details about the victim model, e.g., the parameters and objective function. We provide a more flexible strategy that does not require model details but still produces an effective attack outcome.

Random Polynomial Neural Networks: Analysis and Design.

In this article, we propose the concept of random polynomial neural networks (RPNNs) realized based on the architecture of polynomial neural networks (PNNs) with random polynomial neurons (RPNs). RPNs exhibit generalized polynomial neurons (PNs) based on random forest (RF) architecture. In the design of RPNs, the target variables are no longer directly used in conventional decision trees, and the polynomial of these target variables is exploited here to determine the average prediction. Unlike the conventional performance index used in the selection of PNs, the correlation coefficient is adopted here to select the RPNs of each layer. When compared with the conventional PNs used in PNNs, the proposed RPNs exhibit the following advantages: first, RPNs are insensitive to outliers; second, RPNs can obtain the importance of each input variable after training; third, RPNs can alleviate the overfitting problem with the use of an RF structure. The overall nonlinearity of a complex system is captured by means of PNNs. Moreover, particle swarm optimization (PSO) is exploited to optimize the parameters when constructing RPNNs. The RPNNs take advantage of both RF and PNNs: it exhibits high accuracy based on ensemble learning used in the RF and is beneficial to describe high-order nonlinear relations between input and output variables stemming from PNNs. Experimental results based on a series of well-known modeling benchmarks illustrate that the proposed RPNNs outperform other state-of-the-art models reported in the literature.

Knowledge-Induced Multiple Kernel Fuzzy Clustering.

The introduction of domain knowledge opens new horizons to fuzzy clustering. Then knowledge-driven and data-driven fuzzy clustering methods come into being. To address the challenges of inadequate extraction mechanism and imperfect fusion mode in such class of methods, we propose the Knowledge-induced Multiple Kernel Fuzzy Clustering (KMKFC) algorithm. First, to extract knowledge points better, the Relative Density-based Knowledge Extraction (RDKE) method is proposed to extract high-density knowledge points close to cluster centers of real data structure, and provide initialized cluster centers. Moreover, the multiple kernel mechanism is introduced to improve the adaptability of clustering algorithm and map data to high-dimensional space, so as to better discover the differences between the data and obtain superior clustering results. Second, knowledge points generated by RDKE are integrated into KMKFC through a knowledge-influence matrix to guide the iterative process of KMKFC. Third, we also provide a strategy of automatically obtaining knowledge points, and thus propose the RDKE with Automatic knowledge acquisition (RDKE-A) method and the corresponding KMKFC-A algorithm. Then we prove the convergence of KMKFC and KMKFC-A. Finally, experimental studies demonstrate that the KMKFC and KMKFC-A algorithms perform better than thirteen comparison algorithms with regard to four evaluation indexes and the convergence speed.

Fuzzy Attention Neural Network to Tackle Discontinuity in Airway Segmentation.

Airway segmentation is crucial for the examination, diagnosis, and prognosis of lung diseases, while its manual delineation is unduly burdensome. To alleviate this time-consuming and potentially subjective manual procedure, researchers have proposed methods to automatically segment airways from computerized tomography (CT) images. However, some small-sized airway branches (e.g., bronchus and terminal bronchioles) significantly aggravate the difficulty of automatic segmentation by machine learning models. In particular, the variance of voxel values and the severe data imbalance in airway branches make the computational module prone to discontinuous and false-negative predictions, especially for cohorts with different lung diseases. The attention mechanism has shown the capacity to segment complex structures, while fuzzy logic can reduce the uncertainty in feature representations. Therefore, the integration of deep attention networks and fuzzy theory, given by the fuzzy attention layer, should be an escalated solution for better generalization and robustness. This article presents an efficient method for airway segmentation, comprising a novel fuzzy attention neural network (FANN) and a comprehensive loss function to enhance the spatial continuity of airway segmentation. The deep fuzzy set is formulated by a set of voxels in the feature map and a learnable Gaussian membership function. Different from the existing attention mechanism, the proposed channel-specific fuzzy attention addresses the issue of heterogeneous features in different channels. Furthermore, a novel evaluation metric is proposed to assess both the continuity and completeness of airway structures. The efficiency, generalization, and robustness of the proposed method have been proved by training on normal lung disease while testing on datasets of lung cancer, COVID-19, and pulmonary fibrosis.

A Fuzzy Clustering Validity Index Induced by Triple Center Relation.

The existing clustering validity indexes (CVIs) show some difficulties to produce the correct cluster number when some cluster centers are close to each other, and the separation processing mechanism appears simple. The results are imperfect in case of noisy data sets. For this reason, in this study, we come up with a novel CVI for fuzzy clustering, referred to as the triple center relation (TCR) index. The originality of this index is twofold. On the one hand, a new fuzzy cardinality is built on the strength of the maximum membership degree, and a novel compactness formula is constructed by combining it with the within-class weighted squared error sum. On the other hand, starting from the minimum distance between different cluster centers, the mean distance as well as the sample variance of cluster centers in the statistical sense are further integrated. These three factors are combined by means of product to form a triple characterization of the relationship between cluster centers, and hence a 3-D expression pattern of separability is formed. Subsequently, the TCR index is put forward by combining the compactness formula with the separability expression pattern. By virtue of the degenerate structure of hard clustering, we show an important property of the TCR index. Finally, based on the fuzzy C -means (FCMs) clustering algorithm, experimental studies were conducted on 36 data sets (incorporating artificial and UCI data sets, images, the Olivetti face database). For comparative purposes, 10 CVIs were also considered. It has been found that the proposed TCR index performs best in finding the correct cluster number, and has excellent stability.

Artificial Proprioceptive Reflex Warning Using EMG in Advanced Driving Assistance System.

A frequent cause of auto accidents is disregarding the proximal traffic of an ego-vehicle during lane changing. Presumably, in a split-second-decision situation we may prevent an accident by predicting the intention of a driver before her action onset using the neural signals data, meanwhile building the perception of surroundings of a vehicle using optical sensors. The prediction of an intended action fused with the perception can generate an instantaneous signal that may replenish the driver's ignorance about the surroundings. This study examines electromyography (EMG) signals to predict intention of a driver along perception building stack of an autonomous driving system (ADS) in building an advanced driving assistant system (ADAS). EMG are classified into left-turn and right-turn intended actions and lanes and object detection with camera and Lidar are used to detect vehicles approaching from behind. A warning issued before the action onset, can alert a driver and may save her from a fatal accident. The use of neural signals for intended action prediction is a novel addition to camera, radar and Lidar based ADAS systems. Furthermore, the study demonstrates efficacy of the proposed idea with experiments designed to classify online and offline EMG data in real-world settings with computation time and the latency of communicated warnings.

An Approach to Semantic-Aware Heterogeneous Network Embedding for Recommender Systems.

Recent studies on heterogeneous information network (HIN) embedding-based recommendations have encountered challenges. These challenges are related to the data heterogeneity of the associated unstructured attribute or content (e.g., text-based summary/description) of users and items in the context of HIN. In order to address these challenges, in this article, we propose a novel approach of semantic-aware HIN embedding-based recommendation, called SemHE4Rec. In our proposed SemHE4Rec model, we define two embedding techniques for efficiently learning the representations of both users and items in the context of HIN. These rich-structural user and item representations are then used to facilitate the matrix factorization (MF) process. The first embedding technique is a traditional co-occurrence representation learning (CoRL) approach which aims to learn the co-occurrence of structural features of users and items. These structural features are represented for their interconnections in terms of meta-paths. In order to do that, we adopt the well-known meta-path-based random walk strategy and heterogeneous Skip-gram architecture. The second embedding approach is a semantic-aware representation learning (SRL) method. The SRL embedding technique is designed to focus on capturing the unstructured semantic relations between users and item content for the recommendation task. Finally, all the learned representations of users and items are then jointly combined and optimized while integrating with the extended MF for the recommendation task. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed SemHE4Rec in comparison with the recent state-of-the-art HIN embedding-based recommendation techniques, and reveal that the joint text-based and co-occurrence-based representation learning can help to improve the recommendation performance.

Coupled Multimodal Emotional Feature Analysis Based on Broad-Deep Fusion Networks in Human-Robot Interaction.

A coupled multimodal emotional feature analysis (CMEFA) method based on broad-deep fusion networks, which divide multimodal emotion recognition into two layers, is proposed. First, facial emotional features and gesture emotional features are extracted using the broad and deep learning fusion network (BDFN). Considering that the bi-modal emotion is not completely independent of each other, canonical correlation analysis (CCA) is used to analyze and extract the correlation between the emotion features, and a coupling network is established for emotion recognition of the extracted bi-modal features. Both simulation and application experiments are completed. According to the simulation experiments completed on the bimodal face and body gesture database (FABO), the recognition rate of the proposed method has increased by 1.15% compared to that of the support vector machine recursive feature elimination (SVMRFE) (without considering the unbalanced contribution of features). Moreover, by using the proposed method, the multimodal recognition rate is 21.22%, 2.65%, 1.61%, 1.54%, and 0.20% higher than those of the fuzzy deep neural network with sparse autoencoder (FDNNSA), ResNet-101 + GFK, C3D + MCB + DBN, the hierarchical classification fusion strategy (HCFS), and cross-channel convolutional neural network (CCCNN), respectively. In addition, preliminary application experiments are carried out on our developed emotional social robot system, where emotional robot recognizes the emotions of eight volunteers based on their facial expressions and body gestures.

Prioritizing tasks in software development: A systematic literature review.

Task prioritization is one of the most researched areas in software development. Given the huge number of papers written on the topic, it might be challenging for IT practitioners-software developers, and IT project managers-to find the most appropriate tools or methods developed to date to deal with this important issue. The main goal of this work is therefore to review the current state of research and practice on task prioritization in the Software Engineering domain and to individuate the most effective ranking tools and techniques used in the industry. For this purpose, we conducted a systematic literature review guided and inspired by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, otherwise known as the PRISMA statement. Based on our analysis, we can make a number of important observations for the field. Firstly, we found that most of the task prioritization approaches developed to date involve a specific type of prioritization strategy-bug prioritization. Secondly, the most recent works we review investigate task prioritization in terms of "pull request prioritization" and "issue prioritization," (and we speculate that the number of such works will significantly increase due to the explosion of version control and issue management software systems). Thirdly, we remark that the most frequently used metrics for measuring the quality of a prioritization model are f-score, precision, recall, and accuracy.

Multi-objective combinatorial optimization analysis of the recycling of retired new energy electric vehicle power batteries in a sustainable dynamic reverse logistics network.

The recycling of retired new energy vehicle power batteries produces economic benefits and promotes the sustainable development of environment and society. However, few attentions have been paid to the design and optimization of sustainable reverse logistics network for the recycling of retired power batteries. To this end, we develop a six-level sustainable dynamic reverse logistics network model from the perspectives of economy, environment, and society. We solve the multi-objective combinatorial optimization model to explore the layout of the sustainable reverse logistics network for retired new energy vehicle power batteries recycling. A case study is implemented to verify the effectiveness of the proposed model. The results show that (a) the facility nodes near the front of the network fluctuate more by opening and closing; (b) the dynamic reverse logistics network is superior to its static counterpart; and (c) cooperation cost changes affect the transaction volume between third-party and cooperative enterprises and total network cost.

Center transfer for supervised domain adaptation.

Domain adaptation (DA) is a popular strategy for pattern recognition and classification tasks. It leverages a large amount of data from the source domain to help train the model applied in the target domain. Supervised domain adaptation (SDA) approaches are desirable when only few labeled samples from the target domain are available. They can be easily adopted in many real-world applications where data collection is expensive. In this study, we propose a new supervision signal, namely center transfer loss (CTL), to efficiently align features under the SDA setting in the deep learning (DL) field. Unlike most previous SDA methods that rely on pairing up training samples, the proposed loss is trainable only using one-stream input based on the mini-batch strategy. The CTL exhibits two main functionalities in training to increase the performance of DL models, i.e., domain alignment and increasing the feature's discriminative power. The hyper-parameter to balance these two functionalities is waived in CTL, which is the second improvement from the previous approaches. Extensive experiments completed on well-known public datasets show that the proposed method performs better than recent state-of-the-art approaches.

Design of Granular Model: A Method Driven by Hyper-Box Iteration Granulation.

Recently, granular models have been highlighted in system modeling and applied to many fields since their outcomes are information granules supporting human-centric comprehension and reasoning. In this study, a design method of granular model driven by hyper-box iteration granulation is proposed. The method is composed mainly of partition of input space, formation of input hyper-box information granules with confidence levels, and granulation of output data corresponding to input hyper-box information granules. Among them, the formation of input hyper-box information granules is realized through performing the hyper-box iteration granulation algorithm governed by information granularity on input space, and the granulation of out data corresponding to input hyper-box information granules is completed by the improved principle of justifiable granularity to produce triangular fuzzy information granules. Compared with the existing granular models, the resulting one can yield the more accurate numeric and preferable granular outcomes simultaneously. Experiments completed on the synthetic and publicly available datasets demonstrate the superiority of the granular model designed by the proposed method at granular and numeric levels. Also, the impact of parameters involved in the proposed design method on the performance of ensuing granular model is explored.

Robust and Precise Facial Landmark Detection by Self-Calibrated Pose Attention Network.

Current fully supervised facial landmark detection methods have progressed rapidly and achieved remarkable performance. However, they still suffer when coping with faces under large poses and heavy occlusions for inaccurate facial shape constraints and insufficient labeled training samples. In this article, we propose a semisupervised framework, that is, a self-calibrated pose attention network (SCPAN) to achieve more robust and precise facial landmark detection in challenging scenarios. To be specific, a boundary-aware landmark intensity (BALI) field is proposed to model more effective facial shape constraints by fusing boundary and landmark intensity field information. Moreover, a self-calibrated pose attention (SCPA) model is designed to provide a self-learned objective function that enforces intermediate supervision without label information by introducing a self-calibrated mechanism and a pose attention mask. We show that by integrating the BALI fields and SCPA model into a novel SCPAN, more facial prior knowledge can be learned and the detection accuracy and robustness of our method for faces with large poses and heavy occlusions have been improved. The experimental results obtained for challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

A Design of Granular Classifier Based on Granular Data Descriptors.

Designing effective and efficient classifiers is a challenging task given the facts that data may exhibit different geometric structures and complex intrarelationships may exist within data. As a fundamental component of granular computing, information granules play a key role in human cognition. Therefore, it is of great interest to develop classifiers based on information granules such that highly interpretable human-centric models with higher accuracy can be constructed. In this study, we elaborate on a novel design methodology of granular classifiers in which information granules play a fundamental role. First, information granules are formed on the basis of labeled patterns following the principle of justifiable granularity. The diversity of samples embraced by each information granule is quantified and controlled in terms of the entropy criterion. This design implies that the information granules constructed in this way form sound homogeneous descriptors characterizing the structure and the diversity of available experimental data. Next, granular classifiers are built in the presence of formed information granules. The classification result for any input instance is determined by summing the contents of the related information granules weighted by membership degrees. The experiments concerning both synthetic data and publicly available datasets demonstrate that the proposed models exhibit better prediction abilities than some commonly encountered classifiers (namely, linear regression, support vector machine, Naïve Bayes, decision tree, and neural networks) and come with enhanced interpretability.

Training Novel Adaptive Fuzzy Cognitive Map by Knowledge-Guidance Learning Mechanism for Large-Scale Time-Series Forecasting.

A fuzzy cognitive map (FCM) is a graph-based knowledge representation model wherein the connections of the nodes (edges) represent casual relationships between the knowledge items associated with the nodes. This model has been applied to solve various modeling tasks including forecasting time series. In the original FCM-based forecasting model, causal relationships among concepts of the FCM remain unchanged. However, causal relationships may change in time. Therefore, we propose a new learning method for training an FCM resulting in an adaptive FCM which consists of several sub-FCMs. It can select different sub-FCMs at different moments. In an active processing scenario, in which we deal with a large-scale time series with new data being continuously generated, a forecasting model built on the old data should be updated when the new data arrive. Furthermore, retraining an FCM from scratch entails increasing computing overhead that will become a serious obstacle in many practical scenarios. To overcome the above-mentioned shortcomings, this study offers an original design setting in which the FCM is updated by knowledge-guidance learning mechanism for the first time. Compared with the existing classical forecasting models, the proposed model shows higher accuracy and efficiency. Its increased performance is demonstrated through a series of reported experimental studies.

KNNENS: A k-Nearest Neighbor Ensemble-Based Method for Incremental Learning Under Data Stream With Emerging New Classes.

In this brief, we investigate the problem of incremental learning under data stream with emerging new classes (SENC). In the literature, existing approaches encounter the following problems: 1) yielding high false positive for the new class; i) having long prediction time; and 3) having access to true labels for all instances, which is unrealistic and unacceptable in real-life streaming tasks. Therefore, we propose the k -Nearest Neighbor ENSemble-based method (KNNENS) to handle these problems. The KNNENS is effective to detect the new class and maintains high classification performance for known classes. It is also efficient in terms of run time and does not require true labels of new class instances for model update, which is desired in real-life streaming classification tasks. Experimental results show that the KNNENS achieves the best performance on four benchmark datasets and three real-world data streams in terms of accuracy and F1-measure and has a relatively fast run time compared to four reference methods. Codes are available at https://github.com/Ntriver/KNNENS.