Feature Selection and Classification of Clinical Datasets Using Bioinspired Algorithms and Super Learner.

A computer-aided diagnosis (CAD) system that employs a super learner to diagnose the presence or absence of a disease has been developed. Each clinical dataset is preprocessed and split into training set (60%) and testing set (40%). A wrapper approach that uses three bioinspired algorithms, namely, cat swarm optimization (CSO), krill herd (KH) ,and bacterial foraging optimization (BFO) with the classification accuracy of support vector machine (SVM) as the fitness function has been used for feature selection. The selected features of each bioinspired algorithm are stored in three separate databases. The features selected by each bioinspired algorithm are used to train three back propagation neural networks (BPNN) independently using the conjugate gradient algorithm (CGA). Classifier testing is performed by using the testing set on each trained classifier, and the diagnostic results obtained are used to evaluate the performance of each classifier. The classification results obtained for each instance of the testing set of the three classifiers and the class label associated with each instance of the testing set will be the candidate instances for training and testing the super learner. The training set comprises of 80% of the instances, and the testing set comprises of 20% of the instances. Experimentation has been carried out using seven clinical datasets from the University of California Irvine (UCI) machine learning repository. The super learner has achieved a classification accuracy of 96.83% for Wisconsin diagnostic breast cancer dataset (WDBC), 86.36% for Statlog heart disease dataset (SHD), 94.74% for hepatocellular carcinoma dataset (HCC), 90.48% for hepatitis dataset (HD), 81.82% for vertebral column dataset (VCD), 84% for Cleveland heart disease dataset (CHD), and 70% for Indian liver patient dataset (ILP).

A neurodynamic optimization approach to supervised feature selection via fractional programming.

Feature selection is an important issue in machine learning and data mining. Most existing feature selection methods are greedy in nature thus are prone to sub-optimality. Though some global feature selection methods based on unsupervised redundancy minimization can potentiate clustering performance improvements, their efficacy for classification may be limited. In this paper, a neurodynamics-based holistic feature selection approach is proposed via feature redundancy minimization and relevance maximization. An information-theoretic similarity coefficient matrix is defined based on multi-information and entropy to measure feature redundancy with respect to class labels. Supervised feature selection is formulated as a fractional programming problem based on the similarity coefficients. A neurodynamic approach based on two one-layer recurrent neural networks is developed for solving the formulated feature selection problem. Experimental results with eight benchmark datasets are discussed to demonstrate the global convergence of the neural networks and superiority of the proposed neurodynamic approach to several existing feature selection methods in terms of classification accuracy, precision, recall, and F-measure.

Multi-scale Attention Convolutional Neural Network for time series classification.

With the rapid increase of data availability, time series classification (TSC) has arisen in a wide range of fields and drawn great attention of researchers. Recently, hundreds of TSC approaches have been developed, which can be classified into two categories: traditional and deep learning based TSC methods. However, it remains challenging to improve accuracy and model generalization ability. Therefore, we investigate a novel end-to-end model based on deep learning named as Multi-scale Attention Convolutional Neural Network (MACNN) to solve the TSC problem. We first apply the multi-scale convolution to capture different scales of information along the time axis by generating different scales of feature maps. Then an attention mechanism is proposed to enhance useful feature maps and suppress less useful ones by learning the importance of each feature map automatically. MACNN addresses the limitation of single-scale convolution and equal weight feature maps. We conduct a comprehensive evaluation of 85 UCR standard datasets and the experimental results show that our proposed approach achieves the best performance and outperforms the other traditional and deep learning based methods by a large margin.

Stacked DeBERT: All attention in incomplete data for text classification.

In this paper, we propose Stacked DeBERT, short for StackedDenoising Bidirectional Encoder Representations from Transformers. This novel model improves robustness in incomplete data, when compared to existing systems, by designing a novel encoding scheme in BERT, a powerful language representation model solely based on attention mechanisms. Incomplete data in natural language processing refer to text with missing or incorrect words, and its presence can hinder the performance of current models that were not implemented to withstand such noises, but must still perform well even under duress. This is due to the fact that current approaches are built for and trained with clean and complete data, and thus are not able to extract features that can adequately represent incomplete data. Our proposed approach consists of obtaining intermediate input representations by applying an embedding layer to the input tokens followed by vanilla transformers. These intermediate features are given as input to novel denoising transformers which are responsible for obtaining richer input representations. The proposed approach takes advantage of stacks of multilayer perceptrons for the reconstruction of missing words' embeddings by extracting more abstract and meaningful hidden feature vectors, and bidirectional transformers for improved embedding representation. We consider two datasets for training and evaluation: the Chatbot Natural Language Understanding Evaluation Corpus and Kaggle's Twitter Sentiment Corpus. Our model shows improved F1-scores and better robustness in informal/incorrect texts present in tweets and in texts with Speech-to-Text error in the sentiment and intent classification tasks.1.

CCFS: A Confidence-Based Cost-Effective Feature Selection Scheme for Healthcare Data Classification.

Feature selection (FS) is one of the fundamental data processing techniques in various machine learning algorithms, especially for classification of healthcare data. However, it is a challenging issue due to the large search space. Binary Particle Swarm Optimization (BPSO) is an efficient evolutionary computation technique, and has been widely used in FS. In this paper, we proposed a Confidence-based and Cost-effective feature selection (CCFS) method using BPSO to improve the performance of healthcare data classification. Specifically, first, CCFS improves search effectiveness by developing a new updating mechanism that designs the feature confidence to explicitly take into account the fine-grained impact of each dimension in the particle on the classification performance. The feature confidence is composed of two measurements: the correlation between feature and categories, and historically selected frequency of each feature. Second, considering the fact that the acquisition costs of different features are naturally different, especially for medical data, and should be fully taken into account in practical applications, besides the classification performance, the feature cost and the feature reduction ratio are comprehensively incorporated into the design of fitness function. The proposed method has been verified in various UCI public datasets and compared with various benchmark schemes. The thoroughly experimental results show the effectiveness of the proposed method, in terms of accuracy and feature selection cost.

DS-GCNs: Connectome Classification using Dynamic Spectral Graph Convolution Networks with Assistant Task Training.

Functional connectivity (FC) matrices measure the regional interactions in the brain and have been widely used in neurological brain disease classification. A brain network, also named as connectome, could form a graph structure naturally, the nodes of which are brain regions and the edges are interregional connectivity. Thus, in this study, we proposed novel graph convolutional networks (GCNs) to extract efficient disease-related features from FC matrices. Considering the time-dependent nature of brain activity, we computed dynamic FC matrices with sliding windows and implemented a graph convolution-based LSTM (long short-term memory) layer to process dynamic graphs. Moreover, the demographics of patients were also used as additional outputs to guide the classification. In this paper, we proposed to utilize the demographic information as extra outputs and to share parameters among three networks predicting subject status, gender, and age, which serve as assistant tasks. We tested the performance of the proposed architecture in ADNI II dataset to classify Alzheimer's disease patients from normal controls. The classification accuracy, sensitivity, and specificity reach 90.0%, 91.7%, and 88.6%, respectively, on ADNI II dataset.

Developing an Index of Medical Conditions Associated with Outcomes after Moderate-to-Severe Traumatic Brain Injury.

Medical conditions co-occurring with traumatic brain injury (TBI) are associated with outcomes, and comorbidity indices such as Charlson and Elixhauser are used in TBI research, but they are not TBI specific. The purpose of this research was to develop an index or indices of medical conditions, identified in acute care after moderate to severe TBI, that are associated with outcomes at rehabilitation discharge. Using the TBI Model Systems National Database, the International Classification of Diseases, Ninth Revision (ICD-9) and Tenth Revision (ICD-10) codes of 8988 participants were converted to Healthcare Cost and Utilization Project (HCUP) diagnostic categories. Poisson regression models were built predicting Disability Rating Scale and Functional Independence Measure Cognitive and Motor subscale scores from HCUP categories after controlling for demographic and injury characteristics. Unweighted, weighted, and anchored indices based on the outcome models predicted 7.5-14.3% of the variance in the observed outcomes. When the indices were applied to a new validation sample of 1613 cases, however, only 2.6-6.6% of the observed outcomes were predicted. Therefore, no models or indices were recommended for future use, but several study findings are highlighted suggesting the importance and the potential for future research in this area.

Modality independent adversarial network for generalized zero shot image classification.

Zero Shot Learning (ZSL) aims to classify images of unseen target classes by transferring knowledge from source classes through semantic embeddings. The core of ZSL research is to embed both visual representation of object instance and semantic description of object class into a joint latent space and learn cross-modal (visual and semantic) latent representations. However, the learned representations by existing efforts often fail to fully capture the underlying cross-modal semantic consistency, and some of the representations are very similar and less discriminative. To circumvent these issues, in this paper, we propose a novel deep framework, called Modality Independent Adversarial Network (MIANet) for Generalized Zero Shot Learning (GZSL), which is an end-to-end deep architecture with three submodules. First, both visual feature and semantic description are embedded into a latent hyper-spherical space, where two orthogonal constraints are employed to ensure the learned latent representations discriminative. Second, a modality adversarial submodule is employed to make the latent representations independent of modalities to make the shared representations grab more cross-modal high-level semantic information during training. Third, a cross reconstruction submodule is proposed to reconstruct latent representations into the counterparts instead of themselves to make them capture more modality irrelevant information. Comprehensive experiments on five widely used benchmark datasets are conducted on both GZSL and standard ZSL settings, and the results show the effectiveness of our proposed method.

Classification of drug use patterns.

Characterizing long-term prescription data is challenging due to the time-varying nature of drug use. Conventional approaches summarize time-varying data into categorical variables based on simple measures, such as cumulative dose, while ignoring patterns of use. The loss of information can lead to misclassification and biased estimates of the exposure-outcome association. We introduce a classification method to characterize longitudinal prescription data with an unsupervised machine learning algorithm. We used administrative databases covering virtually all 1.3 million residents of Manitoba and explicitly designed features to describe the average dose, proportion of days covered (PDC), dose change, and dose variability, and clustered the resulting feature space using K-means clustering. We applied this method to metformin use in diabetes patients. We identified 27,786 metformin users and showed that the feature distributions of their metformin use are stable for varying the lengths of follow-up and that these distributions have clear interpretations. We found six distinct metformin user groups: patients with intermittent use, decreasing dose, increasing dose, high dose, and two medium dose groups (one with stable dose and one with highly variable use). Patients in the varying and decreasing dose groups had a higher chance of progression of diabetes than other patients. The method presented in this paper allows for characterization of drug use into distinct and clinically relevant groups in a way that cannot be obtained from merely classifying use by quantiles of overall use.

Data classification based on fractional order gradient descent with momentum for RBF neural network.

The weight-updating methods have played an important role in improving the performance of neural networks. To ameliorate the oscillating phenomenon in training radial basis function (RBF) neural network, a fractional order gradient descent with momentum method for updating the weights of RBF neural network (FOGDM-RBF) is proposed for data classification. Its convergence is proved. In order to speed up the convergence process, an adaptive learning rate is used to adjust the training process. The Iris data set and MNIST data set are used to test the proposed algorithm. The results verify the theoretical results of the proposed algorithm such as its monotonicity and convergence. Some non-parametric statistical tests such as Friedman test and Quade test are taken for the comparison of the proposed algorithm with other algorithms. The influence of fractional order, learning rate and batch size is analysed and compared. Error analysis shows that the algorithm can effectively accelerate the convergence speed of gradient descent method and improve its performance with high accuracy and validity.

Mechanisms for the reduction of caffeine consumption: What, how and why.

BACKGROUND: Anecdotal evidence suggests consumers of caffeine self-administer strategies to reduce consumption, but little is known of what these strategies are or how they are implemented. This study aimed to understand the lived experience of reducing caffeine consumption including specific techniques (what) and implementation strategies (how), harm and withdrawal symptoms (why). METHODS: We developed a classification system through an inductive and deductive approach and applied it to a large dataset derived from online sources. RESULTS: A total of 112 internet sources were identified, containing 2,682 different strategies. The classification system identified 22 categories of Behaviour Change Techniques (BCT): 10 categories were directly aligned with a BCT, one was split into two categories (substance and behavioural substitution), six represented a cluster of BCT's (e.g., withdrawal management and maintaining momentum) and four appeared to uniquely represent a consumer perspective (e.g., realisation of a problem). The most common techniques were substance substitution, seek knowledge and information, avoidance of caffeine and identify prompts for change. The most frequently perceived benefit was the stimulating effects of caffeine and a feeling of mental alertness. The most frequently cited harms were sleep problems including insomnia and concerns about dependence (or addiction) to caffeine. We found 16 categories of withdrawal symptoms. The most frequently endorsed symptom was headaches, followed by fatigue, exhaustion and low energy. CONCLUSIONS: Consumers use a wide range of techniques when attempting to reduce caffeine consumption. Treatment approaches are focused on fading, but the current study found consumers most frequently focus on substance and behavioural substitution.

Cognitive phenotypes in temporal lobe epilepsy utilizing data- and clinically driven approaches: Moving toward a new taxonomy.

OBJECTIVE: To identify cognitive phenotypes in temporal lobe epilepsy (TLE) and test their reproducibility in a large, multi-site cohort of patients using both data-driven and clinically driven approaches. METHOD: Four-hundred seven patients with TLE who underwent a comprehensive neuropsychological evaluation at one of four epilepsy centers were included. Scores on tests of verbal memory, naming, fluency, executive function, and psychomotor speed were converted into z-scores based on 151 healthy controls (HCs). For the data-driven method, cluster analysis (k-means) was used to determine the optimal number of clusters. For the clinically driven method, impairment was defined as >1.5 standard deviations below the mean of the HC, and patients were classified into groups based on the pattern of impairment. RESULTS: Cluster analysis revealed a three-cluster solution characterized by (a) generalized impairment (29%), (b) language and memory impairment (28%), and (c) no impairment (43%). Based on the clinical criteria, the same broad categories were identified, but with a different distribution: (a) generalized impairment (37%), (b) language and memory impairment (30%), and (c) no impairment (33%). There was a 82.6% concordance rate with good agreement (κ = .716) between the methods. Forty-eight patients classified as having a normal profile based on cluster analysis were classified as having generalized impairment (n = 16) or an isolated language/memory impairment (n = 32) based on the clinical criteria. Patients with generalized impairment had a longer disease duration and patients with no impairment had more years of education. However, patients demonstrating the classic TLE profile (ie, language and memory impairment) were not more likely to have an earlier age at onset or mesial temporal sclerosis. SIGNIFICANCE: We validate previous findings from single-site studies that have identified three unique cognitive phenotypes in TLE and offer a means of translating the patterns into a clinical diagnostic criteria, representing a novel taxonomy of neuropsychological status in TLE.

Classification of Current Procedural Terminology Codes from Electronic Health Record Data Using Machine Learning.

BACKGROUND: Accurate anesthesiology procedure code data are essential to quality improvement, research, and reimbursement tasks within anesthesiology practices. Advanced data science techniques, including machine learning and natural language processing, offer opportunities to develop classification tools for Current Procedural Terminology codes across anesthesia procedures. METHODS: Models were created using a Train/Test dataset including 1,164,343 procedures from 16 academic and private hospitals. Five supervised machine learning models were created to classify anesthesiology Current Procedural Terminology codes, with accuracy defined as first choice classification matching the institutional-assigned code existing in the perioperative database. The two best performing models were further refined and tested on a Holdout dataset from a single institution distinct from Train/Test. A tunable confidence parameter was created to identify cases for which models were highly accurate, with the goal of at least 95% accuracy, above the reported 2018 Centers for Medicare and Medicaid Services (Baltimore, Maryland) fee-for-service accuracy. Actual submitted claim data from billing specialists were used as a reference standard. RESULTS: Support vector machine and neural network label-embedding attentive models were the best performing models, respectively, demonstrating overall accuracies of 87.9% and 84.2% (single best code), and 96.8% and 94.0% (within top three). Classification accuracy was 96.4% in 47.0% of cases using support vector machine and 94.4% in 62.2% of cases using label-embedding attentive model within the Train/Test dataset. In the Holdout dataset, respective classification accuracies were 93.1% in 58.0% of cases and 95.0% among 62.0%. The most important feature in model training was procedure text. CONCLUSIONS: Through application of machine learning and natural language processing techniques, highly accurate real-time models were created for anesthesiology Current Procedural Terminology code classification. The increased processing speed and a priori targeted accuracy of this classification approach may provide performance optimization and cost reduction for quality improvement, research, and reimbursement tasks reliant on anesthesiology procedure codes.

Understanding Opioid Use Disorder (OUD) using tree-based classifiers.

BACKGROUND: Opioid Use Disorder (OUD), defined as a physical or psychological reliance on opioids, is a public health epidemic. Identifying adults likely to develop OUD can help public health officials in planning effective intervention strategies. The aim of this paper is to develop a machine learning approach to predict adults at risk for OUD and to identify interactions between various characteristics that increase this risk. METHODS: In this approach, a data set was curated using the responses from the 2016 edition of the National Survey on Drug Use and Health (NSDUH). Using this data set, tree-based classifiers (decision tree and random forest) were trained, while employing downsampling to handle class imbalance. Predictions from the tree-based classifiers were also compared to the results from a logistic regression model. The results from the three classifiers were then interpreted synergistically to highlight individual characteristics and their interplay that pose a risk for OUD. RESULTS: Random forest predicted adults at risk for OUD with remarkable accuracy, with the average area under the Receiver-Operating-Characteristics curve (AUC) over 0.89, even though the prevalence of OUD was only about 1 %. It showed a slight improvement over logistic regression. Logistic regression identified statistically significant characteristics, while random forest ranked the predictors in order of their contribution to OUD prediction. Early initiation of marijuana (before 18 years) emerged as the dominant predictor. Decision trees revealed that early marijuana initiation especially increased the risk if individuals: (i) were between 18-34 years of age, or (ii) had incomes less than $49,000, or (iii) were of Hispanic and White heritage, or (iv) were on probation, or (v) lived in neighborhoods with easy access to drugs. CONCLUSIONS: Machine learning can accurately predict adults at risk for OUD, and identify interactions among the factors that pronounce this risk. Curbing early initiation of marijuana may be an effective prevention strategy against opioid addiction, especially in high risk groups.

Heterogeneous Multilayer Generalized Operational Perceptron.

The traditional multilayer perceptron (MLP) using a McCulloch-Pitts neuron model is inherently limited to a set of neuronal activities, i.e., linear weighted sum followed by nonlinear thresholding step. Previously, generalized operational perceptron (GOP) was proposed to extend the conventional perceptron model by defining a diverse set of neuronal activities to imitate a generalized model of biological neurons. Together with GOP, a progressive operational perceptron (POP) algorithm was proposed to optimize a predefined template of multiple homogeneous layers in a layerwise manner. In this paper, we propose an efficient algorithm to learn a compact, fully heterogeneous multilayer network that allows each individual neuron, regardless of the layer, to have distinct characteristics. Based on the complexity of the problem, the proposed algorithm operates in a progressive manner on a neuronal level, searching for a compact topology, not only in terms of depth but also width, i.e., the number of neurons in each layer. The proposed algorithm is shown to outperform other related learning methods in extensive experiments on several classification problems.

Deep Neural Architectures for Highly Imbalanced Data in Bioinformatics.

In the postgenome era, many problems in bioinformatics have arisen due to the generation of large amounts of imbalanced data. In particular, the computational classification of precursor microRNA (pre-miRNA) involves a high imbalance in the classes. For this task, a classifier is trained to identify RNA sequences having the highest chance of being miRNA precursors. The big issue is that well-known pre-miRNAs are usually just a few in comparison to the hundreds of thousands of candidate sequences in a genome, which results in highly imbalanced data. This imbalance has a strong influence on most standard classifiers and, if not properly addressed, the classifier is not able to work properly in a real-life scenario. This work provides a comparative assessment of recent deep neural architectures for dealing with the large imbalanced data issue in the classification of pre-miRNAs. We present and analyze recent architectures in a benchmark framework with genomes of animals and plants, with increasing imbalance ratios up to 1:2000. We also propose a new graphical way for comparing classifiers performance in the context of high-class imbalance. The comparative results obtained show that, at a very high imbalance, deep belief neural networks can provide the best performance.

Online Incremental Classification Resonance Network and Its Application to Human-Robot Interaction.

In human-robot interaction (HRI), classification is one of the most important problems, and it is essential particularly when the robot recognizes the surroundings and chooses a reaction based on a certain situation. Each interaction is different since new people appear or the environment changes, and the robot should be able to adapt to different situations during a brief interaction. Thus, it is imperative that the classification is performed incrementally in real time. In this sense, we propose an online incremental classification resonance network (OICRN) that enables incremental class learning in multi-class classification with high performance online. In OICRN, a scale-preserving projection process is introduced to use the raw input vectors online without a normalization process in advance. The integrated network of the convolutional neural network (CNN) for feature extraction and the OICRN for classification is applied to a robotic system that learns human identities through HRIs. To demonstrate the effectiveness of our network, experiments are carried out on benchmark data sets and on a humanoid robot, Mybot, developed in the Robot Intelligence Technology Laboratory, KAIST.

Neural network-based approaches for biomedical relation classification: A review.

The explosive growth of biomedical literature has created a rich source of knowledge, such as that on protein-protein interactions (PPIs) and drug-drug interactions (DDIs), locked in unstructured free text. Biomedical relation classification aims to automatically detect and classify biomedical relations, which has great benefits for various biomedical research and applications. In the past decade, significant progress has been made in biomedical relation classification. With the advance of neural network methodology, neural network-based approaches have been applied in biomedical relation classification and achieved state-of-the-art performance for some public datasets and shared tasks. In this review, we describe the recent advancement of neural network-based approaches for classifying biomedical relations. We summarize the available corpora and introduce evaluation metrics. We present the general framework for neural network-based approaches in biomedical relation extraction and pretrained word embedding resources. We discuss neural network-based approaches, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs). We conclude by describing the remaining challenges and outlining future directions.

Subcortical heterotopic gray matter brain malformations: Classification study of 107 individuals.

OBJECTIVE: To better evaluate the imaging spectrum of subcortical heterotopic gray matter brain malformations (subcortical heterotopia [SUBH]), we systematically reviewed neuroimaging and clinical data of 107 affected individuals. METHODS: SUBH is defined as heterotopic gray matter, located within the white matter between the cortex and lateral ventricles. Four large brain malformation databases were searched for individuals with these malformations; data on imaging, clinical outcomes, and results of molecular testing were systematically reviewed and integrated with all previously published subtypes to create a single classification system. RESULTS: Review of the databases revealed 107 patients with SUBH, the large majority scanned during childhood (84%), including more than half before 4 years (59%). Although most individuals had cognitive or motor disability, 19% had normal development. Epilepsy was documented in 69%. Additional brain malformations were common and included abnormalities of the corpus callosum (65/102 [64%]), and, often, brainstem or cerebellum (47/106 [44%]). Extent of the heterotopic gray matter brain malformations (unilateral or bilateral) did not influence the presence or age at onset of seizures. Although genetic testing was not systematically performed in this group, the sporadic occurrence and frequent asymmetry suggests either postzygotic mutations or prenatal disruptive events. Several rare, bilateral forms are caused by mutations in genes associated with cell proliferation and polarity (EML1, TUBB, KATNB1, CENPJ, GPSM2). CONCLUSION: This study reveals a broad clinical and imaging spectrum of heterotopic malformations and provides a framework for their classification.