A Hybrid EEG-based Emotion Recognition Approach Using Wavelet Convolutional Neural Networks and Support Vector Machine.

Introduction: Nowadays, deep learning and convolutional neural networks (CNNs) have become widespread tools in many biomedical engineering studies. CNN is an end-to-end tool, which makes the processing procedure integrated, but in some situations, this processing tool requires to be fused with machine learning methods to be more accurate. Methods: In this paper, a hybrid approach based on deep features extracted from wavelet CNNs (WCNNs) weighted layers and multiclass support vector machine (MSVM) was proposed to improve the recognition of emotional states from electroencephalogram (EEG) signals. First, EEG signals were preprocessed and converted to Time-Frequency (T-F) color representation or scalogram using the continuous wavelet transform (CWT) method. Then, scalograms were fed into four popular pre-trained CNNs, AlexNet, ResNet-18, VGG-19, and Inception-v3 to fine-tune them. Then, the best feature layer from each one was used as input to the MSVM method to classify four quarters of the valence-arousal model. Finally, the subject-independent leave-one-subject-out criterion was used to evaluate the proposed method on DEAP and MAHNOB-HCI databases. Results: Results showed that extracting deep features from the earlier convolutional layer of ResNet-18 (Res2a) and classifying using the MSVM increased the average accuracy, precision, and recall by about 20% and 12% for MAHNOB-HCI and DEAP databases, respectively. Also, combining scalograms from four regions of pre-frontal, frontal, parietal, and parietal-occipital and two regions of frontal and parietal achieved the higher average accuracy of 77.47% and 87.45% for MAHNOB-HCI and DEAP databases, respectively. Conclusion: Combining CNN and MSVM increased the recognition of emotion from EEG signals and the results were comparable to state-of-the art studies.

Sentiment Analysis of Insomnia-Related Tweets via a Combination of Transformers Using Dempster-Shafer Theory: Pre- and Peri-COVID-19 Pandemic Retrospective Study.

BACKGROUND: The COVID-19 pandemic has imposed additional stress on population health that may result in a change of sleeping behavior. OBJECTIVE: In this study, we hypothesized that using natural language processing to explore social media would help with assessing the mental health conditions of people experiencing insomnia after the outbreak of COVID-19. METHODS: We designed a retrospective study that used public social media content from Twitter. We categorized insomnia-related tweets based on time, using the following two intervals: the prepandemic (January 1, 2019, to January 1, 2020) and peripandemic (January 1, 2020, to January 1, 2021) intervals. We performed a sentiment analysis by using pretrained transformers in conjunction with Dempster-Shafer theory (DST) to classify the polarity of emotions as positive, negative, and neutral. We validated the proposed pipeline on 300 annotated tweets. Additionally, we performed a temporal analysis to examine the effect of time on Twitter users' insomnia experiences, using logistic regression. RESULTS: We extracted 305,321 tweets containing the word insomnia (prepandemic tweets: n=139,561; peripandemic tweets: n=165,760). The best combination of pretrained transformers (combined via DST) yielded 84% accuracy. By using this pipeline, we found that the odds of posting negative tweets (odds ratio [OR] 1.39, 95% CI 1.37-1.41; P<.001) were higher in the peripandemic interval compared to those in the prepandemic interval. The likelihood of posting negative tweets after midnight was 21% higher than that before midnight (OR 1.21, 95% CI 1.19-1.23; P<.001). In the prepandemic interval, while the odds of posting negative tweets were 2% higher after midnight compared to those before midnight (OR 1.02, 95% CI 1.00-1.07; P=.008), they were 43% higher (OR 1.43, 95% CI 1.40-1.46; P<.001) in the peripandemic interval. CONCLUSIONS: The proposed novel sentiment analysis pipeline, which combines pretrained transformers via DST, is capable of classifying the emotions and sentiments of insomnia-related tweets. Twitter users shared more negative tweets about insomnia in the peripandemic interval than in the prepandemic interval. Future studies using a natural language processing framework could assess tweets about other types of psychological distress, habit changes, weight gain resulting from inactivity, and the effect of viral infection on sleep.

Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals.

Convolutional Neural Networks (CNN) have recently made considerable advances in the field of biomedical signal processing. These methodologies can assist in emotion recognition for affective brain computer interface. In this paper, a novel emotion recognition system based on the effective connectivity and the fine-tuned CNNs from multichannel Electroencephalogram (EEG) signal is presented. After preprocessing EEG signals, the relationships among 32 channels of EEG in the form of effective brain connectivity analysis which represents information flow between regions are computed by direct Directed Transfer Function (dDTF) method which yields a 32*32 image. Then, these constructed images from EEG signals for each subject were fed as input to four versions of pre-trained CNN models, AlexNet, ResNet-50, Inception-v3 and VGG-19 and the parameters of these models are fine-tuned, independently. The proposed deep learning architectures automatically learn patterns in the constructed image of the EEG signals in frequency bands. The efficiency of the proposed approach is evaluated on MAHNOB-HCI and DEAP databases. The experiments for classifying five emotional states show that the ResNet-50 applied on dDTF images in alpha band achieves best results due to specific architecture which captures the brain connectivity, efficiently. The accuracy and F1-score values for MAHNOB-HCI were obtained 99.41, 99.42 and for DEAP databases, 98.17, and 98.23. Newly proposed model is capable of effectively analyzing the brain function using information flow from multichannel EEG signals using effective connectivity measure of dDTF and ResNet-50.

Hand Motor Imagery Classification Using Effective Connectivity and Hierarchical Machine Learning in EEG Signals.

Background: Motor Imagery (MI) Brain Computer Interface (BCI) directly links central nervous system to a computer or a device. Most MI-BCI structures rely on features of a single channel of Electroencephalogram (EEG). However, to provide more valuable features, the relationships among EEG channels in the form of effective brain connectivity analysis must be considered. Objective: This study aims to identify a set of robust and discriminative effective connectivity features from EEG signals and to develop a hierarchical machine learning structure for discrimination of left and right hand MI task effectively. Material and Methods: In this analytical study, we estimated effective connectivity using Granger Causality (GC) methods namely, Generalized Partial Directed Coherence (GPDC), Directed Transfer Function (DTF) and direct Directed Transfer Function (dDTF). These measures determine the transient causal relation between different brain areas. Then a feature subset selection method based on Kruskal-Wallis test was performed to choose most significant directed causal connection between channels. Moreover, the minimal-redundancy-maximal-relevance feature selection method is applied to discard non-significance features. Finally, support vector machine method is used for classification. Results: The maximum value of the classification accuracies using GC methods over different frequency bands in 29 subjects in 60 trial is approximately 84% in Mu (8-12 Hz) - Beta1 (12 - 15 Hz) frequency band using GPDC method. Conclusion: This new hierarchical automated BCI system could be applied for discrimination of left and right hand MI tasks from EEG signal, effectively.

An Interpretable Machine Learning Method for the Detection of Schizophrenia Using EEG Signals.

In this work we propose a machine learning (ML) method to aid in the diagnosis of schizophrenia using electroencephalograms (EEGs) as input data. The computational algorithm not only yields a proposal of diagnostic but, even more importantly, it provides additional information that admits clinical interpretation. It is based on an ML model called random forest that operates on connectivity metrics extracted from the EEG signals. Specifically, we use measures of generalized partial directed coherence (GPDC) and direct directed transfer function (dDTF) to construct the input features to the ML model. The latter allows the identification of the most performance-wise relevant features which, in turn, provide some insights about EEG signals and frequency bands that are associated with schizophrenia. Our preliminary results on real data show that signals associated with the occipital region seem to play a significant role in the diagnosis of the disease. Moreover, although every frequency band might yield useful information for the diagnosis, the beta and theta (frequency) bands provide features that are ultimately more relevant for the ML classifier that we have implemented.

Major depressive disorder diagnosis based on effective connectivity in EEG signals: a convolutional neural network and long short-term memory approach.

Deep learning techniques have recently made considerable advances in the field of artificial intelligence. These methodologies can assist psychologists in early diagnosis of mental disorders and preventing severe trauma. Major Depression Disorder (MDD) is a common and serious medical condition whose exact manifestations are not fully understood. So, early discovery of MDD patients helps to cure or limit the adverse effects. Electroencephalogram (EEG) is prominently used to study brain diseases such as MDD due to having high temporal resolution information, and being a noninvasive, inexpensive and portable method. This paper has proposed an EEG-based deep learning framework that automatically discriminates MDD patients from healthy controls. First, the relationships among EEG channels in the form of effective brain connectivity analysis are extracted by Generalized Partial Directed Coherence (GPDC) and Direct directed transfer function (dDTF) methods. A novel combination of sixteen connectivity methods (GPDC and dDTF in eight frequency bands) was used to construct an image for each individual. Finally, the constructed images of EEG signals are applied to the five different deep learning architectures. The first and second algorithms were based on one and two-dimensional convolutional neural network (1DCNN-2DCNN). The third method is based on long short-term memory (LSTM) model, while the fourth and fifth algorithms utilized a combination of CNN with LSTM model namely, 1DCNN-LSTM and 2DCNN-LSTM. The proposed deep learning architectures automatically learn patterns in the constructed image of the EEG signals. The efficiency of the proposed algorithms is evaluated on resting state EEG data obtained from 30 healthy subjects and 34 MDD patients. The experiments show that the 1DCNN-LSTM applied on constructed image of effective connectivity achieves best results with accuracy of 99.24% due to specific architecture which captures the presence of spatial and temporal relations in the brain connectivity. The proposed method as a diagnostic tool is able to help clinicians for diagnosing the MDD patients for early diagnosis and treatment.

Mental Arithmetic Task Recognition Using Effective Connectivity and Hierarchical Feature Selection From EEG Signals.

Introduction: Mental arithmetic analysis based on Electroencephalogram (EEG) signals can help understand disorders, such as attention-deficit hyperactivity, dyscalculia, or autism spectrum disorder where the difficulty in learning or understanding the arithmetic exists. Most mental arithmetic recognition systems rely on features of a single channel of EEG; however, the relationships between EEG channels in the form of effective brain connectivity analysis can contain valuable information. This study aims to find distinctive, effective brain connectivity features and create a hierarchical feature selection for effectively classifying mental arithmetic and baseline tasks. Methods: We estimated effective connectivity using Directed Transfer Function (DTF), direct DTF (dDTF) and Generalized Partial Directed Coherence (GPDC) methods. These measures determine the causal relationship between different brain areas. A hierarchical feature subset selection method selects the most significant effective connectivity features. Initially, Kruskal- Wallis test was performed. Consequently, five feature selection algorithms, namely, Support Vector Machine (SVM) method based on Recursive Feature Elimination, Fisher score, mutual information, minimum Redundancy Maximum Relevance (RMR), and concave minimization and SVM are used to select the best discriminative features. Finally, the SVM method was used for classification. Results: The obtained results indicated that the best EEG classification performance in 29 participants and 60 trials is obtained using GPDC and feature selection via concave minimization method in Beta2 (15-22Hz) frequency band with 89% accuracy. Conclusion: This new hierarchical automated system could be helpful in the discrimination of mental arithmetic and baseline tasks from EEG signals effectively. Highlights: Propose effective connectivity to describe EEG signals during mental arithmetic task.Most significant connectivity features from generalized partial directed coherence method.Hierarchical feature selection from Kruskal-Wallis test and concave minimization method. Plain Language Summary: Brain analysis methods by Electroencephalogram (EEG) signals provide a suitable method to monitor human brain activity due to having high temporal resolution, being noninvasive, inexpensive, and portable method. Analysis of mental arithmetic based EEG signal is helpful for psychological disorders like dyscalculia where they have learning understanding arithmetic, attention deficit hyperactivity, and autism spectrum disorders with attention deficit problem. This study finds distinctive effective brain connectivity features and creates a hierarchical feature selection for classification of mental arithmetic and baseline tasks effectively. Best EEG classification performance in 29 participants and 60 trials is obtained using Generalized Partial Directed Coherence (GPDC) methods and feature selection via concave minimization method in Beta2 (15-22Hz) frequency band with 89% accuracy. Thus, this new hierarchical automated system is useful for discrimination of mental arithmetic and baseline tasks from EEG signal effectively.

A Novel Method for Individual Age Group Determination Based on the Hand Muscle Synergy.

BACKGROUND: As people get older, muscles become more synchronized and cooperate to accomplish an activity, so the main purpose of this research is to determine the relationship between changes in age and the amount of muscle synergy. The presence of muscle synergies has been long considered in the movement control as a mechanism for reducing the degree of freedom of the motor system. METHODS: By combining these synergies, a wide range of complex movements can be produced. Muscle synergies are often extracted from the electromyogram (EMG) signal. One of the most common methods for extracting synergies is the nonnegative matrix factorization. In this research, the EMG signal is obtained from individuals from different age groups (namely 15-20 years, 25-30 years, and 35-40 years), and after preprocessing, the muscular synergies are extracted. By processing and studying these synergies. RESULTS: It was observed that there is a significant difference between the muscular synergy of different age groups. Furthermore, there was a significant difference in the mean value of synergy coefficients in each group, especially in motions that were accompanied by force. CONCLUSION: This result candidates this parameter as a biomarker to differentiate and recognize the effects of age on the individual's muscular signal. In the best case, using the synergy tool, classification of the age of persons can be done by 77%.

Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals.

Schizophrenia (SZ) is a severe disorder of the human brain which disturbs behavioral characteristics such as interruption in thinking, memory, perception, speech and other living activities. If the patient suffering from SZ is not diagnosed and treated in the early stages, damage to human behavioral abilities in its later stages could become more severe. Therefore, early discovery of SZ may help to cure or limit the effects. Electroencephalogram (EEG) is prominently used to study brain diseases such as SZ due to having high temporal resolution information, and being a noninvasive and inexpensive method. This paper introduces an automatic methodology based on transfer learning with deep convolutional neural networks (CNNs) for the diagnosis of SZ patients from healthy controls. First, EEG signals are converted into images by applying a time-frequency approach called continuous wavelet transform (CWT) method. Then, the images of EEG signals are applied to the four popular pre-trained CNNs: AlexNet, ResNet-18, VGG-19 and Inception-v3. The output of convolutional and pooling layers of these models are used as deep features and are fed into the support vector machine (SVM) classifier. We have tuned the parameters of SVM to classify SZ patients and healthy subjects. The efficiency of the proposed method is evaluated on EEG signals from 14 healthy subjects and 14 SZ patients. The experiments showed that the combination of frontal, central, parietal, and occipital regions applied to the ResNet-18-SVM achieved best results with accuracy, sensitivity and specificity of 98.60% ± 2.29, 99.65% ± 2.35 and 96.92% ± 2.25, respectively. Therefore, the proposed method as a diagnostic tool can help clinicians in detection of the SZ patients for early diagnosis and treatment.

Major depressive disorder assessment via enhanced k-nearest neighbor method and EEG signals.

The aim of this paper is to introduce a novel method using short-term EEG signals to separate depressed patients from healthy controls. Five common frequency bands (delta, theta, alpha, beta and gamma) were extracted from the signals as linear features, as well as, wavelet packet decomposition to break down signals into certain frequency bands. Afterwards, two entropy measures, namely sample entropy and approximate entropy were applied on the wavelet packet coefficients as nonlinear features, and significant features were selected via genetic algorithm (GA). Three machine-learning algorithms were used for classification; including support vector machine (SVM), multilayer perceptron (MLP) a novel enhanced K-nearest neighbors (E-KNN), which uses GA to optimize the feature-space distances and provides a feature importance index. The highest accuracy obtained by using frequency-based features was from gamma oscillations which resulted in 91.38%. Performance of nonlinear features were better compared to the frequency-based features and the results showed 94.28% accuracy. The combination of the features showed 98.44% accuracy with the new proposed E-KNN classifier.

Model-based needle control in prostate percutaneous procedures.

In percutaneous applications, needle insertion into soft tissue is considered as a challenging procedure, and hence, it has been the subject of many recent studies. This study considers a model-based dynamics equation to evaluate the needle movement through prostate soft tissue. The proposed model estimates the applied force to the needle using the tissue deformation data and finite element model of the tissue. To address the role of mechanical properties of the soft tissue, an inverse dynamics control method based on sliding mode approach is used to demonstrate system performance in the presence of uncertainties. Furthermore, to deal with inaccurate estimation of mechanical parameters of the soft tissue, an adaptive controller is developed. Moreover, through a sensitivity analysis, it is shown that the uncertainty in the tissue mechanical parameters affects the system performance. Our results indicate that the adaptive controller approach performs slightly better than inverse dynamics method at the expense of fine-tuning the additional gain parameter.

Adjustable primitive pattern generator: a novel cerebellar model for reaching movements.

Cerebellum has been assumed as an array of adjustable pattern generators (APGs). In recent years, electrophysiological researches have suggested the existence of modular structures in spinal cord called motor primitives. In our proposed model, each "adjustable primitive pattern generator" (APPG) module in the cerebellum is consisted of a large number of parallel APGs, the output of each module being the weighted sum of the outputs of these APGs. Each spinal field is tuned by a coefficient, representing a descending supraspinal command, which is modulated by ith APPG correspondingly. According to this model, motor control can be interpreted in terms of the modification of these coefficients. Vector summation of force fields implies that the complex nonlinearities in neuronal behavior are eliminated, causing our model to be simple and linear. The force field vectors, derived from motor primitives, depend on the state of movement and its derivative and the time that causes different repertoire of movement. This is physiologically plausible. Our model agrees with virtual trajectory hypothesis, stating that dynamics are not computed explicitly in central nervous system, but the desired trajectory, is fed into the spinal cord. We think that the dysmetria and the ataxia seen in some cerebellar diseases may be the result of local disruption of some APPGs. Accordingly, determining the exact location of related motor primitives in human spinal cord and stimulating them by functional neurostimulation may provide a good management for these clinical signs. Surely, experimental researches and clinical trials are needed to validate our hypothesis.