Classification of mental workload with EEG analysis by using effective connectivity and a hybrid model of CNN and LSTM.

Estimation of mental workload from electroencephalogram (EEG) signals aims to accurately measure the cognitive demands placed on an individual during multitasking mental activities. By analyzing the brain activity of the subject, we can determine the level of mental effort required to perform a task and optimize the workload to prevent cognitive overload or underload. This information can be used to enhance performance and productivity in various fields such as healthcare, education, and aviation. In this paper, we propose a method that uses EEG and deep neural networks to estimate the mental workload of human subjects during multitasking mental activities. Notably, our proposed method employs subject-independent classification. We use the "STEW" dataset, which consists of two tasks, namely "No task" and "simultaneous capacity (SIMKAP)-based multitasking activity". We estimate the different workload levels of two tasks using a composite framework consisting of brain connectivity and deep neural networks. After the initial preprocessing of EEG signals, an analysis of the relationships between the 14 EEG channels is conducted to evaluate effective brain connectivity. This assessment illustrates the information flow between various brain regions, utilizing the direct Directed Transfer Function (dDTF) method. Then, we propose a deep hybrid model based on pre-trained Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) for the classification of workload levels. The accuracy of the proposed deep model achieved 83.12% according to the subject-independent leave-subject-out (LSO) approach. The pre-trained CNN + LSTM approaches to EEG data have been found to be an accurate method for assessing the mental workload.

Classification of mental workload using brain connectivity and machine learning on electroencephalogram data.

Mental workload refers to the cognitive effort required to perform tasks, and it is an important factor in various fields, including system design, clinical medicine, and industrial applications. In this paper, we propose innovative methods to assess mental workload from EEG data that use effective brain connectivity for the purpose of extracting features, a hierarchical feature selection algorithm to select the most significant features, and finally machine learning models. We have used the Simultaneous Task EEG Workload (STEW) dataset, an open-access collection of raw EEG data from 48 subjects. We extracted brain-effective connectivities by the direct directed transfer function and then selected the top 30 connectivities for each standard frequency band. Then we applied three feature selection algorithms (forward feature selection, Relief-F, and minimum-redundancy-maximum-relevance) on the top 150 features from all frequencies. Finally, we applied sevenfold cross-validation on four machine learning models (support vector machine (SVM), linear discriminant analysis, random forest, and decision tree). The results revealed that SVM as the machine learning model and forward feature selection as the feature selection method work better than others and could classify the mental workload levels with accuracy equal to 89.53% (± 1.36).

A Hybrid EEG-based Emotion Recognition Approach Using Wavelet Convolutional Neural Networks and Support Vector Machine.

Introduction: Nowadays, deep learning and convolutional neural networks (CNNs) have become widespread tools in many biomedical engineering studies. CNN is an end-to-end tool, which makes the processing procedure integrated, but in some situations, this processing tool requires to be fused with machine learning methods to be more accurate. Methods: In this paper, a hybrid approach based on deep features extracted from wavelet CNNs (WCNNs) weighted layers and multiclass support vector machine (MSVM) was proposed to improve the recognition of emotional states from electroencephalogram (EEG) signals. First, EEG signals were preprocessed and converted to Time-Frequency (T-F) color representation or scalogram using the continuous wavelet transform (CWT) method. Then, scalograms were fed into four popular pre-trained CNNs, AlexNet, ResNet-18, VGG-19, and Inception-v3 to fine-tune them. Then, the best feature layer from each one was used as input to the MSVM method to classify four quarters of the valence-arousal model. Finally, the subject-independent leave-one-subject-out criterion was used to evaluate the proposed method on DEAP and MAHNOB-HCI databases. Results: Results showed that extracting deep features from the earlier convolutional layer of ResNet-18 (Res2a) and classifying using the MSVM increased the average accuracy, precision, and recall by about 20% and 12% for MAHNOB-HCI and DEAP databases, respectively. Also, combining scalograms from four regions of pre-frontal, frontal, parietal, and parietal-occipital and two regions of frontal and parietal achieved the higher average accuracy of 77.47% and 87.45% for MAHNOB-HCI and DEAP databases, respectively. Conclusion: Combining CNN and MSVM increased the recognition of emotion from EEG signals and the results were comparable to state-of-the art studies.

Emotion recognition in EEG signals using deep learning methods: A review.

Emotions are a critical aspect of daily life and serve a crucial role in human decision-making, planning, reasoning, and other mental states. As a result, they are considered a significant factor in human interactions. Human emotions can be identified through various sources, such as facial expressions, speech, behavior (gesture/position), or physiological signals. The use of physiological signals can enhance the objectivity and reliability of emotion detection. Compared with peripheral physiological signals, electroencephalogram (EEG) recordings are directly generated by the central nervous system and are closely related to human emotions. EEG signals have the great spatial resolution that facilitates the evaluation of brain functions, making them a popular modality in emotion recognition studies. Emotion recognition using EEG signals presents several challenges, including signal variability due to electrode positioning, individual differences in signal morphology, and lack of a universal standard for EEG signal processing. Moreover, identifying the appropriate features for emotion recognition from EEG data requires further research. Finally, there is a need to develop more robust artificial intelligence (AI) including conventional machine learning (ML) and deep learning (DL) methods to handle the complex and diverse EEG signals associated with emotional states. This paper examines the application of DL techniques in emotion recognition from EEG signals and provides a detailed discussion of relevant articles. The paper explores the significant challenges in emotion recognition using EEG signals, highlights the potential of DL techniques in addressing these challenges, and suggests the scope for future research in emotion recognition using DL techniques. The paper concludes with a summary of its findings.

Detection of schizophrenia using hybrid of deep learning and brain effective connectivity image from electroencephalogram signal.

Detection of mental disorders such as schizophrenia (SZ) through investigating brain activities recorded via Electroencephalogram (EEG) signals is a promising field in neuroscience. This study presents a hybrid brain effective connectivity and deep learning framework for SZ detection on multichannel EEG signals. First, the effective connectivity matrix is measured based on the Transfer Entropy (TE) method that estimates directed causalities in terms of brain information flow from 19 EEG channels for each subject. Then, TE effective connectivity elements were represented by colors and formed a 19 × 19 connectivity image which, simultaneously, represents the time and spatial information of EEG signals. Created images are used to be fed into the five pre-trained Convolutional Neural Networks (CNN) models named VGG-16, ResNet50V2, InceptionV3, EfficientNetB0, and DenseNet121 as Transfer Learning (TL) models. Finally, deep features from these TL models equipped with the Long Short-Term Memory (LSTM) model for the extraction of most discriminative spatiotemporal features are used to classify 14 SZ patients from 14 healthy controls. Results show that the hybrid framework of pre-trained CNN-LSTM models achieved higher accuracy than pre-trained CNN models. The highest average accuracy and F1-score were achieved using the EfficientNetB0-LSTM model through the 10-fold cross-validation method equal to 99.90% and 99.93%, respectively. Therefore, the superior performance of the hybrid framework of brain effective connectivity images from EEG signals and pre-trained CNN-LSTM models show that the proposed method is highly capable of detecting SZ patients from healthy controls.

Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals.

Convolutional Neural Networks (CNN) have recently made considerable advances in the field of biomedical signal processing. These methodologies can assist in emotion recognition for affective brain computer interface. In this paper, a novel emotion recognition system based on the effective connectivity and the fine-tuned CNNs from multichannel Electroencephalogram (EEG) signal is presented. After preprocessing EEG signals, the relationships among 32 channels of EEG in the form of effective brain connectivity analysis which represents information flow between regions are computed by direct Directed Transfer Function (dDTF) method which yields a 32*32 image. Then, these constructed images from EEG signals for each subject were fed as input to four versions of pre-trained CNN models, AlexNet, ResNet-50, Inception-v3 and VGG-19 and the parameters of these models are fine-tuned, independently. The proposed deep learning architectures automatically learn patterns in the constructed image of the EEG signals in frequency bands. The efficiency of the proposed approach is evaluated on MAHNOB-HCI and DEAP databases. The experiments for classifying five emotional states show that the ResNet-50 applied on dDTF images in alpha band achieves best results due to specific architecture which captures the brain connectivity, efficiently. The accuracy and F1-score values for MAHNOB-HCI were obtained 99.41, 99.42 and for DEAP databases, 98.17, and 98.23. Newly proposed model is capable of effectively analyzing the brain function using information flow from multichannel EEG signals using effective connectivity measure of dDTF and ResNet-50.

Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals.

Schizophrenia (SZ) is a severe disorder of the human brain which disturbs behavioral characteristics such as interruption in thinking, memory, perception, speech and other living activities. If the patient suffering from SZ is not diagnosed and treated in the early stages, damage to human behavioral abilities in its later stages could become more severe. Therefore, early discovery of SZ may help to cure or limit the effects. Electroencephalogram (EEG) is prominently used to study brain diseases such as SZ due to having high temporal resolution information, and being a noninvasive and inexpensive method. This paper introduces an automatic methodology based on transfer learning with deep convolutional neural networks (CNNs) for the diagnosis of SZ patients from healthy controls. First, EEG signals are converted into images by applying a time-frequency approach called continuous wavelet transform (CWT) method. Then, the images of EEG signals are applied to the four popular pre-trained CNNs: AlexNet, ResNet-18, VGG-19 and Inception-v3. The output of convolutional and pooling layers of these models are used as deep features and are fed into the support vector machine (SVM) classifier. We have tuned the parameters of SVM to classify SZ patients and healthy subjects. The efficiency of the proposed method is evaluated on EEG signals from 14 healthy subjects and 14 SZ patients. The experiments showed that the combination of frontal, central, parietal, and occipital regions applied to the ResNet-18-SVM achieved best results with accuracy, sensitivity and specificity of 98.60% ± 2.29, 99.65% ± 2.35 and 96.92% ± 2.25, respectively. Therefore, the proposed method as a diagnostic tool can help clinicians in detection of the SZ patients for early diagnosis and treatment.