EEG classification model for virtual reality motion sickness based on multi-scale CNN feature correlation.

BACKGROUND: Virtual reality motion sickness (VRMS) is a key issue hindering the development of virtual reality technology, and accurate detection of its occurrence is the first prerequisite for solving the issue. OBJECTIVE: In this paper, a convolutional neural network (CNN) EEG detection model based on multi-scale feature correlation is proposed for detecting VRMS. METHODS: The model uses multi-scale 1D convolutional layers to extract multi-scale temporal features from the multi-lead EEG data, and then calculates the feature correlations of the extracted multi-scale features among all the leads to form the feature adjacent matrixes, which converts the time-domain features to correlation-based brain network features, thus strengthen the feature representation. Finally, the correlation features of each layer are fused. The fused features are then fed into the channel attention module to filter the channels and classify them using a fully connected network. Finally, we recruit subjects to experience 6 different modes of virtual roller coaster scenes, and collect resting EEG data before and after the task to verify the model. RESULTS: The results show that the accuracy, precision, recall and F1-score of this model for the recognition of VRMS are 98.66 %, 98.65 %, 98.68 %, and 98.66 %, respectively. The proposed model outperforms the current classic and advanced EEG recognition models. SIGNIFICANCE: It shows that this model can be used for the recognition of VRMS based on the resting state EEG.

Detection of virtual reality motion sickness based on EEG using asymmetry of entropy and cross-frequency coupling.

The existence of Virtual Reality Motion Sickness (VRMS) is a key factor restricting the further development of the VR industry, and the premise to solve this problem is to be able to accurately and effectively detect its occurrence. In view of the current lack of high-accuracy and effective detection methods, this paper proposes a VRMS detection method based on entropy asymmetry and cross-frequency coupling value asymmetry of EEG. First of all, the EEG of the four selected pairs of electrodes on the bilateral brain are subjected to Multivariate Variational Mode Decomposition (MVMD) respectively, and three types of entropy values on the low-frequency and high-frequency components are calculated, namely approximate entropy, fuzzy entropy and permutation entropy, as well as three types of phase-amplitude coupling features between the low-frequency and high-frequency components, namely the mean value, standard deviation and correlation coefficient; Secondly, the difference of the entropies and the cross-frequency coupling features between the left electrodes and the right electrodes are calculated; Finally, the final feature set are selected via t-test and fed into the SVM for classification, thus realizing the automatic detection of VRMS. The results show that the three classification indexes under this method, i.e., accuracy, sensitivity and specificity, reach 99.5 %, 99.3 % and 99.7 %, respectively, and the value of the area under the ROC curve reached 1, which proves that this method can be an effective indicator for detecting the occurrence of VRMS.

Study of Human Tacit Knowledge Based on Electroencephalogram Signal Characteristics.

Tacit knowledge is the kind of knowledge that is difficult to transfer to another person by means of writing it down or verbalizing it. In the mineral grinding process, the proficiency of the operators depends on the tacit knowledge gained from their experience and training rather than on knowledge learned from a handbook. This article proposed a method combining the electroencephalogram (EEG) signals and the industrial process to detect the proficiency of the operators in the mineral grinding process to reveal the effect of tacit knowledge on the functional cortical connection. The functional brain networks of operators were established based on partial direct coherence and directed transfer function of EEG, and the multi-classifiers were used with the graph-theoretic indexes of the FBNs as input to distinguish the trained operators (Hps) from the non-trained operators (Lps). The results showed that the brain networks of Hps had a better connectivity than those of Lps (p < 0.01), and the accuracy of classification was up to 94.2%. Our studies confirm that based on the performance of EEG features and the combination of industrial operational operation and cognitive processes, the proficiency of the operators can be detected.

Using convolutional neural networks to decode EEG-based functional brain network with different severity of acrophobia.

Objective.The prevalence of acrophobia is high, especially with the rise of many high-rise buildings. In the recent few years, researchers have begun to analyze acrophobia from the neuroscience perspective, especially to improve the virtual reality exposure therapy (VRET). Electroencephalographic (EEG) is an informative neuroimaging technique, but it is rarely used for acrophobia. The purpose of this study is to evaluate the effectiveness of using EEGs to identify the degree of acrophobia objectively.Approach.EEG data were collected by virtual reality (VR) exposure experiments. We classified all subjects' degrees of acrophobia into three categories, where their questionnaire scores and behavior data showed significant differences. Using synchronization likelihood, we computed the functional connectivity between each pair of channels and then obtained complex networks named functional brain networks (FBNs). Basic topological features and community structure features were extracted from the FBNs. Statistical results demonstrated that FBN features can be used to distinguish different groups of subjects. We trained machine learning (ML) algorithms with FBN features as inputs and trained convolutional neural networks (CNNs) with FBNs directly as inputs.Main results.It turns out that using FBN to identify the severity of acrophobia is feasible. For ML algorithms, the community structure features of some cerebral cortex regions outperform typical topological features of the whole brain, in terms of classification accuracy. The performances of CNN algorithms are better than ML algorithms. The CNN with ResNet performs the best (accuracy reached 98.46 ± 0.42%).Significance.These observations indicate that community structures of certain cerebral cortex regions could be used to identify the degree of acrophobia. The proposed CNN framework can provide objective feedback, which could help build closed-loop VRET portable systems.

EEG based functional connectivity analysis of human pain empathy towards humans and robots.

Humans can show emotional reactions toward humanoid robots, such as empathy. Previous neuroimaging studies have indicated that neural responses of empathy for others' pain are modulated by an early automatic emotional sharing and a late controlled cognitive evaluation process. Recent studies about pain empathy for robots found humans present similar empathy process towards humanoid robots under painful stimuli as well as to humans. However, the whole-brain functional connectivity and the spatial dynamics of neural activities underlying empathic processes are still unknown. In the present study, the functional connectivity was investigated for ERPs recorded from 18 healthy adults who were presented with pictures of human hand and robot hand under painful and non-painful situations. Functional brain networks for both early and late empathy responses were constructed and a new parameter, empathy index (EI), was proposed to represent the empathy ability of humans quantitatively. We found that the mutual dependences between early ERP components was significantly decreased, but for the late components, there were no significant changes. The mutual dependences for human hand stimuli were larger than to robot hand stimuli for early components, but not for late components. The connectivity weights for early components were larger than late components. EI value shows significant difference between painful and non-painful stimuli, indicating it is a good indicator to represent the empathy of humans. This study enriches our understanding of the neurological mechanisms implicated in human empathy, and provides evidence of functional connectivity for both early and late responses of pain empathy towards humans and robots.

Exploring the fatigue affecting electroencephalography based functional brain networks during real driving in young males.

In recent years, a large proportion of traffic accidents are caused by driver fatigue. The brain has been conceived as a complex network, whose function can be assessed with EEG. Hence, in this research, fourteen subjects participated in the real driving experiments, and a comprehensive EEG-based expert system was designed for detecting driver fatigue. Collected EEG signals were first decomposed into delta-range, theta-range, alpha-range and beta-range by wavelet packet transform (WPT). Unlike other approaches, a multi-channel network construction method based on Phase Lag Index (PLI) was then proposed in this paper. Finally, the functional connectivity between alert state (at the beginning of the drive) and fatigue state (at the end of the drive) in multiple frequency bands were analyzed. The results indicate that functional connectivity of the brain area was significantly different between alert and fatigue states, especially in alpha-range and beta-range. Particularly, the frontal-to-parietal functional connectivity was weakened. Meanwhile, lower clustering coefficient (C) values and higher characteristic path length (L) values were observed in fatigue state in comparison with alert state. Based on this, two new EEG feature selection approaches, C and L in the corresponding sub-frequency range were applied to feature recognition and classification system. Using a support vector machine (SVM) machine learning algorithm, these features were combined to distinguish between alert and fatigue states, achieving an accuracy of 94.4%, precision of 94.3%, sensitivity of 94.6% and false alarm rate of 5.7%. The results suggest that brain network analysis approaches combined with SVM are helpful to alert drivers while being sleepy or even fatigue.

A Novel Method of Building Functional Brain Network Using Deep Learning Algorithm with Application in Proficiency Detection.

Functional brain network (FBN) has become very popular to analyze the interaction between cortical regions in the last decade. But researchers always spend a long time to search the best way to compute FBN for their specific studies. The purpose of this study is to detect the proficiency of operators during their mineral grinding process controlling based on FBN. To save the search time, a novel semi-data-driven method of computing functional brain connection based on stacked autoencoder (BCSAE) is proposed in this paper. This method uses stacked autoencoder (SAE) to encode the multi-channel EEG data into codes and then computes the dissimilarity between the codes from every pair of electrodes to build FBN. The highlight of this method is that the SAE has a multi-layered structure and is semi-supervised, which means it can dig deeper information and generate better features. Then an experiment was performed, the EEG of the operators were collected while they were operating and analyzed to detect their proficiency. The results show that the BCSAE method generated more number of separable features with less redundancy, and the average accuracy of classification (96.18%) is higher than that of the control methods: PLV (92.19%) and PLI (78.39%).

Assessment of driver drowsiness using electroencephalogram signals based on multiple functional brain networks.

This paper proposes a comprehensive approach to explore whether functional brain network (FBN) changes from the alert state to the drowsy state and to find out ideal neurophysiology indicators able to detect driver drowsiness in terms of FBN. A driving simulation experiment consisting of two driving tasks is designed and conducted using fifteen participant drivers. Collected EEG signals are then decomposed into multiple frequency bands by wavelet packet transform (WPT). Based on this, two novel FBN approaches, synchronization likelihood (SL) and minimum spanning tree (MST) are combined and applied to feature recognition and classification system. Unlike other methods, our approaches focus on the interaction and correlation between different brain regions. Statistical analysis of network features indicates that the difference between alert state and drowsy state are significant and further confirmed that brain network configuration should be related to drowsiness. For classification, these brain network features are selected and then fed into four classifiers considered namely Support Vector Machines (SVM), K Nearest Neighbors classifier (KNN), Logistic Regression (LR) and Decision Trees (DT). It is found that combining MST method and SL method is actually increasing the classification accuracy with all classifiers considered in this work especially the KNN classifier from 95.4% to 98.6%. Moreover, KNN classifier also gives the highest precision of 98.3%, sensitivity of 98.8% and specificity of 98.9%. Thus this kind of methodology might be a useful tool for further understanding the neurophysiology mechanisms of driver drowsiness, and as a reference work for future studies or future 'systems'.

Graph analysis of functional brain network topology using minimum spanning tree in driver drowsiness.

A large number of traffic accidents due to driver drowsiness have been under more attention of many countries. The organization of the functional brain network is associated with drowsiness, but little is known about the brain network topology that is modulated by drowsiness. To clarify this problem, in this study, we introduce a novel approach to detect driver drowsiness. Electroencephalogram (EEG) signals have been measured during a simulated driving task, in which participants are recruited to undergo both alert and drowsy states. The filtered EEG signals are then decomposed into multiple frequency bands by wavelet packet transform. Functional connectivity between all pairs of channels for multiple frequency bands is assessed using the phase lag index (PLI). Based on this, PLI-weighted networks are subsequently calculated, from which minimum spanning trees are constructed-a graph method that corrects for comparison bias. Statistical analyses are performed on graph-derived metrics as well as on the PLI connectivity values. The major finding is that significant differences in the delta frequency band for three graph metrics and in the theta frequency band for five graph metrics suggesting network integration and communication between network nodes are increased from alertness to drowsiness. Together, our findings also suggest a more line-like configuration in alert states and a more star-like topology in drowsy states. Collectively, our findings point to a more proficient configuration in drowsy state for lower frequency bands. Graph metrics relate to the intrinsic organization of functional brain networks, and these graph metrics may provide additional insights on driver drowsiness detection for reducing and preventing traffic accidents and further understanding the neural mechanisms of driver drowsiness.