Subject matching for cross-subject EEG-based recognition of driver states related to situation awareness.

Situation awareness (SA) has received much attention in recent years because of its importance for operators of dynamic systems. Electroencephalography (EEG) can be used to measure mental states of operators related to SA. However, cross-subject EEG-based SA recognition is a critical challenge, as data distributions of different subjects vary significantly. Subject variability is considered as a domain shift problem. Several attempts have been made to find domain-invariant features among subjects, where subject-specific information is neglected. In this work, we propose a simple but efficient subject matching framework by finding a connection between a target (test) subject and source (training) subjects. Specifically, the framework includes two stages: (1) we train the model with multi-source domain alignment layers to collect source domain statistics. (2) During testing, a distance is computed to perform subject matching in the latent representation space. We use a reciprocal exponential function as a similarity measure to dynamically select similar source subjects. Experiment results show that our framework achieves a state-of-the-art accuracy 74.32% for the Taiwan driving dataset.

A compact and interpretable convolutional neural network for cross-subject driver drowsiness detection from single-channel EEG.

Driver drowsiness is one of the main factors leading to road fatalities and hazards in the transportation industry. Electroencephalography (EEG) has been considered as one of the best physiological signals to detect drivers' drowsy states, since it directly measures neurophysiological activities in the brain. However, designing a calibration-free system for driver drowsiness detection with EEG is still a challenging task, as EEG suffers from serious mental and physical drifts across different subjects. In this paper, we propose a compact and interpretable Convolutional Neural Network (CNN) to discover shared EEG features across different subjects for driver drowsiness detection. We incorporate the Global Average Pooling (GAP) layer in the model structure, allowing the Class Activation Map (CAM) method to be used for localizing regions of the input signal that contribute most for classification. Results show that the proposed model can achieve an average accuracy of 73.22% on 11 subjects for 2-class cross-subject EEG signal classification, which is higher than conventional machine learning methods and other state-of-art deep learning methods. It is revealed by the visualization technique that the model has learned biologically explainable features, e.g., Alpha spindles and Theta burst, as evidence for the drowsy state. It is also interesting to see that the model uses artifacts that usually dominate the wakeful EEG, e.g., muscle artifacts and sensor drifts, to recognize the alert state. The proposed model illustrates a potential direction to use CNN models as a powerful tool to discover shared features related to different mental states across different subjects from EEG signals.

Learning Polychronous Neuronal Groups Using Joint Weight-Delay Spike-Timing-Dependent Plasticity.

Polychronous neuronal group (PNG), a type of cell assembly, is one of the putative mechanisms for neural information representation. According to the reader-centric definition, some readout neurons can become selective to the information represented by polychronous neuronal groups under ongoing activity. Here, in computational models, we show that the frequently activated polychronous neuronal groups can be learned by readout neurons with joint weight-delay spike-timing-dependent plasticity. The identity of neurons in the group and their expected spike timing at millisecond scale can be recovered from the incoming weights and delays of the readout neurons. The detection performance can be further improved by two layers of readout neurons. In this way, the detection of polychronous neuronal groups becomes an intrinsic part of the network, and the readout neurons become differentiated members in the group to indicate whether subsets of the group have been activated according to their spike timing. The readout spikes representing this information can be used to analyze how PNGs interact with each other or propagate to downstream networks for higher-level processing.

EEG-Based Cross-Subject Driver Drowsiness Recognition With an Interpretable Convolutional Neural Network.

In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this article, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.

Sample-Based Data Augmentation Based on Electroencephalogram Intrinsic Characteristics.

Deep learning for electroencephalogram-based classification is confronted with data scarcity, due to the time-consuming and expensive data collection procedure. Data augmentation has been shown as an effective way to improve data efficiency. In addition, contrastive learning has recently been shown to hold great promise in learning effective representations without human supervision, which has the potential to improve the electroencephalogram-based recognition performance with limited labeled data. However, heavy data augmentation is a key ingredient of contrastive learning. In view of the limited number of sample-based data augmentation in electroencephalogram processing, three methods, performance-measure-based time warp, frequency noise addition and frequency masking, are proposed based on the characteristics of electroencephalogram signal. These methods are parameter learning free, easy to implement, and can be applied to individual samples. In the experiment, the proposed data augmentation methods are evaluated on three electroencephalogram-based classification tasks, including situation awareness recognition, motor imagery classification and brain-computer interface steady-state visually evoked potentials speller system. Results demonstrated that the convolutional models trained with the proposed data augmentation methods yielded significantly improved performance over baselines. In overall, this work provides more potential methods to cope with the problem of limited data and boost the classification performance in electroencephalogram processing.

EEG-based "serious" games and monitoring tools for pain management.

EEG-based "serious games" for medical applications attracted recently more attention from the research community and industry as wireless EEG reading devices became easily available on the market. EEG-based technology has been applied in anesthesiology, psychology, etc. In this paper, we proposed and developed EEG-based "serious" games and doctor's monitoring tools that could be used for pain management. As EEG signal is considered to have a fractal nature, we proposed and develop a novel spatio-temporal fractal based algorithm for brain state quantification. The algorithm is implemented with blobby visualization tools for patient monitoring and in EEG-based "serious" games. Such games could be used by patient even at home convenience for pain management as an alternative to traditional drug treatment.

Large-Scale Automated Sleep Staging.

Study Objectives: Automated sleep staging has been previously limited by a combination of clinical and physiological heterogeneity. Both factors are in principle addressable with large data sets that enable robust calibration. However, the impact of sample size remains uncertain. The objectives are to investigate the extent to which machine learning methods can approximate the performance of human scorers when supplied with sufficient training cases and to investigate how staging performance depends on the number of training patients, contextual information, model complexity, and imbalance between sleep stage proportions. Methods: A total of 102 features were extracted from six electroencephalography (EEG) channels in routine polysomnography. Two thousand nights were partitioned into equal (n = 1000) training and testing sets for validation. We used epoch-by-epoch Cohen's kappa statistics to measure the agreement between classifier output and human scorer according to American Academy of Sleep Medicine scoring criteria. Results: Epoch-by-epoch Cohen's kappa improved with increasing training EEG recordings until saturation occurred (n = ~300). The kappa value was further improved by accounting for contextual (temporal) information, increasing model complexity, and adjusting the model training procedure to account for the imbalance of stage proportions. The final kappa on the testing set was 0.68. Testing on more EEG recordings leads to kappa estimates with lower variance. Conclusion: Training with a large data set enables automated sleep staging that compares favorably with human scorers. Because testing was performed on a large and heterogeneous data set, the performance estimate has low variance and is likely to generalize broadly.

Human electroencephalograms seen as fractal time series: mathematical analysis and visualization.

The paper presents a novel technique of nonlinear spectral analysis, which has been used for processing encephalograms of humans. This technique is based on the concept of generalized entropy of a given probability distribution, known as the Rényi entropy that allows defining the set of generalized fractal dimensions of encephalogram (EEG) and determining fractal spectra of encephalographic signals. Unlike the Fourier spectra, the spectra of fractal dimensions contain information of both frequency and amplitude characteristics of EEG and can be used together with well-accepted techniques of EEG analysis as an enhancement of the latter. Powered by volume visualization of the brain activity, the method provides new clues for understanding the mental processes in humans.

Real-time mental arithmetic task recognition from EEG signals.

Electroencephalography (EEG)-based monitoring the state of the user's brain functioning and giving her/him the visual/audio/tactile feedback is called neurofeedback technique, and it could allow the user to train the corresponding brain functions. It could provide an alternative way of treatment for some psychological disorders such as attention deficit hyperactivity disorder (ADHD), where concentration function deficit exists, autism spectrum disorder (ASD), or dyscalculia where the difficulty in learning and comprehending the arithmetic exists. In this paper, a novel method for multifractal analysis of EEG signals named generalized Higuchi fractal dimension spectrum (GHFDS) was proposed and applied in mental arithmetic task recognition from EEG signals. Other features such as power spectrum density (PSD), autoregressive model (AR), and statistical features were analyzed as well. The usage of the proposed fractal dimension spectrum of EEG signal in combination with other features improved the mental arithmetic task recognition accuracy in both multi-channel and one-channel subject-dependent algorithms up to 97.87% and 84.15% correspondingly. Based on the channel ranking, four channels were chosen which gave the accuracy up to 97.11%. Reliable real-time neurofeedback system could be implemented based on the algorithms proposed in this paper.