Multi-scale 3D-CRU for EEG emotion recognition.

In this paper, we propose a novel multi-scale 3D-CRU model, with the goal of extracting more discriminative emotion feature from EEG signals. By concurrently exploiting the relative electrode locations and different frequency subbands of EEG signals, a three-dimensional feature representation is reconstructed wherein the Delta (δ) frequency pattern is included. We employ a multi-scale approach, termed 3D-CRU, to concurrently extract frequency and spatial features at varying levels of granularity within each time segment. In the proposed 3D-CRU, we introduce a multi-scale 3D Convolutional Neural Network (3D-CNN) to effectively capture discriminative information embedded within the 3D feature representation. To model the temporal dynamics across consecutive time segments, we incorporate a Gated Recurrent Unit (GRU) module to extract temporal representations from the time series of combined frequency-spatial features. Ultimately, the 3D-CRU model yields a global feature representation, encompassing comprehensive information across time, frequency, and spatial domains. Numerous experimental assessments conducted on publicly available DEAP and SEED databases provide empirical evidence supporting the enhanced performance of our proposed model in the domain of emotion recognition. These findings underscore the efficacy of the features extracted by the proposed multi-scale 3D-GRU model, particularly with the incorporation of the Delta (δ) frequency pattern. Specifically, on the DEAP dataset, the accuracy of Valence and Arousal are 93.12% and 94.31%, respectively, while on the SEED dataset, the accuracy is 92.25%.

Spatial reconstructed local attention Res2Net with F0 subband for fake speech detection.

The rhythm of bonafide speech is often difficult to replicate, which causes that the fundamental frequency (F0) of synthetic speech is significantly different from that of real speech. It is expected that the F0 feature contains the discriminative information for the fake speech detection (FSD) task. In this paper, we propose a novel F0 subband for FSD. In addition, to effectively model the F0 subband so as to improve the performance of FSD, the spatial reconstructed local attention Res2Net (SR-LA Res2Net) is proposed. Specifically, Res2Net is used as a backbone network to obtain multiscale information, and enhanced with a spatial reconstruction mechanism to avoid losing important information when the channel group is constantly superimposed. In addition, local attention is designed to make the model focus on the local information of the F0 subband. Experimental results on the ASVspoof 2019 LA dataset show that our proposed method obtains an equal error rate (EER) of 0.47% and a minimum tandem detection cost function (min t-DCF) of 0.0159, achieving the state-of-the-art performance among all of the single systems.

Multimodal brain-controlled system for rehabilitation training: Combining asynchronous online brain-computer interface and exoskeleton.

BACKGROUND: Traditional therapist-based rehabilitation training for patients with movement impairment is laborious and expensive. In order to reduce the cost and improve the treatment effect of rehabilitation, many methods based on human-computer interaction (HCI) technology have been proposed, such as robot-assisted therapy and functional electrical stimulation (FES). However, due to the lack of active participation of brain, these methods have limited effects on the promotion of damaged nerve remodeling. NEW METHOD: Based on the neurofeedback training provided by the combination of brain-computer interface (BCI) and exoskeleton, this paper proposes a multimodal brain-controlled active rehabilitation system to help improve limb function. The joint control mode of steady-state visual evoked potential (SSVEP) and motor imagery (MI) is adopted to achieve self-paced control and thus maximize the degree of brain involvement, and a requirement selection function based on SSVEP design is added to facilitate communication with aphasia patients. COMPARISON WITH EXISTING METHODS: In addition, the Transformer is introduced as the MI decoder in the asynchronous online BCI to improve the global perception of electroencephalogram (EEG) signals and maintain the sensitivity and efficiency of the system. RESULTS: In two multi-task online experiments for left hand, right hand, foot and idle states, subject achieves 91.25% and 92.50% best accuracy, respectively. CONCLUSION: Compared with previous studies, this paper aims to establish a high-performance and low-latency brain-controlled rehabilitation system, and provide an independent and autonomous control mode of the brain, so as to improve the effect of neural remodeling. The performance of the proposed method is evaluated through offline and online experiments.

CompNet: Complementary network for single-channel speech enhancement.

Recent multi-domain processing methods have demonstrated promising performance for monaural speech enhancement tasks. However, few of them explain why they behave better over single-domain approaches. As an attempt to fill this gap, this paper presents a complementary single-channel speech enhancement network (CompNet) that demonstrates promising denoising capabilities and provides a unique perspective to understand the improvements introduced by multi-domain processing. Specifically, the noisy speech is initially enhanced through a time-domain network. However, despite the waveform can be feasibly recovered, the distribution of the time-frequency bins may still be partly different from the target spectrum when we reconsider the problem in the frequency domain. To solve this problem, we design a dedicated dual-path network as a post-processing module to independently filter the magnitude and refine the phase. This further drives the estimated spectrum to closely approximate the target spectrum in the time-frequency domain. We conduct extensive experiments with the WSJ0-SI84 and VoiceBank + Demand datasets. Objective test results show that the performance of the proposed system is highly competitive with existing systems.

An Investigation of Olfactory-Enhanced Video on EEG-Based Emotion Recognition.

Collecting emotional physiological signals is significant in building affective Human-Computer Interactions (HCI). However, how to evoke subjects' emotions efficiently in EEG-related emotional experiments is still a challenge. In this work, we developed a novel experimental paradigm that allows odors dynamically participate in different stages of video-evoked emotions, to investigate the efficiency of olfactory-enhanced videos in inducing subjects' emotions; According to the period that the odors participated in, the stimuli were divided into four patterns, i.e., the olfactory-enhanced video in early/later stimulus periods (OVEP/OVLP), and the traditional videos in early/later stimulus periods (TVEP/TVLP). The differential entropy (DE) feature and four classifiers were employed to test the efficiency of emotion recognition. The best average accuracies of the OVEP, OVLP, TVEP, and TVLP were 50.54%, 51.49%, 40.22%, and 57.55%, respectively. The experimental results indicated that the OVEP significantly outperformed the TVEP on classification performance, while there was no significant difference between the OVLP and TVLP. Besides, olfactory-enhanced videos achieved higher efficiency in evoking negative emotions than traditional videos. Moreover, we found that the neural patterns in response to emotions under different stimulus methods were stable, and for Fp1, FP2, and F7, there existed significant differences in whether adopt the odors.

Dynamic Domain Adaptation for Class-Aware Cross-Subject and Cross-Session EEG Emotion Recognition.

It is vital to develop general models that can be shared across subjects and sessions in the real-world deployment of electroencephalogram (EEG) emotion recognition systems. Many prior studies have exploited domain adaptation algorithms to alleviate the inter-subject and inter-session discrepancies of EEG distributions. However, these methods only aligned the global domain divergence, but overlooked the local domain divergence with respect to each emotion category. This degenerates the emotion-discriminating ability of the domain invariant features. In this paper, we argue that aligning the EEG data within the same emotion categories is important for generalizable and discriminative features. Hence, we propose the dynamic domain adaptation (DDA) algorithm where the global and local divergences are disposed by minimizing the global domain discrepancy and local subdomain discrepancy, respectively. To tackle the absence of emotion labels in the target domain, we introduce a dynamic training strategy where the model focuses on optimizing the global domain discrepancy in the early training steps, and then gradually switches to the local subdomain discrepancy. The DDA algorithm is formally implemented as an unsupervised version and a semi-supervised version for different experimental settings. Based on the coarse-to-fine alignment, our model achieves the average peak accuracy of 91.08%, 92.89% on SEED, and 81.58%, 80.82% on SEED-IV in the cross-subject and cross-session scenarios, respectively.

Low-quality training data detection method of EEG signals for motor imagery BCI system.

BACKGROUND: The design and implementation of high-performance motor imagery-based brain computer interface (MI-BCI) requires high-quality training samples. However, fluctuation in subjects' physiological and mental states as well as artifacts can produce the low-quality motor imagery electroencephalogram (EEG) signal, which will damage the performance of MI-BCI system. NEW METHOD: In order to select high-quality MI-EEG training data, this paper proposes a low-quality training data detection method combining independent component analysis (ICA) and weak classifier cluster. we also design and implement a new online BCI system based on motor imagery to verify the online processing performance of the proposed method. RESULT: In order to verify the effectiveness of the proposed method, we conducted offline experiments on the public dataset called BCI Competition IV Data Set 2b. Furthermore, in order to verify the processing performance of the online system, we designed 60 groups of online experiments on 12 subjects. The online experimental results show that the twelve subjects can complete the system task efficiently (the best experiment is 135.6 s with 9 trials of subject S1). CONCLUSION: This paper demonstrated that the proposed low-quality training data detection method can effectively screen out low-quality training samples, so as to improve the performance of the MI-BCI system.

MS-MDA: Multisource Marginal Distribution Adaptation for Cross-Subject and Cross-Session EEG Emotion Recognition.

As an essential element for the diagnosis and rehabilitation of psychiatric disorders, the electroencephalogram (EEG) based emotion recognition has achieved significant progress due to its high precision and reliability. However, one obstacle to practicality lies in the variability between subjects and sessions. Although several studies have adopted domain adaptation (DA) approaches to tackle this problem, most of them treat multiple EEG data from different subjects and sessions together as a single source domain for transfer, which either fails to satisfy the assumption of domain adaptation that the source has a certain marginal distribution, or increases the difficulty of adaptation. We therefore propose the multi-source marginal distribution adaptation (MS-MDA) for EEG emotion recognition, which takes both domain-invariant and domain-specific features into consideration. First, we assume that different EEG data share the same low-level features, then we construct independent branches for multiple EEG data source domains to adopt one-to-one domain adaptation and extract domain-specific features. Finally, the inference is made by multiple branches. We evaluate our method on SEED and SEED-IV for recognizing three and four emotions, respectively. Experimental results show that the MS-MDA outperforms the comparison methods and state-of-the-art models in cross-session and cross-subject transfer scenarios in our settings. Codes at https://github.com/VoiceBeer/MS-MDA.