Precise detection of awareness in disorders of consciousness using deep learning framework.

Diagnosis of disorders of consciousness (DOC) remains a formidable challenge. Deep learning methods have been widely applied in general neurological and psychiatry disorders, while limited in DOC domain. Considering the successful use of resting-state functional MRI (rs-fMRI) for evaluating patients with DOC, this study seeks to explore the conjunction of deep learning techniques and rs-fMRI in precisely detecting awareness in DOC. We initiated our research with a benchmark dataset comprising 140 participants, including 76 unresponsive wakefulness syndrome (UWS), 25 minimally conscious state (MCS), and 39 Controls, from three independent sites. We developed a cascade 3D EfficientNet-B3-based deep learning framework tailored for discriminating MCS from UWS patients, referred to as "DeepDOC", and compared its performance against five state-of-the-art machine learning models. We also included an independent dataset consists of 11 DOC patients to test whether our model could identify patients with cognitive motor dissociation (CMD), in which DOC patients were behaviorally diagnosed unconscious but could be detected conscious by brain computer interface (BCI) method. Our results demonstrate that DeepDOC outperforms the five machine learning models, achieving an area under curve (AUC) value of 0.927 and accuracy of 0.861 for distinguishing MCS from UWS patients. More importantly, DeepDOC excels in CMD identification, achieving an AUC of 1 and accuracy of 0.909. Using gradient-weighted class activation mapping algorithm, we found that the posterior cortex, encompassing the visual cortex, posterior middle temporal gyrus, posterior cingulate cortex, precuneus, and cerebellum, as making a more substantial contribution to classification compared to other brain regions. This research offers a convenient and accurate method for detecting covert awareness in patients with MCS and CMD using rs-fMRI data.

RpS25 is required for sperm elongation and individualization during Drosophila spermatogenesis.

Ribosomal protein 25 (RPS25) has been related to male fertility diseases in humans. However, the role of RPS25 in spermatogenesis has yet to be well understood. RpS25 is evolutionarily highly conserved from flies to humans through sequence alignment and phylogenetic tree construction. In this study, we found that RpS25 plays a critical role in Drosophila spermatogenesis and its knockdown leads to male sterility. Examination of each stage of spermatogenesis from RpS25-knockdown flies showed that RpS25 was not required for initial germline cell divisions, but was required for spermatid elongation and individualization. In RpS25-knockdown testes, the average length of cyst elongation was shortened, the spermatid nuclei bundling was disrupted, and the assembly of individualization complex from actin cones failed, resulting in the failure of mature sperm production. Our data revealed an essential role of RpS25 during Drosophila spermatogenesis through regulating spermatid elongation and individualization.

Design and Experiment of an Unoccupied Control System for a Tracked Grain Vehicle.

In order to enhance crop harvesting efficiency, an automatic-driving tracked grain vehicle system was designed. Based on the harvester chassis, we designed the mechanical structure of a tracked grain vehicle with a loading capacity of 4.5 m3 and a grain unloading hydraulic system. Using the BODAS hydraulic controller, we implemented the design of an electronic control system that combines the manual and automatic operation of the chassis walking mechanism and grain unloading mechanism. We utilized a hybrid A* algorithm to plan the traveling path of the tracked grain vehicle, and the path-tracking controller of the tracked grain vehicle was designed by combining fuzzy control and pure pursuit algorithms. Leveraging binocular vision technology and semantic segmentation technology, we designed an automatic grain unloading control system with functions of grain tank recognition and grain unloading regulation control. Finally, we conducted experiments on automatic grain unloading control and automatic navigation control in the field. The results showed that both the precision of the path-tracking control and the automatic unloading system meet the requirements for practical unoccupied operations of the tracked grain vehicle.

Identifying patients with cognitive motor dissociation using resting-state temporal stability.

Using task-dependent neuroimaging techniques, recent studies discovered a fraction of patients with disorders of consciousness (DOC) who had no command-following behaviors but showed a clear sign of awareness as healthy controls, which was defined as cognitive motor dissociation (CMD). However, existing task-dependent approaches might fail when CMD patients have cognitive function (e.g., attention, memory) impairments, in which patients with covert awareness cannot perform a specific task accurately and are thus wrongly considered unconscious, which leads to false-negative findings. Recent studies have suggested that sustaining a stable functional organization over time, i.e., high temporal stability, is crucial for supporting consciousness. Thus, temporal stability could be a powerful tool to detect the patient's cognitive functions (e.g., consciousness), while its alteration in the DOC and its capacity for identifying CMD were unclear. The resting-state fMRI (rs-fMRI) study included 119 participants from three independent research sites. A sliding-window approach was used to investigate global and regional temporal stability, which measured how stable the brain's functional architecture was across time. The temporal stability was compared in the first dataset (36/16 DOC/controls), and then a Support Vector Machine (SVM) classifier was built to discriminate DOC from controls. Furthermore, the generalizability of the SVM classifier was tested in the second independent dataset (35/21 DOC/controls). Finally, the SVM classifier was applied to the third independent dataset, where patients underwent rs-fMRI and brain-computer interface assessment (4/7 CMD/potential non-CMD), to test its performance in identifying CMD. Our results showed that global and regional temporal stability was impaired in DOC patients, especially in regions of the cingulo-opercular task control network, default-mode network, fronto-parietal task control network, and salience network. Using temporal stability as the feature, the SVM model not only showed good performance in the first dataset (accuracy = 90%), but also good generalizability in the second dataset (accuracy = 84%). Most importantly, the SVM model generalized well in identifying CMD in the third dataset (accuracy = 91%). Our preliminary findings suggested that temporal stability could be a potential tool to assist in diagnosing CMD. Furthermore, the temporal stability investigated in this study also contributed to a deeper understanding of the neural mechanism of consciousness.

Efficacy and safety of thrombolytic therapy for portal venous system thrombosis: A systematic review and meta-analysis.

BACKGROUND AND AIMS: The role of thrombolytic therapy in patients with portal venous system thrombosis (PVST) remains ambiguous. This study aimed to systematically collect available evidence and evaluate the efficacy and safety of thrombolysis for PVST. METHODS: Eligible studies were searched via PubMed, EMBASE, and Cochrane Library databases. Among the cohort studies, meta-analyses were performed to assess the outcomes of PVST patients receiving thrombolysis. Pooled proportions were calculated. Among the case reports and case series, logistic regression analyses were performed to identify the risk factors for outcomes of PVST patients receiving thrombolysis. Odds ratios (ORs) were calculated. RESULTS: Among the 2134 papers initially identified, 29 cohort studies and 131 case reports or case series were included. Based on the cohort studies, the pooled rates of overall response to thrombolytic therapy, complete recanalization of PVST, bleeding events during thrombolysis, further bowel resection, thrombosis recurrence, and 30-day mortality were 93%, 58%, 18%, 3%, 1%, and 4%, respectively. Based on the case reports and case series, acute pancreatitis (OR = 0.084), history of liver transplantation (OR = 13.346), and interval between onset of symptoms and initiation of thrombolysis ≤14 days (OR = 3.105) were significantly associated with complete recanalization of PVST; acute pancreatitis (OR = 6.556) was significantly associated with further bowel resection; but no factors associated with the overall response to thrombolytic therapy, bleeding events during thrombolysis, thrombosis recurrence, and 30-day mortality were identified or could be calculated. CONCLUSION: Early initiation of thrombolysis should be effective for the treatment of PVST. But its benefits for PVST secondary to acute pancreatitis are weakened.

Application of Bayesian phylogenetic inference modelling for evolutionary genetic analysis and dynamic changes in 2019-nCoV.

The novel coronavirus (2019-nCoV) has recently caused a large-scale outbreak of viral pneumonia both in China and worldwide. In this study, we obtained the entire genome sequence of 777 new coronavirus strains as of 29 February 2020 from a public gene bank. Bioinformatics analysis of these strains indicated that the mutation rate of these new coronaviruses is not high at present, similar to the mutation rate of the severe acute respiratory syndrome (SARS) virus. The similarities of 2019-nCoV and SARS virus suggested that the S and ORF6 proteins shared a low similarity, while the E protein shared the higher similarity. The 2019-nCoV sequence has similar potential phosphorylation sites and glycosylation sites on the surface protein and the ORF1ab polyprotein as the SARS virus; however, there are differences in potential modification sites between the Chinese strain and some American strains. At the same time, we proposed two possible recombination sites for 2019-nCoV. Based on the results of the skyline, we speculate that the activity of the gene population of 2019-nCoV may be before the end of 2019. As the scope of the 2019-nCoV infection further expands, it may produce different adaptive evolutions due to different environments. Finally, evolutionary genetic analysis can be a useful resource for studying the spread and virulence of 2019-nCoV, which are essential aspects of preventive and precise medicine.

AGL-Net: An Efficient Neural Network for EEG-Based Driver Fatigue Detection.

BACKGROUND: In recent years, road traffic safety has become a prominent issue due to the worldwide proliferation of vehicles on roads. The challenge of driver fatigue detection involves balancing the efficiency and accuracy of the detection process. While various detection methods are available, electroencephalography (EEG) is considered the gold standard due to its high precision in terms of detecting fatigue. However, deep learning models for EEG-based fatigue detection are limited by their large numbers of parameters and low computational efficiency levels, making it difficult to implement them on mobile devices. METHODS: To overcome this challenge, an attention-based Ghost-LSTM neural network (AGL-Net) is proposed for EEG-based fatigue detection in this paper. AGL-Net utilizes an attention mechanism to focus on relevant features and incorporates Ghost bottlenecks to efficiently extract spatial EEG fatigue information. Temporal EEG fatigue features are extracted using a long short-term memory (LSTM) network. We establish two types of models: regression and classification models. In the regression model, we use linear regression to obtain regression values. In the classification model, we classify features based on the predicted values obtained from regression. RESULTS: AGL-Net exhibits improved computational efficiency and a more lightweight design than existing deep learning models, as evidenced by its floating-point operations per second (FLOPs) and Params values of 2.67 M and 103,530, respectively. Furthermore, AGL-Net achieves an average accuracy of approximately 87.3% and an average root mean square error (RMSE) of approximately 0.0864 with the Shanghai Jiao Tong University (SJTU) Emotion EEG Dataset (SEED)-VIG fatigued driving dataset, indicating its advanced performance capabilities. CONCLUSIONS: The experiments conducted with the SEED-VIG dataset demonstrate the feasibility and advanced performance of the proposed fatigue detection method. The effectiveness of each AGL-Net module is verified through thorough ablation experiments. Additionally, the implementation of the Ghost bottleneck module greatly enhances the computational efficiency of the model. Overall, the proposed method has higher accuracy and computational efficiency than prior fatigue detection methods, demonstrating its considerable practical application value.

Brain-Computer Interfaces for Awareness Detection, Auxiliary Diagnosis, Prognosis, and Rehabilitation in Patients with Disorders of Consciousness.

In recent years, neuroimaging studies have remarkably demonstrated the presence of cognitive motor dissociation in patients with disorders of consciousness (DoC). These findings accelerated the development of brain-computer interfaces (BCIs) as clinical tools for behaviorally unresponsive patients. This article reviews the recent progress of BCIs in patients with DoC and discusses the open challenges. In view of the practical application of BCIs in patients with DoC, four aspects of the relevant literature are introduced: consciousness detection, auxiliary diagnosis, prognosis, and rehabilitation. For each aspect, the paradigm design, brain signal processing methods, and experimental results of representative BCI systems are analyzed. Furthermore, this article provides guidance for BCI design for patients with DoC and discusses practical challenges for future research.

Genome-Wide Analysis of the Peroxidase Gene Family and Verification of Lignin Synthesis-Related Genes in Watermelon.

Watermelon (Citrullus lanatus) is an important horticultural crop worldwide, but peel cracking caused by peel hardness severely decreases its quality. Lignification is one of the important functions of class III peroxidase (PRX), and its accumulation in the plant cell wall leads to cell thickening and wood hardening. For in-depth physiological and genetical understanding, we studied the relationship between peel hardness and lignin accumulation and the role of PRXs affecting peel lignin biosynthesis using genome-wide bioinformatics analysis. The obtained results showed that lignin accumulation gradually increased to form the peel stone cell structure, and tissue lignification led to peel hardness. A total of 79 ClPRXs (class III) were identified using bioinformatics analysis, which were widely distributed on 11 chromosomes. The constructed phylogenetics indicated that ClPRXs were divided into seven groups and eleven subclasses, and gene members of each group had highly conserved intron structures. Repeated pattern analysis showed that deletion and replication events occurred during the process of ClPRX amplification. However, in the whole-protein sequence alignment analysis, high homology was not observed, although all contained four conserved functional sites. Repeated pattern analysis showed that deletion and replication events occurred during ClPRXs' amplification process. The prediction of the promoter cis-acting element and qRT-PCR analysis in four tissues (leaf, petiole, stem, and peel) showed different expression patterns for tissue specificity, abiotic stress, and hormone response by providing a genetic basis of the ClPRX gene family involved in a variety of physiological processes in plants. To our knowledge, we for the first time report the key roles of two ClPRXs in watermelon peel lignin synthesis. In conclusion, the extensive data collected in this study can be used for additional functional analysis of ClPRXs in watermelon growth and development and hormone and abiotic stress response.

Characteristics of the Distance and Space Required for Knife Slashing Using a Motion Capture System.

OBJECTIVES: To investigate the relationship between the perpetrator's sex, victim's position and slashing location as well as anthropometric parameters on distance and space required for slashing, to provide the theoretical basis for the judgment of whether the crime scene was consistent with the criminal activity space. METHODS: The kinematics data of 12 male and 12 female subjects slashing the neck of standing and supine mannequins as well as the chest of the standing mannequins with a kitchen knife were obtained by using a 3D motion capture system. The relationship between the perpetrator's sex-victim's position, the perpetrator's sex-slashing location, and anthropometric parameters and the distance and space required for the slashing were analyzed by two-factor repeated measures ANOVA and Pearson correlation analysis respectively. RESULTS: Compared with slashing the neck of supine mannequins, the distance (L) and normalized L (l) of slashing the neck of standing mannequins were greater, while vertical distance (LVR) and normalized LVR (lVR) of the knife side were smaller. Compared with slashing the neck of standing mannequins, the L and l slashing the chest of standing mannequins were greater, while LVR and lVR were smaller. Horizontal distance (LHR) and normalized LHR (lHR) of the knife side in males were greater than that in females. Height and arm length were positively correlated with L, LHR, and LVR when striking the standing mannequins. CONCLUSIONS: When slashing the neck of supine or standing victims, the slashing distance is shorter and the slashing height is greater. Furthermore, the distance and space required for slashing are correlate with anthropometric parameters.

Prognosis for patients with cognitive motor dissociation identified by brain-computer interface.

Cognitive motor dissociation describes a subset of patients with disorders of consciousness who show neuroimaging evidence of consciousness but no detectable command-following behaviours. Although essential for family counselling, decision-making, and the design of rehabilitation programmes, the prognosis for patients with cognitive motor dissociation remains under-investigated. The current study included 78 patients with disorders of consciousness who showed no detectable command-following behaviours. These patients included 45 patients with unresponsive wakefulness syndrome and 33 patients in a minimally conscious state, as diagnosed using the Coma Recovery Scale-Revised. Each patient underwent an EEG-based brain-computer interface experiment, in which he or she was instructed to perform an item-selection task (i.e. select a photograph or a number from two candidates). Patients who achieved statistically significant brain-computer interface accuracies were identified as cognitive motor dissociation. Two evaluations using the Coma Recovery Scale-Revised, one before the experiment and the other 3 months later, were carried out to measure the patients' behavioural improvements. Among the 78 patients with disorders of consciousness, our results showed that within the unresponsive wakefulness syndrome patient group, 15 of 18 patients with cognitive motor dissociation (83.33%) regained consciousness, while only five of the other 27 unresponsive wakefulness syndrome patients without significant brain-computer interface accuracies (18.52%) regained consciousness. Furthermore, within the minimally conscious state patient group, 14 of 16 patients with cognitive motor dissociation (87.5%) showed improvements in their Coma Recovery Scale-Revised scores, whereas only four of the other 17 minimally conscious state patients without significant brain-computer interface accuracies (23.53%) had improved Coma Recovery Scale-Revised scores. Our results suggest that patients with cognitive motor dissociation have a better outcome than other patients. Our findings extend current knowledge of the prognosis for patients with cognitive motor dissociation and have important implications for brain-computer interface-based clinical diagnosis and prognosis for patients with disorders of consciousness.

The MindGomoku: An Online P300 BCI Game Based on Bayesian Deep Learning.

In addition to helping develop products that aid the disabled, brain-computer interface (BCI) technology can also become a modality of entertainment for all people. However, most BCI games cannot be widely promoted due to the poor control performance or because they easily cause fatigue. In this paper, we propose a P300 brain-computer-interface game (MindGomoku) to explore a feasible and natural way to play games by using electroencephalogram (EEG) signals in a practical environment. The novelty of this research is reflected in integrating the characteristics of game rules and the BCI system when designing BCI games and paradigms. Moreover, a simplified Bayesian convolutional neural network (SBCNN) algorithm is introduced to achieve high accuracy on limited training samples. To prove the reliability of the proposed algorithm and system control, 10 subjects were selected to participate in two online control experiments. The experimental results showed that all subjects successfully completed the game control with an average accuracy of 90.7% and played the MindGomoku an average of more than 11 min. These findings fully demonstrate the stability and effectiveness of the proposed system. This BCI system not only provides a form of entertainment for users, particularly the disabled, but also provides more possibilities for games.

[The potential role of mirnas in the pathogenesis of hemorrhagic fever with renal syndrome].

Hemorrhagic fever with renal syndrome (HFRS) is an acute natural focal viral disease caused by viruses of the genus hantavirus, characterized by damage to small blood vessels, kidneys, lungs and other organs of a person. MicroRNAs (miRNAs) are 18-22 nucleotide endogenously expressed RNA molecules that inhibit gene expression at the post-transcriptional level by binding to the 3-untranslated region of the target mRNA. It has been proven that miRNAs play a significant role in various biological processes, including the cell cycle, apoptosis, cell proliferation and differentiation. It has been proven that miRNAs may be involved in the pathogenesis of infectious diseases, including HFRS. Hantavirus infection predominantly affects endothelial cells and causes dysfunction of the endothelium of capillaries and small vessels. It is known that the immune response induced by Hantavirus infection plays an important role in disrupting the endothelial barrier. In a few studies, both in vitro and in vivo, it has been shown that endothelial dysfunction and the immune response after infection with Hantavirus can be partially regulated by miRNAs by acting on certain genes. Most of the miRNAs is expressed within the cells themselves. However, in some biological fluids of the human body, for example, plasma or blood serum, numerous miRNAs, called circulating miRNAs, have been found. Circulating miRNAs can be secreted by cells into human biological fluids as part of extracellular vesicles as exosomes or be part of an RNA-bound protein complex as miRNA-Argonaute 2 (Ago2). These miRNAs are resistant to nucleases, which makes them attractive as potential biomarkers in various human diseases. There is no specific antiviral therapy for HFRS, and the determination of laboratory parameters that are used to diagnose, assess the severity, and predict the course of the disease remains a challenge due to the peculiarities of the pathophysiology and clinical course of the disease. Studying the role of miRNAs in HFRS seems to be expedient for the development of specific and effective therapy, as well as for use as diagnostic and prognostic biomarkers (in relation to circulating miRNAs).

Trehalose targets Nrf2 signal to alleviate d-galactose induced aging and improve behavioral ability.

As an important factor leading to aging and chronic diseases, oxidative stress has become a hot research topic. Trehalose is a natural sugar widely found in many edible plants, animals and natural microorganisms, and recent studies have suggested that trehalose is an antioxidant, although its underlying molecular mechanism is unclear. Therefore, we evaluated the protective mechanism of trehalose against oxidative stress-induced senescence. In the mouse model of d-galactose (D-gal) induced aging, we found that trehalose significantly reversed the learning and memory impairment caused by D-gal and improved the ability to explore unknown things, which was associated with a significant reduction in brain tissue damage. Further studies have shown that trehalose activates the expressions of downstream target genes HO-1, NQO1, SOD, GSH and CAT by promoting the nuclear translocation of Nrf2 in the liver. The detoxification ability of organs is increased, antioxidant enzyme activity is enhanced, lipid peroxidation is reduced, and the secretion of inflammatory factors TNF-α, IL-1ß, il-6 is decreased. In conclusion, trehalose play an anti-aging role by activating genes related to Nrf2 pathway.

Enhancing BCI-Based Emotion Recognition Using an Improved Particle Swarm Optimization for Feature Selection.

：Electroencephalogram (EEG) signals have been widely used in emotion recognition. However, the current EEG-based emotion recognition has low accuracy of emotion classification, and its real-time application is limited. In order to address these issues, in this paper, we proposed an improved feature selection algorithm to recognize subjects' emotion states based on EEG signal, and combined this feature selection method to design an online emotion recognition brain-computer interface (BCI) system. Specifically, first, different dimensional features from the time-domain, frequency domain, and time-frequency domain were extracted. Then, a modified particle swarm optimization (PSO) method with multi-stage linearly-decreasing inertia weight (MLDW) was purposed for feature selection. The MLDW algorithm can be used to easily refine the process of decreasing the inertia weight. Finally, the emotion types were classified by the support vector machine classifier. We extracted different features from the EEG data in the DEAP data set collected by 32 subjects to perform two offline experiments. Our results showed that the average accuracy of four-class emotion recognition reached 76.67%. Compared with the latest benchmark, our proposed MLDW-PSO feature selection improves the accuracy of EEG-based emotion recognition. To further validate the efficiency of the MLDW-PSO feature selection method, we developed an online two-class emotion recognition system evoked by Chinese videos, which achieved good performance for 10 healthy subjects with an average accuracy of 89.5%. The effectiveness of our method was thus demonstrated.

A gaze-independent audiovisual brain-computer Interface for detecting awareness of patients with disorders of consciousness.

BACKGROUND: Currently, it is challenging to detect the awareness of patients who suffer disorders of consciousness (DOC). Brain-computer interfaces (BCIs), which do not depend on the behavioral response of patients, may serve for detecting the awareness in patients with DOC. However, we must develop effective BCIs for these patients because their ability to use BCIs does not as good as healthy users. METHODS: Because patients with DOC generally do not exhibit eye movements, a gaze-independent audiovisual BCI is put forward in the study where semantically congruent and incongruent audiovisual number stimuli were sequentially presented to evoke event-related potentials (ERPs). Subjects were required to pay attention to congruent audiovisual stimuli (target) and ignore the incongruent audiovisual stimuli (non-target). The BCI system was evaluated by analyzing online and offline data from 10 healthy subjects followed by being applied to online awareness detection in 8 patients with DOC. RESULTS: According to the results on healthy subjects, the audiovisual BCI system outperformed the corresponding auditory-only and visual-only systems. Multiple ERP components, including the P300, N400 and late positive complex (LPC), were observed using the audiovisual system, strengthening different brain responses to target stimuli and non-target stimuli. The results revealed the abilities of three of eight patients to follow commands and recognize numbers. CONCLUSIONS: This gaze-independent audiovisual BCI system represents a useful auxiliary bedside tool to detect the awareness of patients with DOC.

[A Target Discrimination Method Based on the Active Polarization Imaging with the Distribution of Polarization Fresnel Ratio].

As the polarization characteristics are the physical property determined by the material itself, its corresponding polarization image contains abundant target's information. Using polarization information to identify the target is always a hot research topic in the field of the target detection. Active polarization imaging has more advantages compared with passive polarization imaging because of its high signal-to-noise ratio and good controllability. In this paper, based on the detailed analysis of the theory of the distribution of polarization Fresnel reflectance ratio, a kind of active polarization imaging method is proposed with detecting the polarization Fresnel ratio of the surface of the object. The proposed method adopts two kind of polarization light with orthogonal polarization direction at the light emission part to exposure to the target scenario alternately. Then two cameras side-by-side at the detecting part respectively equipped with two orthogonal polarization direction filters to capture the polarization images. Meanwhile, the detectors are placed in different detecting direction to acquire the polarization imaging with active polarization light source illuminating. Finally, with transmitting the data to the calculating center, optical constants can be recovered from the polarization data by the optimization fitting technique. Because the materials of target's surface are different, the corresponding optical constants are different. Then the purpose of discriminating the targets with different materials is achieved. The simulated and actual measured experiments are explored to verify the effectiveness of the proposed method. Simulation experiment shows it is not only scientific but also more convenient and effective in that the proposed method can distinguish the different materials using the calculated optical constants. The actual measured data further shows that the method is able to do better in recover optical constants of targets, especially in the distinction between metal and dielectric materials. Furthermore, the system has great application prospect in the field of target detection and camouflage recognition with its simple structure and practicability.

Detecting number processing and mental calculation in patients with disorders of consciousness using a hybrid brain-computer interface system.

BACKGROUND: For patients with disorders of consciousness such as coma, a vegetative state or a minimally conscious state, one challenge is to detect and assess the residual cognitive functions in their brains. Number processing and mental calculation are important brain functions but are difficult to detect in patients with disorders of consciousness using motor response-based clinical assessment scales such as the Coma Recovery Scale-Revised due to the patients' motor impairments and inability to provide sufficient motor responses for number- and calculation-based communication. METHODS: In this study, we presented a hybrid brain-computer interface that combines P300 and steady state visual evoked potentials to detect number processing and mental calculation in Han Chinese patients with disorders of consciousness. Eleven patients with disorders of consciousness who were in a vegetative state (n = 6) or in a minimally conscious state (n = 3) or who emerged from a minimally conscious state (n = 2) participated in the brain-computer interface-based experiment. During the experiment, the patients with disorders of consciousness were instructed to perform three tasks, i.e., number recognition, number comparison, and mental calculation, including addition and subtraction. In each experimental trial, an arithmetic problem was first presented. Next, two number buttons, only one of which was the correct answer to the problem, flickered at different frequencies to evoke steady state visual evoked potentials, while the frames of the two buttons flashed in a random order to evoke P300 potentials. The patients needed to focus on the target number button (the correct answer). Finally, the brain-computer interface system detected P300 and steady state visual evoked potentials to determine the button to which the patients attended, further presenting the results as feedback. RESULTS: Two of the six patients who were in a vegetative state, one of the three patients who were in a minimally conscious state, and the two patients that emerged from a minimally conscious state achieved accuracies significantly greater than the chance level. Furthermore, P300 potentials and steady state visual evoked potentials were observed in the electroencephalography signals from the five patients. CONCLUSIONS: Number processing and arithmetic abilities as well as command following were demonstrated in the five patients. Furthermore, our results suggested that through brain-computer interface systems, many cognitive experiments may be conducted in patients with disorders of consciousness, although they cannot provide sufficient behavioral responses.

An Intersubject Brain-Computer Interface Based on Domain-Adversarial Training of Convolutional Neural Network for Online Attention Decoding.

Attention decoding plays a vital role in daily life, where electroencephalography (EEG) has been widely involved. However, training a universally effective model for everyone is impractical due to substantial interindividual variability in EEG signals. To tackle the above challenge, we propose an end-to-end brain-computer interface (BCI) framework, including temporal and spatial one-dimensional (1D) convolutional neural network and domain-adversarial training strategy, namely DA-TSnet. Specifically, DA-TSnet extracts temporal and spatial features of EEG, while it is jointly supervised by task loss and domain loss. During training, DA-TSnet aims to maximize the domain loss while simultaneously minimizing the task loss. We conduct an offline analysis, simulate online experiments on a self-collected dataset of 85 subjects, and real online experiments on 22 subjects. Main results: DA-TSnet achieves a leave-one-subject-out (LOSO) cross-validation (CV) classification accuracy of 89.40% ± 9.96%, outperforming several state-of-the-art attention EEG decoding methods. In simulated online experiments, DA-TSnet achieves an outstanding accuracy of 88.07% ± 11.22%. In real online experiments, it achieves an average accuracy surpassing 86%. Significance: An end-to-end network framework does not rely on elaborate preprocessing and feature extraction steps, which saves time and human workload. Moreover, our framework utilizes domain-adversarial training neural network (DANN) to tackle the challenge posed by the high interindividual variability in EEG signals, which has significant reference value for handling other EEG signal decoding issues. Last, the performance of the DA-TSnet framework in offline and online experiments underscores its potential to facilitate more reliable applications.

Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG.

Goal: As an essential human-machine interactive task, emotion recognition has become an emerging area over the decades. Although previous attempts to classify emotions have achieved high performance, several challenges remain open: 1) How to effectively recognize emotions using different modalities remains challenging. 2) Due to the increasing amount of computing power required for deep learning, how to provide real-time detection and improve the robustness of deep neural networks is important. Method: In this paper, we propose a deep learning-based multimodal emotion recognition (MER) called Deep-Emotion, which can adaptively integrate the most discriminating features from facial expressions, speech, and electroencephalogram (EEG) to improve the performance of the MER. Specifically, the proposed Deep-Emotion framework consists of three branches, i.e., the facial branch, speech branch, and EEG branch. Correspondingly, the facial branch uses the improved GhostNet neural network proposed in this paper for feature extraction, which effectively alleviates the overfitting phenomenon in the training process and improves the classification accuracy compared with the original GhostNet network. For work on the speech branch, this paper proposes a lightweight fully convolutional neural network (LFCNN) for the efficient extraction of speech emotion features. Regarding the study of EEG branches, we proposed a tree-like LSTM (tLSTM) model capable of fusing multi-stage features for EEG emotion feature extraction. Finally, we adopted the strategy of decision-level fusion to integrate the recognition results of the above three modes, resulting in more comprehensive and accurate performance. Result and Conclusions: Extensive experiments on the CK+, EMO-DB, and MAHNOB-HCI datasets have demonstrated the advanced nature of the Deep-Emotion method proposed in this paper, as well as the feasibility and superiority of the MER approach.