Cauchy non-convex sparse feature selection method for the high-dimensional small-sample problem in motor imagery EEG decoding.

Introduction: The time, frequency, and space information of electroencephalogram (EEG) signals is crucial for motor imagery decoding. However, these temporal-frequency-spatial features are high-dimensional small-sample data, which poses significant challenges for motor imagery decoding. Sparse regularization is an effective method for addressing this issue. However, the most commonly employed sparse regularization models in motor imagery decoding, such as the least absolute shrinkage and selection operator (LASSO), is a biased estimation method and leads to the loss of target feature information. Methods: In this paper, we propose a non-convex sparse regularization model that employs the Cauchy function. By designing a proximal gradient algorithm, our proposed model achieves closer-to-unbiased estimation than existing sparse models. Therefore, it can learn more accurate, discriminative, and effective feature information. Additionally, the proposed method can perform feature selection and classification simultaneously, without requiring additional classifiers. Results: We conducted experiments on two publicly available motor imagery EEG datasets. The proposed method achieved an average classification accuracy of 82.98% and 64.45% in subject-dependent and subject-independent decoding assessment methods, respectively. Conclusion: The experimental results show that the proposed method can significantly improve the performance of motor imagery decoding, with better classification performance than existing feature selection and deep learning methods. Furthermore, the proposed model shows better generalization capability, with parameter consistency over different datasets and robust classification across different training sample sizes. Compared with existing sparse regularization methods, the proposed method converges faster, and with shorter model training time.

A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding.

Motor imagery (MI) decoding plays a crucial role in the advancement of electroencephalography (EEG)-based brain-computer interface (BCI) technology. Currently, most researches focus on complex deep learning structures for MI decoding. The growing complexity of networks may result in overfitting and lead to inaccurate decoding outcomes due to the redundant information. To address this limitation and make full use of the multi-domain EEG features, a multi-domain temporal-spatial-frequency convolutional neural network (TSFCNet) is proposed for MI decoding. The proposed network provides a novel mechanism that utilize the spatial and temporal EEG features combined with frequency and time-frequency characteristics. This network enables powerful feature extraction without complicated network structure. Specifically, the TSFCNet first employs the MixConv-Residual block to extract multiscale temporal features from multi-band filtered EEG data. Next, the temporal-spatial-frequency convolution block implements three shallow, parallel and independent convolutional operations in spatial, frequency and time-frequency domain, and captures high discriminative representations from these domains respectively. Finally, these features are effectively aggregated by average pooling layers and variance layers, and the network is trained with the joint supervision of the cross-entropy and the center loss. Our experimental results show that the TSFCNet outperforms the state-of-the-art models with superior classification accuracy and kappa values (82.72% and 0.7695 for dataset BCI competition IV 2a, 86.39% and 0.7324 for dataset BCI competition IV 2b). These competitive results demonstrate that the proposed network is promising for enhancing the decoding performance of MI BCIs.

Sparse Bayesian Learning for End-to-End EEG Decoding.

Decoding brain activity from non-invasive electroencephalography (EEG) is crucial for brain-computer interfaces (BCIs) and the study of brain disorders. Notably, end-to-end EEG decoding has gained widespread popularity in recent years owing to the remarkable advances in deep learning research. However, many EEG studies suffer from limited sample sizes, making it difficult for existing deep learning models to effectively generalize to highly noisy EEG data. To address this fundamental limitation, this paper proposes a novel end-to-end EEG decoding algorithm that utilizes a low-rank weight matrix to encode both spatio-temporal filters and the classifier, all optimized under a principled sparse Bayesian learning (SBL) framework. Importantly, this SBL framework also enables us to learn hyperparameters that optimally penalize the model in a Bayesian fashion. The proposed decoding algorithm is systematically benchmarked on five motor imagery BCI EEG datasets ( N=192) and an emotion recognition EEG dataset ( N=45), in comparison with several contemporary algorithms, including end-to-end deep-learning-based EEG decoding algorithms. The classification results demonstrate that our algorithm significantly outperforms the competing algorithms while yielding neurophysiologically meaningful spatio-temporal patterns. Our algorithm therefore advances the state-of-the-art by providing a novel EEG-tailored machine learning tool for decoding brain activity.

Shared Three-Dimensional Robotic Arm Control Based on Asynchronous BCI and Computer Vision.

OBJECTIVE: A brain-computer interface (BCI) can be used to translate neuronal activity into commands to control external devices. However, using noninvasive BCI to control a robotic arm for movements in three-dimensional (3D) environments and accomplish complicated daily tasks, such as grasping and drinking, remains a challenge. APPROACH: In this study, a shared robotic arm control system based on hybrid asynchronous BCI and computer vision was presented. The BCI model, which combines steady-state visual evoked potentials (SSVEPs) and blink-related electrooculography (EOG) signals, allows users to freely choose from fifteen commands in an asynchronous mode corresponding to robot actions in a 3D workspace and reach targets with a wide movement range, while computer vision can identify objects and assist a robotic arm in completing more precise tasks, such as grasping a target automatically. RESULTS: Ten subjects participated in the experiments and achieved an average accuracy of more than 92% and a high trajectory efficiency for robot movement. All subjects were able to perform the reach-grasp-drink tasks successfully using the proposed shared control method, with fewer error commands and shorter completion time than with direct BCI control. SIGNIFICANCE: Our results demonstrated the feasibility and efficiency of generating practical multidimensional control of an intuitive robotic arm by merging hybrid asynchronous BCI and computer vision-based recognition.

Bayesian Uncertainty Modeling for P300-Based Brain-Computer Interface.

P300 potential is important to cognitive neuroscience research, and has also been widely applied in brain-computer interfaces (BCIs). To detect P300, many neural network models, including convolutional neural networks (CNNs), have achieved outstanding results. However, EEG signals are usually high-dimensional. Moreover, since collecting EEG signals is time-consuming and expensive, EEG datasets are typically small. Therefore, data-sparse regions usually exist within EEG dataset. However, most existing models compute predictions based on point-estimate. They cannot evaluate prediction uncertainty and tend to make overconfident decisions on samples located in data-sparse regions. Hence, their predictions are unreliable. To solve this problem, we propose a Bayesian convolutional neural network (BCNN) for P300 detection. The network places probability distributions over weights to capture model uncertainty. In prediction phase, a set of neural networks can be obtained by Monte Carlo sampling. Integrating the predictions of these networks implies ensembling. Therefore, the reliability of prediction can be improved. Experimental results demonstrate that BCNN can achieve better P300 detection performance than point-estimate networks. In addition, placing a prior distribution over the weight acts as a regularization technique. Experimental results show that it improves the robustness of BCNN to overfitting on small dataset. More importantly, with BCNN, both weight uncertainty and prediction uncertainty can be obtained. The weight uncertainty is then used to optimize the network through pruning, and the prediction uncertainty is applied to reject unreliable decisions so as to reduce detection error. Therefore, uncertainty modeling provides important information to further improve BCI systems.

Eliminating or Shortening the Calibration for a P300 Brain-Computer Interface Based on a Convolutional Neural Network and Big Electroencephalography Data: An Online Study.

A brain-computer interface (BCI) measures and analyzes brain activity and converts it into computer commands to control external devices. Traditional BCIs usually require full calibration, which is time-consuming and makes BCI systems inconvenient to use. In this study, we propose an online P300 BCI spelling system with zero or shortened calibration based on a convolutional neural network (CNN) and big electroencephalography (EEG) data. Specifically, three methods are proposed to train CNNs for the online detection of P300 potentials: (i) training a subject-independent CNN with data collected from 150 subjects; (ii) adapting the CNN online via a semisupervised learning/self-training method based on unlabeled data collected during the user's online operation; and (iii) fine-tuning the CNN with a transfer learning method based on a small quantity of labeled data collected before the user's online operation. Note that the calibration process is eliminated in the first two methods and dramatically shortened in the third method. Based on these methods, an online P300 spelling system is developed. Twenty subjects participated in our online experiments. Average accuracies of 89.38%, 94.00% and 93.50% were obtained by the subject-independent CNN, the self-training-based CNN and the transfer learning-based CNN, respectively. These results demonstrate the effectiveness of our methods, and thus, the convenience of the online P300-based BCI system is substantially improved.

Bayesian Collaborative Learning for Whole-Slide Image Classification.

Whole-slide image (WSI) classification is fundamental to computational pathology, which is challenging in extra-high resolution, expensive manual annotation, data heterogeneity, etc. Multiple instance learning (MIL) provides a promising way towards WSI classification, which nevertheless suffers from the memory bottleneck issue inherently, due to the gigapixel high resolution. To avoid this issue, the overwhelming majority of existing approaches have to decouple the feature encoder and the MIL aggregator in MIL networks, which may largely degrade the performance. Towards this end, this paper presents a Bayesian Collaborative Learning (BCL) framework to address the memory bottleneck issue with WSI classification. Our basic idea is to introduce an auxiliary patch classifier to interact with the target MIL classifier to be learned, so that the feature encoder and the MIL aggregator in the MIL classifier can be learned collaboratively while preventing the memory bottleneck issue. Such a collaborative learning procedure is formulated under a unified Bayesian probabilistic framework and a principled Expectation-Maximization algorithm is developed to infer the optimal model parameters iteratively. As an implementation of the E-step, an effective quality-aware pseudo labeling strategy is also suggested. The proposed BCL is extensively evaluated on three publicly available WSI datasets, i.e., CAMELYON16, TCGA-NSCLC and TCGA-RCC, achieving an AUC of 95.6%, 96.0% and 97.5% respectively, which consistently outperforms all the methods compared. Comprehensive analysis and discussion will also be presented for in-depth understanding of the method. To promote future work, our source code is released at: https://github.com/Zero-We/BCL.

Identifying Solitary Granulomatous Nodules from Solid Lung Adenocarcinoma: Exploring Robust Image Features with Cross-Domain Transfer Learning.

PURPOSE: This study aimed to find suitable source domain data in cross-domain transfer learning to extract robust image features. Then, a model was built to preoperatively distinguish lung granulomatous nodules (LGNs) from lung adenocarcinoma (LAC) in solitary pulmonary solid nodules (SPSNs). METHODS: Data from 841 patients with SPSNs from five centres were collected retrospectively. First, adaptive cross-domain transfer learning was used to construct transfer learning signatures (TLS) under different source domain data and conduct a comparative analysis. The Wasserstein distance was used to assess the similarity between the source domain and target domain data in cross-domain transfer learning. Second, a cross-domain transfer learning radiomics model (TLRM) combining the best performing TLS, clinical factors and subjective CT findings was constructed. Finally, the performance of the model was validated through multicentre validation cohorts. RESULTS: Relative to other source domain data, TLS based on lung whole slide images as source domain data (TLS-LW) had the best performance in all validation cohorts (AUC range: 0.8228-0.8984). Meanwhile, the Wasserstein distance of TLS-LW was 1.7108, which was minimal. Finally, TLS-LW, age, spiculated sign and lobulated shape were used to build the TLRM. In all validation cohorts, The AUC ranges were 0.9074-0.9442. Compared with other models, decision curve analysis and integrated discrimination improvement showed that TLRM had better performance. CONCLUSIONS: The TLRM could assist physicians in preoperatively differentiating LGN from LAC in SPSNs. Furthermore, compared with other images, cross-domain transfer learning can extract robust image features when using lung whole slide images as source domain data and has a better effect.

Efficient Informative Path Planning via Normalized Utility in Unknown Environments Exploration.

Exploration is an important aspect of autonomous robotics, whether it is for target searching, rescue missions, or reconnaissance in an unknown environment. In this paper, we propose a solution to efficiently explore the unknown environment by unmanned aerial vehicles (UAV). Innovatively, a topological road map is incrementally built based on Rapidly-exploring Random Tree (RRT) and maintained along with the whole exploration process. The topological structure can provide a set of waypoints for searching an optimal informative path. To evaluate the path, we consider the information measurement based on prior map uncertainty and the distance cost of the path, and formulate a normalized utility to describe information-richness along the path. The informative path is determined in every period by a local planner, and the robot executes the planned path to collect measurements of the unknown environment and restructure a map. The proposed framework and its composed modules are verified in two 3-D environments, which exhibit better performance in improving the exploration efficiency than other methods.

Predicting lymphovascular invasion in clinically node-negative breast cancer detected by abbreviated magnetic resonance imaging: Transfer learning vs. radiomics.

Objective: To compare the performance of abbreviated breast magnetic resonance imaging (AB-MRI)-based transfer learning (TL) algorithm and radionics analysis for lymphovascular invasion (LVI) prediction in patients with clinically node-negative invasive breast cancer (IBC). Methods: Between November 2017 and October 2020, 233 clinically node-negative IBCs detected by AB-MRI were retrospectively enrolled. One hundred thirty IBCs from center 1 (37 LVI-positive and 93 LVI-negative) were assigned as the training cohort and 103 from center 2 (25 LVI-positive and 78 LVI-negative) as the validation cohort. Based on AB-MRI, a TL signature (TLS) and a radiomics signature (RS) were built with the least absolute shrinkage and selection operator (LASSO) logistic regression. Their diagnostic performances were validated and compared using areas under the receiver operating curve (AUCs), net reclassification improvement (NRI), integrated discrimination improvement (IDI), decision curve analysis (DCA), and stratification analysis. A convolutional filter visualization technique was used to map the response areas of LVI on the AB-MRI. Results: In the validation cohort, compared with RS, the TLS showed better capability in discriminating LVI-positive from LVI-negative lesions (AUC: 0.852 vs. 0.726, p < 0.001; IDI = 0.092, p < 0.001; NRI = 0.554, p < 0.001). The diagnostic performance of TLS was not affected by the menstrual state, molecular subtype, or contrast agent type (all p > 0.05). Moreover, DCA showed that the TLS added more net benefit than RS for clinical utility. Conclusions: An AB-MRI-based TLS was superior to RS for preoperative LVI prediction in patients with clinically node-negative IBC.

Toward Assessment of Sound Localization in Disorders of Consciousness Using a Hybrid Audiovisual Brain-Computer Interface.

Behavioral assessment of sound localization in the Coma Recovery Scale-Revised (CRS-R) poses a significant challenge due to motor disability in patients with disorders of consciousness (DOC). Brain-computer interfaces (BCIs), which can directly detect brain activities related to external stimuli, may thus provide an approach to assess DOC patients without the need for any physical behavior. In this study, a novel audiovisual BCI system was developed to simulate sound localization evaluation in CRS-R. Specifically, there were two alternatively flashed buttons on the left and right sides of the graphical user interface, one of which was randomly chosen as the target. The auditory stimuli of bell sounds were simultaneously presented by the ipsilateral loudspeaker during the flashing of the target button, which prompted patients to selectively attend to the target button. The recorded electroencephalography data were analyzed in real time to detect event-related potentials evoked by the target and further to determine whether the target was attended to or not. A significant BCI accuracy for a patient implied that he/she had sound localization. Among eighteen patients, eleven and four showed sound localization in the BCI and CRS-R, respectively. Furthermore, all patients showing sound localization in the CRS-R were among those detected by our BCI. The other seven patients who had no sound localization behavior in CRS-R were identified by the BCI assessment, and three of them showed improvements in the second CRS-R assessment after the BCI experiment. Thus, the proposed BCI system is promising for assisting the assessment of sound localization and improving the clinical diagnosis of DOC patients.

A BCI-Based Study on the Relationship Between the SSVEP and Retinal Eccentricity in Overt and Covert Attention.

Covert attention aids us in monitoring the environment and optimizing performance in visual tasks. Past behavioral studies have shown that covert attention can enhance spatial resolution. However, electroencephalography (EEG) activity related to neural processing between central and peripheral vision has not been systematically investigated. Here, we conducted an EEG study with 25 subjects who performed covert attentional tasks at different retinal eccentricities ranging from 0.75° to 13.90°, as well as tasks involving overt attention and no attention. EEG signals were recorded with a single stimulus frequency to evoke steady-state visual evoked potentials (SSVEPs) for attention evaluation. We found that the SSVEP response in fixating at the attended location was generally negatively correlated with stimulus eccentricity as characterized by Euclidean distance or horizontal and vertical distance. Moreover, more pronounced characteristics of SSVEP analysis were also acquired in overt attention than in covert attention. Furthermore, offline classification of overt attention, covert attention, and no attention yielded an average accuracy of 91.42%. This work contributes to our understanding of the SSVEP representation of attention in humans and may also lead to brain-computer interfaces (BCIs) that allow people to communicate with choices simply by shifting their attention to them.

Learning Invariant Patterns Based on a Convolutional Neural Network and Big Electroencephalography Data for Subject-Independent P300 Brain-Computer Interfaces.

A brain-computer interface (BCI) measures and analyzes brain activity and converts this activity into computer commands to control external devices. In contrast to traditional BCIs that require a subject-specific calibration process before being operated, a subject-independent BCI learns a subject-independent model and eliminates subject-specific calibration for new users. However, building subject-independent BCIs remains difficult because electroencephalography (EEG) is highly noisy and varies by subject. In this study, we propose an invariant pattern learning method based on a convolutional neural network (CNN) and big EEG data for subject-independent P300 BCIs. The CNN was trained using EEG data from a large number of subjects, allowing it to extract subject-independent features and make predictions for new users. We collected EEG data from 200 subjects in a P300-based spelling task using two different types of amplifiers. The offline analysis showed that almost all subjects obtained significant cross-subject and cross-amplifier effects, with an average accuracy of more than 80%. Furthermore, more than half of the subjects achieved accuracies above 85%. These results indicated that our method was effective for building a subject-independent P300 BCI, with which more than 50% of users could achieve high accuracies without subject-specific calibration.

Capsule Network for ERP Detection in Brain-Computer Interface.

Event-related potential (ERP) is bioelectrical activity that occurs in the brain in response to specific events or stimuli, reflecting the electrophysiological changes in the brain during cognitive processes. ERP is important in cognitive neuroscience and has been applied to brain-computer interfaces (BCIs). However, because ERP signals collected on the scalp are weak, mixed with spontaneous electroencephalogram (EEG) signals, and their temporal and spatial features are complex, accurate ERP detection is challenging. Compared to traditional neural networks, the capsule network (CapsNet) replaces scalar-output neurons with vector-output capsules, allowing the various input information to be well preserved in the capsules. In this study, we expect to utilize CapsNet to extract the discriminative spatial-temporal features of ERP and encode them in capsules to reduce the loss of valuable information, thereby improving the ERP detection performance for BCI. Therefore, we propose ERP-CapsNet to perform ERP detection in a BCI speller application. The experimental results on BCI Competition datasets and the Akimpech dataset show that ERP-CapsNet achieves better classification performances than do the state-of-the-art techniques. We also use a decoder to investigate the attributes of ERPs encoded in capsules. The results show that ERP-CapsNet relies on the P300 and P100 components to detect ERP. Therefore, ERP-CapsNet not only acts as an outstanding method for ERP detection, but also provides useful insights into the ERP detection mechanism.

A P300-Based BCI System Using Stereoelectroencephalography and Its Application in a Brain Mechanistic Study.

Stereoelectroencephalography (SEEG) signals can be obtained by implanting deep intracranial electrodes. SEEG depth electrodes can record brain activity from the shallow cortical layer and deep brain structures, which is not achievable through other recording techniques. Moreover, SEEG has the advantage of a high signal-to-noise ratio (SNR). Therefore, it provides a potential way to establish a highly efficient brain-computer interface (BCI) and aid in understanding human brain activity. In this study, we implemented a P300-based BCI using SEEG signals. A single-character oddball paradigm was applied to elicit P300. To predict target characters, we fed the feature vectors extracted from the signals collected by five SEEG contacts into a Bayesian linear discriminant analysis (BLDA) classifier. Thirteen epileptic patients implanted with SEEG electrodes participated in the experiment and achieved an average online spelling accuracy of 93.85%. Moreover, through single-contact decoding analysis and simulated online analysis, we found that the SEEG-based BCI system achieved a high performance even when using a single signal channel. Furthermore, contacts with high decoding accuracies were mainly distributed in the visual ventral pathway, especially the fusiform gyrus (FG) and lingual gyrus (LG), which played an important role in building P300-based SEEG BCIs. These results might provide new insights into P300 mechanistic studies and the corresponding BCIs.

Spatiotemporal-Filtering-Based Channel Selection for Single-Trial EEG Classification.

Achieving high classification performance in electroencephalogram (EEG)-based brain-computer interfaces (BCIs) often entails a large number of channels, which impedes their use in practical applications. Despite the previous efforts, it remains a challenge to determine the optimal subset of channels in a subject-specific manner without heavily compromising the classification performance. In this article, we propose a new method, called spatiotemporal-filtering-based channel selection (STECS), to automatically identify a designated number of discriminative channels by leveraging the spatiotemporal information of the EEG data. In STECS, the channel selection problem is cast under the framework of spatiotemporal filter optimization by incorporating a group sparsity constraints, and a computationally efficient algorithm is developed to solve the optimization problem. The performance of STECS is assessed on three motor imagery EEG datasets. Compared with state-of-the-art spatiotemporal filtering algorithms using full EEG channels, STECS yields comparable classification performance with only half of the channels. Moreover, STECS significantly outperforms the existing channel selection methods. These results suggest that this algorithm holds promise for simplifying BCI setups and facilitating practical utility.

A Transfer Learning Radiomics Nomogram for Preoperative Prediction of Borrmann Type IV Gastric Cancer From Primary Gastric Lymphoma.

OBJECTIVE: This study aims to differentiate preoperative Borrmann type IV gastric cancer (GC) from primary gastric lymphoma (PGL) by transfer learning radiomics nomogram (TLRN) with whole slide images of GC as source domain data. MATERIALS AND METHODS: This study retrospectively enrolled 438 patients with histopathologic diagnoses of Borrmann type IV GC and PGL. They received CT examinations from three hospitals. Quantitative transfer learning features were extracted by the proposed transfer learning radiopathomic network and used to construct transfer learning radiomics signatures (TLRS). A TLRN, which integrates TLRS, clinical factors, and CT subjective findings, was developed by multivariate logistic regression. The diagnostic TLRN performance was assessed by clinical usefulness in the independent validation set. RESULTS: The TLRN was built by TLRS and a high enhanced serosa sign, which showed good agreement by the calibration curve. The TLRN performance was superior to the clinical model and TLRS. Its areas under the curve (AUC) were 0.958 (95% confidence interval [CI], 0.883-0.991), 0.867 (95% CI, 0.794-0.922), and 0.921 (95% CI, 0.860-0.960) in the internal and two external validation cohorts, respectively. Decision curve analysis (DCA) showed that the TLRN was better than any other model. TLRN has potential generalization ability, as shown in the stratification analysis. CONCLUSIONS: The proposed TLRN based on gastric WSIs may help preoperatively differentiate PGL from Borrmann type IV GC.Borrmann type IV gastric cancer, primary gastric lymphoma, transfer learning, whole slide image, deep learning.

The CSP-Based New Features Plus Non-Convex Log Sparse Feature Selection for Motor Imagery EEG Classification.

The common spatial pattern (CSP) is a very effective feature extraction method in motor imagery based brain computer interface (BCI), but its performance depends on the selection of the optimal frequency band. Although a lot of research works have been proposed to improve CSP, most of these works have the problems of large computation costs and long feature extraction time. To this end, three new feature extraction methods based on CSP and a new feature selection method based on non-convex log regularization are proposed in this paper. Firstly, EEG signals are spatially filtered by CSP, and then three new feature extraction methods are proposed. We called them CSP-wavelet, CSP-WPD and CSP-FB, respectively. For CSP-Wavelet and CSP-WPD, the discrete wavelet transform (DWT) or wavelet packet decomposition (WPD) is used to decompose the spatially filtered signals, and then the energy and standard deviation of the wavelet coefficients are extracted as features. For CSP-FB, the spatially filtered signals are filtered into multiple bands by a filter bank (FB), and then the logarithm of variances of each band are extracted as features. Secondly, a sparse optimization method regularized with a non-convex log function is proposed for the feature selection, which we called LOG, and an optimization algorithm for LOG is given. Finally, ensemble learning is used for secondary feature selection and classification model construction. Combing feature extraction and feature selection methods, a total of three new EEG decoding methods are obtained, namely CSP-Wavelet+LOG, CSP-WPD+LOG, and CSP-FB+LOG. Four public motor imagery datasets are used to verify the performance of the proposed methods. Compared to existing methods, the proposed methods achieved the highest average classification accuracy of 88.86, 83.40, 81.53, and 80.83 in datasets 1-4, respectively. The feature extraction time of CSP-FB is the shortest. The experimental results show that the proposed methods can effectively improve the classification accuracy and reduce the feature extraction time. With comprehensive consideration of classification accuracy and feature extraction time, CSP-FB+LOG has the best performance and can be used for the real-time BCI system.