Corrigendum: Multi-Modal Integration of EEG-fNIRS for Brain-Computer Interfaces - Current Limitations and Future Directions.

[This corrects the article DOI: 10.3389/fnhum.2017.00503.].

User's Self-Prediction of Performance in Motor Imagery Brain-Computer Interface.

Performance variation is a critical issue in motor imagery brain-computer interface (MI-BCI), and various neurophysiological, psychological, and anatomical correlates have been reported in the literature. Although the main aim of such studies is to predict MI-BCI performance for the prescreening of poor performers, studies which focus on the user's sense of the motor imagery process and directly estimate MI-BCI performance through the user's self-prediction are lacking. In this study, we first test each user's self-prediction idea regarding motor imagery experimental datasets. Fifty-two subjects participated in a classical, two-class motor imagery experiment and were asked to evaluate their easiness with motor imagery and to predict their own MI-BCI performance. During the motor imagery experiment, an electroencephalogram (EEG) was recorded; however, no feedback on motor imagery was given to subjects. From EEG recordings, the offline classification accuracy was estimated and compared with several questionnaire scores of subjects, as well as with each subject's self-prediction of MI-BCI performance. The subjects' performance predictions during motor imagery task showed a high positive correlation (r = 0.64, p < 0.01). Interestingly, it was observed that the self-prediction became more accurate as the subjects conducted more motor imagery tasks in the Correlation coefficient (pre-task to 2nd run: r = 0.02 to r = 0.54, p < 0.01) and root mean square error (pre-task to 3rd run: 17.7% to 10%, p < 0.01). We demonstrated that subjects may accurately predict their MI-BCI performance even without feedback information. This implies that the human brain is an active learning system and, by self-experiencing the endogenous motor imagery process, it can sense and adopt the quality of the process. Thus, it is believed that users may be able to predict MI-BCI performance and results may contribute to a better understanding of low performance and advancing BCI.

Multi-Modal Integration of EEG-fNIRS for Brain-Computer Interfaces - Current Limitations and Future Directions.

Multi-modal integration, which combines multiple neurophysiological signals, is gaining more attention for its potential to supplement single modality's drawbacks and yield reliable results by extracting complementary features. In particular, integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) is cost-effective and portable, and therefore is a fascinating approach to brain-computer interface (BCI). However, outcomes from the integration of these two modalities have yielded only modest improvement in BCI performance because of the lack of approaches to integrate the two different features. In addition, mismatch of recording locations may hinder further improvement. In this literature review, we surveyed studies of the integration of EEG/fNIRS in BCI thoroughly and discussed its current limitations. We also suggested future directions for efficient and successful multi-modal integration of EEG/fNIRS in BCI systems.

Multi-Scale Computational Models for Electrical Brain Stimulation.

Electrical brain stimulation (EBS) is an appealing method to treat neurological disorders. To achieve optimal stimulation effects and a better understanding of the underlying brain mechanisms, neuroscientists have proposed computational modeling studies for a decade. Recently, multi-scale models that combine a volume conductor head model and multi-compartmental models of cortical neurons have been developed to predict stimulation effects on the macroscopic and microscopic levels more precisely. As the need for better computational models continues to increase, we overview here recent multi-scale modeling studies; we focused on approaches that coupled a simplified or high-resolution volume conductor head model and multi-compartmental models of cortical neurons, and constructed realistic fiber models using diffusion tensor imaging (DTI). Further implications for achieving better precision in estimating cellular responses are discussed.

Exploring Neuro-Physiological Correlates of Drivers' Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data.

Investigations of the neuro-physiological correlates of mental loads, or states, have attracted significant attention recently, as it is particularly important to evaluate mental fatigue in drivers operating a motor vehicle. In this research, we collected multimodal EEG/ECG/EOG and fNIRS data simultaneously to develop algorithms to explore neuro-physiological correlates of drivers' mental states. Each subject performed simulated driving under two different conditions (well-rested and sleep-deprived) on different days. During the experiment, we used 68 electrodes for EEG/ECG/EOG and 8 channels for fNIRS recordings. We extracted the prominent features of each modality to distinguish between the well-rested and sleep-deprived conditions, and all multimodal features, except EOG, were combined to quantify mental fatigue during driving. Finally, a novel driving condition level (DCL) was proposed that distinguished clearly between the features of well-rested and sleep-deprived conditions. This proposed DCL measure may be applicable to real-time monitoring of the mental states of vehicle drivers. Further, the combination of methods based on each classifier yielded substantial improvements in the classification accuracy between these two conditions.

Gamma band activity associated with BCI performance: simultaneous MEG/EEG study.

While brain computer interface (BCI) can be employed with patients and healthy subjects, there are problems that must be resolved before BCI can be useful to the public. In the most popular motor imagery (MI) BCI system, a significant number of target users (called "BCI-Illiterates") cannot modulate their neuronal signals sufficiently to use the BCI system. This causes performance variability among subjects and even among sessions within a subject. The mechanism of such BCI-Illiteracy and possible solutions still remain to be determined. Gamma oscillation is known to be involved in various fundamental brain functions, and may play a role in MI. In this study, we investigated the association of gamma activity with MI performance among subjects. Ten simultaneous MEG/EEG experiments were conducted; MI performance for each was estimated by EEG data, and the gamma activity associated with BCI performance was investigated with MEG data. Our results showed that gamma activity had a high positive correlation with MI performance in the prefrontal area. This trend was also found across sessions within one subject. In conclusion, gamma rhythms generated in the prefrontal area appear to play a critical role in BCI performance.

Bayesian brain source imaging based on combined MEG/EEG and fMRI using MCMC.

A number of brain imaging techniques have been developed in order to investigate brain function and to develop diagnostic tools for various brain disorders. Each modality has strengths as well as weaknesses compared to the others. Recent work has explored how multiple modalities can be integrated effectively so that they complement one another while maintaining their individual strengths. Bayesian inference employing Markov Chain Monte Carlo (MCMC) techniques provides a straightforward way to combine disparate forms of information while dealing with the uncertainty in each. In this paper we introduce methods of Bayesian inference as a way to integrate different forms of brain imaging data in a probabilistic framework. We formulate Bayesian integration of magnetoencephalography (MEG) data and functional magnetic resonance imaging (fMRI) data by incorporating fMRI data into a spatial prior. The usefulness and feasibility of the method are verified through testing with both simulated and empirical data.

Probabilistic forward model for electroencephalography source analysis.

Source localization by electroencephalography (EEG) requires an accurate model of head geometry and tissue conductivity. The estimation of source time courses from EEG or from EEG in conjunction with magnetoencephalography (MEG) requires a forward model consistent with true activity for the best outcome. Although MRI provides an excellent description of soft tissue anatomy, a high resolution model of the skull (the dominant resistive component of the head) requires CT, which is not justified for routine physiological studies. Although a number of techniques have been employed to estimate tissue conductivity, no present techniques provide the noninvasive 3D tomographic mapping of conductivity that would be desirable. We introduce a formalism for probabilistic forward modeling that allows the propagation of uncertainties in model parameters into possible errors in source localization. We consider uncertainties in the conductivity profile of the skull, but the approach is general and can be extended to other kinds of uncertainties in the forward model. We and others have previously suggested the possibility of extracting conductivity of the skull from measured electroencephalography data by simultaneously optimizing over dipole parameters and the conductivity values required by the forward model. Using Cramer-Rao bounds, we demonstrate that this approach does not improve localization results nor does it produce reliable conductivity estimates. We conclude that the conductivity of the skull has to be either accurately measured by an independent technique, or that the uncertainties in the conductivity values should be reflected in uncertainty in the source location estimates.

Spatiotemporal noise covariance estimation from limited empirical magnetoencephalographic data.

The performance of parametric magnetoencephalography (MEG) and electroencephalography (EEG) source localization approaches can be degraded by the use of poor background noise covariance estimates. In general, estimation of the noise covariance for spatiotemporal analysis is difficult mainly due to the limited noise information available. Furthermore, its estimation requires a large amount of storage and a one-time but very large (and sometimes intractable) calculation or its inverse. To overcome these difficulties, noise covariance models consisting of one pair or a sum of multi-pairs of Kronecker products of spatial covariance and temporal covariance have been proposed. However, these approaches cannot be applied when the noise information is very limited, i.e., the amount of noise information is less than the degrees of freedom of the noise covariance models. A common example of this is when only averaged noise data are available for a limited prestimulus region (typically at most a few hundred milliseconds duration). For such cases, a diagonal spatiotemporal noise covariance model consisting of sensor variances with no spatial or temporal correlation has been the common choice for spatiotemporal analysis. In this work, we propose a different noise covariance model which consists of diagonal spatial noise covariance and Toeplitz temporal noise covariance. It can easily be estimated from limited noise information, and no time-consuming optimization and data-processing are required. Thus, it can be used as an alternative choice when one-pair or multi-pair noise covariance models cannot be estimated due to lack of noise information. To verify its capability we used Bayesian inference dipole analysis and a number of simulated and empirical datasets. We compared this covariance model with other existing covariance models such as conventional diagonal covariance, one-pair and multi-pair noise covariance models, when noise information is sufficient to estimate them. We found that our proposed noise covariance model yields better localization performance than a diagonal noise covariance, while it performs slightly worse than one-pair or multi-pair noise covariance models - although these require much more noise information. Finally, we present some localization results on median nerve stimulus empirical MEG data for our proposed noise covariance model.

Improving source detection and separation in a spatiotemporal Bayesian inference dipole analysis.

Most existing spatiotemporal multi-dipole approaches for MEG/EEG source localization assume that the dipoles are active for the full time range being analysed. If the actual time range of activity of sources is significantly shorter than the time range being analysed, the detectability, localization and time-course determination of such sources may be adversely affected, especially for weak sources. In order to improve detectability and reconstruction of such sources, it is natural to add active time range information (starting time point and ending time point of source activation) for each candidate source as unknown parameters in the analysis. However, this adds additional nonlinear free parameters that could burden the analysis and could be unfeasible for some methods. Recently, we described a spatiotemporal Bayesian inference multi-dipole analysis for the MEG/EEG inverse problem. This approach treated the number of dipoles as a free parameter, produced realistic uncertainty estimates using a Markov chain Monte Carlo numerical sampling of the posterior distribution and included a method to reduce the unwanted effects of local minima. In this paper, our spatiotemporal Bayesian inference multi-dipole analysis is extended to incorporate active time range parameters of starting and stopping time points. The properties of this analysis in comparison to the previous one without active time range parameters are demonstrated through extensive studies using both simulated and empirical MEG data.

Spatiotemporal Bayesian inference dipole analysis for MEG neuroimaging data.

Recently, we described a Bayesian inference approach to the MEG/EEG inverse problem that used numerical techniques to estimate the full posterior probability distributions of likely solutions upon which all inferences were based [Schmidt, D.M., George, J.S., Wood, C.C., 1999. Bayesian inference applied to the electromagnetic inverse problem. Human Brain Mapping 7, 195; Schmidt, D.M., George, J.S., Ranken, D.M., Wood, C.C., 2001. Spatial-temporal bayesian inference for MEG/EEG. In: Nenonen, J., Ilmoniemi, R. J., Katila, T. (Eds.), Biomag 2000: 12th International Conference on Biomagnetism. Espoo, Norway, p. 671]. Schmidt et al. (1999) focused on the analysis of data at a single point in time employing an extended region source model. They subsequently extended their work to a spatiotemporal Bayesian inference analysis of the full spatiotemporal MEG/EEG data set. Here, we formulate spatiotemporal Bayesian inference analysis using a multi-dipole model of neural activity. This approach is faster than the extended region model, does not require use of the subject's anatomical information, does not require prior determination of the number of dipoles, and yields quantitative probabilistic inferences. In addition, we have incorporated the ability to handle much more complex and realistic estimates of the background noise, which may be represented as a sum of Kronecker products of temporal and spatial noise covariance components. This reduces the effects of undermodeling noise. In order to reduce the rigidity of the multi-dipole formulation which commonly causes problems due to multiple local minima, we treat the given covariance of the background as uncertain and marginalize over it in the analysis. Markov Chain Monte Carlo (MCMC) was used to sample the many possible likely solutions. The spatiotemporal Bayesian dipole analysis is demonstrated using simulated and empirical whole-head MEG data.