Simultaneous multimodal fNIRS-EEG recordings reveal new insights in neural activity during motor execution, observation, and imagery.

Motor execution, observation, and imagery are important skills used in motor learning and rehabilitation. The neural mechanisms underlying these cognitive-motor processes are still poorly understood. We used a simultaneous recording of functional near-infrared spectroscopy (fNIRS) and electroencephalogram (EEG) to elucidate the differences in neural activity across three conditions requiring these processes. Additionally, we used a new method called structured sparse multiset Canonical Correlation Analysis (ssmCCA) to fuse the fNIRS and EEG data and determine the brain regions of neural activity consistently detected by both modalities. Unimodal analyses revealed differentiated activation between conditions; however, the activated regions did not fully overlap across the two modalities (fNIRS: left angular gyrus, right supramarginal gyrus, as well as right superior and inferior parietal lobes; EEG: bilateral central, right frontal, and parietal). These discrepancies might be because fNIRS and EEG detect different signals. Using fused fNIRS-EEG data, we consistently found activation over the left inferior parietal lobe, superior marginal gyrus, and post-central gyrus during all three conditions, suggesting that our multimodal approach identifies a shared neural region associated with the Action Observation Network (AON). This study highlights the strengths of using the multimodal fNIRS-EEG fusion technique for studying AON. Neural researchers should consider using the multimodal approach to validate their findings.

Structured sparse multiset canonical correlation analysis of simultaneous fNIRS and EEG provides new insights into the human action-observation network.

The action observation network (AON) is a network of brain regions involved in the execution and observation of a given action. The AON has been investigated in humans using mostly electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), but shared neural correlates of action observation and action execution are still unclear due to lack of ecologically valid neuroimaging measures. In this study, we used concurrent EEG and functional Near Infrared Spectroscopy (fNIRS) to examine the AON during a live-action observation and execution paradigm. We developed structured sparse multiset canonical correlation analysis (ssmCCA) to perform EEG-fNIRS data fusion. MCCA is a generalization of CCA to more than two sets of variables and is commonly used in medical multimodal data fusion. However, mCCA suffers from multi-collinearity, high dimensionality, unimodal feature selection, and loss of spatial information in interpreting the results. A limited number of participants (small sample size) is another problem in mCCA, which leads to overfitted models. Here, we adopted graph-guided (structured) fused least absolute shrinkage and selection operator (LASSO) penalty to mCCA to conduct feature selection, incorporating structural information amongst the variables (i.e., brain regions). Benefitting from concurrent recordings of brain hemodynamic and electrophysiological responses, the proposed ssmCCA finds linear transforms of each modality such that the correlation between their projections is maximized. Our analysis of 21 right-handed participants indicated that the left inferior parietal region was active during both action execution and action observation. Our findings provide new insights into the neural correlates of AON which are more fine-tuned than the results from each individual EEG or fNIRS analysis and validate the use of ssmCCA to fuse EEG and fNIRS datasets.

Gaming behavior and brain activation using functional near-infrared spectroscopy, Iowa gambling task, and machine learning techniques.

INTRODUCTION: The current study investigates the utilization and performance of machine learning (ML) algorithms in the cognitive task of finding the correlation between numerical parameters of the human brain activation during gaming. We hypothesize that our integrated feature extraction platform is able to distinguish between different psychosomatic conditions in the gaming process as measured by the functional near-infrared brain imaging technique. METHODS: For demonstration, the decision-making process was constructed in the experiment environment that combined gaming simulator, such as the Iowa Gaming Task (IGT), with functional near-infrared spectroscopy (fNIRS) as the neuroimaging technique. Features of fNIRS levels were extracted, averaged, and synchronized by time with the IGT dataset to predict the task score inside ML algorithms, such as multiple regression, classification and regression trees, support vector machine, artificial neural network, and random forest. For findings validation, the experiment data were resampled by training and testing sets. Further, a training dataset was used to train the ML algorithms, and prediction accuracy was estimated by repeated cross-validation methods and compared by R squared and root mean square error (RMSE). The model with the best accuracy was used with the testing dataset and finalized the experiment. RESULTS: During the experiment, the highest correlation was identified in the fourth block between the oxy-hemoglobin signal and IGT score in average value (0.24) and signal feature (0.57). Such relationship is due to block 4 characterization as "conceptual" period when participants task experience reaches the maximum, and rewards raise accordingly. Simultaneously, ML algorithms, constructed based on training data set, demonstrate acceptable performance, and RMSE as the primary performance metric dynamically increases from block 1 to block 5, from the state of uncertainty and unknown to the certainty and risky. In contrast, R squared decreases during the same transition. In most IGT blocks, the best fitted model was determined as support vector machine with radial bases function kernel, and predictions were made with the highest accuracy (lowest RMSE) than in training models. CONCLUSION: Obtained findings showed the applicability and capability of ML models as a powerful technique to evaluate the cognitive neuroimaging task result. Moreover, in terms of features it was identified that the hemodynamic response reacts to the acceleration decision-making process and raises more significance than it was observed before.

Cerebral hemodynamic response during a live action-observation and action-execution task: A fNIRS study.

Although many studies have examined the location of the action observation network (AON) in human adults, the shared neural correlates of action-observation and action-execution are still unclear partially due to lack of ecologically valid neuroimaging measures. In this study, we aim to demonstrate the feasibility of using functional near infrared spectroscopy (fNIRS) to measure the neural correlates of action-observation and action execution regions during a live task. Thirty adults reached for objects or observed an experimenter reaching for objects while their cerebral hemodynamic responses including oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) were recorded in the sensorimotor and parietal regions. Our results indicated that the parietal regions, including bilateral superior parietal lobule (SPL), bilateral inferior parietal lobule (IPL), right supra-marginal region (SMG) and right angular gyrus (AG) share neural activity during action-observation and action-execution. Our findings confirm the applicability of fNIRS for the study of the AON and lay the foundation for future work with developmental and clinical populations.

Canonical correlation analysis of brain prefrontal activity measured by functional near infra-red spectroscopy (fNIRS) during a moral judgment task.

Individuals differ in the extent to which they make decisions in different moral dilemmas. In this study, we investigated the relationship between functional brain activities during moral decision making and psychopathic personality traits in a healthy population. We measured the hemodynamic activities of the brain by functional near-infrared spectroscopy (fNIRS). FNIRS is an evolving non-invasive neuroimaging modality which is relatively inexpensive, patient friendly and robust to subject movement. Psychopathic traits were evaluated through a self-report questionnaire called the Psychopathic Personality Inventory Revised (PPI-R). We recorded functional brain activities of 30 healthy subjects while they performed a moral judgment (MJ) task. Regularized canonical correlation analysis (R-CCA) was applied to find the relationships between activation in different regions of prefrontal cortex (PFC) and the core psychopathic traits. Our results showed a significant canonical correlation between PFC activation and PPI-R content scale (PPI-R-CS). Specifically, coldheartedness and carefree non-planfulness were the only PPI-R-CS factors that were highly correlated with PFC activation during personal (emotionally salient) MJ, while Machiavellian egocentricity, rebellious nonconformity, coldheartedness, and carefree non-planfulness were the core traits that exhibited the same dynamics as PFC activation during impersonal (more logical) MJ. Furthermore, ventromedial prefrontal cortex (vmPFC) and left lateral PFC were the most positively correlated regions with PPI-R-CS traits during personal MJ, and the right vmPFC and right lateral PFC in impersonal MJ.

The role of prefrontal cortex in a moral judgment task using functional near-infrared spectroscopy.

BACKGROUND: Understanding the neural basis of moral judgment (MJ) and human decision-making has been the subject of numerous studies because of their impact on daily life activities and social norms. Here, we aimed to investigate the neural process of MJ using functional near-infrared spectroscopy (fNIRS), a noninvasive, portable, and affordable neuroimaging modality. METHODS: We examined prefrontal cortex (PFC) activation in 33 healthy participants engaging in MJ exercises. We hypothesized that participants presented with personal (emotionally salient) and impersonal (less emotional) dilemmas would exhibit different brain activation observable through fNIRS. We also investigated the effects of utilitarian and nonutilitarian responses to MJ scenarios on PFC activation. Utilitarian responses are those that favor the greatest good while nonutilitarian responses favor moral actions. Mixed effect models were applied to model the cerebral hemodynamic changes that occurred during MJ dilemmas. RESULTS AND CONCLUSIONS: Our analysis found significant differences in PFC activation during personal versus impersonal dilemmas. Specifically, the left dorsolateral PFC was highly activated during impersonal MJ when a nonutilitarian decision was made. This is consistent with the majority of relevant fMRI studies, and demonstrates the feasibility of using fNIRS, with its portable and motion tolerant capacities, to investigate the neural basis of MJ dilemmas.

Exploring the role of task performance and learning style on prefrontal hemodynamics during a working memory task.

Existing literature outlines the quality and location of activation in the prefrontal cortex (PFC) during working memory (WM) tasks. However, the effects of individual differences on the underlying neural process of WM tasks are still unclear. In this functional near infrared spectroscopy study, we administered a visual and auditory n-back task to examine activation in the PFC while considering the influences of task performance, and preferred learning strategy (VARK score). While controlling for age, results indicated that high performance (HP) subjects (accuracy > 90%) showed task dependent lower activation compared to normal performance subjects in PFC region Specifically HP groups showed lower activation in left dorsolateral PFC (DLPFC) region during performance of auditory task whereas during visual task they showed lower activation in the right DLPFC. After accounting for learning style, we found a correlation between visual and aural VARK score and level of activation in the PFC. Subjects with higher visual VARK scores displayed lower activation during auditory task in left DLPFC, while those with higher visual scores exhibited higher activation during visual task in bilateral DLPFC. During performance of auditory task, HP subjects had higher visual VARK scores compared to NP subjects indicating an effect of learning style on the task performance and activation. The results of this study show that learning style and task performance can influence PFC activation, with applications toward neurological implications of learning style and populations with deficits in auditory or visual processing.

A machine learning approach to identify functional biomarkers in human prefrontal cortex for individuals with traumatic brain injury using functional near-infrared spectroscopy.

BACKGROUND: We have explored the potential prefrontal hemodynamic biomarkers to characterize subjects with Traumatic Brain Injury (TBI) by employing the multivariate machine learning approach and introducing a novel task-related hemodynamic response detection followed by a heuristic search for optimum set of hemodynamic features. To achieve this goal, the hemodynamic response from a group of 31 healthy controls and 30 chronic TBI subjects were recorded as they performed a complexity task. METHODS: To determine the optimum hemodynamic features, we considered 11 features and their combinations in characterizing TBI subjects. We investigated the significance of the features by utilizing a machine learning classification algorithm to score all the possible combinations of features according to their predictive power. RESULTS AND CONCLUSIONS: The identified optimum feature elements resulted in classification accuracy, sensitivity, and specificity of 85%, 85%, and 84%, respectively. Classification improvement was achieved for TBI subject classification through feature combination. It signified the major advantage of the multivariate analysis over the commonly used univariate analysis suggesting that the features that are individually irrelevant in characterizing the data may become relevant when used in combination. We also conducted a spatio-temporal classification to identify regions within the prefrontal cortex (PFC) that contribute in distinguishing between TBI and healthy subjects. As expected, Brodmann areas (BA) 10 within the PFC were isolated as the region that healthy subjects (unlike subjects with TBI), showed major hemodynamic activity in response to the High Complexity task. Overall, our results indicate that identified temporal and spatio-temporal features from PFC's hemodynamic activity are promising biomarkers in classifying subjects with TBI.