Identifying the Effect of Cognitive Motivation with the Method Based on Temporal Association Rule Mining Concept.

Being motivated has positive influences on task performance. However, motivation could result from various motives that affect different parts of the brain. Analyzing the motivation effect from all affected areas requires a high number of EEG electrodes, resulting in high cost, inflexibility, and burden to users. In various real-world applications, only the motivation effect is required for performance evaluation regardless of the motive. Analyzing the relationships between the motivation-affected brain areas associated with the task's performance could limit the required electrodes. This study introduced a method to identify the cognitive motivation effect with a reduced number of EEG electrodes. The temporal association rule mining (TARM) concept was used to analyze the relationships between attention and memorization brain areas under the effect of motivation from the cognitive motivation task. For accuracy improvement, the artificial bee colony (ABC) algorithm was applied with the central limit theorem (CLT) concept to optimize the TARM parameters. From the results, our method can identify the motivation effect with only FCz and P3 electrodes, with 74.5% classification accuracy on average with individual tests.

A simultaneous EEG-fNIRS dataset of the visual cognitive motivation study in healthy adults.

This article described a publicly available dataset of the visual cognitive motivation study in healthy adults. To gain an in-depth understanding and insights into motivation, Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were measured simultaneously at shared locations while participants performed a visual cognitive motivation task. The participants' choices in the cognitive motivation task were recorded. The effects of their motivation were identified in the recognition test afterward. This dataset comprised EEG and fNIRS data from sixteen healthy adults (age: 21- 37 years; 14 males and 2 females) during the cognitive motivation task with visual scenic stimuli. In addition, the motivation and the corresponding motivation effect were also provided. This dataset provides understanding and analyzing opportunities for the process of attention and decision while the brain undergoes an induced motivated state and its effect on the recognition performance.

Cross Domain Correlation Maximization for Enhancing the Target Recognition of SSVEP-Based Brain-Computer Interfaces.

The target recognition performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces can be significantly improved with a training-based approach. However, the training procedure is time consuming and often causes fatigue. Consequently, the number of training data should be limited, which may reduce the classification performance. Thus, how to improve classification accuracy without increasing the training time is crucial to SSVEP-based BCI system. This study proposes a transfer-related component analysis (TransRCA) method for addressing the above issue. In this method, the SSVEP-related components are extracted from a small number of training data of the current individual and combined with those extracted from a large number of existing training data of other individuals. The TransRCA method maximizes not only the inter-trial covariances between the source and target subjects, but also the correlation between the reference signals and SSVEP signals from the source and target subjects. The proposed method was validated on the SSVEP public Benchmark and BETA datasets, and the classification accuracy and information transmission rate of the ensemble version of the proposed TransRCA method were compared with those of the state-of-the-art eCCA, eTRCA, ttCCA, LSTeTRCA, and eIISMC methods on both datasets. The comparison results indicate that the proposed method provides a superior performance compared with these state-of-the-art methods, and thus has high potential for the development of a SSVEP-based brain-computer interface system with high classification performance that only uses a small number of training data.

Block Distributed Joint Temporal-Frequency-Phase Modulation for Steady-State Visual Evoked Potential Based Brain-Computer Interface With a Limited Number of Frequencies.

How to encode as many targets as possible with limited frequency resources is a grave problem that restricts the application of steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs). In the current study, we propose a novel block-distributed joint temporal-frequency-phase modulation method for a virtual speller based on SSVEP-based BCI. A 48-target speller keyboard array is virtually divided into eight blocks and each block contains six targets. The coding cycle consists of two sessions: in the first session, each block flashes at different frequencies while all the targets in the same block flicker at the same frequency; in the second session, all the targets in the same block flash at different frequencies. Using this method, 48 targets can be coded with only eight frequencies, which greatly reduces the frequency resources required, and average accuracies of 86.81  ± 9.41% and 91.36  ± 6.41% were obtained for both the offline and online experiments. This study provides a new coding approach for a large number of targets with a small number of frequencies, which can further expand the application potential of SSVEP-based BCI.

Effective Connectivity Analysis and Classification of Action Observation From Different Perspectives: An fMRI Study.

OBJECTIVE: Analyzing the effective connectivity characteristics of brain networks in the process of action observation is helpful for understanding the neurodynamic mechanisms during action observation. METHOD: In this study, functional magnetic resonance imaging (fMRI) images were obtained from 20 participants who performed hand-object interaction observation tasks from the first-person perspective (1PP) and third-person perspective (3PP). On the basis of a meta-analysis, 11 key brain regions were extracted as nodes to build an action observation network. The weighted and directional connections between all of the nodes were investigated using partial directional coherence (PDC) analysis in five narrow frequency bands. RESULTS: The statistical analysis indicated that the ultra-low frequency band ( ≤ 0.04 Hz) exhibited significant activation compared with other frequency bands for both 1PP and 3PP. In addition, it was found that 3PP induced significantly stronger brain activation than 1PP in the ultra-low frequency band. Moreover, this study attempted to classify fMRI data corresponding to different perspectives using brain network features. A comparative analysis revealed that the weighted and binary PDC matrix methods achieved classification accuracies of 86.3% and 80.8%, respectively. SIGNIFICANCE: The weighted PDC analysis exhibits a more comprehensive understanding of neural mechanisms during action observation in different visual perspectives. It also has potential applications value in human-computer interaction in the future.

The comparison of electroencephalography power and event related potential in success and failure during multitask game.

The first game-based treatment for children with attention-deficit hyperactivity disorder (ADHD) was approved by the United States Food and Drug Administration (FDA) in 2020. This game was developed for use at home as part of everyday training and can be used along with one's usual training plan. In this game, two tasks are performed in parallel: (1) a perceptual discrimination targeting task (response and not response and avoiding responding to sudden pop-up targets) and (2) a sensory-motor navigation task (players continuously adjust their location to interact with or avoid positional targets). However, the brain activity of people playing this game was not examined, and the immersive environment (3D virtual world) was not considered. Therefore, we aimed to develop a system to investigate brain activity using electroencephalography (EEG) during multitask gameplay in virtual reality (VR). In this experiment, we focused on the difference between the success and failure of the Go/No-Go task in a multitask game. We created a color discrimination task and a target tracking task in VR. The content of this game task was designed using previous multitask training. EEG and event data were recorded. Using event data, we can analyze the data in detail. We divided the trial types (Go and No-Go) and results (success and failure). We then compared the success and failure of each task. In the Go trial, the relative theta power in success at Fz was significantly higher than that of failure. However, no difference in power was observed in the No-Go trial. On the other hand, theta power was no different between success and failure in the other task. These results of the Go trial suggest that the participants were attentive to processing both tasks. Thus, it is possible that theta power in the frontal area 1 s before stimulation could predict the success or failure of the Go trial. On the other hand, the results of the No-Go trial may be due to the low number of No-Go failure trials and the fact that stimulus oversight is one of the factors for success.

Spatio-Temporal Neural Dynamics of Observing Non-Tool Manipulable Objects and Interactions.

Previous studies have reported that a series of sensory-motor-related cortical areas are affected when a healthy human is presented with images of tools. This phenomenon has been explained as familiar tools launching a memory-retrieval process to provide a basis for using the tools. Consequently, we postulated that this theory may also be applicable if images of tools were replaced with images of daily objects if they are graspable (i.e., manipulable). Therefore, we designed and ran experiments with human volunteers (participants) who were visually presented with images of three different daily objects and recorded their electroencephalography (EEG) synchronously. Additionally, images of these objects being grasped by human hands were presented to the participants. Dynamic functional connectivity between the visual cortex and all the other areas of the brain was estimated to find which of them were influenced by visual stimuli. Next, we compared our results with those of previous studies that investigated brain response when participants looked at tools and concluded that manipulable objects caused similar cerebral activity to tools. We also looked into mu rhythm and found that looking at a manipulable object did not elicit a similar activity to seeing the same object being grasped.

Using Mean Pupil Diameter Change to Analyze Behavioral Performance in Multitasking Training Game.

To explore the actual behavioral performance of subjects in multitasking training games, we designed a VR game including a Target-tracking task (TTT) of continuously moving "Player" to track "Targets" and a randomly appearing Color-discrimination task (CDT) requiring discriminating whether "Player" and "Monster" have the same color, and recorded subjects' pupillary changes to reflect mental effort. By analyzing the mean pupil diameter change (MPDC) of different groups, we found that the high group presented pupil dilation during the post-event stage, reflecting that they engaged in psychological processing of CDT during the event, whereas the low group had no pupil dilation during part of the post-event stage, reflecting the possibility of ignoring the appearance of CDT, and such behaviors hardly raise good expectations for training effect. Our study suggests that MPDC mirrors not only the actual behavior of the different groups treating the multitasking paradigm, but also the influence of game design.

EEG-EMG Correlation Analysis with Linear and Nonlinear Coupling Methods Across Four Motor Tasks.

Correlation between brain and muscle signal is referred to as functional coupling. The amount of correlation between two signals greatly depends on the motor task performance. In this study, we designed the experimental paradigm with four types of motor tasks such as real hand grasping movement (RM), movement intention (Inten), motor imagery (MI) and only looking at virtual hand in three dimensional head mounted display (OL). We aimed to investigate EEG-EMG correlation with linear and nonlinear coupling methods. The results proved that high correlation could be occurred in RM and Inten tasks rather than MI and OL tasks in both linear and nonlinear methods. High coherence occurred in beta and gamma bands of RM and Inten tasks whereas no coherence was detected in MI and OL tasks. In terms of nonlinear correlation, the high mutual information was detected in RM and Inten tasks. There was slight mutual information in MI and OL tasks. The results showed that the coherence in the contralateral brain cortex was higher than in the ipsilateral motor cortex during motor tasks. Furthermore, the amount of EEG-EMG functional coupling changed according to the motor task executed.

Electroencephalogram-Electromyogram Functional Coupling and Delay Time Change Based on Motor Task Performance.

Synchronous correlation brain and muscle oscillations during motor task execution is termed as functional coupling. Functional coupling between two signals appears with a delay time which can be used to infer the directionality of information flow. Functional coupling of brain and muscle depends on the type of muscle contraction and motor task performance. Although there have been many studies of functional coupling with types of muscle contraction and force level, there has been a lack of investigation with various motor task performances. Motor task types play an essential role that can reflect the amount of functional interaction. Thus, we examined functional coupling under four different motor tasks: real movement, intention, motor imagery and movement observation tasks. We explored interaction of two signals with linear and nonlinear information flow. The aim of this study is to investigate the synchronization between brain and muscle signals in terms of functional coupling and delay time. The results proved that brain-muscle functional coupling and delay time change according to motor tasks. Quick synchronization of localized cortical activity and motor unit firing causes good functional coupling and this can lead to short delay time to oscillate between signals. Signals can flow with bidirectionality between efferent and afferent pathways.

Investigation on Identifying Implicit Learning Event from EEG Signal Using Multiscale Entropy and Artificial Bee Colony.

The way people learn will play an essential role in the sustainable development of the educational system for the future. Utilizing technology in the age of information and incorporating it into how people learn can produce better learners. Implicit learning is a type of learning of the underlying rules without consciously seeking or understanding the rules; it is commonly seen in small children while learning how to speak their native language without learning grammar. This research aims to introduce a processing system that can systematically identify the relationship between implicit learning events and their Encephalogram (EEG) signal characteristics. This study converted the EEG signal from participants while performing cognitive task experiments into Multiscale Entropy (MSE) data. Using MSE data from different frequency bands and channels as features, the system explored a wide range of classifiers and observed their performance to see how they classified the features related to participants' performance. The Artificial Bee Colony (ABC) method was used for feature selection to improve the process to make the system more efficient. The results showed that the system could correctly identify the differences between participants' performance using MSE data and the ABC method with 95% confidence.

SSVEP-Based Brain-Computer Interface With a Limited Number of Frequencies Based on Dual-Frequency Biased Coding.

How to encode as many targets as possible with a limited-frequency resource is a difficult problem in the practical use of a steady-state visual evoked potential (SSVEP) based brain-computer interface (BCI) speller. To solve this problem, this study developed a novel method called dual-frequency biased coding (DFBC) to tag targets in a SSVEP-based 48-character virtual speller, in which each target is encoded with a permutation sequence consisting of two permuted flickering periods that flash at different frequencies. The proposed paradigm was validated by 11 participants in an offline experiment and 7 participants in an online experiment. Three occipital channels (O1, Oz, and O2) were used to obtain the SSVEP signals for identifying the targets. Based on the coding characteristics of the DFBC method, the proposed approach has the ability of self-correction and thus achieves an accuracy of 76.6% and 79.3% for offline and online experiments, respectively, which outperforms the traditional multiple frequencies sequential coding (MFSC) method. This study demonstrates that DFBC is an efficient method for coding a high number of SSVEP targets with a small number of available frequencies.

Multiscale Entropy as a New Feature for EEG and fNIRS Analysis.

The present study aims to apply multiscale entropy (MSE) to analyse brain activity in terms of brain complexity levels and to use simultaneous electroencephalogram and functional near-infrared spectroscopy (EEG/fNIRS) recordings for brain functional analysis. A memory task was selected to demonstrate the potential of this multimodality approach since memory is a highly complex neurocognitive process, and the mechanisms governing selective retention of memories are not fully understood by other approaches. In this study, 15 healthy participants with normal colour vision participated in the visual memory task, which involved the making the executive decision of remembering or forgetting the visual stimuli based on his/her own will. In a continuous stimulus set, 250 indoor/outdoor scenes were presented at random, between periods of fixation on a black background. The participants were instructed to make a binary choice indicating whether they wished to remember or forget the image; both stimulus and response times were stored for analysis. The participants then performed a scene recognition test to confirm whether or not they remembered the images. The results revealed that the participants intentionally memorising a visual scene demonstrate significantly greater brain complexity levels in the prefrontal and frontal lobe than when purposefully forgetting a scene; p < 0.05 (two-tailed). This suggests that simultaneous EEG and fNIRS can be used for brain functional analysis, and MSE might be the potential indicator for this multimodality approach.

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement.

In a social world, observing the actions of others is fundamental to understanding what they are doing, as well as their intentions and feelings. Studies of the neural basis and decoding of action observation are important for understanding action-related processes and have implications for cognitive, social neuroscience, and human-machine interaction (HMI). In the current study, we first investigated temporal-spatial dynamics during action observation using a combined 64-channel electroencephalography (EEG) and 48-channel functional near-infrared spectroscopy (fNIRS) system. We measured brain activation while 16 healthy participants observed three action tasks: (1) grasping a cup with the intention of drinking; (2) grasping a cup with the intention of moving it; and (3) touching a cup with an unclear intention. The EEG and fNIRS source analysis results revealed the dynamic involvement of both the mirror neuron system (MNS) and the theory of mind (ToM)/mentalizing network during action observation. The source analysis results suggested that the extent to which these two systems were engaged was determined by the clarity of the intention of the observed action. Based on the difference in neural activity observed among different action-observation tasks in the first experiment, we conducted a second experiment to classify the neural processes underlying action observation using a feature classification method. We constructed complex brain networks based on the EEG and fNIRS data. Fusing features from both EEG and fNIRS complex brain networks resulted in a classification accuracy of 72.7% for the three action observation tasks. This study provides a theoretical and empirical basis for elucidating the neural mechanisms of action observation and intention understanding, and a feasible method for decoding the underlying neural processes.

Consistency and dynamical changes of directional information flow in different brain states: A comparison of working memory and resting-state using EEG.

Several recent studies have reported a frequency-dependent directional information flow loop in resting-state networks by phase transfer entropy, comprising an anterior-to-posterior information flow in the theta band and a posterior-to-anterior information flow in the alpha band. However, the functional roles of this information flow loop remain unclear. In the current study, we compared information flow patterns in four different brain states using electroencephalography: resting-state, fixation, working memory (WM) encoding and WM maintenance. An auditory (pure tones) WM span task was carried out. A consistent anterior-to-posterior information flow in the theta band and an opposite pattern in the alpha band were found in all four segments. Flows in both patterns were enhanced during WM encoding. In contrast, a prefrontal-to-central information flow in the alpha band was dominant during the resting-state. In addition, enhanced information flows from right temporal to other brain regions in the theta band were found during WM processing (WM encoding and maintenance). Comparison of the consistency and dynamical changes of information flows in these four brain states indicated their functional roles in central executive processes, internal attention, WM information maintenance, and the right-hemisphere advantage in pure tone processing.

Auditory White Noise Affects Left/Right Visual Working Memory in an Opposite Pattern.

Adding auditory white noise (WN) to the environment has been considered to be a promising way to enhance the memory performance of children with attention deficit/hyperactivity disorder (ADHD) but disrupt that of non-ADHD children. To explore the exact mechanism behind WN benefits, we did a bilateral color-memory task with different WN conditions. A bilateral color-square array was displayed on one display. Only one side colors were asked to be remembered in a trial. Our experiment found that the memory accuracy of left visual memory was improved with WN, especially when WN was displayed via left earphone at encoding and maintenance periods. However, the right visual memory showed a reduced performance tendency with WN. Thus, the WN affects left/right visual working memory in an opposite pattern. Using time-frequency analysis, we found an enhanced lower-alpha activity over the left occipitotemporal lobe. We conclude that the induced lower-alpha activity at the left occipitotemporal lobe might be helpful to inhibit information processing of left hemisphere.

Investigating the Effect of Intrinsic Motivation on Alpha Desynchronization Using Sample Entropy.

The effect of motivation and attention could play an important role in providing personalized learning services and improving learners toward smart education. These effects on brain activity could be quantified by EEG and open the path to analyze the efficiency of services during the learning process. Many studies reported the appearance of EEG alpha desynchronization during the attention period, resulting in better cognitive performance. Motivation was also found to be reflected in EEG. This study investigated the effect of intrinsic motivation on the alpha desynchronization pattern in terms of the complexity of time series data. The sample entropy method was used to quantify the complexity of event-related spectral perturbation (ERSP) of EEG data. We found that when participants can remember the stimulus, ERSP was significantly less complex than when they cannot. However, the effect of intrinsic motivation cannot be defined by using sample entropy directly. ERSP's main effect showed that motivation affects the complexity of ERSP data; longer continuous alpha desynchronization patterns were found when participants were motivated. Therefore, we introduced an algorithm to identify the longest continuous alpha desynchronization pattern. The method allowed us to understand that intrinsic motivation has an effect on recognition at the frontal and left parietal area directly.

Hybrid System for Engagement Recognition During Cognitive Tasks Using a CFS + KNN Algorithm.

Engagement is described as a state in which an individual involved in an activity can ignore other influences. The engagement level is important to obtaining good performance especially under study conditions. Numerous methods using electroencephalograph (EEG), electrocardiograph (ECG), and near-infrared spectroscopy (NIRS) for the recognition of engagement have been proposed. However, the results were either unsatisfactory or required many channels. In this study, we introduce the implementation of a low-density hybrid system for engagement recognition. We used a two-electrode wireless EEG, a wireless ECG, and two wireless channels NIRS to measure engagement recognition during cognitive tasks. We used electrooculograms (EOG) and eye tracking to record eye movements for data labeling. We calculated the recognition accuracy using the combination of correlation-based feature selection and k-nearest neighbor algorithm. Following that, we did a comparative study against a stand-alone system. The results show that the hybrid system had an acceptable accuracy for practical use (71.65 ± 0.16%). In comparison, the accuracy of a pure EEG system was (65.73 ± 0.17%), pure ECG (67.44 ± 0.19%), and pure NIRS (66.83 ± 0.17%). Overall, our results demonstrate that the proposed method can be used to improve performance in engagement recognition.

Sinusoidal Signal Assisted Multivariate Empirical Mode Decomposition for Brain-Computer Interfaces.

A brain-computer interface (BCI) is a communication approach that permits cerebral activity to control computers or external devices. Brain electrical activity recorded with electroencephalography (EEG) is most commonly used for BCI. Noise-assisted multivariate empirical mode decomposition (NA-MEMD) is a data-driven time-frequency analysis method that can be applied to nonlinear and nonstationary EEG signals for BCI data processing. However, because white Gaussian noise occupies a broad range of frequencies, some redundant components are introduced. To solve this leakage problem, in this study, we propose using a sinusoidal assisted signal that occupies the same frequency ranges as the original signals to improve MEMD performance. To verify the effectiveness of the proposed sinusoidal signal assisted MEMD (SA-MEMD) method, we compared the decomposition performances of MEMD, NA-MEMD, and the proposed SA-MEMD using synthetic signals and a real-world BCI dataset. The spectral decomposition results indicate that the proposed SA-MEMD can avoid the generation of redundant components and over decomposition, thus, substantially reduce the mode mixing and misalignment that occurs in MEMD and NA-MEMD. Moreover, using SA-MEMD as a signal preprocessing method instead of MEMD or NA-MEMD can significantly improve BCI classification accuracy and reduce calculation time, which indicates that SA-MEMD is a powerful spectral decomposition method for BCI.

Relationship between rTMS effects and MEP features before rTMS.

Repetitive transcranial magnetic stimulation (rTMS) is a promising method for use in the clinical field, as it can induce modulation of cortical excitability. Generally, rTMS inhibits the motor cortex when delivered at less than 1 Hz. However, it has been indicated that a facilitative effect is induced by 1 Hz rTMS, depending on the stimulation parameters and the individual. Therefore, the aim of this study was to investigate the features of the subject that could affect the 1 Hz rTMS effect when rTMS stimulus conditions change. First, motor evoked potentials (MEP) were measured under rTMS conditions with a variety of stimulus intensities and numbers of pulses. The MEP features before rTMS and the MEP modulation by the rTMS were then analyzed. Furthermore, correlations between the MEP features and the rTMS effect were investigated. It was found that the MEP amplitude and MEP onset before rTMS can influence the rTMS effect. Furthermore, negative correlations were found between these MEP features and the rTMS effect. MEPs with a small amplitude and early latency were little influenced by the inhibitive effect of 1 Hz rTMS, while MEPs with a large amplitude and late latency were readily affected by the inhibitive effect of 1 Hz rTMS. In this study, we focused on the MEP features before rTMS and identified the features of the subject that could influence the rTMS effect when the rTMS stimulus condition was changed.