An auditory brain-computer interface based on selective attention to multiple tone streams.

In this study, we attempted to improve brain-computer interface (BCI) systems by means of auditory stream segregation in which alternately presented tones are perceived as sequences of various different tones (streams). A 3-class BCI using three tone sequences, which were perceived as three different tone streams, was investigated and evaluated. Each presented musical tone was generated by a software synthesizer. Eleven subjects took part in the experiment. Stimuli were presented to each user's right ear. Subjects were requested to attend to one of three streams and to count the number of target stimuli in the attended stream. In addition, 64-channel electroencephalogram (EEG) and two-channel electrooculogram (EOG) signals were recorded from participants with a sampling frequency of 1000 Hz. The measured EEG data were classified based on Riemannian geometry to detect the object of the subject's selective attention. P300 activity was elicited by the target stimuli in the segregated tone streams. In five out of eleven subjects, P300 activity was elicited only by the target stimuli included in the attended stream. In a 10-fold cross validation test, a classification accuracy over 80% for five subjects and over 75% for nine subjects was achieved. For subjects whose accuracy was lower than 75%, either the P300 was also elicited for nonattended streams or the amplitude of P300 was small. It was concluded that the number of selected BCI systems based on auditory stream segregation can be increased to three classes, and these classes can be detected by a single ear without the aid of any visual modality.

Left Lateral Prefrontal Activity Reflects a Change of Behavioral Tactics to Cope with a Given Rule: An fNIRS Study.

Rules prescribe human behavior and our attempts to choose appropriate behavior under a given rule. Cognitive control, a mechanism to choose and evaluate actions under a rule, is required to determine the appropriate behavior within the limitations of that rule. Consequently, such cognitive control increases mental workload. However, the workload caused by a cognitive task might be different when an additional rule must be considered in choosing the action. The present study was a functional near-infrared spectroscopy (fNIRS) investigation of an experimental task, in which the difficulty of an operation and existence of an additional rule were manipulated to dissociate the influence of that additional rule on cognitive processing. Twenty healthy Japanese volunteers participated. The participants performed an experimental task, in which the player caught one of five colored balls from the upper part of a computer screen by operating a mouse. Four task conditions were prepared to manipulate the task difficulty, which was defined in terms of operational difficulty. In turn, operational difficulty was determined by the width of the playable space and the existence of an additional rule, which reduced the score when a red ball was not caught. The 52-channel fNIRS data were collected from the forehead. Two regions of interest (ROIs) associated with the bilateral lateral prefrontal cortices (LPFCs) were determined, and a three-way repeated-measures analysis of variance (ANOVA) was performed using the task-related signal changes from each ROI. The fNIRS results revealed that bilateral LPFCs showed large signal changes with the increase in mental workload. The ANOVA showed a significant interaction between the existence of an additional rule and the location of the ROIs; that is, the left lateral prefrontal area showed a significant increase in signal intensity when the additional rule existed, and the participant occasionally decided to avoid catching a ball to successfully catch the red-colored ball. Thus, activation of the left LPFC corresponded more closely to the increase in cognitive control underlying the behavioral change made to cope with the additional rule.

Generation of spatial filters by ICA for detecting motor-related oscillatory EEG.

To detect the imagined limb movement from EEG for the use in BCI, the increase (ERS) and decrease (ERD) of the band power of the EEG originated from the sensorimotor cortex are commonly used. A spatial filter using neighboring channels is generally applied to the measured EEG for detecting such brain activity related to the motor imagery. However, the configuration and location of the spatial filter have been selected by the empirical method on trial-and-error basis. In this study, we recorded the EEG during motor imagery of left hand, right hand and feet from five subjects, and the ICA (independent component analysis) was applied to discover the spatial filters for extracting event-related EEG components of the motor imagery. It was suggested that the application of ICA might offer the experimenters appropriate local spatial filters, or at least, the "initial guess" for designing or selecting custom local spatial filters.

Neural substrate of sound duration discrimination during an auditory sequence in the guinea pig primary auditory cortex.

Mismatch negativity (MMN) is a negative component of event-related brain potentials elicited by stimulus transitions. Stimulus duration transition also elicits MMN (duration MMN), with a magnitude that is related to the degree of duration change and the discrimination ability. The neural substrates of duration MMN have not yet been investigated. We therefore studied how duration transitions in an auditory stimulus train are represented in neurons in the primary auditory cortex of anesthetized guinea pigs. Two types of neuronal responses to the context of changes in stimulus duration were found. One was a reduced response as the duration of the preceding stimulus was increased. Second was an enhancement of the late components of the response including sustained and offset responses at the duration transition. The former may be explained by the previously proposed two-tone suppression, which is dependent on the preceding stimulus duration. The latter is likely to be caused by stimulus-specific adaptation that could be a possible neural generator of duration MMN.

A NIRS-based brain-computer interface system during motor imagery: system development and online feedback training.

A brain-computer interface (BCI) to detect motor imagery from cerebrum hemodynamic data measured by NIRS (near-infrared spectroscopy) was constructed and the effect of the online feedback training for subjects was evaluated. Concentrations of Oxy- and deOxy-hemoglobin in the motor cortex during motor imagery of subject's right hand was measured by 52-channel NIRS system, and the mean magnitude of measured signal near C3 in the International 10-20 System was visually fed back online to the subject. On two out of three subjects, it was shown that the ratio between the averaged value and the standard deviation over trials (S/N ratio) of Oxy-hemoglobin signal elicited by the imagery of subject's right hand was increased by the 5-day online feedback training. Detailed investigation of the effect of the online feedback training on brain activities was left for further study.

A brain-computer interface (BCI) system based on auditory stream segregation.

An auditory brain-computer interface (BCI) which detected event-related potential (ERP) elicited by selective attention to one of the tone streams was proposed. Each tone in two kinds of frequency oddball tone sequences with different tone frequency range was presented alternatively to subjects, and they were perceived by subjects as two kinds of segregated streams. Event-related potentials elicited by two kinds of deviant tones were classified by linear discriminant analysis (LDA) to find which streams subjects paid selective attention. By the experiments to six subjects, it was shown that this system could realize binary selection from two kinds of segregated tone streams.

Characteristics of human luminance discrimination and modeling a neural network based on the response properties of the visual cortex.

Reaction time (RT) and error rate that depend on stimulus duration were measured in a luminance-discrimination reaction time task. Two patches of light with different luminance were presented to participants for 'short' (150 ms) or 'long' (1 s) period on each trial. When the stimulus duration was 'short', the participants responded more rapidly with poorer discrimination performance than they did in the longer duration. The results suggested that different sensory responses in the visual cortices were responsible for the dependence of response speed and accuracy on the stimulus duration during the luminance-discrimination reaction time task. It was shown that the simple winner-take-all-type neural network model receiving transient and sustained stimulus information from the primary visual cortex successfully reproduced RT distributions for correct responses and error rates. Moreover, temporal spike sequences obtained from the model network closely resembled to the neural activity in the monkey prefrontal or parietal area during other visual decision tasks such as motion discrimination and oddball detection tasks.

Change detection and difference detection of tone duration discrimination.

An event-related potential called mismatch negativity is known to exhibit physiological evidence of sensory memory. Mismatch negativity is believed to represent complicated neuronal mechanisms in a variety of animals and in humans. We employed the auditory oddball paradigm varying sound durations and observed two types of duration mismatch negativity in anesthetized guinea pigs. One was a duration mismatch negativity whose increase in peak amplitude occurred immediately after onset of the stimulus difference in a decrement oddball paradigm. The other exhibited a peak amplitude increase closer to the offset of the longer stimulus in an increment oddball paradigm. These results demonstrated a mechanism to percept the difference of duration change and revealed the importance of the end of a stimulus for this perception.

Sequential grouping of tone sequence as reflected by the mismatch negativity.

The human sequential grouping that organizes parts of tones into a group was examined by the mismatch negativity (MMN), a component of event-related potentials that reveals the sensory memory process. The sequential grouping is accomplished by the combinations of some factors, e.g., temporal and frequency proximity principles. In this study, auditory oddball stimuli in which each of the stimuli consisted of series of tone bursts, were applied to the subjects, and the MMN elicited by the deviation of the frequency of the last tone in the stimulus was investigated. The relationship between the expected phenomena of sequential grouping of tones and observed magnitudes of MMN was evaluated. It was shown that the magnitudes of MMN changed according to the configuration (number of tones, frequency) of tone sequence to be stored. This result suggested that the sequential grouping of presented tones was achieved on the preattentive auditory sensory memory process. It was also shown that the relative change of MMN magnitudes corresponded to the conditions of sequential grouping, which had been proposed by the auditory psychophysical studies. The investigation of MMN properties could reveal the nature of auditory sequential grouping.