Understanding how the human brain tracks emitted speech sounds to execute fluent speech production.

Auditory feedback of one's own speech is used to monitor and adaptively control fluent speech production. A new study in PLOS Biology using electrocorticography (ECoG) in listeners whose speech was artificially delayed identifies regions involved in monitoring speech production.

Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting.

Individuals often have reduced ability to hear alarms in real world situations (e.g., anesthesia monitoring, flying airplanes) when attention is focused on another task, sometimes with devastating consequences. This phenomenon is called inattentional deafness and usually occurs under critical high workload conditions. It is difficult to simulate the critical nature of these tasks in the laboratory. In this study, dry electroencephalography is used to investigate inattentional deafness in real flight while piloting an airplane. The pilots participating in the experiment responded to audio alarms while experiencing critical high workload situations. It was found that missed relative to detected alarms were marked by reduced stimulus evoked phase synchrony in theta and alpha frequencies (6-14 Hz) from 120 to 230 ms poststimulus onset. Correlation of alarm detection performance with intertrial coherence measures of neural phase synchrony showed different frequency and time ranges for detected and missed alarms. These results are consistent with selective attentional processes actively disrupting oscillatory coherence in sensory networks not involved with the primary task (piloting in this case) under critical high load conditions. This hypothesis is corroborated by analyses of flight parameters showing greater maneuvering associated with difficult phases of flight occurring during missed alarms. Our results suggest modulation of neural oscillation is a general mechanism of attention utilizing enhancement of phase synchrony to sharpen alarm perception during successful divided attention, and disruption of phase synchrony in brain networks when attentional demands of the primary task are great, such as in the case of inattentional deafness.

Neural signature of inattentional deafness.

Inattentional deafness is the failure to hear otherwise audible sounds (usually alarms) that may occur under high workload conditions. One potential cause for its occurrence could be an attentional bottleneck that occurs when task demands are high, resulting in lack of resources for processing of additional tasks. In this fMRI experiment, we explore the brain regions active during the occurrence of inattentional deafness using a difficult perceptual-motor task in which the participants fly through a simulated Red Bull air race course and at the same time push a button on the joystick to the presence of audio alarms. Participants were instructed to focus on the difficult piloting task and to press the button on the joystick quickly when they noticed an audio alarm. The fMRI results revealed that audio misses relative to hits had significantly greater activity in the right inferior frontal gyrus IFG and the superior medial frontal cortex. Consistent with an attentional bottleneck, activity in these regions was also present for poor flying performance (contrast of gates missed versus gates passed for the flying task). A psychophysiological interaction analysis from the IFG identified reduced effective connectivity to auditory processing regions in the right superior temporal gyrus for missed audio alarms relative to audio alarms that were heard. This study identifies a neural signature of inattentional deafness in an ecologically valid situation by directly measuring differences in brain activity and effective connectivity between audio alarms that were not heard compared to those that were heard. Hum Brain Mapp 38:5440-5455, 2017. © 2017 Wiley Periodicals, Inc.

Neural processes distinguishing elite from expert and novice athletes.

This commentary builds on a companion article in which Kim et al compare brain activation in elite, expert, and novice archers during a simulated target aiming task (Kim et al. 2014. Cogn Behav Neurol. 27:173-182). With the archery study as our starting point, we address 4 neural processes that may be responsible in general for elite athletes' superior performance over experts and novices: neural efficiency, cortical expansion, specialized processes, and internal models. In Kim et al's study, the elite archers' brains showed more activity in the supplementary motor area and the cerebellum than those of the novices and experts, and showed minimal widespread activity, especially in frontal areas involved with executive control. Kim et al's results are consistent with the idea of specialized neural processes that help coordinate motor planning and control. As athletes become more skilled, these processes may mediate the reduction in widespread activity in regions mapping executive control, and may produce a shift toward more automated processing. Kim et al's finding that activity in the cerebellum rose with increasing skill is consistent both with expansion of the finger representational area in the cerebellum and with internal models that simulate how archers manipulate the bow and arrow when aiming. Kim et al prepare the way for testing of neuromodulation techniques to improve athletic performance, refine highly technical job skills, and rehabilitate patients.

Shredding artifacts: extracting brain activity in EEG from extreme artifacts during skateboarding using ASR and ICA.

Introduction: To understand brain function in natural real-world settings, it is crucial to acquire brain activity data in noisy environments with diverse artifacts. Electroencephalography (EEG), while susceptible to environmental and physiological artifacts, can be cleaned using advanced signal processing techniques like Artifact Subspace Reconstruction (ASR) and Independent Component Analysis (ICA). This study aims to demonstrate that ASR and ICA can effectively extract brain activity from the substantial artifacts occurring while skateboarding on a half-pipe ramp. Methods: A dual-task paradigm was used, where subjects were presented with auditory stimuli during skateboarding and rest conditions. The effectiveness of ASR and ICA in cleaning artifacts was evaluated using a support vector machine to classify the presence or absence of a sound stimulus in single-trial EEG data. The study evaluated the effectiveness of ASR and ICA in artifact cleaning using five different pipelines: (1) Minimal cleaning (bandpass filtering), (2) ASR only, (3) ICA only, (4) ICA followed by ASR (ICAASR), and (5) ASR preceding ICA (ASRICA). Three skateboarders participated in the experiment. Results: Results showed that all ICA-containing pipelines, especially ASRICA (69%, 68%, 63%), outperformed minimal cleaning (55%, 52%, 50%) in single-trial classification during skateboarding. The ASRICA pipeline performed significantly better than other pipelines containing ICA for two of the three subjects, with no other pipeline performing better than ASRICA. The superior performance of ASRICA likely results from ASR removing non-stationary artifacts, enhancing ICA decomposition. Evidenced by ASRICA identifying more brain components via ICLabel than ICA alone or ICAASR for all subjects. For the rest condition, with fewer artifacts, the ASRICA pipeline (71%, 82%, 75%) showed slight improvement over minimal cleaning (73%, 70%, 72%), performing significantly better for two subjects. Discussion: This study demonstrates that ASRICA can effectively clean artifacts to extract single-trial brain activity during skateboarding. These findings affirm the feasibility of recording brain activity during physically demanding tasks involving substantial body movement, laying the groundwork for future research into the neural processes governing complex and coordinated body movements.

Neural correlates of sound externalization.

When we listen to sounds through headphones without utilizing special transforms, sound sources seem to be located inside our heads. The sound sources are said to be lateralized to one side or the other to varying degree. This internal lateralization is different than sound source localization in the natural environment in which the sound is localized distal to the head. We used fMRI to investigate difference in neural responses between lateralization and localization. Individualized binaural recordings were used as externalized auditory stimuli and stereo recordings were used as internalized auditory stimuli. Brain activity was measured while 14 participants performed an active auditory localization task and while 12 participants performed a stimulus type identification task. Irrespective of the task condition, we observed enhanced activity in the bilateral posterior temporal gyri (pSTG) for the externalized stimuli relative to the internalized stimuli. Region of interest analysis indicated that both left and right pSTG were more sensitive to sound sources in contra- than ipsilateral hemifields. Moreover, greater back than front activity was also found in the left pSTG. Compared to impoverished spatial auditory stimuli, realistic spatial auditory stimuli enhance neural responses in the pSTG. This may be why we could observe contralateral hemifield preference in bilateral pSTG that many previous studies have failed to observe. Overall, the results indicate the importance of using ecologically valid stimuli for investigating neural processes in human cortex.

Differential activation of brain regions involved with error-feedback and imitation based motor simulation when observing self and an expert's actions in pilots and non-pilots on a complex glider landing task.

In this fMRI study we investigate neural processes related to the action observation network using a complex perceptual-motor task in pilots and non-pilots. The task involved landing a glider (using aileron, elevator, rudder, and dive brake) as close to a target as possible, passively observing a replay of one's own previous trial, passively observing a replay of an expert's trial, and a baseline do nothing condition. The objective of this study is to investigate two types of motor simulation processes used during observation of action: imitation based motor simulation and error-feedback based motor simulation. It has been proposed that the computational neurocircuitry of the cortex is well suited for unsupervised imitation based learning, whereas, the cerebellum is well suited for error-feedback based learning. Consistent with predictions, pilots (to a greater extent than non-pilots) showed significant differential activity when observing an expert landing the glider in brain regions involved with imitation based motor simulation (including premotor cortex PMC, inferior frontal gyrus IFG, anterior insula, parietal cortex, superior temporal gyrus, and middle temporal MT area) than when observing one's own previous trial which showed significant differential activity in the cerebellum (only for pilots) thought to be concerned with error-feedback based motor simulation. While there was some differential brain activity for pilots in regions involved with both Execution and Observation of the flying task (potential Mirror System sites including IFG, PMC, superior parietal lobule) the majority was adjacent to these areas (Observation Only Sites) (predominantly in PMC, IFG, and inferior parietal loblule). These regions showing greater activity for observation than for action may be involved with processes related to motor-based representational transforms that are not necessary when actually carrying out the task.

The role of brain-localized gamma and alpha oscillations in inattentional deafness: implications for understanding human attention.

Introduction: The processes involved in how the attention system selectively focuses on perceptual and motor aspects related to a specific task, while suppressing features of other tasks and/or objects in the environment, are of considerable interest for cognitive neuroscience. The goal of this experiment was to investigate neural processes involved in selective attention and performance under multi-task situations. Several studies have suggested that attention-related gamma-band activity facilitates processing in task-specific modalities, while alpha-band activity inhibits processing in non-task-related modalities. However, investigations into the phenomenon of inattentional deafness/blindness (inability to observe stimuli in non-dominant task when primary task is demanding) have yet to observe gamma-band activity. Methods: This EEG experiment utilizes an engaging whole-body perceptual motor task while carrying out a secondary auditory detection task to investigate neural correlates of inattentional deafness in natural immersive high workload conditions. Differences between hits and misses on the auditory detection task in the gamma (30-50 Hz) and alpha frequency (8-12 Hz) range were carried out at the cortical source level using LORETA. Results: Participant auditory task performance correlated with an increase in gamma-band activity for hits over misses pre- and post-stimulus in left auditory processing regions. Alpha-band activity was greater for misses relative to hits in right auditory processing regions pre- and post-stimulus onset. These results are consistent with the facilitatory/inhibitory role of gamma/alpha-band activity for neural processing. Additional gamma- and alpha-band activity was found in frontal and parietal brain regions which are thought to reflect various attentional monitoring, selection, and switching processes. Discussion: The results of this study help to elucidate the role of gamma and alpha frequency bands in frontal and modality-specific regions involved with selective attention in multi-task immersive situations.

Simultaneous fMRI and tDCS for Enhancing Training of Flight Tasks.

There is a gap in our understanding of how best to apply transcranial direct-current stimulation (tDCS) to enhance learning in complex, realistic, and multifocus tasks such as aviation. Our goal is to assess the effects of tDCS and feedback training on task performance, brain activity, and connectivity using functional magnetic resonance imaging (fMRI). Experienced glider pilots were recruited to perform a one-day, three-run flight-simulator task involving varying difficulty conditions and a secondary auditory task, mimicking real flight requirements. The stimulation group (versus sham) received 1.5 mA high-definition HD-tDCS to the right dorsolateral prefrontal cortex (DLPFC) for 30 min during the training. Whole-brain fMRI was collected before, during, and after stimulation. Active stimulation improved piloting performance both during and post-training, particularly in novice pilots. The fMRI revealed a number of tDCS-induced effects on brain activation, including an increase in the left cerebellum and bilateral basal ganglia for the most difficult conditions, an increase in DLPFC activation and connectivity to the cerebellum during stimulation, and an inhibition in the secondary task-related auditory cortex and Broca's area. Here, we show that stimulation increases activity and connectivity in flight-related brain areas, particularly in novices, and increases the brain's ability to focus on flying and ignore distractors. These findings can guide applied neurostimulation in real pilot training to enhance skill acquisition and can be applied widely in other complex perceptual-motor real-world tasks.

Neural correlates of the spacing effect in explicit verbal semantic encoding support the deficient-processing theory.

Spaced presentations of to-be-learned items during encoding leads to superior long-term retention over massed presentations. Despite over a century of research, the psychological and neural basis of this spacing effect however is still under investigation. To test the hypotheses that the spacing effect results either from reduction in encoding-related verbal maintenance rehearsal in massed relative to spaced presentations (deficient processing hypothesis) or from greater encoding-related elaborative rehearsal of relational information in spaced relative to massed presentations (encoding variability hypothesis), we designed a vocabulary learning experiment in which subjects encoded paired-associates, each composed of a known word paired with a novel word, in both spaced and massed conditions during functional magnetic resonance imaging. As expected, recall performance in delayed cued-recall tests was significantly better for spaced over massed conditions. Analysis of brain activity during encoding revealed that the left frontal operculum, known to be involved in encoding via verbal maintenance rehearsal, was associated with greater performance-related increased activity in the spaced relative to massed condition. Consistent with the deficient processing hypothesis, a significant decrease in activity with subsequent episodes of presentation was found in the frontal operculum for the massed but not the spaced condition. Our results suggest that the spacing effect is mediated by activity in the frontal operculum, presumably by encoding-related increased verbal maintenance rehearsal, which facilitates binding of phonological and word level verbal information for transfer into long-term memory.

Cerebellum, Basal Ganglia, and Cortex Mediate Performance of an Aerial Pursuit Task.

The affordance competition hypothesis is an ethologically inspired theory from cognitive neuroscience that provides an integrative neural account of continuous, real-time behavior, and will likely become increasingly relevant to the growing field of neuroergonomics. In the spirit of neuroergonomics in aviation, we designed a three-dimensional, first-person, continuous, and real-time fMRI task during which human subjects maneuvered a simulated airplane in pursuit of a target airplane along constantly changing headings. We introduce a pseudo-event-related, parametric fMRI analysis approach to begin testing the affordance competition hypothesis in neuroergonomic contexts, and attempt to identify regions of the brain that exhibit a linear metabolic relationship with the continuous variables of task performance and distance-from-target. In line with the affordance competition hypothesis, our results implicate the cooperation of the cerebellum, basal ganglia, and cortex in such a task, with greater involvement of the basal ganglia during good performance, and greater involvement of cortex and cerebellum during poor performance and when distance-from-target closes. We briefly review the somatic marker and dysmetria of thought hypotheses, in addition to the affordance competition hypothesis, to speculate on the intricacies of the cooperation of these brain regions in a task such as ours. In doing so, we demonstrate how the affordance competition hypothesis and other cognitive neuroscience theories are ready for testing in continuous, real-time tasks such as ours, and in other neuroergonomic settings more generally.

Neural correlates of resolving uncertainty in driver's decision making.

Neural correlates of driving and of decision making have been investigated separately, but little is known about the underlying neural mechanisms of decision making in driving. Previous research discusses two types of decision making: reward-weighted decision making and cost-weighted decision making. There are many reward-weighted decision making neuroimaging studies but there are few cost-weighted studies. Considering that driving involves serious risk, it is assumed that decision making in driving is cost weighted. Therefore, neural substrates of cost-weighted decision making can be assessed by investigation of driver's decision making. In this study, neural correlates of resolving uncertainty in driver's decision making were investigated. Turning right in left-hand traffic at a signalized intersection was simulated by computer graphic animation based videos. When the driver's view was occluded by a big truck, the uncertainty of the oncoming traffic was resolved by an in-car video assist system that presented the driver's occluded view. Resolving the uncertainty reduced activity in a distributed area including the amygdala and anterior cingulate. These results implicate the amygdala and anterior cingulate as serving a role in cost-weighted decision making.

Giving speech a hand: gesture modulates activity in auditory cortex during speech perception.

Viewing hand gestures during face-to-face communication affects speech perception and comprehension. Despite the visible role played by gesture in social interactions, relatively little is known about how the brain integrates hand gestures with co-occurring speech. Here we used functional magnetic resonance imaging (fMRI) and an ecologically valid paradigm to investigate how beat gesture-a fundamental type of hand gesture that marks speech prosody-might impact speech perception at the neural level. Subjects underwent fMRI while listening to spontaneously-produced speech accompanied by beat gesture, nonsense hand movement, or a still body; as additional control conditions, subjects also viewed beat gesture, nonsense hand movement, or a still body all presented without speech. Validating behavioral evidence that gesture affects speech perception, bilateral nonprimary auditory cortex showed greater activity when speech was accompanied by beat gesture than when speech was presented alone. Further, the left superior temporal gyrus/sulcus showed stronger activity when speech was accompanied by beat gesture than when speech was accompanied by nonsense hand movement. Finally, the right planum temporale was identified as a putative multisensory integration site for beat gesture and speech (i.e., here activity in response to speech accompanied by beat gesture was greater than the summed responses to speech alone and beat gesture alone), indicating that this area may be pivotally involved in synthesizing the rhythmic aspects of both speech and gesture. Taken together, these findings suggest a common neural substrate for processing speech and gesture, likely reflecting their joint communicative role in social interactions.

Music Improvisation Is Characterized by Increase EEG Spectral Power in Prefrontal and Perceptual Motor Cortical Sources and Can be Reliably Classified From Non-improvisatory Performance.

This study expores neural activity underlying creative processes through the investigation of music improvisation. Fourteen guitar players with a high level of improvisation skill participated in this experiment. The experimental task involved playing 32-s alternating blocks of improvisation and scales on guitar. electroencephalography (EEG) data was measured continuously throughout the experiment. In order to remove potential artifacts and extract brain-related activity the following signal processing techniques were employed: bandpass filtering, Artifact Subspace Reconstruction, and Independent Component Analysis (ICA). For each participant, artifact related independent components (ICs) were removed from the EEG data and only ICs found to be from brain activity were retained. Source localization using this brain-related activity was carried out using sLORETA. Greater activity for improvisation over scale was found in multiple frequency bands (theta, alpha, and beta) localized primarily in the medial frontal cortex (MFC), Middle frontal gyrus (MFG), anterior cingulate, polar medial prefrontal cortex (MPFC), premotor cortex (PMC), pre and postcentral gyrus (PreCG and PostCG), superior temporal gyrus (STG), inferior parietal lobule (IPL), and the temporal-parietal junction. Together this collection of brain regions suggests that improvisation was mediated by processes involved in coordinating planned sequences of movement that are modulated in response to ongoing environmental context through monitoring and feedback of sensory states in relation to internal plans and goals. Machine-learning using Common Spatial Patterns (CSP) for EEG feature extraction attained a mean of over 75% classification performance for improvisation vs. scale conditions across participants. These machine-learning results are a step towards the development of a brain-computer interface that could be used for neurofeedback training to improve creativity.

Attentional changes in pre-stimulus oscillatory activity within early visual cortex are predictive of human visual performance.

Physiological and neuroimaging studies provide evidence to suggest that attentional mechanisms operating within the fronto-parietal network may exert top-down control on early visual areas, priming them for forthcoming sensory events. The believed consequence of such priming is enhanced task performance. Using the technique of magnetoencephalography (MEG), we investigated this possibility by examining whether attention-driven changes in cortical activity are correlated with performance on a line-orientation judgment task. We observed that, approximately 200 ms after a covert attentional shift towards the impending visual stimulus, the level of phase-resetting (transient neural coherence) within the calcarine significantly increased for 2-10 Hz activity. This was followed by a suppression of alpha activity (near 10 Hz) which persisted until the onset of the stimulus. The levels of phase-resetting, alpha suppression and subsequent behavioral performance varied between subjects in a systematic fashion. The magnitudes of phase-resetting and alpha-band power were negatively correlated, with high levels of coherence associated with high levels of performance. We propose that top-down attentional control mechanisms exert their initial effects within the calcarine through a phase-resetting within the 2-10 Hz band, which in turn triggers a suppression of alpha activity, priming early visual areas for incoming information and enhancing behavioral performance.

Individual differences in learning correlate with modulation of brain activity induced by transcranial direct current stimulation.

Transcranial direct current stimulation (tDCS) has been shown to enhance cognitive performance on a variety of tasks. It is hypothesized that tDCS enhances performance by affecting task related cortical excitability changes in networks underlying or connected to the site of stimulation facilitating long term potentiation. However, many recent studies have called into question the reliability and efficacy of tDCS to induce modulatory changes in brain activity. In this study, our goal is to investigate the individual differences in tDCS induced modulatory effects on brain activity related to the degree of enhancement in performance, providing insight into this lack of reliability. In accomplishing this goal, we used functional magnetic resonance imaging (fMRI) concurrently with tDCS stimulation (1 mA, 30 minutes duration) using a visual search task simulating real world conditions. The experiment consisted of three fMRI sessions: pre-training (no performance feedback), training (performance feedback which included response accuracy and target location and either real tDCS or sham stimulation given), and post-training (no performance feedback). The right posterior parietal cortex was selected as the site of anodal tDCS based on its known role in visual search and spatial attention processing. Our results identified a region in the right precentral gyrus, known to be involved with visual spatial attention and orienting, that showed tDCS induced task related changes in cortical excitability that were associated with individual differences in improved performance. This same region showed greater activity during the training session for target feedback of incorrect (target-error feedback) over correct trials for the tDCS stim over sham group indicating greater attention to target features during training feedback when trials were incorrect. These results give important insight into the nature of neural excitability induced by tDCS as it relates to variability in individual differences in improved performance shedding some light the apparent lack of reliability found in tDCS research.

Speech and song: the role of the cerebellum.

An exploration into cerebellar activity during the perception and production of speech and song may elucidate general underlying cerebellar functions. Recently, the cerebellum has been hypothesized to be involved with sharpening sensory input, temporal coordination and processing of motor articulation and perception, as well as instantiation of internal models that simulate the input-output characteristics of a specific system. Sung language and spoken language share many common features (physiology for articulation and perception as well as phonology, phonotactics, syntax, and semantics of the underlying language), although they differ in certain vocal and prosodic aspects. A review of the literature on perception and production of singing and speech reveals considerable overlap in the lateral aspect of the VI lobule of the posterior cerebellum, a region known to somatotopically represent the lips and tongue. This region may instantiate internal models of vocal tract articulation that simulate well learned phonological and/or segmental articulatory - auditory/orosensory mappings utilized for both speech and singing. Recent results show tendencies for left cerebellar hemispheric specialization for processing of singing and right specialization for processing of speech, both in the VI lobule of the cerebellum, inferior to that found for representing both speech and singing. Given the crossed pattern of cerebellar-cortical anatomical connectivity the findings are consistent with the hypothesis that the right cerebellum differentially processes high pass filtered information (segmental properties) and the left cerebellum differentially processes low pass filtered information (prosodic, melodic properties). Further research is necessary to examine these hypotheses and their alternatives directly.

Neural processes involved with perception of non-native durational contrasts.

Neural processes underlying identification of durational contrasts were studied by comparing English and Japanese speakers for Japanese short/long vowel identification relative to consonant identification. Enhanced activities for non-native contrast (Japanese short/long vowel identification by English speakers) were observed in brain regions involved with articulatory-auditory mapping (Broca's area, superior temporal gyrus, planum temporale, and cerebellum), but not in the supramarginal gyrus. Greater activity in the supramarginal gyrus found for the consonant identification over short/long vowel identification by Japanese speakers implies that it is more important for phonetic contrasts differing in place of articulation than for vowel duration. These results support the hypothesis that neural processes used to facilitate perception depend on the relative contribution of information important for articulatory planning control.

Simultaneous tDCS-fMRI Identifies Resting State Networks Correlated with Visual Search Enhancement.

This study uses simultaneous transcranial direct current stimulation (tDCS) and functional MRI (fMRI) to investigate tDCS modulation of resting state activity and connectivity that underlies enhancement in behavioral performance. The experiment consisted of three sessions within the fMRI scanner in which participants conducted a visual search task: Session 1: Pre-training (no performance feedback), Session 2: Training (performance feedback given), Session 3: Post-training (no performance feedback). Resting state activity was recorded during the last 5 min of each session. During the 2nd session one group of participants underwent 1 mA tDCS stimulation and another underwent sham stimulation over the right posterior parietal cortex. Resting state spontaneous activity, as measured by fractional amplitude of low frequency fluctuations (fALFF), for session 2 showed significant differences between the tDCS stim and sham groups in the precuneus. Resting state functional connectivity from the precuneus to the substantia nigra, a subcortical dopaminergic region, was found to correlate with future improvement in visual search task performance for the stim over the sham group during active stimulation in session 2. The after-effect of stimulation on resting state functional connectivity was measured following a post-training experimental session (session 3). The left cerebellum Lobule VIIa Crus I showed performance related enhancement in resting state functional connectivity for the tDCS stim over the sham group. The ability to determine the relationship that the relative strength of resting state functional connectivity for an individual undergoing tDCS has on future enhancement in behavioral performance has wide ranging implications for neuroergonomic as well as therapeutic, and rehabilitative applications.

The Brain Is Faster than the Hand in Split-Second Intentions to Respond to an Impending Hazard: A Simulation of Neuroadaptive Automation to Speed Recovery to Perturbation in Flight Attitude.

The goal of this research is to test the potential for neuroadaptive automation to improve response speed to a hazardous event by using a brain-computer interface (BCI) to decode perceptual-motor intention. Seven participants underwent four experimental sessions while measuring brain activity with magnetoencephalograpy. The first three sessions were of a simple constrained task in which the participant was to pull back on the control stick to recover from a perturbation in attitude in one condition and to passively observe the perturbation in the other condition. The fourth session consisted of having to recover from a perturbation in attitude while piloting the plane through the Grand Canyon constantly maneuvering to track over the river below. Independent component analysis was used on the first two sessions to extract artifacts and find an event related component associated with the onset of the perturbation. These two sessions were used to train a decoder to classify trials in which the participant recovered from the perturbation (motor intention) vs. just passively viewing the perturbation. The BCI-decoder was tested on the third session of the same simple task and found to be able to significantly distinguish motor intention trials from passive viewing trials (mean = 69.8%). The same BCI-decoder was then used to test the fourth session on the complex task. The BCI-decoder significantly classified perturbation from no perturbation trials (73.3%) with a significant time savings of 72.3 ms (Original response time of 425.0-352.7 ms for BCI-decoder). The BCI-decoder model of the best subject was shown to generalize for both performance and time savings to the other subjects. The results of our off-line open loop simulation demonstrate that BCI based neuroadaptive automation has the potential to decode motor intention faster than manual control in response to a hazardous perturbation in flight attitude while ignoring ongoing motor and visual induced activity related to piloting the airplane.