Differential Representation of Articulatory Gestures and Phonemes in Precentral and Inferior Frontal Gyri.

Speech is a critical form of human communication and is central to our daily lives. Yet, despite decades of study, an understanding of the fundamental neural control of speech production remains incomplete. Current theories model speech production as a hierarchy from sentences and phrases down to words, syllables, speech sounds (phonemes), and the actions of vocal tract articulators used to produce speech sounds (articulatory gestures). Here, we investigate the cortical representation of articulatory gestures and phonemes in ventral precentral and inferior frontal gyri in men and women. Our results indicate that ventral precentral cortex represents gestures to a greater extent than phonemes, while inferior frontal cortex represents both gestures and phonemes. These findings suggest that speech production shares a common cortical representation with that of other types of movement, such as arm and hand movements. This has important implications both for our understanding of speech production and for the design of brain-machine interfaces to restore communication to people who cannot speak.SIGNIFICANCE STATEMENT Despite being studied for decades, the production of speech by the brain is not fully understood. In particular, the most elemental parts of speech, speech sounds (phonemes) and the movements of vocal tract articulators used to produce these sounds (articulatory gestures), have both been hypothesized to be encoded in motor cortex. Using direct cortical recordings, we found evidence that primary motor and premotor cortices represent gestures to a greater extent than phonemes. Inferior frontal cortex (part of Broca's area) appears to represent both gestures and phonemes. These findings suggest that speech production shares a similar cortical organizational structure with the movement of other body parts.

High-Gamma Activity Is Coupled to Low-Gamma Oscillations in Precentral Cortices and Modulates with Movement and Speech.

Planning and executing motor behaviors requires coordinated neural activity among multiple cortical and subcortical regions of the brain. Phase-amplitude coupling between the high-gamma band amplitude and the phase of low frequency oscillations (theta, alpha, beta) has been proposed to reflect neural communication, as has synchronization of low-gamma oscillations. However, coupling between low-gamma and high-gamma bands has not been investigated. Here, we measured phase-amplitude coupling between low- and high-gamma in monkeys performing a reaching task and in humans either performing finger-flexion or word-reading tasks. We found significant coupling between low-gamma phase and high-gamma amplitude in multiple sensorimotor and premotor cortices of both species during all tasks. This coupling modulated with the onset of movement. These findings suggest that interactions between the low and high gamma bands are markers of network dynamics related to movement and speech generation.

High-gamma activity is coupled to low-gamma oscillations in precentral cortices and modulates with movement and speech.

Planning and executing motor behaviors requires coordinated neural activity among multiple cortical and subcortical regions of the brain. Phase-amplitude coupling between the high-gamma band amplitude and the phase of low frequency oscillations (theta, alpha, beta) has been proposed to reflect neural communication, as has synchronization of low-gamma oscillations. However, coupling between low-gamma and high-gamma bands has not been investigated. Here, we measured phase-amplitude coupling between low- and high-gamma in monkeys performing a reaching task and in humans either performing finger movements or speaking words aloud. We found significant coupling between low-gamma phase and high-gamma amplitude in multiple sensorimotor and premotor cortices of both species during all tasks. This coupling modulated with the onset of movement. These findings suggest that interactions between the low and high gamma bands are markers of network dynamics related to movement and speech generation.

Identifying Naturalistic Movies from Human Brain Activity with High-Density Diffuse Optical Tomography.

Modern neuroimaging modalities, particularly functional MRI (fMRI), can decode detailed human experiences. Thousands of viewed images can be identified or classified, and sentences can be reconstructed. Decoding paradigms often leverage encoding models that reduce the stimulus space into a smaller yet generalizable feature set. However, the neuroimaging devices used for detailed decoding are non-portable, like fMRI, or invasive, like electrocorticography, excluding application in naturalistic use. Wearable, non-invasive, but lower-resolution devices such as electroencephalography and functional near-infrared spectroscopy (fNIRS) have been limited to decoding between stimuli used during training. Herein we develop and evaluate model-based decoding with high-density diffuse optical tomography (HD-DOT), a higher-resolution expansion of fNIRS with demonstrated promise as a surrogate for fMRI. Using a motion energy model of visual content, we decoded the identities of novel movie clips outside the training set with accuracy far above chance for single-trial decoding. Decoding was robust to modulations of testing time window, different training and test imaging sessions, hemodynamic contrast, and optode array density. Our results suggest that HD-DOT can translate detailed decoding into naturalistic use.

Ultra-high density imaging arrays for diffuse optical tomography of human brain improve resolution, signal-to-noise, and information decoding.

Functional magnetic resonance imaging (fMRI) has dramatically advanced non-invasive human brain mapping and decoding. Functional near-infrared spectroscopy (fNIRS) and high-density diffuse optical tomography (HD-DOT) non-invasively measure blood oxygen fluctuations related to brain activity, like fMRI, at the brain surface, using more-lightweight equipment that circumvents ergonomic and logistical limitations of fMRI. HD-DOT grids have smaller inter-optode spacing (∼13 mm) than sparse fNIRS (∼30 mm) and therefore provide higher image quality, with spatial resolution ∼1/2 that of fMRI. Herein, simulations indicated reducing inter-optode spacing to 6.5 mm would further improve image quality and noise-resolution tradeoff, with diminishing returns below 6.5 mm. We then constructed an ultra-high-density DOT system (6.5-mm spacing) with 140 dB dynamic range that imaged stimulus-evoked activations with 30-50% higher spatial resolution and repeatable multi-focal activity with excellent agreement with participant-matched fMRI. Further, this system decoded visual stimulus position with 19-35% lower error than previous HD-DOT, throughout occipital cortex.

Scalable, modular continuous wave functional near-infrared spectroscopy system (Spotlight).

Significance: We present a fiberless, portable, and modular continuous wave-functional near-infrared spectroscopy system, Spotlight, consisting of multiple palm-sized modules-each containing high-density light-emitting diode and silicon photomultiplier detector arrays embedded in a flexible membrane that facilitates optode coupling to scalp curvature. Aim: Spotlight's goal is to be a more portable, accessible, and powerful functional near-infrared spectroscopy (fNIRS) device for neuroscience and brain-computer interface (BCI) applications. We hope that the Spotlight designs we share here can spur more advances in fNIRS technology and better enable future non-invasive neuroscience and BCI research. Approach: We report sensor characteristics in system validation on phantoms and motor cortical hemodynamic responses in a human finger-tapping experiment, where subjects wore custom 3D-printed caps with two sensor modules. Results: The task conditions can be decoded offline with a median accuracy of 69.6%, reaching 94.7% for the best subject, and at a comparable accuracy in real time for a subset of subjects. We quantified how well the custom caps fitted to each subject and observed that better fit leads to more observed task-dependent hemodynamic response and better decoding accuracy. Conclusions: The advances presented here should serve to make fNIRS more accessible for BCI applications.

Multivariate Kalman filter regression of confounding physiological signals for real-time classification of fNIRS data.

Significance: Functional near-infrared spectroscopy (fNIRS) is a noninvasive technique for measuring hemodynamic changes in the human cortex related to neural function. Due to its potential for miniaturization and relatively low cost, fNIRS has been proposed for applications, such as brain-computer interfaces (BCIs). The relatively large magnitude of the signals produced by the extracerebral physiology compared with the ones produced by evoked neural activity makes real-time fNIRS signal interpretation challenging. Regression techniques incorporating physiologically relevant auxiliary signals such as short separation channels are typically used to separate the cerebral hemodynamic response from the confounding components in the signal. However, the coupling of the extra-cerebral signals is often noninstantaneous, and it is necessary to find the proper delay to optimize nuisance removal. Aim: We propose an implementation of the Kalman filter with time-embedded canonical correlation analysis for the real-time regression of fNIRS signals with multivariate nuisance regressors that take multiple delays into consideration. Approach: We tested our proposed method on a previously acquired finger tapping dataset with the purpose of classifying the neural responses as left or right. Results: We demonstrate computationally efficient real-time processing of 24-channel fNIRS data (400 samples per second per channel) with a two order of selective magnitude decrease in cardiac signal power and up to sixfold increase in the contrast-to-noise ratio compared with the nonregressed signals. Conclusion: The method provides a way to obtain better distinction of brain from non-brain signals in real time for BCI application with fNIRS.

Myoelectric Computer Interface Training for Reducing Co-Activation and Enhancing Arm Movement in Chronic Stroke Survivors: A Randomized Trial.

BACKGROUND: Abnormal muscle co-activation contributes to impairment after stroke. We developed a myoelectric computer interface (MyoCI) training paradigm to reduce abnormal co-activation. MyoCI provides intuitive feedback about muscle activation patterns, enabling decoupling of these muscles. OBJECTIVE: To investigate tolerability and effects of MyoCI training of 3 muscle pairs on arm motor recovery after stroke, including effects of training dose and isometric versus movement-based training. METHODS: We randomized chronic stroke survivors with moderate-to-severe arm impairment to 3 groups. Two groups tested different doses of isometric MyoCI (60 vs 90 minutes), and one group tested MyoCI without arm restraint (90 minutes), over 6 weeks. Primary outcome was arm impairment (Fugl-Meyer Assessment). Secondary outcomes included function, spasticity, and elbow range-of-motion at weeks 6 and 10. RESULTS: Over all 32 subjects, MyoCI training of 3 muscle pairs significantly reduced impairment (Fugl-Meyer Assessment) by 3.3 ± 0.6 and 3.1 ± 0.7 ( P < 10-4) at weeks 6 and 10, respectively. Each group improved significantly from baseline; no significant differences were seen between groups. Participants' lab-based and home-based function also improved at weeks 6 and 10 ( P ≤ .01). Spasticity also decreased over all subjects, and elbow range-of-motion improved. Both moderately and severely impaired patients showed significant improvement. No participants had training-related adverse events. MyoCI reduced abnormal co-activation, which appeared to transfer to reaching in the movement group. CONCLUSIONS: MyoCI is a well-tolerated, novel rehabilitation tool that enables stroke survivors to reduce abnormal co-activation. It may reduce impairment and spasticity and improve arm function, even in severely impaired patients.

Hemicraniectomy in Traumatic Brain Injury: A Noninvasive Platform to Investigate High Gamma Activity for Brain Machine Interfaces.

Brain-machine interfaces (BMIs) translate brain signals into control signals for an external device, such as a computer cursor or robotic limb. These signals can be obtained either noninvasively or invasively. Invasive recordings, using electrocorticography (ECoG) or intracortical microelectrodes, provide higher bandwidth and more informative signals. Rehabilitative BMIs, which aim to drive plasticity in the brain to enhance recovery after brain injury, have almost exclusively used non-invasive recordings, such electroencephalography (EEG) or magnetoencephalography (MEG), which have limited bandwidth and information content. Invasive recordings provide more information and spatiotemporal resolution, but do incur risk, and thus are not usually investigated in people with stroke or traumatic brain injury (TBI). Here, in this paper, we describe a new BMI paradigm to investigate the use of higher frequency signals in brain-injured subjects without incurring significant risk. We recorded EEG in TBI subjects who required hemicraniectomies (removal of a part of the skull). EEG over the hemicraniectomy (hEEG) contained substantial information in the high gamma frequency range (65-115 Hz). Using this information, we decoded continuous finger flexion force with moderate to high accuracy (variance accounted for 0.06 to 0.52), which at best approaches that using epidural signals. These results indicate that people with hemicraniectomies can provide a useful resource for developing BMI therapies for the treatment of brain injury.

The effect of myoelectric computer interface training on arm kinematics and function after stroke.

Abnormal co-activation patterns of arm muscles is a substantial cause of impaired arm function after stroke. We designed a myoelectric computer interface (MCI) training paradigm to help stroke survivors reduce this abnormal coactivation. Here, we evaluated the effects of MCI training on function and arm kinematics in 32 chronic stroke survivors. We compared the effects of training duration and isometric vs. movement-based training conditions in 3 different groups. All groups reduced abnormal co-activation in targeted muscles, and showed reduced arm impairment after 6 weeks of training. They also showed improvements in arm kinematics as well as functional scores. Moreover, the gains persisted, though most were reduced, at one month after training stopped. These results suggest that MCI training holds promise to improve arm function after stroke.

Corneal Potential Maps Measured With Multi-Electrode Electroretinography in Rat Eyes With Experimental Lesions.

Purpose: Conventional full-field flash electroretinography (ERG) yields a single response waveform that can be useful in the early detection and diagnosis of many diseases affecting the retina. It is an objective measurement that probes the entire retina. However, localized areas of dysfunction have relatively small influence on ERG amplitudes compared to normal ranges. Here we evaluate the use of corneal potential maps obtained in response to full-field flash stimuli for sensitivity to local areas of retinal damage. Methods: A contact lens electrode array was used to record 25 ERG waveforms simultaneously following saturating full-field flash stimuli (multi-electrode electroretinography, meERG) in rats. Waveforms were evaluated for a-wave and b-wave amplitudes; these values were normalized and further evaluated for spatial differences across the corneal surface. Cluster analysis and a support vector machine approach were used to classify meERG responses from healthy eyes and eyes with central (photocoagulation) or peripheral (cryocoagulation) experimental lesions. Results: A normative normalized corneal potential map was obtained from healthy eyes (n = 26). Corneal potential maps from eyes with experimental lesions (n = 13) could be classified with sensitivity and specificity of approximately 80% based solely on the normalized spatial distribution of corneal potentials, that is, with no knowledge of absolute amplitudes. Conclusions: Corneal potential maps obtained in response to full-field flash stimuli are altered in eyes with scotomas in the central and far-peripheral retina. The meERG approach yields useful spatial information following a single brief flash, analogous to body-surface potential maps used to evaluate heart and brain.

Decoding of articulatory gestures during word production using speech motor and premotor cortical activity.

Brain-machine interfaces that directly translate attempted speech from the speech motor areas could change the lives of people with complete paralysis. However, it remains uncertain exactly how speech production is encoded in cortex. Improving this understanding could greatly improve brain-machine interface design. Specifically, it is not clear to what extent the different levels of speech production (phonemes, or speech sounds, and articulatory gestures, which describe the movements of the articulator muscles) are represented in the motor cortex. Using electrocorticographic (ECoG) electrodes on the cortical surface, we recorded neural activity from speech motor and premotor areas during speech production. We decoded both gestures and phonemes using the neural signals. Overall classification accuracy was higher for gestures than phonemes. In particular, gestures were better represented in the primary sensorimotor cortices, while phonemes were better represented in more anterior areas.

Cortical encoding of phonemic context during word production.

Brain-computer interfaces that directly decode speech could restore communication to locked-in individuals. However, decoding speech from brain signals still faces many challenges. We investigated decoding of phonemes - the smallest separable parts of speech - from ECoG signals during word production. We expanded on previous efforts to identify specific phoneme by identifying phonemes by where in the word they were formed. We evaluated how the context of phonemes in words affects classification results using linear discriminant analysis. The decoding accuracy of our linear classifier indicated the degree to which the context of a phoneme can be determined from the cortical signal significantly greater than chance. Further, we identified the spectrotemporal features that contributed most to successful decoding of phonemic classes. Finally, we discuss how this can augment speech decoding for neural interfaces.

Direct classification of all American English phonemes using signals from functional speech motor cortex.

OBJECTIVE: Although brain-computer interfaces (BCIs) can be used in several different ways to restore communication, communicative BCI has not approached the rate or efficiency of natural human speech. Electrocorticography (ECoG) has precise spatiotemporal resolution that enables recording of brain activity distributed over a wide area of cortex, such as during speech production. In this study, we sought to decode elements of speech production using ECoG. APPROACH: We investigated words that contain the entire set of phonemes in the general American accent using ECoG with four subjects. Using a linear classifier, we evaluated the degree to which individual phonemes within each word could be correctly identified from cortical signal. MAIN RESULTS: We classified phonemes with up to 36% accuracy when classifying all phonemes and up to 63% accuracy for a single phoneme. Further, misclassified phonemes follow articulation organization described in phonology literature, aiding classification of whole words. Precise temporal alignment to phoneme onset was crucial for classification success. SIGNIFICANCE: We identified specific spatiotemporal features that aid classification, which could guide future applications. Word identification was equivalent to information transfer rates as high as 3.0 bits s(-1) (33.6 words min(-1)), supporting pursuit of speech articulation for BCI control.

Design and implementation of a P300-based brain-computer interface for controlling an internet browser.

An electroencephalographic (EEG) brain-computer interface (BCI) internet browser was designed and evaluated with 10 healthy volunteers and three individuals with advanced amyotrophic lateral sclerosis (ALS), all of whom were given tasks to execute on the internet using the browser. Participants with ALS achieved an average accuracy of 73% and a subsequent information transfer rate (ITR) of 8.6 bits/min and healthy participants with no prior BCI experience over 90% accuracy and an ITR of 14.4 bits/min. We define additional criteria for unrestricted internet access for evaluation of the presented and future internet browsers, and we provide a review of the existing browsers in the literature. The P300-based browser provides unrestricted access and enables free web surfing for individuals with paralysis.