The speech neuroprosthesis.

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

A high-performance neuroprosthesis for speech decoding and avatar control.

Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive1. Here we use high-density surface recordings of the speech cortex in a clinical-trial participant with severe limb and vocal paralysis to achieve high-performance real-time decoding across three complementary speech-related output modalities: text, speech audio and facial-avatar animation. We trained and evaluated deep-learning models using neural data collected as the participant attempted to silently speak sentences. For text, we demonstrate accurate and rapid large-vocabulary decoding with a median rate of 78 words per minute and median word error rate of 25%. For speech audio, we demonstrate intelligible and rapid speech synthesis and personalization to the participant's pre-injury voice. For facial-avatar animation, we demonstrate the control of virtual orofacial movements for speech and non-speech communicative gestures. The decoders reached high performance with less than two weeks of training. Our findings introduce a multimodal speech-neuroprosthetic approach that has substantial promise to restore full, embodied communication to people living with severe paralysis.

Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria.

BACKGROUND: Technology to restore the ability to communicate in paralyzed persons who cannot speak has the potential to improve autonomy and quality of life. An approach that decodes words and sentences directly from the cerebral cortical activity of such patients may represent an advancement over existing methods for assisted communication. METHODS: We implanted a subdural, high-density, multielectrode array over the area of the sensorimotor cortex that controls speech in a person with anarthria (the loss of the ability to articulate speech) and spastic quadriparesis caused by a brain-stem stroke. Over the course of 48 sessions, we recorded 22 hours of cortical activity while the participant attempted to say individual words from a vocabulary set of 50 words. We used deep-learning algorithms to create computational models for the detection and classification of words from patterns in the recorded cortical activity. We applied these computational models, as well as a natural-language model that yielded next-word probabilities given the preceding words in a sequence, to decode full sentences as the participant attempted to say them. RESULTS: We decoded sentences from the participant's cortical activity in real time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%. In post hoc analyses, we detected 98% of the attempts by the participant to produce individual words, and we classified words with 47.1% accuracy using cortical signals that were stable throughout the 81-week study period. CONCLUSIONS: In a person with anarthria and spastic quadriparesis caused by a brain-stem stroke, words and sentences were decoded directly from cortical activity during attempted speech with the use of deep-learning models and a natural-language model. (Funded by Facebook and others; ClinicalTrials.gov number, NCT03698149.).

A Neurosurgical Functional Dissection of the Middle Precentral Gyrus during Speech Production.

Classical models have traditionally focused on the left posterior inferior frontal gyrus (Broca's area) as a key region for motor planning of speech production. However, converging evidence suggests that it is not critical for either speech motor planning or execution. Alternative cortical areas supporting high-level speech motor planning have yet to be defined. In this review, we focus on the precentral gyrus, whose role in speech production is often thought to be limited to lower-level articulatory muscle control. In particular, we highlight neurosurgical investigations that have shed light on a cortical region anatomically located near the midpoint of the precentral gyrus, hence called the middle precentral gyrus (midPrCG). The midPrCG is functionally located between dorsal hand and ventral orofacial cortical representations and exhibits unique sensorimotor and multisensory functions relevant for speech processing. This includes motor control of the larynx, auditory processing, as well as a role in reading and writing. Furthermore, direct electrical stimulation of midPrCG can evoke complex movements, such as vocalization, and selective injury can cause deficits in verbal fluency, such as pure apraxia of speech. Based on these findings, we propose that midPrCG is essential to phonological-motoric aspects of speech production, especially syllabic-level speech sequencing, a role traditionally ascribed to Broca's area. The midPrCG is a cortical brain area that should be included in contemporary models of speech production with a unique role in speech motor planning and execution.

Detection of aberrant hippocampal mossy fiber connections: Ex vivo mesoscale diffusion MRI and microtractography with histological validation in a patient with uncontrolled temporal lobe epilepsy.

Understanding the neurobiology and functional connectivity of hippocampal structures is essential for improving the treatment of mesial temporal lobe epilepsy. At the macroscale, in vivo MRI often reveals hippocampal atrophy and decreased fractional anisotropy, whereas at the microscopic scale, there frequently is evidence of neuronal loss and gliosis. Mossy fiber sprouting in the dentate gyrus (DG), with evidence of glutamatergic synapses in the stratum moleculare (SM) putatively originating from granule cell neurons, may also be observed. This aberrant connection between the DG and SM could produce a reverberant excitatory circuit. However, this hypothesis cannot easily be evaluated using macroscopic or microscopic techniques. We here demonstrate that the ex vivo mesoscopic MRI of surgically excised hippocampi can bridge the explanatory and analytical gap between the macro- and microscopic scale. Specifically, diffusion- and T2 -weighted MRI can be integrated to visualize a cytoarchitecture that is akin to immunohistochemistry. An appropriate spatial resolution to discern individual cell layers can then be established. Processing of diffusion tensor images using tractography detects extra- and intrahippocampal connections, hence providing a unique systems view of the hippocampus and its connected regions. Here, this approach suggests that there is indeed an aberrant connection between the DG and SM, supporting the sprouting hypothesis of a reverberant excitatory network. Mesoscopic ex vivo MR imaging hence provides an exciting new avenue to study hippocampi from treatment-resistant patients and allows exploration of existing hypotheses, as well as the development of new treatment strategies based on these novel insights. Hum Brain Mapp 37:780-795, 2016. © 2015 Wiley Periodicals, Inc.

A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages.

Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish-English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.

Apraxia of speech with phonological alexia and agraphia following resection of the left middle precentral gyrus: illustrative case.

BACKGROUND: Apraxia of speech is a disorder of speech-motor planning in which articulation is effortful and error-prone despite normal strength of the articulators. Phonological alexia and agraphia are disorders of reading and writing disproportionately affecting unfamiliar words. These disorders are almost always accompanied by aphasia. OBSERVATIONS: A 36-year-old woman underwent resection of a grade IV astrocytoma based in the left middle precentral gyrus, including a cortical site associated with speech arrest during electrocortical stimulation mapping. Following surgery, she exhibited moderate apraxia of speech and difficulty with reading and spelling, both of which improved but persisted 6 months after surgery. A battery of speech and language assessments was administered, revealing preserved comprehension, naming, cognition, and orofacial praxis, with largely isolated deficits in speech-motor planning and the spelling and reading of nonwords. LESSONS: This case describes a specific constellation of speech-motor and written language symptoms-apraxia of speech, phonological agraphia, and phonological alexia in the absence of aphasia-which the authors theorize may be attributable to disruption of a single process of "motor-phonological sequencing." The middle precentral gyrus may play an important role in the planning of motorically complex phonological sequences for production, independent of output modality.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis.

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.

Quantification of the Extracellular Matrix Molecule Thrombospondin 1 and Its Pericellular Association in the Brain Using a Semiautomated Computerized Approach.

The structure and functions of the extracellular matrix (ECM), its spatial distribution and pericellular association of ECM molecules remain poorly understood. Colocalization of ECM molecules with cell phenotypes through immunohistochemistry can provide crucial insights into their juxtacrine signaling role as well as their structural relevance to tissue architecture. As manual quantification of images introduces intra- and inter-user bias and is cumbersome for high-throughput approaches, we implemented an automated high-throughput method to quantify the spatial distribution and cellular association of one ECM molecule, thrombospondin 1 (TSP1) with two major cell phenotypes, neurons, and astrocytes. The distribution of TSP1 was homogeneous throughout the striatum and cortex along the anterior-posterior axis. TSP1 occupied 8.85% of the striatum and 7.40% in the cortex. TSP1 also associated with 94.58% and 88.45% of neurons in the striatum and cortex. The association with astrocytes was significantly lower at 47.55% and 28.09%. These findings highlight the key role that TSP1 plays in neuron physiology in a healthy brain, but also highlights key regional difference in astrocytes secreting ECM molecules. The semiautomated approach implemented here will improve the throughput and reliability of measuring the distribution and cellular colocalization of ECM molecules.

Ex vivo biomechanical characterization of syringe-needle ejections for intracerebral cell delivery.

Intracerebral implantation of cell suspensions is finding its clinical translation with encouraging results in patients with stroke. However, the survival of cells in the brain remains poor. Although the biological potential of neural stem cells (NSCs) is widely documented, the biomechanical effects of delivering cells through a syringe-needle remain poorly understood. We here detailed the biomechanical forces (pressure, shear stress) that cells are exposed to during ejection through different sized needles (20G, 26G, 32G) and syringes (10, 50, 250 µL) at relevant flow rates (1, 5, 10 µL/min). A comparison of 3 vehicles, Phosphate Buffered Saline (PBS), Hypothermosol (HTS), and Pluronic, indicated that less viscous vehicles are favorable for suspension with a high cell volume fraction to minimize sedimentation. Higher suspension viscosity was associated with greater shear stress. Higher flow rates with viscous vehicle, such as HTS reduced viability by ~10% and also produced more apoptotic cells (28%). At 5 µL/min ejection using a 26G needle increased neuronal differentiation for PBS and HTS suspensions. These results reveal the biological impact of biomechanical forces in the cell delivery process. Appropriate engineering strategies can be considered to mitigate these effects to ensure the efficacious translation of this promising therapy.

A Comparison of Exogenous Labels for the Histological Identification of Transplanted Neural Stem Cells.

The interpretation of cell transplantation experiments is often dependent on the presence of an exogenous label for the identification of implanted cells. The exogenous labels Hoechst 33342, 5-bromo-2'-deoxyuridine (BrdU), PKH26, and Qtracker were compared for their labeling efficiency, cellular effects, and reliability to identify a human neural stem cell (hNSC) line implanted intracerebrally into the rat brain. Hoechst 33342 (2 mg/ml) exhibited a delayed cytotoxicity that killed all cells within 7 days. This label was hence not progressed to in vivo studies. PKH26 (5 µM), Qtracker (15 nM), and BrdU (0.2 µM) labeled 100% of the cell population at day 1, although BrdU labeling declined by day 7. BrdU and Qtracker exerted effects on proliferation and differentiation. PKH26 reduced viability and proliferation at day 1, but this normalized by day 7. In an in vitro coculture assay, all labels transferred to unlabeled cells. After transplantation, the reliability of exogenous labels was assessed against the gold standard of a human-specific nuclear antigen (HNA) antibody. BrdU, PKH26, and Qtracker resulted in a very small proportion (<2%) of false positives, but a significant amount of false negatives (∼30%), with little change between 1 and 7 days. Exogenous labels can therefore be reliable to identify transplanted cells without exerting major cellular effects, but validation is required. The interpretation of cell transplantation experiments should be presented in the context of the label's limitations.

Long-term retention of ECM hydrogel after implantation into a sub-acute stroke cavity reduces lesion volume.

Salvaging or functional replacement of damaged tissue caused by stroke in the brain remains a major therapeutic challenge. In situ gelation and retention of a hydrogel bioscaffold composed of 8mg/mL extracellular matrix (ECM) can induce a robust invasion of cells within 24h and potentially promote a structural remodeling to replace lost tissue. Herein, we demonstrate a long-term retention of ECM hydrogel within the lesion cavity. A decrease of approximately 32% of ECM volume is observed over 12weeks. Lesion volume, as measured by magnetic resonance imaging and histology, was reduced by 28%, but a battery of behavioral tests (bilateral asymmetry test; footfault; rotameter) did not reveal a therapeutic or detrimental effect of the hydrogel. Glial scarring and peri-infarct astrocytosis were equivalent between untreated and treated animals, potentially indicating that permeation into host tissue is required to exert therapeutic effects. These results reveal a marked difference of biodegradation of ECM hydrogel in the stroke-damaged brain compared to peripheral soft tissue repair. Further exploration of these structure-function relationships is required to achieve a structural remodeling of the implanted hydrogel, as seen in peripheral tissues, to replace lost tissue and promote behavioral recovery. STATEMENT OF SIGNIFICANCE: In situ gelation of ECM is essential for its retention within a tissue cavity. The brain is a unique environment with restricted access that necessitates image-guided delivery through a thin needle to access tissue cavities caused by stroke, as well as other conditions, such as traumatic brain injury or glioma resection. Knowledge about a brain tissue response to implanted hydrogels remains limited, especially in terms of long-term effects and potential impact on behavioral function. We here address the long-term retention of hydrogel within the brain environment, its impact on behavioral function, as well as its ability to reduce further tissue deformation caused by stroke. This study highlights considerable differences in the brain's long-term response to an ECM hydrogel compared to peripheral soft tissue. It underlines the importance of understanding the effect of the structural presence of a hydrogel within a cavity upon host brain tissue and behavioral function. As demonstrated herein, ECM hydrogel can fill a cavity long-term to reduce further progression of the cavity, while potentially serving as a reservoir for local drug or cell delivery.