The speech neuroprosthesis.

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore natural communication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinical epilepsy monitoring and subsequently in individuals with paralysis as part of early feasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapid and substantial progress in decoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

A high-performance neuroprosthesis for speech decoding and avatar control.

Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive1. Here we use high-density surface recordings of the speech cortex in a clinical-trial participant with severe limb and vocal paralysis to achieve high-performance real-time decoding across three complementary speech-related output modalities: text, speech audio and facial-avatar animation. We trained and evaluated deep-learning models using neural data collected as the participant attempted to silently speak sentences. For text, we demonstrate accurate and rapid large-vocabulary decoding with a median rate of 78 words per minute and median word error rate of 25%. For speech audio, we demonstrate intelligible and rapid speech synthesis and personalization to the participant's pre-injury voice. For facial-avatar animation, we demonstrate the control of virtual orofacial movements for speech and non-speech communicative gestures. The decoders reached high performance with less than two weeks of training. Our findings introduce a multimodal speech-neuroprosthetic approach that has substantial promise to restore full, embodied communication to people living with severe paralysis.

Intrinsically disordered regions are poised to act as sensors of cellular chemistry.

Intrinsically disordered proteins and protein regions (IDRs) are abundant in eukaryotic proteomes and play a wide variety of essential roles. Instead of folding into a stable structure, IDRs exist in an ensemble of interconverting conformations whose structure is biased by sequence-dependent interactions. The absence of a stable 3D structure, combined with high solvent accessibility, means that IDR conformational biases are inherently sensitive to changes in their environment. Here, we argue that IDRs are ideally poised to act as sensors and actuators of cellular physicochemistry. We review the physical principles that underlie IDR sensitivity, the molecular mechanisms that translate this sensitivity to function, and recent studies where environmental sensing by IDRs may play a key role in their downstream function.

Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria.

BACKGROUND: Technology to restore the ability to communicate in paralyzed persons who cannot speak has the potential to improve autonomy and quality of life. An approach that decodes words and sentences directly from the cerebral cortical activity of such patients may represent an advancement over existing methods for assisted communication. METHODS: We implanted a subdural, high-density, multielectrode array over the area of the sensorimotor cortex that controls speech in a person with anarthria (the loss of the ability to articulate speech) and spastic quadriparesis caused by a brain-stem stroke. Over the course of 48 sessions, we recorded 22 hours of cortical activity while the participant attempted to say individual words from a vocabulary set of 50 words. We used deep-learning algorithms to create computational models for the detection and classification of words from patterns in the recorded cortical activity. We applied these computational models, as well as a natural-language model that yielded next-word probabilities given the preceding words in a sequence, to decode full sentences as the participant attempted to say them. RESULTS: We decoded sentences from the participant's cortical activity in real time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%. In post hoc analyses, we detected 98% of the attempts by the participant to produce individual words, and we classified words with 47.1% accuracy using cortical signals that were stable throughout the 81-week study period. CONCLUSIONS: In a person with anarthria and spastic quadriparesis caused by a brain-stem stroke, words and sentences were decoded directly from cortical activity during attempted speech with the use of deep-learning models and a natural-language model. (Funded by Facebook and others; ClinicalTrials.gov number, NCT03698149.).

Use of psychoactive substances among patients presenting at the emergency department of a tertiary hospital.

BACKGROUND: Psychoactive substance use is frequently encountered in hospitals' emergency departments (EDs). It accounts for major health-care problems frequently leading to accident and ED admissions, yet it is frequently unidentified. The aim of this study was to determine the prevalence and pattern of psychoactive substance use among patients presenting in the Accident and EDs and to compare the case detection rate of psychoactive substance use between self-report questionnaire and biochemical markers (e.g., urine toxicology). METHODS: To achieve this, 200 consenting participants attending the accident and emergency unit of a tertiary hospital were consecutively enlisted into the study within 2 weeks. They were screened for psychoactive substance use with the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) and the urine drug test (UDT). RESULTS: The lifetime prevalence of psychoactive substance use was 45.5%, while the past 3 months (recent use) prevalence was 27.0%. The pattern of psychoactive substance use revealed that alcohol was the predominant psychoactive substance use with a lifetime prevalence of 13.0% and recent use of 12.0%. The UDT significantly detected more patients who used psychoactive substance compared to self-report (P < 0.001). CONCLUSION: The prevalence of drug use recorded among attendees of the accident and emergency unit was high in this study. The UDT significantly detected more patients who used psychoactive substances compared to self-report (P < 0.001). Several patients with major health problems as a result of psychoactive substance use were identified with the aid of these screening tools.

Teratogenic Effects of Topiramate in a Zebrafish Model.

Topiramate is commonly used for treating epilepsy in both children and adults. Recent clinical data suggests that administration of topiramate to women during pregnancy increases the risk of oral clefts in their offspring. To better understand the potential effects of topiramate, we dosed adult female zebrafish with topiramate, and investigated the altered morphologies in adult females and their offspring. It showed that topiramate-treated female fish had reduced oocyte maturation, and the survival rates of their offspring were seriously decreased during embryogenesis. In addition, around 23% of offspring displayed cartilage malformation in the craniofacial area, such as loss of ceratobranchial cartilages as well as impaired ceratohyal, Meckel's cartilage and ethmoid plate development. Moreover, mineralization of ceratohyal, Meckel's cartilage, and vertebrae were downregulated during bone development. Taken together, we concluded that topiramate impaired oogenesis in the maternal reproductive system, and then caused offspring cartilage malformation or bone dysplasia.

A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages.

Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish-English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.

Structural biases in disordered proteins are prevalent in the cell.

Intrinsically disordered proteins and protein regions (IDPs) are prevalent in all proteomes and are essential to cellular function. Unlike folded proteins, IDPs exist in an ensemble of dissimilar conformations. Despite this structural plasticity, intramolecular interactions create sequence-specific structural biases that determine an IDP ensemble's three-dimensional shape. Such structural biases can be key to IDP function and are often measured in vitro, but whether those biases are preserved inside the cell is unclear. Here we show that structural biases in IDP ensembles found in vitro are recapitulated inside human-derived cells. We further reveal that structural biases can change in a sequence-dependent manner due to changes in the intracellular milieu, subcellular localization, and intramolecular interactions with tethered well-folded domains. We propose that the structural sensitivity of IDP ensembles can be leveraged for biological function, can be the underlying cause of IDP-driven pathology or can be used to design disorder-based biosensors and actuators.

Recommendations for promoting user agency in the design of speech neuroprostheses.

Brain-computer interfaces (BCI) that directly decode speech from brain activity aim to restore communication in people with paralysis who cannot speak. Despite recent advances, neural inference of speech remains imperfect, limiting the ability for speech BCIs to enable experiences such as fluent conversation that promote agency - that is, the ability for users to author and transmit messages enacting their intentions. Here, we make recommendations for promoting agency based on existing and emerging strategies in neural engineering. The focus is on achieving fast, accurate, and reliable performance while ensuring volitional control over when a decoder is engaged, what exactly is decoded, and how messages are expressed. Additionally, alongside neuroscientific progress within controlled experimental settings, we argue that a parallel line of research must consider how to translate experimental successes into real-world environments. While such research will ultimately require input from prospective users, here we identify and describe design choices inspired by human-factors work conducted in existing fields of assistive technology, which address practical issues likely to emerge in future real-world speech BCI applications.

Electrochemical activation and inhibition of neuromuscular systems through modulation of ion concentrations with ion-selective membranes.

Conventional functional electrical stimulation aims to restore functional motor activity of patients with disabilities resulting from spinal cord injury or neurological disorders. However, intervention with functional electrical stimulation in neurological diseases lacks an effective implantable method that suppresses unwanted nerve signals. We have developed an electrochemical method to activate and inhibit a nerve by electrically modulating ion concentrations in situ along the nerve. Using ion-selective membranes to achieve different excitability states of the nerve, we observe either a reduction of the electrical threshold for stimulation by up to approximately 40%, or voluntary, reversible inhibition of nerve signal propagation. This low-threshold electrochemical stimulation method is applicable in current implantable neuroprosthetic devices, whereas the on-demand nerve-blocking mechanism could offer effective clinical intervention in disease states caused by uncontrolled nerve activation, such as epilepsy and chronic pain syndromes.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis.

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.

Harnessing the Power of Artificial Intelligence in Otolaryngology and the Communication Sciences.

Use of artificial intelligence (AI) is a burgeoning field in otolaryngology and the communication sciences. A virtual symposium on the topic was convened from Duke University on October 26, 2020, and was attended by more than 170 participants worldwide. This review presents summaries of all but one of the talks presented during the symposium; recordings of all the talks, along with the discussions for the talks, are available at https://www.youtube.com/watch?v=ktfewrXvEFg and https://www.youtube.com/watch?v=-gQ5qX2v3rg . Each of the summaries is about 2500 words in length and each summary includes two figures. This level of detail far exceeds the brief summaries presented in traditional reviews and thus provides a more-informed glimpse into the power and diversity of current AI applications in otolaryngology and the communication sciences and how to harness that power for future applications.

Intrinsically disordered protein biosensor tracks the physical-chemical effects of osmotic stress on cells.

Cell homeostasis is perturbed when dramatic shifts in the external environment cause the physical-chemical properties inside the cell to change. Experimental approaches for dynamically monitoring these intracellular effects are currently lacking. Here, we leverage the environmental sensitivity and structural plasticity of intrinsically disordered protein regions (IDRs) to develop a FRET biosensor capable of monitoring rapid intracellular changes caused by osmotic stress. The biosensor, named SED1, utilizes the Arabidopsis intrinsically disordered AtLEA4-5 protein expressed in plants under water deficit. Computational modeling and in vitro studies reveal that SED1 is highly sensitive to macromolecular crowding. SED1 exhibits large and near-linear osmolarity-dependent changes in FRET inside living bacteria, yeast, plant, and human cells, demonstrating the broad utility of this tool for studying water-associated stress. This study demonstrates the remarkable ability of IDRs to sense the cellular environment across the tree of life and provides a blueprint for their use as environmentally-responsive molecular tools.

Machine translation of cortical activity to text with an encoder-decoder framework.

A decade after speech was first decoded from human brain signals, accuracy and speed remain far below that of natural speech. Here we show how to decode the electrocorticogram with high accuracy and at natural-speech rates. Taking a cue from recent advances in machine translation, we train a recurrent neural network to encode each sentence-length sequence of neural activity into an abstract representation, and then to decode this representation, word by word, into an English sentence. For each participant, data consist of several spoken repeats of a set of 30-50 sentences, along with the contemporaneous signals from ~250 electrodes distributed over peri-Sylvian cortices. Average word error rates across a held-out repeat set are as low as 3%. Finally, we show how decoding with limited data can be improved with transfer learning, by training certain layers of the network under multiple participants' data.

Revealing the Hidden Sensitivity of Intrinsically Disordered Proteins to their Chemical Environment.

Intrinsically disordered protein-regions (IDRs) make up roughly 30% of the human proteome and are central to a wide range of biological processes. Given a lack of persistent tertiary structure, all residues in IDRs are, to some extent, solvent exposed. This extensive surface area, coupled with the absence of strong intramolecular contacts, makes IDRs inherently sensitive to their chemical environment. We report a combined experimental, computational, and analytical framework for high-throughput characterization of IDR sensitivity. Our framework reveals that IDRs can expand or compact in response to changes in their solution environment. Importantly, the direction and magnitude of conformational change depend on both protein sequence and cosolute identity. For example, some solutes such as short polyethylene glycol chains exert an expanding effect on some IDRs and a compacting effect on others. Despite this complex behavior, we can rationally interpret IDR responsiveness to solution composition changes using relatively simple polymer models. Our results imply that solution-responsive IDRs are ubiquitous and can provide an additional layer of regulation to biological systems.

Real-time decoding of question-and-answer speech dialogue using human cortical activity.

Natural communication often occurs in dialogue, differentially engaging auditory and sensorimotor brain regions during listening and speaking. However, previous attempts to decode speech directly from the human brain typically consider listening or speaking tasks in isolation. Here, human participants listened to questions and responded aloud with answers while we used high-density electrocorticography (ECoG) recordings to detect when they heard or said an utterance and to then decode the utterance's identity. Because certain answers were only plausible responses to certain questions, we could dynamically update the prior probabilities of each answer using the decoded question likelihoods as context. We decode produced and perceived utterances with accuracy rates as high as 61% and 76%, respectively (chance is 7% and 20%). Contextual integration of decoded question likelihoods significantly improves answer decoding. These results demonstrate real-time decoding of speech in an interactive, conversational setting, which has important implications for patients who are unable to communicate.

Real-time classification of auditory sentences using evoked cortical activity in humans.

OBJECTIVE: Recent research has characterized the anatomical and functional basis of speech perception in the human auditory cortex. These advances have made it possible to decode speech information from activity in brain regions like the superior temporal gyrus, but no published work has demonstrated this ability in real-time, which is necessary for neuroprosthetic brain-computer interfaces. APPROACH: Here, we introduce a real-time neural speech recognition (rtNSR) software package, which was used to classify spoken input from high-resolution electrocorticography signals in real-time. We tested the system with two human subjects implanted with electrode arrays over the lateral brain surface. Subjects listened to multiple repetitions of ten sentences, and rtNSR classified what was heard in real-time from neural activity patterns using direct sentence-level and HMM-based phoneme-level classification schemes. MAIN RESULTS: We observed single-trial sentence classification accuracies of [Formula: see text] or higher for each subject with less than 7 minutes of training data, demonstrating the ability of rtNSR to use cortical recordings to perform accurate real-time speech decoding in a limited vocabulary setting. SIGNIFICANCE: Further development and testing of the package with different speech paradigms could influence the design of future speech neuroprosthetic applications.

Porcine epidemic diarrhea virus papain-like protease 2 can be noncompetitively inhibited by 6-thioguanine.

Porcine epidemic diarrhea virus (PEDV) is a coronavirus (CoV) discovered in the 1970s that infects the intestinal tract of pigs, resulting in diarrhea and vomiting. It can cause extreme dehydration and death in neonatal piglets. In Asia, modified live attenuated vaccines have been used to control PEDV infection in recent years. However, a new strain of PEDV that belongs to genogroup 2a appeared in the USA in 2013 and then quickly spread to Canada and Mexico as well as Asian and European countries. Due to the less effective protective immunity provided by the vaccines against this new strain, it has caused considerable agricultural and economic loss worldwide. The emergence of this new strain increases the importance of understanding PEDV as well as strategies for inhibiting it. Coronaviral proteases, including main proteases and papain-like proteases, are ideal antiviral targets because of their essential roles in viral maturation. Here we provide a first description of the expression, purification and structural characteristics of recombinant PEDV papain-like protease 2, moreover present our finding that 6-thioguanine, a chemotherapeutic drug, in contrast to its competitive inhibition on SARS- and MERS-CoV papain-like proteases, is a noncompetitive inhibitor of PEDV papain-like protease 2.