A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages.

Advancements in decoding speech from brain activity have focused on decoding a single language. Hence, the extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear. Here, we leveraged electrocorticography, along with deep-learning and statistical natural-language models of English and Spanish, to record and decode activity from speech-motor cortex of a Spanish-English bilingual with vocal-tract and limb paralysis into sentences in either language. This was achieved without requiring the participant to manually specify the target language. Decoding models relied on shared vocal-tract articulatory representations across languages, which allowed us to build a syllable classifier that generalized across a shared set of English and Spanish syllables. Transfer learning expedited training of the bilingual decoder by enabling neural data recorded in one language to improve decoding in the other language. Overall, our findings suggest shared cortical articulatory representations that persist after paralysis and enable the decoding of multiple languages without the need to train separate language-specific decoders.

A high-performance neuroprosthesis for speech decoding and avatar control.

Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive1. Here we use high-density surface recordings of the speech cortex in a clinical-trial participant with severe limb and vocal paralysis to achieve high-performance real-time decoding across three complementary speech-related output modalities: text, speech audio and facial-avatar animation. We trained and evaluated deep-learning models using neural data collected as the participant attempted to silently speak sentences. For text, we demonstrate accurate and rapid large-vocabulary decoding with a median rate of 78 words per minute and median word error rate of 25%. For speech audio, we demonstrate intelligible and rapid speech synthesis and personalization to the participant's pre-injury voice. For facial-avatar animation, we demonstrate the control of virtual orofacial movements for speech and non-speech communicative gestures. The decoders reached high performance with less than two weeks of training. Our findings introduce a multimodal speech-neuroprosthetic approach that has substantial promise to restore full, embodied communication to people living with severe paralysis.

Associations of Lean Mass, Muscular Strength, and Physical Function with Trabecular Bone Score in Older Adults.

INTRODUCTION/BACKGROUND: Trabecular bone score (TBS) is an indirect measurement of bone quality and microarchitecture determined from dual-energy X-ray absorptiometry (DXA) imaging of the lumbar spine. TBS predicts fracture risk independent of bone mass/density, suggesting this assessment of bone quality adds value to the understanding of patients' bone health. While lean mass and muscular strength have been associated with higher bone density and lower fracture risk among older adults, the literature is limited regarding the relationship of lean mass and strength with TBS. The purpose of this study was to determine associations of DXA-determined total body and trunk lean mass, maximal muscular strength, and gait speed as a measure of physical function, with TBS in 141 older adults (65-84 yr, 72.5 +/- 5.1 yr, 74% women). METHODOLOGY: Assessments included lumbar spine (L1-L4) bone density and total body and trunk lean mass by DXA, lower body (leg press) and upper body (seated row) strength by one repetition maximum tests, hand grip strength, and usual gait speed. TBS was derived from the lumbar spine DXA scan. Multivariable linear regression determined the contribution of proposed predictors to TBS. RESULTS: After adjusting for age, sex, and lumbar spine bone density, upper body strength significantly predicted TBS (unadjusted/adjusted R2= 0.16/ 0.11, ß coefficient =0.378, p=0.005), while total body lean mass index showed a trend in the expected direction (ß coefficient =0.243, p=0.053). Gait speed and grip strength were not associated with TBS (p>0.05). CONCLUSION: Maximum strength of primarily back muscles measured as the seated row appears important to bone quality as measured by TBS, independent of bone density. Additional research on exercise training targeting back strength is needed to determine its clinical utility in preventing vertebral fractures among older adults.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis.

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding.