Automatic Speech Recognition of Conversational Speech in Individuals With Disordered Speech.

PURPOSE: This study examines the effectiveness of automatic speech recognition (ASR) for individuals with speech disorders, addressing the gap in performance between read and conversational ASR. We analyze the factors influencing this disparity and the effect of speech mode-specific training on ASR accuracy. METHOD: Recordings of read and conversational speech from 27 individuals with various speech disorders were analyzed using both (a) one speaker-independent ASR system trained and optimized for typical speech and (b) multiple ASR models that were personalized to the speech of the participants with disordered speech. Word error rates were calculated for each speech model, read versus conversational, and subject. Linear mixed-effects models were used to assess the impact of speech mode and disorder severity on ASR accuracy. We investigated nine variables, classified as technical, linguistic, or speech impairment factors, for their potential influence on the performance gap. RESULTS: We found a significant performance gap between read and conversational speech in both personalized and unadapted ASR models. Speech impairment severity notably impacted recognition accuracy in unadapted models for both speech modes and in personalized models for read speech. Linguistic attributes of utterances were the most influential on accuracy, though atypical speech characteristics also played a role. Including conversational speech samples in model training notably improved recognition accuracy. CONCLUSIONS: We observed a significant performance gap in ASR accuracy between read and conversational speech for individuals with speech disorders. This gap was largely due to the linguistic complexity and unique characteristics of speech disorders in conversational speech. Training personalized ASR models using conversational speech significantly improved recognition accuracy, demonstrating the importance of domain-specific training and highlighting the need for further research into ASR systems capable of handling disordered conversational speech effectively.

The relationship of rate and pause features to the communicative participation of people living with ALS.

INTRODUCTION/AIMS: Many people living with amyotrophic lateral sclerosis (PALS) report restrictions in their day-to-day communication (communicative participation). However, little is known about which speech features contribute to these restrictions. This study evaluated the effects of common speech symptoms in PALS (reduced overall speaking rate, slowed articulation rate, and increased pausing) on communicative participation restrictions. METHODS: Participants completed surveys (the Communicative Participation Item Bank-short form; the self-entry version of the ALS Functional Rating Scale-Revised) and recorded themselves reading the Bamboo Passage aloud using a smartphone app. Rate and pause measures were extracted from the recordings. The association of various demographic, clinical, self-reported, and acoustic speech features with communicative participation was evaluated with bivariate correlations. The contribution of salient rate and pause measures to communicative participation was assessed using multiple linear regression. RESULTS: Fifty seven people living with ALS participated in the study (mean age = 61.1 years). Acoustic and self-report measures of speech and bulbar function were moderately to highly associated with communicative participation (Spearman rho coefficients ranged from rs = 0.48 to rs = 0.77). A regression model including participant age, sex, articulation rate, and percent pause time accounted for 57% of the variance of communicative participation ratings. DISCUSSION: Even though PALS with slowed articulation rate and increased pausing may convey their message clearly, these speech features predict communicative participation restrictions. The identification of quantitative speech features, such as articulation rate and percent pause time, is critical to facilitating early and targeted intervention and for monitoring bulbar decline in ALS.

Listener effort quantifies clinically meaningful progression of dysarthria in people living with amyotrophic lateral sclerosis.

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative motor neuron disease that causes progressive muscle weakness. Progressive bulbar dysfunction causes dysarthria and thus social isolation, reducing quality of life. The Everything ALS Speech Study obtained longitudinal clinical information and speech recordings from 292 participants. In a subset of 120 participants, we measured speaking rate (SR) and listener effort (LE), a measure of dysarthria severity rated by speech pathologists from recordings. LE intra- and inter-rater reliability was very high (ICC 0.88 to 0.92). LE correlated with other measures of dysarthria at baseline. LE changed over time in participants with ALS (slope 0.77 pts/month; p<0.001) but not controls (slope 0.005 pts/month; p=0.807). The slope of LE progression was similar in all participants with ALS who had bulbar dysfunction at baseline, regardless of ALS site of onset. LE could be a remotely collected clinically meaningful clinical outcome assessment for ALS clinical trials.

Suprachoroidal gene transfer with nonviral nanoparticles in large animal eyes.

Suprachoroidal nonviral gene therapy with biodegradable poly(ß-amino ester) nanoparticles (NPs) provides widespread expression in photoreceptors and retinal pigmented epithelial (RPE) cells and therapeutic benefits in rodents. Here, we show in a human-sized minipig eye that suprachoroidal injection of 50 µl of NPs containing 19.2 µg of GFP expression plasmid caused GFP expression in photoreceptors and RPE throughout the entire eye with no toxicity. Two weeks after injection of 50, 100, or 200 µl, there was considerable within-eye and between-eye variability in expression that was reduced 3 months after injection of 200 µl and markedly reduced after three suprachoroidal injections at different locations around the eye. Reduction of bacterial CpG sequences in the expression plasmid resulted in a trend toward higher expression. These data indicate that nonviral suprachoroidal gene therapy with optimized polymer, expression plasmid, and injection approach has potential for treating photoreceptors throughout the entire retina of a human-sized eye.

Exploring Motor Speech Disorders in Low and Minimally Verbal Autistic Individuals: An Auditory-Perceptual Analysis.

PURPOSE: Motor deficits are widely documented among autistic individuals, and speech characteristics consistent with a motor speech disorder have been reported in prior literature. We conducted an auditory-perceptual analysis of speech production skills in low and minimally verbal autistic individuals as a step toward clarifying the nature of speech production impairments in this population and the potential link between oromotor functioning and language development. METHOD: Fifty-four low or minimally verbal autistic individuals aged 4-18 years were video-recorded performing nonspeech oromotor tasks and producing phonemes, syllables, and words in imitation. Three trained speech-language pathologists provided auditory perceptual ratings of 11 speech features reflecting speech subsystem performance and overall speech production ability. The presence, attributes, and severity of signs of oromotor dysfunction were analyzed, as were relative performance on nonspeech and speech tasks and correlations between perceptual speech features and language skills. RESULTS AND CONCLUSIONS: Our findings provide evidence of a motor speech disorder in this population, characterized by perceptual speech features including reduced intelligibility, decreased consonant and vowel precision, and impairments of speech coordination and consistency. Speech deficits were more associated with articulation than with other speech subsystems. Speech production was more impaired than nonspeech oromotor abilities in a subgroup of the sample. Oromotor deficits were significantly associated with expressive and receptive language skills. Findings are interpreted in the context of known characteristics of the pediatric motor speech disorders childhood apraxia of speech and childhood dysarthria. These results, if replicated in future studies, have significant potential to improve the early detection of language impairments, inform the development of speech and language interventions, and aid in the identification of neurobiological mechanisms influencing communication development.

Individualized white matter connectivity of the articulatory pathway: An ultra-high field study.

In current sensorimotor theories pertaining to speech perception, there is a notable emphasis on the involvement of the articulatory-motor system in the processing of speech sounds. Using ultra-high field diffusion-weighted imaging at 7 Tesla, we visualized the white matter tracts connected to areas activated during a simple speech-sound production task in 18 healthy right-handed adults. Regions of interest for white matter tractography were individually determined through 7T functional MRI (fMRI) analyses, based on activations during silent vocalization tasks. These precentral seed regions, activated during the silent production of a lip-vowel sound, demonstrated anatomical connectivity with posterior superior temporal gyrus areas linked to the auditory perception of phonetic sounds. Our study provides a macrostructural foundation for understanding connections in speech production and underscores the central role of the articulatory motor system in speech perception. These findings highlight the value of ultra-high field 7T MR acquisition in unraveling the neural underpinnings of speech.

Grand Challenges at the Interface of Engineering and Medicine.

Over the past two decades Biomedical Engineering has emerged as a major discipline that bridges societal needs of human health care with the development of novel technologies. Every medical institution is now equipped at varying degrees of sophistication with the ability to monitor human health in both non-invasive and invasive modes. The multiple scales at which human physiology can be interrogated provide a profound perspective on health and disease. We are at the nexus of creating "avatars" (herein defined as an extension of "digital twins") of human patho/physiology to serve as paradigms for interrogation and potential intervention. Motivated by the emergence of these new capabilities, the IEEE Engineering in Medicine and Biology Society, the Departments of Biomedical Engineering at Johns Hopkins University and Bioengineering at University of California at San Diego sponsored an interdisciplinary workshop to define the grand challenges that face biomedical engineering and the mechanisms to address these challenges. The Workshop identified five grand challenges with cross-cutting themes and provided a roadmap for new technologies, identified new training needs, and defined the types of interdisciplinary teams needed for addressing these challenges. The themes presented in this paper include: 1) accumedicine through creation of avatars of cells, tissues, organs and whole human; 2) development of smart and responsive devices for human function augmentation; 3) exocortical technologies to understand brain function and treat neuropathologies; 4) the development of approaches to harness the human immune system for health and wellness; and 5) new strategies to engineer genomes and cells.

Vowel distinctiveness as a concurrent predictor of expressive language function in autistic children.

Speech ability may limit spoken language development in some minimally verbal autistic children. In this study, we aimed to determine whether an acoustic measure of speech production, vowel distinctiveness, is concurrently related to expressive language (EL) for autistic children. Syllables containing the vowels [i] and [a] were recorded remotely from 27 autistic children (4;1-7;11) with a range of spoken language abilities. Vowel distinctiveness was calculated using automatic formant tracking software. Robust hierarchical regressions were conducted with receptive language (RL) and vowel distinctiveness as predictors of EL. Hierarchical regressions were also conducted within a High EL and a Low EL subgroup. Vowel distinctiveness accounted for 29% of the variance in EL for the entire group, RL for 38%. For the Low EL group, only vowel distinctiveness was significant, accounting for 38% of variance in EL. Conversely, in the High EL group, only RL was significant and accounted for 26% of variance in EL. Replicating previous results, speech production and RL significantly predicted concurrent EL in autistic children, with speech production being the sole significant predictor for the Low EL group and RL the sole significant predictor for the High EL group. Further work is needed to determine whether vowel distinctiveness longitudinally, as well as concurrently, predicts EL. Findings have important implications for the early identification of language impairment and in developing language interventions for autistic children.

Oral diadochokinetic markers of X-linked dystonia-parkinsonism.

INTRODUCTION: X-linked dystonia-parkinsonism (XDP) is a neurodegenerative disorder that may result in severe speech impairment. The literature suggests that there are differences in the speech of individuals with XDP and healthy controls. This study aims to examine the motor speech characteristics of the mixed dystonia-parkinsonism phase of XDP. METHOD: We extracted acoustic features representing coordination, consistency, speed, precision, and rate from 26 individuals with XDP and 26 controls using Praat, MATLAB, and R software. Group demographics were compared using descriptive statistics. A one-way analysis of variance (ANOVA) with Tukey's post hoc test was used to test for acoustic differences between the two groups. RESULTS: The XDP group had significantly lower consistency, speed, precision, and rate than controls (p < 0.05). For coordination, the XDP group had a smaller ratio of pause duration during transitions when compared to controls. DISCUSSION: To our knowledge, this study is the first to describe the motor speech characteristics of the mixed dystonia-parkinsonism phase of XDP. The motor speech of mixed dystonia-parkinsonism XDP is similar to prior characterizations of mixed hyperkinetic-hypokinetic dysarthria with noted differences in articulatory coordination, consistency, speed, precision, and rate from healthy controls. Identifying the motor speech components of all three phenotypes of XDP (i.e., dystonia-dominant phase, parkinsonism-dominant phase, and mixed dystonia-parkinsonism phase) is needed to establish markers of speech impairment to track disease progression.

The association between longitudinal declines in speech sound accuracy and speech intelligibility in speakers with amyotrophic lateral sclerosis.

The purpose of this study was to examine how neurodegeneration secondary to amyotrophic lateral sclerosis (ALS) impacts speech sound accuracy over time and how speech sound accuracy, in turn, is related to speech intelligibility. Twenty-one participants with ALS read the Bamboo Passage over multiple data collection sessions across several months. Phonemic and orthographic transcriptions were completed for all speech samples. The percentage of phonemes accurately produced was calculated across each phoneme, sound class (i.e. consonants versus vowels), and distinctive feature (i.e. features involved in Manner of Articulation, Place of Articulation, Laryngeal Voicing, Tongue Height, and Tongue Advancement). Intelligibility was determined by calculating the percentage of words correctly transcribed orthographically by naive listeners. Linear mixed effects models were conducted to assess the decline of each distinctive feature over time and its impact on intelligibility. The results demonstrated that overall phonemic production accuracy had a nonlinear relationship with speech intelligibility and that a subset of features (i.e. those dependent on precise lingual and labial constriction and/or extensive lingual and labial movement) were more important for intelligibility and were more impacted over time than other features. Furthermore, findings revealed that consonants were more strongly associated with intelligibility than vowels, but consonants did not significantly differ from vowels in their decline over time. These findings have the potential to (1) strengthen mechanistic understanding of the physiological constraints imposed by neuronal degeneration on speech production and (2) inform the timing and selection of treatment and assessment targets for individuals with ALS.

Screening for lipid nanoparticles that modulate the immune activity of helper T cells towards enhanced antitumour activity.

Lipid nanoparticles (LNPs) can be designed to potentiate cancer immunotherapy by promoting their uptake by antigen-presenting cells, stimulating the maturation of these cells and modulating the activity of adjuvants. Here we report an LNP-screening method for the optimization of the type of helper lipid and of lipid-component ratios to enhance the delivery of tumour-antigen-encoding mRNA to dendritic cells and their immune-activation profile towards enhanced antitumour activity. The method involves screening for LNPs that enhance the maturation of bone-marrow-derived dendritic cells and antigen presentation in vitro, followed by assessing immune activation and tumour-growth suppression in a mouse model of melanoma after subcutaneous or intramuscular delivery of the LNPs. We found that the most potent antitumour activity, especially when combined with immune checkpoint inhibitors, resulted from a coordinated attack by T cells and NK cells, triggered by LNPs that elicited strong immune activity in both type-1 and type-2 T helper cells. Our findings highlight the importance of optimizing the LNP composition of mRNA-based cancer vaccines to tailor antigen-specific immune-activation profiles.

An engineered immunocytokine with collagen affinity improves the tumor bioavailability, tolerability, and therapeutic efficacy of IL-2.

The clinical utility of human interleukin-2 (hIL-2) is limited by its short serum half-life, preferential activation of regulatory T (TReg) over immune effector cells, and dose-limiting toxicities. We previously engineered F10 immunocytokine (IC), an intramolecularly assembled cytokine/antibody fusion protein that linked hIL-2 to an anti-IL-2 antibody (denoted F10) that extended IL-2 half-life and augmented the immune effector to TReg ratio. Here, we leveraged molecular engineering to improve the anti-tumor therapeutic efficacy and tolerability of F10 IC by developing an iteration, denoted F10 IC-CBD (collagen binding domain), designed for intratumoral administration and in situ retention based on collagen affinity. F10 IC-CBD retained IL-2 bioactivity exclusively in the tumor and eliminated IL-2-associated toxicities. Furthermore, F10 IC exhibited potent single-agent therapeutic efficacy and synergy with systemic immune checkpoint blockade and elicited an abscopal response in mouse tumors models. This engineered fusion protein presents a prototype for the design of intratumoral therapies.

Remote Assessment for ALS using Multimodal Dialog Agents: Data Quality, Feasibility and Task Compliance.

We investigate the feasibility, task compliance and audiovisual data quality of a multimodal dialog-based solution for remote assessment of Amyotrophic Lateral Sclerosis (ALS). 53 people with ALS and 52 healthy controls interacted with Tina, a cloud-based conversational agent, in performing speech tasks designed to probe various aspects of motor speech function while their audio and video was recorded. We rated a total of 250 recordings for audio/video quality and participant task compliance, along with the relative frequency of different issues observed. We observed excellent compliance (98%) and audio (95.2%) and visual quality rates (84.8%), resulting in an overall yield of 80.8% recordings that were both compliant and of high quality. Furthermore, recording quality and compliance were not affected by level of speech severity and did not differ significantly across end devices. These findings support the utility of dialog systems for remote monitoring of speech in ALS.

Feedback From Automatic Speech Recognition to Elicit Clear Speech in Healthy Speakers.

PURPOSE: This study assessed the effectiveness of feedback generated by automatic speech recognition (ASR) for eliciting clear speech from young, healthy individuals. As a preliminary step toward exploring a novel method for eliciting clear speech in patients with dysarthria, we investigated the effects of ASR feedback in healthy controls. If successful, ASR feedback has the potential to facilitate independent, at-home clear speech practice. METHOD: Twenty-three healthy control speakers (ages 23-40 years) read sentences aloud in three speaking modes: Habitual, Clear (over-enunciated), and in response to ASR feedback (ASR). In the ASR condition, we used Mozilla DeepSpeech to transcribe speech samples and provide participants with a value indicating the accuracy of the ASR's transcription. For speakers who achieved sufficiently high ASR accuracy, noise was added to their speech at a participant-specific signal-to-noise ratio to ensure that each participant had to over-enunciate to achieve high ASR accuracy. RESULTS: Compared to habitual speech, speech produced in the ASR and Clear conditions was clearer, as rated by speech-language pathologists, and more intelligible, per speech-language pathologist transcriptions. Speech in the Clear and ASR conditions aligned on several acoustic measures, particularly those associated with increased vowel distinctiveness and decreased speaking rate. However, ASR accuracy, intelligibility, and clarity were each correlated with different speech features, which may have implications for how people change their speech for ASR feedback. CONCLUSIONS: ASR successfully elicited outcomes similar to clear speech in healthy speakers. Future work should investigate its efficacy in eliciting clear speech in people with dysarthria.

O-GlcNAcylation regulates neurofilament-light assembly and function and is perturbed by Charcot-Marie-Tooth disease mutations.

The neurofilament (NF) cytoskeleton is critical for neuronal morphology and function. In particular, the neurofilament-light (NF-L) subunit is required for NF assembly in vivo and is mutated in subtypes of Charcot-Marie-Tooth (CMT) disease. NFs are highly dynamic, and the regulation of NF assembly state is incompletely understood. Here, we demonstrate that human NF-L is modified in a nutrient-sensitive manner by O-linked-ß-N-acetylglucosamine (O-GlcNAc), a ubiquitous form of intracellular glycosylation. We identify five NF-L O-GlcNAc sites and show that they regulate NF assembly state. NF-L engages in O-GlcNAc-mediated protein-protein interactions with itself and with the NF component α-internexin, implying that O-GlcNAc may be a general regulator of NF architecture. We further show that NF-L O-GlcNAcylation is required for normal organelle trafficking in primary neurons. Finally, several CMT-causative NF-L mutants exhibit perturbed O-GlcNAc levels and resist the effects of O-GlcNAcylation on NF assembly state, suggesting a potential link between dysregulated O-GlcNAcylation and pathological NF aggregation. Our results demonstrate that site-specific glycosylation regulates NF-L assembly and function, and aberrant NF O-GlcNAcylation may contribute to CMT and other neurodegenerative disorders.

Role of articulatory motor networks in perceptual categorization of speech signals: a 7T fMRI study.

Speech and language processing involve complex interactions between cortical areas necessary for articulatory movements and auditory perception and a range of areas through which these are connected and interact. Despite their fundamental importance, the precise mechanisms underlying these processes are not fully elucidated. We measured BOLD signals from normal hearing participants using high-field 7 Tesla fMRI with 1-mm isotropic voxel resolution. The subjects performed 2 speech perception tasks (discrimination and classification) and a speech production task during the scan. By employing univariate and multivariate pattern analyses, we identified the neural signatures associated with speech production and perception. The left precentral, premotor, and inferior frontal cortex regions showed significant activations that correlated with phoneme category variability during perceptual discrimination tasks. In addition, the perceived sound categories could be decoded from signals in a region of interest defined based on activation related to production task. The results support the hypothesis that articulatory motor networks in the left hemisphere, typically associated with speech production, may also play a critical role in the perceptual categorization of syllables. The study provides valuable insights into the intricate neural mechanisms that underlie speech processing.

BME 2.0: Engineering the Future of Medicine.

If the 20th century was the age of mapping and controlling the external world, the 21st century is the biomedical age of mapping and controlling the biological internal world. The biomedical age is bringing new technological breakthroughs for sensing and controlling human biomolecules, cells, tissues, and organs, which underpin new frontiers in the biomedical discovery, data, biomanufacturing, and translational sciences. This article reviews what we believe will be the next wave of biomedical engineering (BME) education in support of the biomedical age, what we have termed BME 2.0. BME 2.0 was announced on October 12 2017 at BMES 49 (https://www.bme.jhu.edu/news-events/news/miller-opens-2017-bmes-annual-meeting-with-vision-for-new-bme-era/). We present several principles upon which we believe the BME 2.0 curriculum should be constructed, and from these principles, we describe what view as the foundations that form the next generations of curricula in support of the BME enterprise. The core principles of BME 2.0 education are (a) educate students bilingually, from day 1, in the languages of modern molecular biology and the analytical modeling of complex biological systems; (b) prepare every student to be a biomedical data scientist; (c) build a unique BME community for discovery and innovation via a vertically integrated and convergent learning environment spanning the university and hospital systems; (d) champion an educational culture of inclusive excellence; and (e) codify in the curriculum ongoing discoveries at the frontiers of the discipline, thus ensuring BME 2.0 as a launchpad for training the future leaders of the biotechnology marketplaces. We envision that the BME 2.0 education is the path for providing every student with the training to lead in this new era of engineering the future of medicine in the 21st century.

Erratum to "BME 2.0: Engineering the Future of Medicine".

[This corrects the article DOI: 10.34133/bmef.0001.].

Biomaterial-Mediated Genetic Reprogramming of Merkel Cell Carcinoma and Melanoma Leads to Targeted Cancer Cell Killing In Vitro and In Vivo.

Tumor immunotherapy is a promising anticancer strategy; however, tumor cells may employ resistance mechanisms, including downregulation of major histocompatibility complex (MHC) molecules to avoid immune recognition. Here, we investigate reprogramming nanoparticles (NPs) that deliver immunostimulatory genes to enhance immunotherapy and address defective antigen presentation in skin cancer in vitro and in vivo. We use a modular poly(beta-amino ester) (PBAE)-based NP to deliver DNA encoding 4-1BBL, IL-12, and IFNγ to reprogram human Merkel cell carcinoma (MCC) cells in vitro and mouse melanoma tumors in vivo to drive adaptive antitumor immune responses. Optimized NP formulations delivering 4-1BBL/IL-12 or 4-1BBL/IL-12/IFNγ DNA successfully transfect MCC and melanoma cells in vitro and in vivo, respectively, resulting in IFNγ-driven upregulation of MHC class I and II molecules on cancer cells. These NPs reprogram the tumor immune microenvironment (TIME) and elicit strong T-cell-driven immune responses, leading to cancer cell killing and T-cell proliferation in vitro and slowing tumor growth and improving survival rates in vivo. Based on expected changes to the tumor immune microenvironment, particularly the importance of IFNγ to the immune response and driving both T-cell function and exhaustion, next-generation NPs codelivering IFNγ were designed. These offered mixed benefits, exchanging improved polyfunctionality for increased T-cell exhaustion and demonstrating higher systemic toxicity in vivo. Further profiling of the immune response with these NPs provides insight into T-cell exhaustion and polyfunctionality induced by different formulations, providing a greater understanding of this immunotherapeutic strategy.