Automated speech analytics in ALS: higher sensitivity of digital articulatory precision over the ALSFRS-R.

Objective: Although studies have shown that digital measures of speech detected ALS speech impairment and correlated with the ALSFRS-R speech item, no study has yet compared their performance in detecting speech changes. In this study, we compared the performances of the ALSFRS-R speech item and an algorithmic speech measure in detecting clinically important changes in speech. Importantly, the study was part of a FDA submission which received the breakthrough device designation for monitoring ALS; we provide this paper as a roadmap for validating other speech measures for monitoring disease progression. Methods: We obtained ALSFRS-R speech subscores and speech samples from participants with ALS. We computed the minimum detectable change (MDC) of both measures; using clinician-reported listener effort and a perceptual ratings of severity, we calculated the minimal clinically important difference (MCID) of each measure with respect to both sets of clinical ratings. Results: For articulatory precision, the MDC (.85) was lower than both MCID measures (2.74 and 2.28), and for the ALSFRS-R speech item, MDC (.86) was greater than both MCID measures (.82 and .72), indicating that while the articulatory precision measure detected minimal clinically important differences in speech, the ALSFRS-R speech item did not. Conclusion: The results demonstrate that the digital measure of articulatory precision effectively detects clinically important differences in speech ratings, outperforming the ALSFRS-R speech item. Taken together, the results herein suggest that this speech outcome is a clinically meaningful measure of speech change.

Wav2DDK: Analytical and Clinical Validation of an Automated Diadochokinetic Rate Estimation Algorithm on Remotely Collected Speech.

PURPOSE: Oral diadochokinesis is a useful task in assessment of speech motor function in the context of neurological disease. Remote collection of speech tasks provides a convenient alternative to in-clinic visits, but scoring these assessments can be a laborious process for clinicians. This work describes Wav2DDK, an automated algorithm for estimating the diadochokinetic (DDK) rate on remotely collected audio from healthy participants and participants with amyotrophic lateral sclerosis (ALS). METHOD: Wav2DDK was developed using a corpus of 970 DDK assessments from healthy and ALS speakers where ground truth DDK rates were provided manually by trained annotators. The clinical utility of the algorithm was demonstrated on a corpus of 7,919 assessments collected longitudinally from 26 healthy controls and 82 ALS speakers. Corpora were collected via the participants' own mobile device, and instructions for speech elicitation were provided via a mobile app. DDK rate was estimated by parsing the character transcript from a deep neural network transformer acoustic model trained on healthy and ALS speech. RESULTS: Algorithm estimated DDK rates are highly accurate, achieving .98 correlation with manual annotation, and an average error of only 0.071 syllables per second. The rate exactly matched ground truth for 83% of files and was within 0.5 syllables per second for 95% of files. Estimated rates achieve a high test-retest reliability (r = .95) and show good correlation with the revised ALS functional rating scale speech subscore (r = .67). CONCLUSION: We demonstrate a system for automated DDK estimation that increases efficiency of calculation beyond manual annotation. Thorough analytical and clinical validation demonstrates that the algorithm is not only highly accurate, but also provides a convenient, clinically relevant metric for tracking longitudinal decline in ALS, serving to promote participation and diversity of participants in clinical research. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.23787033.

A speech-based prognostic model for dysarthria progression in ALS.

Objective: We demonstrated that it was possible to predict ALS patients' degree of future speech impairment based on past data. We used longitudinal data from two ALS studies where participants recorded their speech on a daily or weekly basis and provided ALSFRS-R speech subscores on a weekly or quarterly basis (quarter-annually). Methods: Using their speech recordings, we measured articulatory precision (a measure of the crispness of pronunciation) using an algorithm that analyzed the acoustic signal of each phoneme in the words produced. First, we established the analytical and clinical validity of the measure of articulatory precision, showing that the measure correlated with perceptual ratings of articulatory precision (r = .9). Second, using articulatory precision from speech samples from each participant collected over a 45-90 day model calibration period, we showed it was possible to predict articulatory precision 30-90 days after the last day of the model calibration period. Finally, we showed that the predicted articulatory precision scores mapped onto ALSFRS-R speech subscores. Results: the mean absolute error was as low as 4% for articulatory precision and 14% for ALSFRS-R speech subscores relative to the total range of their respective scales. Conclusion: Our results demonstrated that a subject-specific prognostic model for speech predicts future articulatory precision and ALSFRS-R speech values accurately.

Language Analytics for Assessment of Mental Health Status and Functional Competency.

BACKGROUND AND HYPOTHESIS: Automated language analysis is becoming an increasingly popular tool in clinical research involving individuals with mental health disorders. Previous work has largely focused on using high-dimensional language features to develop diagnostic and prognostic models, but less work has been done to use linguistic output to assess downstream functional outcomes, which is critically important for clinical care. In this work, we study the relationship between automated language composites and clinical variables that characterize mental health status and functional competency using predictive modeling. STUDY DESIGN: Conversational transcripts were collected from a social skills assessment of individuals with schizophrenia (n = 141), bipolar disorder (n = 140), and healthy controls (n = 22). A set of composite language features based on a theoretical framework of speech production were extracted from each transcript and predictive models were trained. The prediction targets included clinical variables for assessment of mental health status and social and functional competency. All models were validated on a held-out test sample not accessible to the model designer. STUDY RESULTS: Our models predicted the neurocognitive composite with Pearson correlation PCC = 0.674; PANSS-positive with PCC = 0.509; PANSS-negative with PCC = 0.767; social skills composite with PCC = 0.785; functional competency composite with PCC = 0.616. Language features related to volition, affect, semantic coherence, appropriateness of response, and lexical diversity were useful for prediction of clinical variables. CONCLUSIONS: Language samples provide useful information for the prediction of a variety of clinical variables that characterize mental health status and functional competency.

Automated semantic relevance as an indicator of cognitive decline: Out-of-sample validation on a large-scale longitudinal dataset.

We developed and evaluated an automatically extracted measure of cognition (semantic relevance) using automated and manual transcripts of audio recordings from healthy and cognitively impaired participants describing the Cookie Theft picture from the Boston Diagnostic Aphasia Examination. We describe the rationale and metric validation. We developed the measure on one dataset and evaluated it on a large database (>2000 samples) by comparing accuracy against a manually calculated metric and evaluating its clinical relevance. The fully automated measure was accurate (r = .84), had moderate to good reliability (intra-class correlation = .73), correlated with Mini-Mental State Examination and improved the fit in the context of other automatic language features (r = .65), and longitudinally declined with age and level of cognitive impairment. This study demonstrates the use of a rigorous analytical and clinical framework for validating automatic measures of speech, and applied it to a measure that is accurate and clinically relevant.

Explorations of Individual Change Processes and Their Determinants: A Novel Approach and Remaining Challenges.

Over the past 40 years there have been great advances in the analysis of individual change and the analyses of between-person differences in change. While conditional growth models are the dominant approach, exploratory models, such as growth mixture models and structural equation modeling trees, allow for greater flexibility in the modeling of between-person differences in change. We continue to push for greater flexibility in the modeling of individual change and its determinants by combining growth mixture modeling with structural equation modeling trees to evaluate how measured covariates predict class membership using a recursive partitioning algorithm. This approach, referred to as growth mixture modeling with membership trees, is illustrated with longitudinal reading data from the Early Childhood Longitudinal Study with the MplusTrees package in R.

Estimation of forced vital capacity using speech acoustics in patients with ALS.

In this study, we present and provide validation data for a tool that predicts forced vital capacity (FVC) from speech acoustics collected remotely via a mobile app without the need for any additional equipment (e.g. a spirometer). We trained a machine learning model on a sample of healthy participants and participants with amyotrophic lateral sclerosis (ALS) to learn a mapping from speech acoustics to FVC and used this model to predict FVC values in a new sample from a different study of participants with ALS. We further evaluated the cross-sectional accuracy of the model and its sensitivity to within-subject change in FVC. We found that the predicted and observed FVC values in the test sample had a correlation coefficient of .80 and mean absolute error between .54 L and .58 L (18.5% to 19.5%). In addition, we found that the model was able to detect longitudinal decline in FVC in the test sample, although to a lesser extent than the observed FVC values measured using a spirometer, and was highly repeatable (ICC = 0.92-0.94), although to a lesser extent than the actual FVC (ICC = .97). These results suggest that sustained phonation may be a useful surrogate for VC in both research and clinical environments.

Altered speech patterns in subjects with post-traumatic headache due to mild traumatic brain injury.

BACKGROUND/OBJECTIVE: Changes in speech can be detected objectively before and during migraine attacks. The goal of this study was to interrogate whether speech changes can be detected in subjects with post-traumatic headache (PTH) attributed to mild traumatic brain injury (mTBI) and whether there are within-subject changes in speech during headaches compared to the headache-free state. METHODS: Using a series of speech elicitation tasks uploaded via a mobile application, PTH subjects and healthy controls (HC) provided speech samples once every 3 days, over a period of 12 weeks. The following speech parameters were assessed: vowel space area, vowel articulation precision, consonant articulation precision, average pitch, pitch variance, speaking rate and pause rate. Speech samples of subjects with PTH were compared to HC. To assess speech changes associated with PTH, speech samples of subjects during headache were compared to speech samples when subjects were headache-free. All analyses were conducted using a mixed-effect model design. RESULTS: Longitudinal speech samples were collected from nineteen subjects with PTH (mean age = 42.5, SD = 13.7) who were an average of 14 days (SD = 32.2) from their mTBI at the time of enrollment and thirty-one HC (mean age = 38.7, SD = 12.5). Regardless of headache presence or absence, PTH subjects had longer pause rates and reductions in vowel and consonant articulation precision relative to HC. On days when speech was collected during a headache, there were longer pause rates, slower sentence speaking rates and less precise consonant articulation compared to the speech production of HC. During headache, PTH subjects had slower speaking rates yet more precise vowel articulation compared to when they were headache-free. CONCLUSIONS: Compared to HC, subjects with acute PTH demonstrate altered speech as measured by objective features of speech production. For individuals with PTH, speech production may have been more effortful resulting in slower speaking rates and more precise vowel articulation during headache vs. when they were headache-free, suggesting that speech alterations were related to PTH and not solely due to the underlying mTBI.

Early detection and tracking of bulbar changes in ALS via frequent and remote speech analysis.

Bulbar deterioration in amyotrophic lateral sclerosis (ALS) is a devastating characteristic that impairs patients' ability to communicate, and is linked to shorter survival. The existing clinical instruments for assessing bulbar function lack sensitivity to early changes. In this paper, using a cohort of N = 65 ALS patients who provided regular speech samples for 3-9 months, we demonstrated that it is possible to remotely detect early speech changes and track speech progression in ALS via automated algorithmic assessment of speech collected digitally.

Restoration of prosocial behavior in rats after heroin self-administration via chemogenetic activation of the anterior insular cortex.

The anterior insular cortex (AIC) mediates various social, emotional, and interoceptive components of addiction. We recently demonstrated a disruption of prosocial behavior following heroin self-administration in rats, as assessed by examining the animals' propensity to rescue its cagemate from a plastic restrainer while having simultaneous access to heroin. To examine the possibility that heroin-induced deficits in prosocial function are mediated by the AIC, the present study examined the effects of chemogenetic activation or inhibition of excitatory AIC pyramidal neurons on heroin-induced prosocial deficits. After establishment of baseline rescuing behavior, rats received bilateral infusions of viral vectors encoding either a control virus (AAV-CaMKIIα-GFP), stimulatory DREADD (AAV-CaMKIIα-hM3Dq-mCherry) (Experiment 1), or inhibitory DREADD (AAV-CaMKIIα-hM4Di-mCherry) (Experiment 2), into the AIC. Rats were then allowed to self-administer heroin (0.06 mg/kg/infusion) 6 hr/day for 2 weeks. Prior to re-assessment of prosocial behavior, animals were administered clozapine-N-oxide (1.5 mg/kg, i.p.) to assess the effects of chemogenetic activation or inhibition of the AIC. Relative to control animals, chemogenetic activation of the AIC reversed deficits in rescuing behavior induced by heroin, whereas chemogenetic inhibition of the AIC had no effect. We hypothesize that stimulatory neuromodulation of the AIC may be a novel approach for restoring prosociality in opiate abuse.

Repeatability of Commonly Used Speech and Language Features for Clinical Applications.

INTRODUCTION: Changes in speech have the potential to provide important information on the diagnosis and progression of various neurological diseases. Many researchers have relied on open-source speech features to develop algorithms for measuring speech changes in clinical populations as they are convenient and easy to use. However, the repeatability of open-source features in the context of neurological diseases has not been studied. METHODS: We used a longitudinal sample of healthy controls, individuals with amyotrophic lateral sclerosis, and individuals with suspected frontotemporal dementia, and we evaluated the repeatability of acoustic and language features separately on these 3 data sets. RESULTS: Repeatability was evaluated using intraclass correlation (ICC) and the within-subjects coefficient of variation (WSCV). In 3 sets of tasks, the median ICC were between 0.02 and 0.55, and the median WSCV were between 29 and 79%. CONCLUSION: Our results demonstrate that the repeatability of speech features extracted using open-source tool kits is low. Researchers should exercise caution when developing digital health models with open-source speech features. We provide a detailed summary of feature-by-feature repeatability results (ICC, WSCV, SE of measurement, limits of agreement for WSCV, and minimal detectable change) in the online supplementary material so that researchers may incorporate repeatability information into the models they develop.

Effects of heroin on rat prosocial behavior.

Opioid use disorders are characterized in part by impairments in social functioning. Previous research indicates that laboratory rats, which are frequently used as animal models of addiction-related behaviors, are capable of prosocial behavior. For example, under normal conditions, when a 'free' rat is placed in the vicinity of rat trapped in a plastic restrainer, the rat will release or 'rescue' the other rat from confinement. The present study was conducted to determine the effects of heroin on prosocial behavior in rats. For 2 weeks, rats were given the opportunity to rescue their cagemate from confinement, and the occurrence of and latency to free the confined rat was recorded. After baseline rescuing behavior was established, rats were randomly selected to self-administer heroin (0.06 mg/kg/infusion i.v.) or sucrose pellets (orally) for 14 days. Next, rats were retested for rescuing behavior once daily for 3 days, during which they were provided with a choice between freeing the trapped cagemate and continuing to self-administer their respective reinforcer. Our results indicate that rats self-administering sucrose continued to rescue their cagemate, whereas heroin rats chose to self-administer heroin and not rescue their cagemate. These findings suggest that rats with a history of heroin self-administration show deficits in prosocial behavior, consistent with specific diagnostic criteria for opioid use disorder. Behavioral paradigms providing a choice between engaging in prosocial behavior and continuing drug use may be useful in modeling and investigating the neural basis of social functioning deficits in opioid addiction.

Modeling change trajectories with count and zero-inflated outcomes: Challenges and recommendations.

The goal of this article is to describe models to examine change over time with an outcome that represents a count, such as the number of alcoholic drinks per day. Common challenges encountered with this type of data are: (1) the outcome is discrete, may have a large number of zeroes, and may be overdispersed, (2) the data are clustered (multiple observations within each individual), (3) the researchers needs to carefully consider and choose an appropriate time metric, and (4) the researcher needs to identify both a proper individual (potentially nonlinear) change model and an appropriate distributional form that captures the properties of the data. In this article, we provide an overview of generalized linear models, generalized estimating equation models, and generalized latent variable (mixed-effects) models for longitudinal count outcomes focusing on the Poisson, negative binomial, zero-inflated, and hurdle distributions. We review common challenges and provide recommendations for identifying an appropriate change trajectory while determining an appropriate distributional form for the outcome (e.g., determining zero-inflation and overdispersion). We demonstrate the process of fitting and choosing a model with empirical longitudinal data on alcohol intake across adolescence collected as part of the National Longitudinal Survey of Youth 1997.

Recursive Partitioning with Nonlinear Models of Change.

In this article, we introduce nonlinear longitudinal recursive partitioning (nLRP) and the R package longRpart2 to carry out the analysis. This method implements recursive partitioning (also known as decision trees) in order to split data based on individual- (i.e., cluster) level covariates with the goal of predicting differences in nonlinear longitudinal trajectories. At each node, a user-specified linear or nonlinear mixed-effects model is estimated. This method is an extension of Abdolell et al.'s (2002) longitudinal recursive partitioning while permitting a nonlinear mixed-effects model in addition to a linear mixed-effects model in each node. We give an overview of recursive partitioning, nonlinear mixed-effects models for longitudinal data, describe nLRP, and illustrate its use with empirical data from the Early Childhood Longitudinal Study-Kindergarten Cohort.