Linguistic changes in neurodegenerative diseases relate to clinical symptoms.

Background: The detection and characterization of speech changes may help in the identification and monitoring of neurodegenerative diseases. However, there is limited research validating the relationship between speech changes and clinical symptoms across a wide range of neurodegenerative diseases. Method: We analyzed speech recordings from 109 patients who were diagnosed with various neurodegenerative diseases, including Alzheimer's disease, Frontotemporal Dementia, and Vascular Cognitive Impairment, in a cognitive neurology memory clinic. Speech recordings of an open-ended picture description task were processed using the Winterlight speech analysis platform which generates >500 speech features, including the acoustics of speech and linguistic properties of spoken language. We investigated the relationship between the speech features and clinical assessments including the Mini Mental State Examination (MMSE), Mattis Dementia Rating Scale (DRS), Western Aphasia Battery (WAB), and Boston Naming Task (BNT) in a heterogeneous patient population. Result: Linguistic features including lexical and syntactic features were significantly correlated with clinical assessments in patients, across diagnoses. Lower MMSE and DRS scores were associated with the use of shorter words and fewer prepositional phrases. Increased impairment on WAB and BNT was correlated with the use of fewer nouns but more pronouns. Patients also differed from healthy adults as their speech duration was significantly shorter with more pauses. Conclusion: Linguistic changes such as the use of simpler vocabularies and syntax were detectable in patients with different neurodegenerative diseases and correlated with cognitive decline. Speech has the potential to be a sensitive measure for detecting cognitive impairments across various neurodegenerative diseases.

Cognitive components of aging-related increase in word-finding difficulty.

Word-finding difficulty (WFD) is a common cognitive complaint in aging, manifesting both in natural speech and in controlled laboratory tests. Various theories of cognitive aging have addressed WFD, and understanding its underlying mechanisms can help to clarify whether it has diagnostic value for neurodegenerative disease. Two influential "information-universal" theories attribute it to rather broad changes in cognition. The processing speed theory posits a general slowdown of all cognitive processes, while the inhibitory deficit hypothesis (IDH) predicts a specific problem in suppressing irrelevant information. One "information specific" theory of language production, the transmission deficit hypothesis (TDH), posits a breakdown in retrieval of phonological word forms from a corresponding lemma. To adjudicate between these accounts, we administered an online gamified covert naming task featuring picture-word interference (PWI), previously validated to elicit similar semantic interference and phonological facilitation effects as overt naming tasks. 125 healthy adults aged 18 to 85 completed the task, along with a battery of executive function tasks and a naturalistic speech sample to quantify WFD in connected speech. PWI effects provided strong support for the TDH but limited support for IDH, in that semantic interference increased and phonological facilitation decreased across the lifespan. However, neither of these effects on single-word retrieval associated with WFD measured in connected speech. Rather, overall reaction time for word retrieval (controlling for psychomotor slowing) was the best predictor of spontaneous WFD and executive function decline, suggesting processing speed as the key factor, and that verbal reaction time may be an important clinical measure.

Detecting bulbar amyotrophic lateral sclerosis (ALS) using automatic acoustic analysis.

Automatic speech assessments have the potential to dramatically improve ALS clinical practice and facilitate patient stratification for ALS clinical trials. Acoustic speech analysis has demonstrated the ability to capture a variety of relevant speech motor impairments, but implementation has been hindered by both the nature of lab-based assessments (requiring travel and time for patients) and also by the opacity of some acoustic feature analysis methods. These challenges and others have obscured the ability to distinguish different ALS disease stages/severities. Validation of automated acoustic analysis tools could enable detection of early signs of ALS, and these tools could be deployed to screen and monitor patients without requiring clinic visits. Here, we sought to determine whether acoustic features gathered using an automated assessment app could detect ALS as well as different levels of speech impairment severity resulting from ALS. Speech samples (readings of a standardized, 99-word passage) from 119 ALS patients with varying degrees of disease severity as well as 22 neurologically healthy participants were analyzed, and 53 acoustic features were extracted. Patients were stratified into early and late stages of disease (ALS-early/ALS-E and ALS-late/ALS-L) based on the ALS Functional Ratings Scale-Revised bulbar score (FRS-bulb) (median [interquartile range] of FRS-bulbar scores: 11[3]). The data were analyzed using a sparse Bayesian logistic regression classifier. It was determined that the current relatively small set of acoustic features could distinguish between ALS and controls well (area under receiver-operating characteristic curve/AUROC = 0.85), that the ALS-E patients could be separated well from control participants (AUROC = 0.78), and that ALS-E and ALS-L patients could be reasonably separated (AUROC = 0.70). These results highlight the potential for automated acoustic analyses to detect and stratify ALS.

Impaired perceptual discrimination of complex objects in older adults at risk for dementia.

Tau pathology accumulates in the perirhinal cortex (PRC) of the medial temporal lobe (MTL) during the earliest stages of the Alzheimer's disease (AD), appearing decades before clinical diagnosis. Here, we leveraged perceptual discrimination tasks that target PRC function to detect subtle cognitive impairment even in nominally healthy older adults. Older adults who did not have a clinical diagnosis or subjective memory complaints were categorized into "at-risk" (score <26; n = 15) and "healthy" (score ≥26; n = 23) groups based on their performance on the Montreal Cognitive Assessment. The task included two conditions known to recruit the PRC: faces and complex objects (greebles). A scene condition, known to recruit the hippocampus, and a size control condition that does not rely on the MTL were also included. Individuals in the at-risk group were less accurate than those in the healthy group for discriminating greebles. Performance on either the face or size control condition did not predict group status above and beyond that of the greeble condition. Visual discrimination tasks that are sensitive to PRC function may detect early cognitive decline associated with AD.

Validation of automated pipeline for the assessment of a motor speech disorder in amyotrophic lateral sclerosis (ALS).

Background and objective: Amyotrophic lateral sclerosis (ALS) frequently causes speech impairments, which can be valuable early indicators of decline. Automated acoustic assessment of speech in ALS is attractive, and there is a pressing need to validate such tools in line with best practices, including analytical and clinical validation. We hypothesized that data analysis using a novel speech assessment pipeline would correspond strongly to analyses performed using lab-standard practices and that acoustic features from the novel pipeline would correspond to clinical outcomes of interest in ALS. Methods: We analyzed data from three standard speech assessment tasks (i.e., vowel phonation, passage reading, and diadochokinesis) in 122 ALS patients. Data were analyzed automatically using a pipeline developed by Winterlight Labs, which yielded 53 acoustic features. First, for analytical validation, data were analyzed using a lab-standard analysis pipeline for comparison. This was followed by univariate analysis (Spearman correlations between individual features in Winterlight and in-lab datasets) and multivariate analysis (sparse canonical correlation analysis (SCCA)). Subsequently, clinical validation was performed. This included univariate analysis (Spearman correlation between automated acoustic features and clinical measures) and multivariate analysis (interpretable autoencoder-based dimensionality reduction). Results: Analytical validity was demonstrated by substantial univariate correlations (Spearman's ρ > 0.70) between corresponding pairs of features from automated and lab-based datasets, as well as interpretable SCCA feature groups. Clinical validity was supported by strong univariate correlations between automated features and clinical measures (Spearman's ρ > 0.70), as well as associations between multivariate outputs and clinical measures. Conclusion: This novel, automated speech assessment feature set demonstrates substantial promise as a valid tool for analyzing impaired speech in ALS patients and for the further development of these technologies.

How Much Speech Data Is Needed for Tracking Language Change in Alzheimer's Disease? A Comparison of Random Length, 5-Min, and 1-Min Spontaneous Speech Samples.

Introduction: Changes in speech can act as biomarkers of cognitive decline in Alzheimer's disease (AD). While shorter speech samples would promote data collection and analysis, the minimum length of informative speech samples remains debated. This study aims to provide insight into the effect of sample length in analyzing longitudinal recordings of spontaneous speech in AD by comparing the original random length, 5- and 1-minute-long samples. We hope to understand whether capping the audio improves the accuracy of the analysis, and whether an extra 4 min conveys necessary information. Methods: 110 spontaneous speech samples were collected from decades of Youtube videos of 17 public figures, 9 of whom eventually developed AD. 456 language features were extracted and their text-length-sensitivity, comparability, and ability to capture change over time were analyzed across three different sample lengths. Results: Capped audio files had advantages over the random length ones. While most extracted features were statistically comparable or highly correlated across the datasets, potential effects of sample length should be acknowledged for some features. The 5-min dataset presented the highest reliability in tracking the evolution of the disease, suggesting that the 4 extra minutes do convey informative data. Conclusion: Sample length seems to play an important role in extracting the language feature values from speech and tracking disease progress over time. We highlight the importance of further research into optimal sample length and standardization of methods when studying speech in AD.

Development of a Speech-based Composite Score for Remotely Quantifying Language Changes in Frontotemporal Dementia.

BACKGROUND: Changes to speech and language are common symptoms across different subtypes of frontotemporal dementia (FTD). These changes affect the ability to communicate, impacting everyday functions. Accurately assessing these changes may help clinicians to track disease progression and detect response to treatment. OBJECTIVE: To determine which aspects of speech show significant change over time and to develop a novel composite score for tracking speech and language decline in individuals with FTD. METHOD: We recruited individuals with FTD to complete remote digital speech assessments based on a picture description task. Speech samples were analyzed to derive acoustic and linguistic measures of speech and language, which were tested for longitudinal change over the course of the study and were used to compute a novel composite score. RESULTS: Thirty-six (16 F, 20 M; M age = 61.3 years) individuals were enrolled in the study, with 27 completing a follow-up assessment 12 months later. We identified eight variables reflecting different aspects of language that showed longitudinal decline in the FTD clinical syndrome subtypes and developed a novel composite score based on these variables. The resulting composite score demonstrated a significant effect of change over time, high test-retest reliability, and a correlation with standard scores on various other speech tasks. CONCLUSION: Remote digital speech assessments have the potential to characterize speech and language abilities in individuals with FTD, reducing the burden of clinical assessments while providing a novel measure of speech and language abilities that is sensitive to disease and relevant to everyday function.

Innovative Technologies in CNS Trials: Promises and Pitfalls for Recruitment, Retention, and Representativeness.

Objective: Recruitment of a sufficiently large and representative patient sample and its retention during central nervous system (CNS) trials presents major challenges for study sponsors. Technological advances are reshaping clinical trial operations to meet these challenges, and the COVID-19 pandemic further accelerated this development. Method of Research: The International Society for CNS Clinical Trials and Methodology (ISCTM; www.isctm.org) Innovative Technologies for CNS Trials Working Group surveyed the state of technological innovations for improved recruitment and retention and assessed their promises and pitfalls. Results: Online advertisement and electronic patient registries can enhance recruitment, but challenges with sample representativeness, conversion rates from eligible prescreening to enrolled patients, data privacy and security, and patient identification remain hurdles for optimal use of these technologies. Electronic medical records (EMR) mining with artificial intelligence (AI)/machine learning (ML) methods is promising but awaits translation into trials. During the study treatment phase, technological innovations increasingly support participant retention, including adherence with the investigational treatment. Digital tools for adherence and retention support take many forms, including patient-centric communication channels between researchers and participants, real-time study reminders, and digital behavioral interventions to increase study compliance. However, such tools add technical complexities to trials, and their impact on the generalizability of results are largely unknown. Conclusion: Overall, the group found a scarcity of systematic data directly assessing the impact of technological innovations on study recruitment and retention in CNS trials, even for strategies with already high adoption, such as online recruitment. Given the added complexity and costs associated with most technological innovations, such data is needed to fully harness technologies for CNS trials and drive further adoption.

Detecting bulbar amyotrophic lateral sclerosis (ALS) using automatic acoustic analysis.

Home-based speech assessments have the potential to dramatically improve ALS clinical practice and facilitate patient stratification for ALS clinical trials. Acoustic speech analysis has demonstrated the ability to capture a variety of relevant speech motor impairments, but implementation has been hindered by both the nature of lab-based assessments (requiring travel and time for patients) and also by the opacity of some acoustic feature analysis methods. Furthermore, these challenges and others have obscured the ability to distinguish different ALS disease stages/severities. Validation of remote-capable acoustic analysis tools could enable detection of early signs of ALS, and these tools could be deployed to screen and monitor patients without requiring clinic visits. Here, we sought to determine whether acoustic features gathered using a remote-capable assessment app could detect ALS as well as different levels of speech impairment severity resulting from ALS. Speech samples (readings of a standardized, 99-word passage) from 119 ALS patients with varying degrees of disease severity as well as 22 neurologically healthy participants were analyzed, and 53 acoustic features were extracted. Patients were stratified into early and late stages of disease (ALS-early/ALS-E and ALS-late/ALS-L) based on the ALS Functional Ratings Scale - Revised bulbar score (FRS-bulb). Data were analyzed using a sparse Bayesian logistic regression classifier. It was determined that the current relatively small set of acoustic features could distinguish between ALS and controls well (area under receiver operating characteristic curve/AUROC = 0.85), that the ALS-E patients could be separated well from control participants (AUROC = 0.78), and that ALS-E and ALS-L patients could be reasonably separated (AUROC = 0.70). These results highlight the potential for remote acoustic analyses to detect and stratify ALS.

Evaluating the utility of daily speech assessments for monitoring depression symptoms.

Objective: Depression is a common mental health disorder and a major public health concern, significantly interfering with the lives of those affected. The complex clinical presentation of depression complicates symptom assessments. Day-to-day fluctuations of depression symptoms within an individual bring an additional barrier, since infrequent testing may not reveal symptom fluctuation. Digital measures such as speech can facilitate daily objective symptom evaluation. Here, we evaluated the effectiveness of daily speech assessment in characterizing speech fluctuations in the context of depression symptoms, which can be completed remotely, at a low cost and with relatively low administrative resources. Methods: Community volunteers (N = 16) completed a daily speech assessment, using the Winterlight Speech App, and Patient Health Questionnaire-9 (PHQ-9) for 30 consecutive business days. We calculated 230 acoustic and 290 linguistic features from individual's speech and investigated their relationship to depression symptoms at the intra-individual level through repeated measures analyses. Results: We observed that depression symptoms were linked to linguistic features, such as less frequent use of dominant and positive words. Greater depression symptomatology was also significantly correlated with acoustic features: reduced variability in speech intensity and increased jitter. Conclusions: Our findings support the feasibility of using acoustic and linguistic features as a measure of depression symptoms and propose daily speech assessment as a tool for better characterization of symptom fluctuations.

PROSA-a multicenter prospective observational study to develop low-burden digital speech biomarkers in ALS and FTD.

Objective: There is a need for novel biomarkers that can indicate disease state, project disease progression, or assess response to treatment for amyotrophic lateral sclerosis (ALS) and associated neurodegenerative diseases such as frontotemporal dementia (FTD). Digital biomarkers are especially promising as they can be collected non-invasively and at low burden for patients. Speech biomarkers have the potential to objectively measure cognitive, motor as well as respiratory symptoms at low-cost and in a remote fashion using widely available technology such as telephone calls. Methods: The PROSA study aims to develop and evaluate low-burden frequent prognostic digital speech biomarkers. The main goal is to create a single, easy-to-perform battery that serves as a valid and reliable proxy for cognitive, respiratory, and motor domains in ALS and FTD. The study will be a multicenter 12-months observational study aiming to include 75 ALS and 75 FTD patients as well as 50 healthy controls and build on three established longitudinal cohorts: DANCER, DESCRIBE-ALS and DESCRIBE-FTD. In addition to the extensive clinical phenotyping in DESCRIBE, PROSA collects a comprehensive speech protocol in fully remote and automated fashion over the telephone at four time points. This longitudinal speech data, together with gold standard measures, will allow advanced speech analysis using artificial intelligence for the development of speech-based phenotypes of ALS and FTD patients measuring cognitive, motor and respiratory symptoms. Conclusion: Speech-based phenotypes can be used to develop diagnostic and prognostic models predicting clinical change. Results are expected to have implications for future clinical trial stratification as well as supporting innovative trial designs in ALS and FTD.

Automated detection of progressive speech changes in early Alzheimer's disease.

Speech and language changes occur in Alzheimer's disease (AD), but few studies have characterized their longitudinal course. We analyzed open-ended speech samples from a prodromal-to-mild AD cohort to develop a novel composite score to characterize progressive speech changes. Participant speech from the Clinical Dementia Rating (CDR) interview was analyzed to compute metrics reflecting speech and language characteristics. We determined the aspects of speech and language that exhibited significant longitudinal change over 18 months. Nine acoustic and linguistic measures were combined to create a novel composite score. The speech composite exhibited significant correlations with primary and secondary clinical endpoints and a similar effect size for detecting longitudinal change. Our results demonstrate the feasibility of using automated speech processing to characterize longitudinal change in early AD. Speech-based composite scores could be used to monitor change and detect response to treatment in future research. HIGHLIGHTS: Longitudinal speech samples were analyzed to characterize speech changes in early AD.Acoustic and linguistic measures showed significant change over 18 months.A novel speech composite score was computed to characterize longitudinal change.The speech composite correlated with primary and secondary trial endpoints.Automated speech analysis could facilitate remote, high frequency monitoring in AD.

The Generalizability of Longitudinal Changes in Speech Before Alzheimer's Disease Diagnosis.

BACKGROUND: Language impairment in Alzheimer's disease (AD) has been widely studied but due to limited data availability, relatively few studies have focused on the longitudinal change in language in the individuals who later develop AD. Significant differences in speech have previously been found by comparing the press conference transcripts of President Bush and President Reagan, who was later diagnosed with AD. OBJECTIVE: In the current study, we explored whether the patterns previously established in the single AD-healthy control (HC) participant pair apply to a larger group of individuals who later receive AD diagnosis. METHODS: We replicated previous methods on two larger corpora of longitudinal spontaneous speech samples of public figures, consisting of 10 and 9 AD-HC participant pairs. As we failed to find generalizable patterns of language change using previous methodology, we proposed alternative methods for data analysis, investigating the benefits of using different language features and their change with age, and compiling the single features into aggregate scores. RESULTS: The single features that showed the strongest results were moving average type:token ratio (MATTR) and pronoun-related features. The aggregate scores performed better than the single features, with lexical diversity capturing a similar change in two-thirds of the participants. CONCLUSION: Capturing universal patterns of language change prior to AD can be challenging, but the decline in lexical diversity and changes in MATTR and pronoun-related features act as promising measures that reflect the cognitive changes in many participants.

Description of the Method for Evaluating Digital Endpoints in Alzheimer Disease Study: Protocol for an Exploratory, Cross-sectional Study.

BACKGROUND: More sensitive and less burdensome efficacy end points are urgently needed to improve the effectiveness of clinical drug development for Alzheimer disease (AD). Although conventional end points lack sensitivity, digital technologies hold promise for amplifying the detection of treatment signals and capturing cognitive anomalies at earlier disease stages. Using digital technologies and combining several test modalities allow for the collection of richer information about cognitive and functional status, which is not ascertainable via conventional paper-and-pencil tests. OBJECTIVE: This study aimed to assess the psychometric properties, operational feasibility, and patient acceptance of 10 promising technologies that are to be used as efficacy end points to measure cognition in future clinical drug trials. METHODS: The Method for Evaluating Digital Endpoints in Alzheimer Disease study is an exploratory, cross-sectional, noninterventional study that will evaluate 10 digital technologies' ability to accurately classify participants into 4 cohorts according to the severity of cognitive impairment and dementia. Moreover, this study will assess the psychometric properties of each of the tested digital technologies, including the acceptable range to assess ceiling and floor effects, concurrent validity to correlate digital outcome measures to traditional paper-and-pencil tests in AD, reliability to compare test and retest, and responsiveness to evaluate the sensitivity to change in a mild cognitive challenge model. This study included 50 eligible male and female participants (aged between 60 and 80 years), of whom 13 (26%) were amyloid-negative, cognitively healthy participants (controls); 12 (24%) were amyloid-positive, cognitively healthy participants (presymptomatic); 13 (26%) had mild cognitive impairment (predementia); and 12 (24%) had mild AD (mild dementia). This study involved 4 in-clinic visits. During the initial visit, all participants completed all conventional paper-and-pencil assessments. During the following 3 visits, the participants underwent a series of novel digital assessments. RESULTS: Participant recruitment and data collection began in June 2020 and continued until June 2021. Hence, the data collection occurred during the COVID-19 pandemic (SARS-CoV-2 virus pandemic). Data were successfully collected from all digital technologies to evaluate statistical and operational performance and patient acceptance. This paper reports the baseline demographics and characteristics of the population studied as well as the study's progress during the pandemic. CONCLUSIONS: This study was designed to generate feasibility insights and validation data to help advance novel digital technologies in clinical drug development. The learnings from this study will help guide future methods for assessing novel digital technologies and inform clinical drug trials in early AD, aiming to enhance clinical end point strategies with digital technologies. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/35442.

Using Digital Speech Assessments to Detect Early Signs of Cognitive Impairment.

Detecting early signs of cognitive decline is crucial for early detection and treatment of Alzheimer's Disease. Most of the current screening tools for Alzheimer's Disease represent a significant burden, requiring invasive procedures, or intensive and costly clinical testing. Recent findings have highlighted changes to speech and language patterns that occur in Alzheimer's Disease, and may be detectable prior to diagnosis. Automated tools to assess speech have been developed that can be used on a smartphone or tablet, from one's home, in under 10 min. In this study, we present the results of a study of older adults who completed a digital speech assessment task over a 6-month period. Participants were grouped according to those who scored above (N = 18) or below (N = 18) the recommended threshold for detecting cognitive impairment on the Montreal Cognitive Assessment (MoCA) and those with diagnoses of mild cognitive impairment (MCI) or early Alzheimer's Disease (AD) (N = 14). Older adults who scored above the MoCA threshold had better performance on speech composites reflecting language coherence, information richness, syntactic complexity, and word finding abilities. Those with MCI and AD showed more rapid decline in the coherence of language from baseline to 6-month follow-up, suggesting that this score may be useful both for detecting cognitive decline and monitoring change over time. This study demonstrates that automated speech assessments have potential as sensitive tools to detect early signs of cognitive impairment and monitor progression over time.

Natural Language Processing as an Emerging Tool to Detect Late-Life Depression.

Late-life depression (LLD) is a major public health concern. Despite the availability of effective treatments for depression, barriers to screening and diagnosis still exist. The use of current standardized depression assessments can lead to underdiagnosis or misdiagnosis due to subjective symptom reporting and the distinct cognitive, psychomotor, and somatic features of LLD. To overcome these limitations, there has been a growing interest in the development of objective measures of depression using artificial intelligence (AI) technologies such as natural language processing (NLP). NLP approaches focus on the analysis of acoustic and linguistic aspects of human language derived from text and speech and can be integrated with machine learning approaches to classify depression and its severity. In this review, we will provide rationale for the use of NLP methods to study depression using speech, summarize previous research using NLP in LLD, compare findings to younger adults with depression and older adults with other clinical conditions, and discuss future directions including the use of complementary AI strategies to fully capture the spectrum of LLD.

Hepatic Encephalopathy is Associated With Slow Speech on Objective Assessment.

INTRODUCTION: There are no available low-burden, point-of-care tests to diagnose, grade, and predict hepatic encephalopathy (HE). METHODS: We evaluated speech as a biomarker of HE in 76 English-speaking adults with cirrhosis. RESULTS: Three speech features significantly correlated with the following neuropsychiatric scores: speech rate, word duration, and use of particles. Patients with low neuropsychiatric scores had slower speech (22 words/min, P = 0.01), longer word duration (0.09 seconds/word, P = 0.01), and used fewer particles (0.85% fewer, P = 0.01). Patients with a history of overt HE had slower speech (23 words/min, P = 0.005) and longer word duration (0.09 seconds/word, P = 0.005). DISCUSSION: HE is associated with slower speech.

Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer's dementia.

BACKGROUND: Language impairment is an important marker of neurodegenerative disorders. Despite this, there is no universal system of terminology used to describe these impairments and large inter-rater variability can exist between clinicians assessing language. The use of natural language processing (NLP) and automated speech analysis (ASA) is emerging as a novel and potentially more objective method to assess language in individuals with mild cognitive impairment (MCI) and Alzheimer's dementia (AD). No studies have analyzed how variables extracted through NLP and ASA might also be correlated to language impairments identified by a clinician. METHODS: Audio recordings (n=30) from participants with AD, MCI, and controls were rated by clinicians for word-finding difficulty, incoherence, perseveration, and errors in speech. Speech recordings were also transcribed, and linguistic and acoustic variables were extracted through NLP and ASA. Correlations between clinician-rated speech characteristics and the variables were compared using Spearman's correlation. Exploratory factor analysis was applied to find common factors between variables for each speech characteristic. RESULTS: Clinician agreement was high in three of the four speech characteristics: word-finding difficulty (ICC = 0.92, p<0.001), incoherence (ICC = 0.91, p<0.001), and perseveration (ICC = 0.88, p<0.001). Word-finding difficulty and incoherence were useful constructs at distinguishing MCI and AD from controls, while perseveration and speech errors were less relevant. Word-finding difficulty as a construct was explained by three factors, including number and duration of pauses, word duration, and syntactic complexity. Incoherence was explained by two factors, including increased average word duration, use of past tense, and changes in age of acquisition, and more negative valence. CONCLUSIONS: Variables extracted through automated acoustic and linguistic analysis of MCI and AD speech were significantly correlated with clinician ratings of speech and language characteristics. Our results suggest that correlating NLP and ASA with clinician observations is an objective and novel approach to measuring speech and language changes in neurodegenerative disorders.

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech.

Introduction: Research related to the automatic detection of Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional diagnostic methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing, and machine learning provide promising techniques for reliably detecting AD. There has been a recent proliferation of classification models for AD, but these vary in the datasets used, model types and training and testing paradigms. In this study, we compare and contrast the performance of two common approaches for automatic AD detection from speech on the same, well-matched dataset, to determine the advantages of using domain knowledge vs. pre-trained transfer models. Methods: Audio recordings and corresponding manually-transcribed speech transcripts of a picture description task administered to 156 demographically matched older adults, 78 with Alzheimer's Disease (AD) and 78 cognitively intact (healthy) were classified using machine learning and natural language processing as "AD" or "non-AD." The audio was acoustically-enhanced, and post-processed to improve quality of the speech recording as well control for variation caused by recording conditions. Two approaches were used for classification of these speech samples: (1) using domain knowledge: extracting an extensive set of clinically relevant linguistic and acoustic features derived from speech and transcripts based on prior literature, and (2) using transfer-learning and leveraging large pre-trained machine learning models: using transcript-representations that are automatically derived from state-of-the-art pre-trained language models, by fine-tuning Bidirectional Encoder Representations from Transformer (BERT)-based sequence classification models. Results: We compared the utility of speech transcript representations obtained from recent natural language processing models (i.e., BERT) to more clinically-interpretable language feature-based methods. Both the feature-based approaches and fine-tuned BERT models significantly outperformed the baseline linguistic model using a small set of linguistic features, demonstrating the importance of extensive linguistic information for detecting cognitive impairments relating to AD. We observed that fine-tuned BERT models numerically outperformed feature-based approaches on the AD detection task, but the difference was not statistically significant. Our main contribution is the observation that when tested on the same, demographically balanced dataset and tested on independent, unseen data, both domain knowledge and pretrained linguistic models have good predictive performance for detecting AD based on speech. It is notable that linguistic information alone is capable of achieving comparable, and even numerically better, performance than models including both acoustic and linguistic features here. We also try to shed light on the inner workings of the more black-box natural language processing model by performing an interpretability analysis, and find that attention weights reveal interesting patterns such as higher attribution to more important information content units in the picture description task, as well as pauses and filler words. Conclusion: This approach supports the value of well-performing machine learning and linguistically-focussed processing techniques to detect AD from speech and highlights the need to compare model performance on carefully balanced datasets, using consistent same training parameters and independent test datasets in order to determine the best performing predictive model.

Evaluation of Speech-Based Digital Biomarkers: Review and Recommendations.

Speech represents a promising novel biomarker by providing a window into brain health, as shown by its disruption in various neurological and psychiatric diseases. As with many novel digital biomarkers, however, rigorous evaluation is currently lacking and is required for these measures to be used effectively and safely. This paper outlines and provides examples from the literature of evaluation steps for speech-based digital biomarkers, based on the recent V3 framework (Goldsack et al., 2020). The V3 framework describes 3 components of evaluation for digital biomarkers: verification, analytical validation, and clinical validation. Verification includes assessing the quality of speech recordings and comparing the effects of hardware and recording conditions on the integrity of the recordings. Analytical validation includes checking the accuracy and reliability of data processing and computed measures, including understanding test-retest reliability, demographic variability, and comparing measures to reference standards. Clinical validity involves verifying the correspondence of a measure to clinical outcomes which can include diagnosis, disease progression, or response to treatment. For each of these sections, we provide recommendations for the types of evaluation necessary for speech-based biomarkers and review published examples. The examples in this paper focus on speech-based biomarkers, but they can be used as a template for digital biomarker development more generally.