Improving Perceptual Speech Ratings: The Effects of Auditory Training on Judgments of Dysarthric Speech.

PURPOSE: Auditory training has been shown to reduce rater variability in perceptual voice assessment. Because rater variability is also a central issue in the auditory-perceptual assessment of dysarthria, this study sought to determine if training produces a meaningful change in rater reliability, criterion validity, and scaling magnitude of four features: overall speech impairment, articulatory imprecision, monotony, and slow rate. METHOD: Forty-four nonexperts randomized to training and nontraining listener groups completed a pretest and posttest. Only the former group underwent auditory training between pre- and posttests. For both testing and training, listeners rated samples from speakers with amyotrophic lateral sclerosis (ALS), speakers with Parkinson's disease (PD), and neurologically healthy control speakers using separate visual analog scales (VASs) for each of the four features. Intraclass correlation coefficients were used to compare inter- and intrarater reliability between pre- and posttest for both listener groups. For criterion validity, severity ratings from the two nonexpert listener groups were compared to those of two experienced listeners for all four features. To determine changes in scaling magnitude, raw VAS scores for each feature were compared from pre- to posttest within the two nonexpert listener groups. Scaling changes were also compared between the two listener groups for the pre- and posttest conditions. RESULTS AND CONCLUSIONS: In the training group, a meaningful improvement in interrater reliability was observed for some features in all three speaker groups, but not in the nontraining group. In contrast, for intrarater reliability, in the nontraining group, a meaningful improvement was observed for many features in all three speaker groups, but only for PD monotony and slow rate in the training group. All ratings from the nonexpert listeners were valid except for monotony. Raw VAS scores did not meaningfully change from pre- to posttest for any of the features, but there was a trend toward lower scores posttraining, mainly for the ALS samples. Modifications to the auditory training paradigm to further improve reliability and validity, along with future goals for optimizing training, are discussed.

Articulatory Performance in Dysarthria: Using a Data-Driven Approach to Estimate Articulatory Demands and Deficits.

This study pursued two goals: (1) to establish range of motion (ROM) demand tiers (i.e., low, moderate, high) specific to the jaw (J), lower lip (LL), posterior tongue (PT), and anterior tongue (AT) for multisyllabic words based on the articulatory performance of neurotypical talkers and (2) to identify demand- and disease-specific articulatory performance characteristics in talkers with amyotrophic lateral sclerosis (ALS) and Parkinson's disease (PD). J, LL, PT, and AT movements of 12 talkers with ALS, 12 talkers with PD, and 12 controls were recorded using electromagnetic articulography. Vertical ROM, average speed, and movement duration were measured. Results showed that in talkers with PD, J and LL ROM were already significantly reduced at the lowest tier whereas PT and AT ROM were only significantly reduced at moderate and high tiers. In talkers with ALS, J ROM was significantly reduced at the moderate tier whereas LL, PT, and AT ROM were only significantly reduced at the highest tier. In both clinical groups, significantly reduced J and LL speeds could already be observed at the lowest tier whereas significantly reduced AT speeds could only be observed at the highest tier. PT speeds were already significantly reduced at the lowest tier in the ALS group but not until the moderate tier in the PD group. Finally, movement duration, but not ROM or speed performance, differentiated between ALS and PD even at the lowest tier. Results suggest that articulatory deficits vary with stimuli-specific motor demands across articulators and clinical groups.

Articulatory Correlates of Stress Pattern Disturbances in Talkers With Dysarthria.

Purpose Reduced stress commonly occurs in talkers with Parkinson's disease (PD), whereas excessive and equal stress is frequently associated with dysarthria of talkers with amyotrophic lateral sclerosis (ALS) and multiple sclerosis (MS). This study sought to identify articulatory impairment patterns that underlie these two impaired stress patterns. We further aimed to determine if talkers with the same stress pattern disturbance but different diseases (ALS and MS) exhibit disease-specific articulatory deficits. Method Fifty-seven talkers participated in the study-33 talkers with dysarthria and 24 controls. Talkers with dysarthria were grouped based on their medical diagnosis: PD (n = 15), ALS (n = 10), MS (n = 8). Participants repeated target words embedded in a carrier phrase. Kinematic data were recorded using electromagnetic articulography. Duration, displacement, peak speed, stiffness, time-to-peak speed, and parameter c were extracted for the initial lower lip opening stroke of each target word, which was either stressed or unstressed. Results Stress effects were significant for all kinematic measures across groups except for stiffness and time-to-peak speed, which were nonsignificant in ALS. For comparisons with controls, more kinematic measures significantly differed in the ALS group than in the PD and MS groups. Additionally, ALS and MS showed mostly similar articulatory impairment patterns. Conclusions In general, significant stress effects were observed in talkers with dysarthria. However, stress-specific between-group differences in articulatory performance, particularly displacement, may explain the perceptual impression of disturbed stress patterns. Furthermore, similar findings for ALS and MS suggest that articulatory deficits underlying similar stress pattern disturbances are not disease-specific.

An exploratory model of speech intelligibility for healthy aging based on phonatory and articulatory measures.

PURPOSE: The aims of the current study were to determine age-related changes to the phonatory and articulatory subsystems and to investigate an exploratory model of intelligibility for healthy aging based on phonatory and articulatory measures. METHOD: Fifteen healthy, older adults (55-81 years) and 15 younger adults (20-35 years) participated in instrumental assessments of the phonatory (aerodynamic, acoustic) and articulatory (kinematic) subsystems. Speech intelligibility was determined by five listeners during multi-talker babble. RESULTS: Older adults displayed shorter maximum phonation time, greater airflow during sentence reading, and lower cepstral peak prominence (CPP) and CPP SD. Additionally, older adults had slower tongue movement speed than younger adults. Speech intelligibility was also significantly reduced in the older group. A generalized estimating equations model combining phonatory and articulatory measures showed that CPP SD, low/high (L/H) spectral ratio mean and SD, Cepstral Spectral Index of Dysphonia (CSID), and maximum tongue movement speed were significant contributors to intelligibility changes in older individuals. While L/H mean and SD and CSID displayed an inverse relationship with intelligibility, CPP SD and maximum tongue speed displayed a direct relationship with intelligibility. DISCUSSION: Aging affects the phonatory and articulatory subsystems with implications for speech intelligibility. Phonatory cepstral/spectral measures (except mean CPP) were associated with speech intelligibility changes, suggesting that changes in voice quality may contribute to reduced intelligibility in older adults. Pertaining to articulation, slower tongue movement speed likely contributed to reduced intelligibility in older individuals.

Detection of Articulatory Deficits in Parkinson's Disease: Can Systematic Manipulations of Phonetic Complexity Help?

Purpose This study sought to determine the feasibility of using phonetic complexity manipulations as a way to systematically assess articulatory deficits in talkers with progressive dysarthria due to Parkinson's disease (PD). Method Articulatory kinematics were recorded using three-dimensional electromagnetic articulography from 15 talkers with PD (58-84 years old) and 15 healthy controls (55-80 years old) while they produced target words embedded in a carrier phrase. Majority of the talkers with PD exhibited a relatively mild dysarthria. For stimuli selection, phonetic complexity was calculated for a variety of words using the framework proposed by Kent (1992), and six words representative of low, medium, and high phonetic complexity were selected as targets. Jaw, posterior tongue, and anterior tongue kinematic measures that were used to test for phonetic complexity effects included movement speed, cumulative path distance, movement range, movement duration, and spatiotemporal variability. Results Significantly smaller movements and slower movement speeds were evident in talkers with PD, predominantly for words with high phonetic complexity. The effect sizes of between-groups differences were larger for several jaw kinematic measures than those of the tongue. Discussion and Conclusion Findings suggest that systematic manipulations of phonetic complexity can support the detection of articulatory deficits in talkers with PD. Phonetic complexity should therefore be leveraged for the assessment of articulatory performance in talkers with progressive dysarthria. Future work will be directed toward linking speech kinematic and auditory-perceptual measures to determine the clinical significance of the current findings.

Voice Conversion for Persons with Amyotrophic Lateral Sclerosis.

Amyotrophic lateral sclerosis (ALS) results in progressive paralysis of voluntary muscles throughout the body. As speech deteriorates, individuals rely on pre-programmed messages available on commercial speech generating devices to communicate using one of the generic electronic voices on the device. To replace these generic voices and restore vocal identity, our aim is to develop personalized voices for people with ALS via the approach of voice conversion. The task is challenging because very few people have large quantities of their premorbid healthy speech recorded. Therefore, we have to rely on small quantities of dysarthric speech concomitant with an individual's disease stage. Further, progressive fatigue prohibits acquisition of large speech datasets and individuals display a range of dysarthria severities resulting from breathing, voice, articulation, resonance, and prosody disturbances. As the first step to address these problems, we use healthy source speakers and propose the approach of combining a structured sparse spectral transform with multiple linear regression-based frequency warping prediction for spectral conversion, and interpolating the transformed spectral frames for speech rate modification. Our experimental data included four healthy source speakers from the ARCTIC dataset, and four target ALS speakers with mild to severe dysarthria, forming 16 speaker pairs. Subjective listening evaluations showed that on average, (i) the proposed approach improved speech intelligibility by about 80% over the target speakers' speech, (ii) the converted voice was 3 times more similar to the target speakers' speech than to the source speakers' speech, and (iii) the converted speech quality was close to the MOS scale "good" relative to the source speakers' speech being "excellent."

A Comparative Study of Auditory-Perceptual Speech Measures for the Early Detection of Mild Speech Impairments.

BACKGROUND AND PURPOSE: The aim of the study was to compare transcription-based speech intelligibility and scaled speech severity for the detection of mild speech impairments, by studying these metrics across talkers with Parkinson's disease (PD), age- and sex-matched older adults, and younger adults. An additional aim was to determine the impact of listener experience on these clinical measures. METHODS: Fifteen speakers from each experimental group were asked to read aloud 11 randomly generated sentences from the Speech Intelligibility Test at their typical speaking rate and loudness. Two groups of four listeners each, stratified as experienced or inexperienced listeners based on their clinical experience, judged the sentence samples. To estimate intelligibility, both listener groups were asked to orthographically transcribe exactly what they heard for each sentence. For severity estimates, the listener groups were asked to rate the sentences for overall quality based on voice, resonance, articulation, and prosody, using a visual analog scale. RESULTS: Transcription-based intelligibility and scaled severity scores of the PD group differed significantly from those of the older and younger adults. Between-age group differences in intelligibility and scaled severity were not observed. Listener experience had an impact on scaled speech severity, but not speech intelligibility. Between-group differences in speech severity were driven by the inexperienced group and not the experienced listener group. IMPLICATIONS: Both transcription-based intelligibility and scaled severity estimates appear to be sensitive to relatively mild speech impairments in PD. Obtaining scaled severity is less labor intensive than transcription; therefore, visual analog scaling may be the preferred paradigm for clinical use. However, listener experience and training are important considerations for scaling techniques to be implemented clinically.

DeepDDK: A Deep Learning based Oral-Diadochokinesis Analysis Software.

Oromotor dysfunction caused by neurological disorders can result in significant speech and swallowing impairments. Current diagnostic methods to assess oromotor function are subjective and rely on perceptual judgments by clinicians. In particular, the widely used oral-diadochokinesis (oral-DDK) test, which requires rapid, alternate repetitions of speech-based syllables, is conducted and interpreted differently among clinicians. It is therefore prone to inaccuracy, which results in poor test reliability and poor clinical application. In this paper, we present a deep learning based software to extract quantitative data from the oral DDK signal, thereby transforming it into an objective diagnostic and treatment monitoring tool. The proposed software consists of two main modules: a fully automated syllable detection module and an interactive visualization and editing module that allows inspection and correction of automated syllable units. The DeepDDK software was evaluated on speech files corresponding to 9 different DDK syllables (e.g., "Pa", "Ta", "Ka"). The experimental results show robustness of both syllable detection and localization across different types of DDK speech tasks.

A Phonetic Complexity-Based Approach for Intelligibility and Articulatory Precision Testing: A Preliminary Study on Talkers With Amyotrophic Lateral Sclerosis.

Purpose: This study describes a phonetic complexity-based approach for speech intelligibility and articulatory precision testing using preliminary data from talkers with amyotrophic lateral sclerosis. Method: Eight talkers with amyotrophic lateral sclerosis and 8 healthy controls produced a list of 16 low and high complexity words. Sixty-four listeners judged the samples for intelligibility, and 2 trained listeners completed phoneme-level analysis to determine articulatory precision. To estimate percent intelligibility, listeners orthographically transcribed each word, and the transcriptions were scored as being either accurate or inaccurate. Percent articulatory precision was calculated based on the experienced listeners' judgments of phoneme distortions, deletions, additions, and/or substitutions for each word. Articulation errors were weighted based on the perceived impact on intelligibility to determine word-level precision. Results: Between-groups differences in word intelligibility and articulatory precision were significant at lower levels of phonetic complexity as dysarthria severity increased. Specifically, more severely impaired talkers showed significant reductions in word intelligibility and precision at both complexity levels, whereas those with milder speech impairments displayed intelligibility reductions only for more complex words. Articulatory precision was less sensitive to mild dysarthria compared to speech intelligibility for the proposed complexity-based approach. Conclusions: Considering phonetic complexity for dysarthria tests could result in more sensitive assessments for detecting and monitoring dysarthria progression.

Structured Sparse Spectral Transforms and Structural Measures for Voice Conversion.

We investigate a structured sparse spectral transform method for voice conversion (VC) to perform frequency warping and spectral shaping simultaneously on high-dimensional (D) STRAIGHT spectra. Learning a large transform matrix for high-D data often results in an overfit matrix with low sparsity, which leads to muffled speech in VC. We address this problem by using the frequency-warping characteristic of a source-target speaker pair to define a region of support (ROS) in a transform matrix, and further optimize it by nonnegative matrix factorization (NMF) to obtain structured sparse transform. We also investigate structural measures of spectral and temporal covariance and variance at different scales for assessing VC speech quality. Our experiments on ARCTIC dataset of 12 speaker pairs show that embedding the ROS in spectral transforms offers flexibility in tradeoffs between spectral distortion and structure preservation, and the structural measures provide quantitatively reasonable results on converted speech. Our subjective listening tests show that the proposed VC method achieves a mean opinion score of "very good" relative to natural speech, and in comparison with three other VC methods, it is the most preferred one in naturalness and in voice similarity to target speakers.

Spatiotemporal movement variability in ALS: Speaking rate effects on tongue, lower lip, and jaw motor control.

PURPOSE: Although it is frequently presumed that bulbar muscle degeneration in Amyotrophic Lateral Sclerosis (ALS) is associated with progressive loss of speech motor control, empirical evidence is limited. Furthermore, because speaking rate slows with disease progression and rate manipulations are used to improve intelligibility in ALS, this study sought to (i) determine between and within-group differences in articulatory motor control as a result of speaking rate changes and (ii) identify the strength of association between articulatory motor control and speech impairment severity. METHOD: Ten talkers with ALS and 11 healthy controls repeated the target sentence at habitual, fast, and slow rates. The spatiotemporal variability index (STI) was calculated to determine tongue, lower lip, and jaw movement variability. RESULTS: During habitual speech, talkers with mild-moderate dysarthria displayed significantly lower tongue and lip movement variability whereas those with severe dysarthria showed greater variability compared to controls. Within-group rate effects were significant only for talkers with ALS. Specifically, lip and tongue movement variability significantly increased during slow speech relative to habitual and fast speech. Finally, preliminary associations between speech impairment severity and movement variability were moderate to strong in talkers with ALS. CONCLUSION: Between-group differences for habitual speech and within-group effects for slow speech replicated previous findings for lower lip and jaw movements. Preliminary findings of moderate to strong associations between speech impairment severity and STI suggest that articulatory variability may vary from pathologically low (possibly indicating articulatory compensation) to pathologically high variability (possibly indicating loss of control) with dysarthria progression in ALS.

An investigation of clear speech effects on articulatory kinematics in talkers with ALS.

Amyotrophic lateral sclerosis (ALS) affects the speech motor system and causes a substantial decline in intelligibility. Clear speech strategies have been found to benefit intelligibility in talkers with dysarthria and the purpose of this study was to examine clear speech effects on tongue, lip, and jaw movements in persons with ALS and healthy controls. For this purpose, participants produced 10 repetitions of the sentence 'I owe you a yoyo' using both habitual and clear speech. Movement variability as well as maximum speed, range, and duration of movement of the articulators were obtained for both conditions. The results show significantly lower jaw movement variability during habitual speech but greater variability for clear speech in ALS relative to controls. The kinematic data was consistent with expectations for individuals with dysarthria. Based on these results, clear speech may be considered a mode of speech that produces high movement variability in talkers with ALS.