Sentence recognition with modulation-filtered speech segments for younger and older adults: Effects of hearing impairment and cognition.

This study investigated word recognition for sentences temporally filtered within and across acoustic-phonetic segments providing primarily vocalic or consonantal cues. Amplitude modulation was filtered at syllabic (0-8 Hz) or slow phonemic (8-16 Hz) rates. Sentence-level modulation properties were also varied by amplifying or attenuating segments. Participants were older adults with normal or impaired hearing. Older adult speech recognition was compared to groups of younger normal-hearing adults who heard speech unmodified or spectrally shaped with and without threshold matching noise that matched audibility to hearing-impaired thresholds. Participants also completed cognitive and speech recognition measures. Overall, results confirm the primary contribution of syllabic speech modulations to recognition and demonstrate the importance of these modulations across vowel and consonant segments. Group differences demonstrated a hearing loss-related impairment in processing modulation-filtered speech, particularly at 8-16 Hz. This impairment could not be fully explained by age or poorer audibility. Principal components analysis identified a single factor score that summarized speech recognition across modulation-filtered conditions; analysis of individual differences explained 81% of the variance in this summary factor among the older adults with hearing loss. These results suggest that a combination of cognitive abilities and speech glimpsing abilities contribute to speech recognition in this group.

Atypical vocal quality in women with the FMR1 premutation: an indicator of impaired sensorimotor control.

Women with the FMR1 premutation are susceptible to motor involvement related to atypical cerebellar function, including risk for developing fragile X tremor ataxia syndrome. Vocal quality analyses are sensitive to subtle differences in motor skills but have not yet been applied to the FMR1 premutation. This study examined whether women with the FMR1 premutation demonstrate differences in vocal quality, and whether such differences relate to FMR1 genetic, executive, motor, or health features of the FMR1 premutation. Participants included 35 women with the FMR1 premutation and 45 age-matched women without the FMR1 premutation who served as a comparison group. Three sustained /a/ vowels were analyzed for pitch (mean F0), variability of pitch (standard deviation of F0), and overall vocal quality (jitter, shimmer, and harmonics-to-noise ratio). Executive, motor, and health indices were obtained from direct and self-report measures and genetic samples were analyzed for FMR1 CGG repeat length and activation ratio. Women with the FMR1 premutation had a lower pitch, larger pitch variability, and poorer vocal quality than the comparison group. Working memory was related to harmonics-to-noise ratio and shimmer in women with the FMR1 premutation. Vocal quality abnormalities differentiated women with the FMR1 premutation from the comparison group and were evident even in the absence of other clinically evident motor deficits. This study supports vocal quality analyses as a tool that may prove useful in the detection of early signs of motor involvement in this population.

Modeling the effect of linguistic predictability on speech intelligibility prediction.

Many existing speech intelligibility prediction (SIP) algorithms can only account for acoustic factors affecting speech intelligibility and cannot predict intelligibility across corpora with different linguistic predictability. To address this, a linguistic component was added to five existing SIP algorithms by estimating linguistic corpus predictability using a pre-trained language model. The results showed improved SIP performance in terms of correlation and prediction error over a mixture of four datasets, each with a different English open-set corpus.

Recognition of spectrally shaped speech in speech-modulated noise: Effects of age, spectral shape, speech level, and vocoding.

This study examined the recognition of spectrally shaped syllables and sentences in speech-modulated noise by younger and older adults. The effect of spectral shaping and speech level on temporal amplitude modulation cues was explored through speech vocoding. Subclinical differences in hearing thresholds in older adults were controlled using threshold matching noise. Older, compared to younger, adults had poorer recognition but similar improvements as the bandwidth of the shaping function increased. Spectral shaping may enhance the sensation level of glimpsed speech, which improves speech recognition in noise, even with mild elevations in hearing thresholds.

The effect of modality onset asynchrony and processing time on the recognition of text-supplemented speech.

This study examined the effect of modality onset asynchrony and response processing time for the recognition of text-supplemented speech. Speech and text were periodically interrupted by noise or black bars, respectively, to preserve 50% of the sentence and presented in unimodal and multimodal conditions. Sentence recognition and response errors were assessed for responses made simultaneous with the stimulus or after its presentation. Increased processing time allowed for the cognitive repair of initial response errors in working memory. Text-supplemented speech was best recognized with minimal temporal asynchrony. Overall, text supplementation facilitated the recognition of degraded speech when provided sufficient processing time.

Recognition of Interrupted Speech, Text, and Text-Supplemented Speech by Older Adults: Effect of Interruption Rate.

PURPOSE: Studies of speech and text interruption indicate that the interruption rate influences the perceptual information available, from whole words at slow rates to subphonemic cues at faster interruptions rates. In young adults, the benefit obtained from text supplementation of speech may depend on the type of perceptual information available in either modality. Age commonly reduces temporal aspects of information processing, which may influence the benefit older adults obtain from text-supplemented speech across interruption rates. METHOD: Older adults were tested unimodally and multimodally with spoken and printed sentences that were interrupted by silence or white space at various rates. RESULTS: Results demonstrate U-shaped performance-rate functions for all modality conditions, with minimal performance around interruption rates of 2-4 Hz. Comparison to previous studies with younger adults indicates overall poorer recognition for interrupted materials by the older adults. However, as a group, older adults can integrate information between the two modalities to a similar degree as younger adults. Individual differences in multimodal integration were noted. CONCLUSION: Overall, these results indicate that older adults, while demonstrating poorer overall performance in comparison to younger adults, successfully combine distributed partial information across speech and text modalities to facilitate sentence recognition.

Spectro-temporal modulation glimpsing for speech intelligibility prediction.

We compare two alternative speech intelligibility prediction algorithms: time-frequency glimpse proportion (GP) and spectro-temporal glimpsing index (STGI). Both algorithms hypothesize that listeners understand speech in challenging acoustic environments by "glimpsing" partially available information from degraded speech. GP defines glimpses as those time-frequency regions whose local signal-to-noise ratio is above a certain threshold and estimates intelligibility as the proportion of the time-frequency regions glimpsed. STGI, on the other hand, applies glimpsing to the spectro-temporal modulation (STM) domain and uses a similarity measure based on the normalized cross-correlation between the STM envelopes of the clean and degraded speech signals to estimate intelligibility as the proportion of the STM channels glimpsed. Our experimental results demonstrate that STGI extends the notion of glimpsing proportion to a wider range of distortions, including non-linear signal processing, and outperforms GP for the additive uncorrelated noise datasets we tested. Furthermore, the results show that spectro-temporal modulation analysis enables STGI to account for the effects of masker type on speech intelligibility, leading to superior performance over GP in modulated noise datasets.

Perception of interrupted speech and text: Listener and modality factors.

Interrupted speech and text are used to measure processes of linguistic closure that are important for recognition under adverse backgrounds. The present study compared recognition of speech and text that had been periodically interrupted with matched amounts of silence or white space, respectively. Recognition thresholds were obtained for younger and older adults with normal or simulated/impaired hearing and correlated with recognition of speech-in-babble. Results demonstrate domain-general, age-related processes in linguistic closure affecting high context sentences and domain-specific, hearing-related processes in speech recognition affecting low context sentences. Text recognition captures domain-general linguistic processes in speech recognition susceptible to age-related effects.

Phonological and semantic similarity of misperceived words in babble: Effects of sentence context, age, and hearing loss.

This study investigated how age and hearing loss influence the misperceptions made when listening to sentences in babble. Open-set responses to final words in sentences with low and high context were analyzed for younger adults with normal hearing and older adults with normal or impaired hearing. All groups performed similarly in overall accuracy but differed in error type. Misperceptions for all groups were analyzed according to phonological and semantic properties. Comparisons between groups indicated that misperceptions for older adults were more influenced by phonological factors. Furthermore, older adults with hearing loss omitted more responses. Overall, across all groups, results suggest that phonological confusions most explain misperceptions in low context sentences. In high context sentences, the meaningful sentence context appears to provide predictive cues that reduce misperceptions. When misperceptions do occur, responses tend to have greater semantic similarity and lesser phonological similarity to the target, compared to low context sentences. In this way, semantic similarity may index a postdictive process by which ambiguities due to phonological confusions are resolved to conform to the semantic context of the sentence. These patterns demonstrate that context, age, and hearing loss affect the misperceptions, and potential sentence interpretation, made when listening to sentences in babble.

Older adult recognition error patterns when listening to interrupted speech and speech in steady-state noise.

This study examined sentence recognition errors made by older adults in degraded listening conditions compared to a previous sample of younger adults. We examined speech recognition errors made by older normal-hearing adults who repeated sentences that were corrupted by steady-state noise (SSN) or periodically interrupted by noise to preserve 33%, 50%, or 66% of the sentence. Responses were transcribed and coded for the number and type of keyword errors. Errors increased with decreasing preservation of the sentence. Similar sentence recognition was observed between SSN and the greatest amount of interruption (33%). Errors were predominately at the word level rather than at the phoneme level and consisted of omission or substitution of keywords. Compared to younger listeners, older listeners made more total errors and omitted more whole words when speech was highly degraded. They also made more whole word substitutions when speech was more preserved. In addition, the semantic relatedness of the substitution errors to the sentence context varied according to the distortion condition, with greater context effects in SSN than interruption. Overall, older listeners made errors reflecting poorer speech representations. Error analyses provide a more detailed account of speech recognition by identifying changes in the type of errors made across listening conditions and listener groups.

Glimpsing keywords across sentences in noise: A microstructural analysis of acoustic, lexical, and listener factors.

This study investigated how acoustic and lexical word-level factors and listener-level factors of auditory thresholds and cognitive-linguistic processing contribute to the microstructure of sentence recognition in unmodulated and speech-modulated noise. The modulation depth of the modulated masker was changed by expanding and compressing the temporal envelope to control glimpsing opportunities. Younger adults with normal hearing (YNH) and older adults with normal and impaired hearing were tested. A second group of YNH was tested under acoustically identical conditions to the hearing-impaired group, who received spectral shaping. For all of the groups, speech recognition declined and masking release increased for later keywords in the sentence, which is consistent with the word position decreases in the signal-to-noise ratio. The acoustic glimpse proportion and lexical word frequency of individual keywords predicted recognition under different noise conditions. For the older adults, better auditory thresholds and better working memory abilities facilitated sentence recognition. Vocabulary knowledge contributed more to sentence recognition for younger than for older adults. These results demonstrate that acoustic and lexical factors contribute to the recognition of individual words within a sentence, but relative contributions vary based on the noise modulation characteristics. Taken together, acoustic, lexical, and listener factors contribute to how individuals recognize keywords during sentences.

Speech Intelligibility Prediction using Spectro-Temporal Modulation Analysis.

Spectro-temporal modulations are believed to mediate the analysis of speech sounds in the human primary auditory cortex. Inspired by humans' robustness in comprehending speech in challenging acoustic environments, we propose an intrusive speech intelligibility prediction (SIP) algorithm, wSTMI, for normal-hearing listeners based on spectro-temporal modulation analysis (STMA) of the clean and degraded speech signals. In the STMA, each of 55 modulation frequency channels contributes an intermediate intelligibility measure. A sparse linear model with parameters optimized using Lasso regression results in combining the intermediate measures of 8 of the most salient channels for SIP. In comparison with a suite of 10 SIP algorithms, wSTMI performs consistently well across 13 datasets, which together cover degradation conditions including modulated noise, noise reduction processing, reverberation, near-end listening enhancement, and speech interruption. We show that the optimized parameters of wSTMI may be interpreted in terms of modulation transfer functions of the human auditory system. Thus, the proposed approach offers evidence affirming previous studies of the perceptual characteristics underlying speech signal intelligibility.

Probing Phonological Processing Differences in Nonword Repetition for Children with Separate or Co-occurring Dyslexia and Developmental Language Disorder.

We assessed nonword repetition (NWR) skills in 7-9 year-old children with dyslexia (dyslexia-only), developmental language disorder (DLD-only), co-occurring DLD+dyslexia, and typical development (TD) with a norm-referenced and an experimental task. The experimental task manipulated phonemic variability (dissimilarity among consonant phonemes within the nonword) and presentation modality (audio-only versus audiovisual) to probe potential phonological processing differences among the groups. Across tasks, the dyslexia-only and DLD-only groups performed similarly to each other and intermediately to the TD and DLD+dyslexia groups. In the experimental task, nonwords with low phonemic variability were produced less accurately in both modalities, and audiovisual presentation facilitated accurate repetition of low phonemic variability nonwords. A lack of a group interaction with phonemic variability or presentation modality suggests similarities, despite group differences, in how underlying phonological representations influence task performance. Overall, results suggest that poor NWR is associated with both dyslexia and DLD, and that co-occurrence compounds this difficulty.

Comparing Speech Recognition for Listeners With Normal and Impaired Hearing: Simulations for Controlling Differences in Speech Levels and Spectral Shape.

Purpose This study investigated methods used to simulate factors associated with reduced audibility, increased speech levels, and spectral shaping for aided older adults with hearing loss. Simulations provided to younger normal-hearing adults were used to investigate the effect of sensation level, speech presentation level, and spectral shape in comparison to older adults with hearing loss. Method Measures were assessed in quiet, steady-state noise, and speech-modulated noise. Older adults with hearing loss listened to speech that was spectrally shaped according to their hearing thresholds. Younger adults with normal hearing listened to speech that simulated the hearing-impaired group's (a) reduced audibility, (b) increased speech levels, and (c) spectral shaping. Group comparisons were made based on speech recognition performance and masking release. Additionally, younger adults completed measures of listening effort and perceived speech quality to assess if differences across simulations in these outcome measures were similar to those for speech recognition. Results Across the various simulations employed, testing in the presence of a threshold matching noise best matched differences in speech recognition and masking release between younger and older adults. This result remained consistent across the other two outcome measures. Conclusions A combination of audibility, speech level, and spectral shape factors is required to simulate differences between listeners with normal and impaired hearing in recognition, listening effort, and perceived speech quality. The use of spectrally shaped and amplified speech in the presence of threshold matching noise best provided this simulated control. Supplemental Material https://doi.org/10.23641/asha.13224632.

Spectro-temporal glimpsing of speech in noise: Regularity and coherence of masking patterns reduces uncertainty and increases intelligibility.

Adverse listening conditions involve glimpses of spectro-temporal speech information. This study investigated if the acoustic organization of the spectro-temporal masking pattern affects speech glimpsing in "checkerboard" noise. The regularity and coherence of the masking pattern was varied. Regularity was reduced by randomizing the spectral or temporal gating of the masking noise. Coherence involved the spectral alignment of frequency bands across time or the temporal alignment of gated onsets/offsets across frequency bands. Experiment 1 investigated the effect of spectral or temporal coherence. Experiment 2 investigated independent and combined factors of regularity and coherence. Performance was best in spectro-temporally modulated noise having larger glimpses. Generally, performance also improved as the regularity and coherence of masker fluctuations increased, with regularity having a stronger effect than coherence. An acoustic glimpsing model suggested that the effect of regularity (but not coherence) could be partially attributed to the availability of glimpses retained after energetic masking. Performance tended to be better with maskers that were spectrally coherent as compared to temporally coherent. Overall, performance was best when the spectro-temporal masking pattern imposed even spectral sampling and minimal temporal uncertainty, indicating that listeners use reliable masking patterns to aid in spectro-temporal speech glimpsing.

The effect of simulated room acoustic parameters on the intelligibility and perceived reverberation of monosyllabic words and sentences.

Individual acoustic parameters of reverberation have the potential to affect both the intelligibility of speech and the degree of perceived reverberation. The current experiments used monaural acoustic simulations to investigate the effect of reverberation time (RT) and direct-to-reverberant ratio (DRR) on word and sentence intelligibility at different levels of analysis (phonemes, words, and sentences). Perceived reverberation and recall of sentences were also assessed. Intelligibility and perceived reverberation decreased with increasing RT and decreasing DRR (particularly between 0 and -10 dB). Results indicate consistent effects of both RT and DRR on the intelligibility and perceived reverberation of words and sentences.

Combining partial information from speech and text.

The current study investigated how partial speech and text information, distributed at various interruption rates, is combined to support sentence recognition in quiet. Speech and text stimuli were interrupted by silence and presented unimodally or combined in multimodal conditions. Across all conditions, performance was best at the highest interruption rates. Listeners were able to gain benefit from most multimodal presentations, even when the rate of interruption was mismatched between modalities. Supplementing partial speech with incomplete visual cues can improve sentence intelligibility and compensate for degraded speech in adverse listening conditions. However, individual variability in benefit depends on unimodal performance.

The Perceptual Contribution of Consonants and Vowels to Sentence Recognition: Effect of Dialect Variation in American English.

In background noise, the amplitude fluctuations of speech commonly provide for momentary glimpses of high intensity portions of speech, predominantly from vowels. Previous investigations have provided glimpses of consonants or vowels to determine the perceptual contribution of different speech acoustics to sentence intelligibility. The present study investigated the consistency of perceptual contributions across eight American English dialects for a group of listeners from the southern United States. Results demonstrated that sentences preserving predominant vowel acoustics were consistently more intelligible across dialects for all participants. The significant contribution of vowels does not appear dependent on familiarity with properties of dialectal variation but may represent the preservation of more general acoustic cues important for sentence recognition. Acoustic analyses of temporal amplitude modulation suggest important cues present during vowels and highlight gradient differences across dialects associated with intelligibility.

Perceptual Organization of Interrupted Speech and Text.

Purpose: Visual recognition of interrupted text may predict speech intelligibility under adverse listening conditions. This study investigated the nature of the linguistic information and perceptual processes underlying this relationship. Method: To directly compare the perceptual organization of interrupted speech and text, we examined the recognition of spoken and printed sentences interrupted at different rates in 14 adults with normal hearing. The interruption method approximated deletion and retention of rate-specific linguistic information (0.5-64 Hz) in speech by substituting either white space or silent intervals for text or speech in the original sentences. Results: A similar U-shaped pattern of cross-rate variation in performance was observed in both modalities, with minima at 2 Hz. However, at the highest and lowest interruption rates, recognition accuracy was greater for text than speech, whereas the reverse was observed at middle rates. An analysis of word duration and the frequency of word sampling across interruption rates suggested that the location of the function minima was influenced by perceptual reconstruction of whole words. Overall, the findings indicate a high degree of similarity in the perceptual organization of interrupted speech and text. Conclusion: The observed rate-specific variation in the perception of speech and text may potentially affect the degree to which recognition accuracy in one modality is predictive of the other.

Explaining intelligibility in speech-modulated maskers using acoustic glimpse analysis.

Intelligibility was measured in speech-modulated noise varying in level and temporal modulation rate (TMR). Acoustic analysis measured glimpses available above a local signal-to-noise ratio criterion (LC). The proportion and rate of glimpses were correlated with intelligibility, particularly in relation to masker level or TMR manipulations, respectively. Intelligibility correlations for each metric were maximized at different analysis LCs. Regression analysis showed that both metrics measured at -2 dB LC were required to best explain the total variance (R2 = 0.49) for individual sentence intelligibility. Acoustic conditions associated with recognizing speech in complex maskers are best explained using multidimensional glimpse metrics.