The Impact of Hearing Aids on Listening Effort and Listening-Related Fatigue - Investigations in a Virtual Realistic Listening Environment.

Participation in complex listening situations such as group conversations in noisy environments sets high demands on the auditory system and on cognitive processing. Reports of hearing-impaired people indicate that strenuous listening situations occurring throughout the day lead to feelings of fatigue at the end of the day. The aim of the present study was to develop a suitable test sequence to evoke and measure listening effort (LE) and listening-related fatigue (LRF), and, to evaluate the influence of hearing aid use on both dimensions in mild to moderately hearing-impaired participants. The chosen approach aims to reconstruct a representative acoustic day (Time Compressed Acoustic Day [TCAD]) by means of an eight-part hearing-test sequence with a total duration of approximately 2½ h. For this purpose, the hearing test sequence combined four different listening tasks with five different acoustic scenarios and was presented to the 20 test subjects using virtual acoustics in an open field measurement in aided and unaided conditions. Besides subjective ratings of LE and LRF, behavioral measures (response accuracy, reaction times), and an attention test (d2-R) were performed prior to and after the TCAD. Furthermore, stress hormones were evaluated by taking salivary samples. Subjective ratings of LRF increased throughout the test sequence. This effect was observed to be higher when testing unaided. In three of the eight listening tests, the aided condition led to significantly faster reaction times/response accuracies than in the unaided condition. In the d2-R test, an interaction in processing speed between time (pre- vs. post-TCAD) and provision (unaided vs. aided) was found suggesting an influence of hearing aid provision on LRF. A comparison of the averaged subjective ratings at the beginning and end of the TCAD shows a significant increase in LRF for both conditions. At the end of the TCAD, subjective fatigue was significantly lower when wearing hearing aids. The analysis of stress hormones did not reveal significant effects.

Brain and grammar: revealing electrophysiological basic structures with competing statistical models.

Acoustic, lexical, and syntactic information are simultaneously processed in the brain requiring complex strategies to distinguish their electrophysiological activity. Capitalizing on previous works that factor out acoustic information, we could concentrate on the lexical and syntactic contribution to language processing by testing competing statistical models. We exploited electroencephalographic recordings and compared different surprisal models selectively involving lexical information, part of speech, or syntactic structures in various combinations. Electroencephalographic responses were recorded in 32 participants during listening to affirmative active declarative sentences. We compared the activation corresponding to basic syntactic structures, such as noun phrases vs. verb phrases. Lexical and syntactic processing activates different frequency bands, partially different time windows, and different networks. Moreover, surprisal models based on part of speech inventory only do not explain well the electrophysiological data, while those including syntactic information do. By disentangling acoustic, lexical, and syntactic information, we demonstrated differential brain sensitivity to syntactic information. These results confirm and extend previous measures obtained with intracranial recordings, supporting our hypothesis that syntactic structures are crucial in neural language processing. This study provides a detailed understanding of how the brain processes syntactic information, highlighting the importance of syntactic surprisal in shaping neural responses during language comprehension.

High segregation and diminished global integration in large-scale brain functional networks enhances the perceptual binding of cross-modal stimuli.

Speech perception requires the binding of spatiotemporally disjoint auditory-visual cues. The corresponding brain network-level information processing can be characterized by two complementary mechanisms: functional segregation which refers to the localization of processing in either isolated or distributed modules across the brain, and integration which pertains to cooperation among relevant functional modules. Here, we demonstrate using functional magnetic resonance imaging recordings that subjective perceptual experience of multisensory speech stimuli, real and illusory, are represented in differential states of segregation-integration. We controlled the inter-subject variability of illusory/cross-modal perception parametrically, by introducing temporal lags in the incongruent auditory-visual articulations of speech sounds within the McGurk paradigm. The states of segregation-integration balance were captured using two alternative computational approaches. First, the module responsible for cross-modal binding of sensory signals defined as the perceptual binding network (PBN) was identified using standardized parametric statistical approaches and their temporal correlations with all other brain areas were computed. With increasing illusory perception, the majority of the nodes of PBN showed decreased cooperation with the rest of the brain, reflecting states of high segregation but reduced global integration. Second, using graph theoretic measures, the altered patterns of segregation-integration were cross-validated.

A Step Toward Precision Audiology: Individual Differences and Characteristic Profiles From Auditory Perceptual and Cognitive Abilities.

Older adults with normal hearing or with age-related hearing loss face challenges when listening to speech in noisy environments. To better serve individuals with communication difficulties, precision diagnostics are needed to characterize individuals' auditory perceptual and cognitive abilities beyond pure tone thresholds. These abilities can be heterogenous across individuals within the same population. The goal of the present study is to consider the suprathreshold variability and develop characteristic profiles for older adults with normal hearing (ONH) and with hearing loss (OHL). Auditory perceptual and cognitive abilities were tested on ONH (n = 20) and OHL (n = 20) on an abbreviated test battery using portable automated rapid testing. Using cluster analyses, three main profiles were revealed for each group, showing differences in auditory perceptual and cognitive abilities despite similar audiometric thresholds. Analysis of variance showed that ONH profiles differed in spatial release from masking, speech-in-babble testing, cognition, tone-in-noise, and binaural temporal processing abilities. The OHL profiles differed in spatial release from masking, speech-in-babble testing, cognition, and tolerance to background noise performance. Correlation analyses showed significant relationships between auditory and cognitive abilities in both groups. This study showed that auditory perceptual and cognitive deficits can be present to varying degrees in the presence of audiometrically normal hearing and among listeners with similar degrees of hearing loss. The results of this study inform the need for taking individual differences into consideration and developing targeted intervention options beyond pure tone thresholds and speech testing.

[Early auditory processing of speech in prelingually-deafened children with cochlear implants based on event-related potentials].

Objective: To investigate the early auditory discrimination of vowels, consonants and lexical tones in prelingually-deafened children with cochlear implants (CI) using auditory event-related potentials. Methods: Nineteen prelingually-deafened CI children and 19 normal hearing (NH) children were recruited in this study. A multi-deviant oddball paradigm was constructed using the monosyllable/ta1/as the standard stimulus and monosyllables/tu1/,/te1/, /da1/,/ra1/,/ta4/and/ta2/as the deviant stimuli. The event-related potentials evoked by vowel, consonant and lexical tone contrasts were recorded and analyzed in the two groups. Results: NH children showed robust mismatch negativities (MMNs) to vowel, consonant and lexical tone contrasts (P<0.05), whereas CI children only showed positive mismatch responses (pMMRs) and P3a responses to the vowel (P<0.05) and consonant contrasts (P<0.05) and no significant event-related potential to the lexical tone contrasts (P>0.05). The longer pMMR and P3a peak latencies (P<0.01) but similar amplitudes (P>0.05) were found in CI children than in NH children. CI children showed weaker phase synchronization of θ oscillations than NH children (P<0.05). The duration of CI use was positively correlated with the scores of Categories of Auditory Performance (CAP) (P=0.004), Speech Intelligibility Rate (SIR) (P=0.044) and Meaningful Auditory Integration Scale (MAIS) (P=0.001) in CI children. Conclusions: Prelingually-deafened CI children can process vowels and consonants at an early stage. However, their ability of processing speech, especially lexical tones, is still more immature compared with their NH peers. The event-related potentials could be objective electrophysiological indicators reflecting the maturity of CI children's auditory speech functions. Long-term CI use is beneficial for prelingually-deafened children to improve auditory and speech performance.

Neural encoding of linguistic speech cues is unaffected by cognitive decline, but decreases with increasing hearing impairment.

The multivariate temporal response function (mTRF) is an effective tool for investigating the neural encoding of acoustic and complex linguistic features in natural continuous speech. In this study, we investigated how neural representations of speech features derived from natural stimuli are related to early signs of cognitive decline in older adults, taking into account the effects of hearing. Participants without ( n = 25 ) and with ( n = 19 ) early signs of cognitive decline listened to an audiobook while their electroencephalography responses were recorded. Using the mTRF framework, we modeled the relationship between speech input and neural response via different acoustic, segmented and linguistic encoding models and examined the response functions in terms of encoding accuracy, signal power, peak amplitudes and latencies. Our results showed no significant effect of cognitive decline or hearing ability on the neural encoding of acoustic and linguistic speech features. However, we found a significant interaction between hearing ability and the word-level segmentation model, suggesting that hearing impairment specifically affects encoding accuracy for this model, while other features were not affected by hearing ability. These results suggest that while speech processing markers remain unaffected by cognitive decline and hearing loss per se, neural encoding of word-level segmented speech features in older adults is affected by hearing loss but not by cognitive decline. This study emphasises the effectiveness of mTRF analysis in studying the neural encoding of speech and argues for an extension of research to investigate its clinical impact on hearing loss and cognition.

Cochlear Implantation in Single-Sided Deafness and Asymmetric Hearing Loss: 12 Months Follow-up Results of a European Multicenter Evaluation.

People with single-sided deafness (SSD) or asymmetric hearing loss (AHL) have particular difficulty understanding speech in noisy listening situations and in sound localization. The objective of this multicenter study is to evaluate the effect of a cochlear implant (CI) in adults with single-sided deafness (SSD) or asymmetric hearing loss (AHL), particularly regarding sound localization and speech intelligibility with additional interest in electric-acoustic pitch matching. A prospective longitudinal study at 7 European tertiary referral centers was conducted including 19 SSD and 16 AHL subjects undergoing cochlear implantation. Sound localization accuracy was investigated in terms of root mean square error and signed bias before and after implantation. Speech recognition in quiet and speech reception thresholds in noise for several spatial configurations were assessed preoperatively and at several post-activation time points. Pitch perception with CI was tracked using pitch matching. Data up to 12 months post activation were collected. In both SSD and AHL subjects, CI significantly improved sound localization for sound sources on the implant side, and thus overall sound localization. Speech recognition in quiet with the implant ear improved significantly. In noise, a significant head shadow effect was found for SSD subjects only. However, the evaluation of AHL subjects was limited by the small sample size. No uniform development of pitch perception with the implant ear was observed. The benefits shown in this study confirm and expand the existing body of evidence for the effectiveness of CI in SSD and AHL. Particularly, improved localization was shown to result from increased localization accuracy on the implant side.

Tracking age-related changes in voice and speech production with Landmark-based analysis of speech.

Voice and speech production change with age, which can lead to potential communication challenges. This study explored the use of Landmark-based analysis of speech (LMBAS), a knowledge-based speech analysis algorithm based on Stevens' Landmark Theory, to describe age-related changes in adult speakers. The speech samples analyzed were sourced from the University of Florida Aging Voice Database, which included recordings of 16 sentences from the Speech Perception in Noise test of Bilger, Rzcezkowski, Nuetzel, and Rabinowitz [J. Acoust. Soc. Am. 65, S98-S98 (1979)] and Bilger, Nuetzel, Rabinowitz, and Rzeczkowski [J. Speech. Lang. Hear. Res. 27, 32-84 (1984)]. These sentences were read in quiet environments by 50 young, 50 middle-aged, and 50 older American English speakers, with an equal distribution of sexes. Acoustic landmarks, specifically, glottal, bursts, and syllabicity landmarks, were extracted using SpeechMark®, MATLAB Toolbox version 1.1.2. The results showed significant age effect on glottal and burst landmarks. Furthermore, the sex effect was significant for burst and syllabicity landmarks. While the results of LMBAS suggest its potential in detecting age-related changes in speech, increase in syllabicity landmarks with age was unexpected. This finding may suggest the need for further refinement and adjustment of this analytical approach.

Entrainment echoes in the cerebellum.

Evidence accumulates that the cerebellum's role in the brain is not restricted to motor functions. Rather, cerebellar activity seems to be crucial for a variety of tasks that rely on precise event timing and prediction. Due to its complex structure and importance in communication, human speech requires a particularly precise and predictive coordination of neural processes to be successfully comprehended. Recent studies proposed that the cerebellum is indeed a major contributor to speech processing, but how this contribution is achieved mechanistically remains poorly understood. The current study aimed to reveal a mechanism underlying cortico-cerebellar coordination and demonstrate its speech-specificity. In a reanalysis of magnetoencephalography data, we found that activity in the cerebellum aligned to rhythmic sequences of noise-vocoded speech, irrespective of its intelligibility. We then tested whether these "entrained" responses persist, and how they interact with other brain regions, when a rhythmic stimulus stopped and temporal predictions had to be updated. We found that only intelligible speech produced sustained rhythmic responses in the cerebellum. During this "entrainment echo," but not during rhythmic speech itself, cerebellar activity was coupled with that in the left inferior frontal gyrus, and specifically at rates corresponding to the preceding stimulus rhythm. This finding represents evidence for specific cerebellum-driven temporal predictions in speech processing and their relay to cortical regions.

Classifying coherent versus nonsense speech perception from EEG using linguistic speech features.

When a person listens to natural speech, the relation between features of the speech signal and the corresponding evoked electroencephalogram (EEG) is indicative of neural processing of the speech signal. Using linguistic representations of speech, we investigate the differences in neural processing between speech in a native and foreign language that is not understood. We conducted experiments using three stimuli: a comprehensible language, an incomprehensible language, and randomly shuffled words from a comprehensible language, while recording the EEG signal of native Dutch-speaking participants. We modeled the neural tracking of linguistic features of the speech signals using a deep-learning model in a match-mismatch task that relates EEG signals to speech, while accounting for lexical segmentation features reflecting acoustic processing. The deep learning model effectively classifies coherent versus nonsense languages. We also observed significant differences in tracking patterns between comprehensible and incomprehensible speech stimuli within the same language. It demonstrates the potential of deep learning frameworks in measuring speech understanding objectively.

Acoustic scene complexity affects motion behavior during speech perception in audio-visual multi-talker virtual environments.

In real-world listening situations, individuals typically utilize head and eye movements to receive and enhance sensory information while exploring acoustic scenes. However, the specific patterns of such movements have not yet been fully characterized. Here, we studied how movement behavior is influenced by scene complexity, varied in terms of reverberation and the number of concurrent talkers. Thirteen normal-hearing participants engaged in a speech comprehension and localization task, requiring them to indicate the spatial location of a spoken story in the presence of other stories in virtual audio-visual scenes. We observed delayed initial head movements when more simultaneous talkers were present in the scene. Both reverberation and a higher number of talkers extended the search period, increased the number of fixated source locations, and resulted in more gaze jumps. The period preceding the participants' responses was prolonged when more concurrent talkers were present, and listeners continued to move their eyes in the proximity of the target talker. In scenes with more reverberation, the final head position when making the decision was farther away from the target. These findings demonstrate that the complexity of the acoustic scene influences listener behavior during speech comprehension and localization in audio-visual scenes.

Impact of age at the second implantation, experience of amplification use, and long-term binaural experience on sound localization of children with bilateral cochlear implants.

OBJECTIVES: To assess the influence of three factors using retrospective chart review: age at which 2nd cochlear implant (CI) is implanted, prior hearing aid (HA) experience in the 2nd CI ear, and long-term experience with bilateral cochlear implants (BICIs) on sound localization in children with sequential BICIs. METHODS: Mean absolute error (MAE) in localizing speech noise of 60 children with sequential BICIs was compared across four age groups of the 2nd CI (1-5.0; 5.1-10.0; 10.1-14.0; & 14.1-19.0 years) and two extents of prior HA experience (more than and less than one year). MAE was also longitudinally analyzed after 4-6 years of experience with BICI involving 18 participants out of 60. RESULTS: Children who received 2nd CI before five years of age demonstrated significantly better localization than those who received it after ten years of age. More than one year of prior HA experience in the 2nd CI ear and extensive experience with sequential BICIs significantly enhanced localization performance. Inter-implant intervals and age at the 2nd CI showed a significant positive correlation with the MAE (poorer localization). CONCLUSION: The results indicate that age at 2nd CI is important in developing sound localization skills. Based on the results, obtaining 2nd CI within the first five years of life and no later than ten years old is recommended. The results also suggest that longer use of amplification before 2nd CI and prolonged BICI experience significantly fosters localization development.

Effects of age and hearing loss on speech emotion discrimination.

Better communication with older people requires not only improving speech intelligibility but also understanding how well emotions can be conveyed and the effect of age and hearing loss (HL) on emotion perception. In this paper, emotion discrimination experiments were conducted using a vocal morphing method and an HL simulator in young normal hearing (YNH) and older participants. Speech sounds were morphed to represent intermediate emotions between all combinations of happiness, sadness, and anger. Discrimination performance was compared when the YNH listened to normal sounds, when the same YNH listened to HL simulated sounds, and when older people listened to the same normal sounds. The results showed that there was no significant difference between discrimination with and without HL simulation, suggesting that peripheral HL may not affect emotion perception. The discrimination performance of the older participants was significantly worse only for the anger-happiness pair than for the other emotion pairs and for the YNH. It was also found that the difficulty increases with age, not just with hearing level.

Neural semantic effects of tone accents.

This study investigated whether the brain utilizes morphologically induced tones for semantic processing during online speech perception. An auditory comprehension task was conducted while measuring event-related potentials (ERPs). The study tested whether a discrepancy between contextual expectations and the tonal realizations of the target word would yield an N400 effect, indicative of semantic processing difficulty. An N400 effect was observed, reflecting integration difficulty due to semantic anomalies caused by incongruent tones. Additionally, the ERPs in the congruent conditions were modulated by the cohort entropy of the target word indicating lexical competition. The late negativity observed in this study encompasses both the N400 and preactivation negativity. This overlap underscores the brain's potential for rapidly connecting form and meaning from different sources within the word, relying on statistically based prediction in semantic processing.

Speech sensorimotor relationships in francophone preschoolers and adults: Adaptation to real-time auditory feedback perturbations.

PURPOSE: This study investigates the development of sensorimotor relationships by examining adaptation to real-time perturbations of auditory feedback. METHOD: Acoustic signals were recorded while preschoolers and adult speakers of Canadian French produced several utterances of the front rounded vowel /ø/ for which F2 was gradually shifted up to a maximum of 40%. RESULTS: The findings indicate that, although preschool-aged children produced overall similar responses to the perturbed feedback, they displayed significantly more trial-to-trial variability than adults. Furthermore, whereas the magnitude of the adaptation in adults was positively correlated with the slope of the perceptual categorical function, the amount of adaptation in children was linked to the variability of their productions in the baseline condition. These patterns suggest that the immature motor control observed in children, which contributes to increased variability in their speech production, plays a role in shaping adaptive behavior, as it allows children to explore articulatory/acoustic spaces and learn sensorimotor relationships.

Recognition of Emotional Prosody in Mandarin-Speaking Children: Effects of Age, Noise, and Working Memory.

Age, babble noise, and working memory have been found to affect the recognition of emotional prosody based on non-tonal languages, yet little is known about how exactly they influence tone-language-speaking children's recognition of emotional prosody. In virtue of the tectonic theory of Stroop effects and the Ease of Language Understanding (ELU) model, this study aimed to explore the effects of age, babble noise, and working memory on Mandarin-speaking children's understanding of emotional prosody. Sixty Mandarin-speaking children aged three to eight years and 20 Mandarin-speaking adults participated in this study. They were asked to recognize the happy or sad prosody of short sentences with different semantics (negative, neutral, or positive) produced by a male speaker. The results revealed that the prosody-semantics congruity played a bigger role in children than in adults for accurate recognition of emotional prosody in quiet, but a less important role in children compared with adults in noise. Furthermore, concerning the recognition accuracy of emotional prosody, the effect of working memory on children was trivial despite the listening conditions. But for adults, it was very prominent in babble noise. The findings partially supported the tectonic theory of Stroop effects which highlights the perceptual enhancement generated by cross-channel congruity, and the ELU model which underlines the importance of working memory in speech processing in noise. These results suggested that the development of emotional prosody recognition is a complex process influenced by the interplay among age, background noise, and working memory.

Band importance for speech-in-speech recognition in the presence of extended high-frequency cues.

Band importance functions for speech-in-noise recognition, typically determined in the presence of steady background noise, indicate a negligible role for extended high frequencies (EHFs; 8-20 kHz). However, recent findings indicate that EHF cues support speech recognition in multi-talker environments, particularly when the masker has reduced EHF levels relative to the target. This scenario can occur in natural auditory scenes when the target talker is facing the listener, but the maskers are not. In this study, we measured the importance of five bands from 40 to 20 000 Hz for speech-in-speech recognition by notch-filtering the bands individually. Stimuli consisted of a female target talker recorded from 0° and a spatially co-located two-talker female masker recorded either from 0° or 56.25°, simulating a masker either facing the listener or facing away, respectively. Results indicated peak band importance in the 0.4-1.3 kHz band and a negligible effect of removing the EHF band in the facing-masker condition. However, in the non-facing condition, the peak was broader and EHF importance was higher and comparable to that of the 3.3-8.3 kHz band in the facing-masker condition. These findings suggest that EHFs contain important cues for speech recognition in listening conditions with mismatched talker head orientations.

Neural Tracking of Speech Acoustics in Noise Is Coupled with Lexical Predictability as Estimated by Large Language Models.

Adults heard recordings of two spatially separated speakers reading newspaper and magazine articles. They were asked to listen to one of them and ignore the other, and EEG was recorded to assess their neural processing. Machine learning extracted neural sources that tracked the target and distractor speakers at three levels: the acoustic envelope of speech (delta- and theta-band modulations), lexical frequency for individual words, and the contextual predictability of individual words estimated by GPT-4 and earlier lexical models. To provide a broader view of speech perception, half of the subjects completed a simultaneous visual task, and the listeners included both native and non-native English speakers. Distinct neural components were extracted for these levels of auditory and lexical processing, demonstrating that native English speakers had greater target-distractor separation compared with non-native English speakers on most measures, and that lexical processing was reduced by the visual task. Moreover, there was a novel interaction of lexical predictability and frequency with auditory processing; acoustic tracking was stronger for lexically harder words, suggesting that people listened harder to the acoustics when needed for lexical selection. This demonstrates that speech perception is not simply a feedforward process from acoustic processing to the lexicon. Rather, the adaptable context-sensitive processing long known to occur at a lexical level has broader consequences for perception, coupling with the acoustic tracking of individual speakers in noise.