Predicting early auditory evoked potentials using a computational model of auditory-nerve processing.

Non-invasive electrophysiological measures, such as auditory evoked potentials (AEPs), play a crucial role in diagnosing auditory pathology. However, the relationship between AEP morphology and cochlear degeneration remains complex and not well understood. Dau [J. Acoust. Soc. Am. 113, 936-950 (2003)] proposed a computational framework for modeling AEPs that utilized a nonlinear auditory-nerve (AN) model followed by a linear unitary response function. While the model captured some important features of the measured AEPs, it also exhibited several discrepancies in response patterns compared to the actual measurements. In this study, an enhanced AEP modeling framework is presented, incorporating an improved AN model, and the conclusions from the original study were reevaluated. Simulation results with transient and sustained stimuli demonstrated accurate auditory brainstem responses (ABRs) and frequency-following responses (FFRs) as a function of stimulation level, although wave-V latencies remained too short, similar to the original study. When compared to physiological responses in animals, the revised model framework showed a more accurate balance between the contributions of auditory-nerve fibers (ANFs) at on- and off-frequency regions to the predicted FFRs. These findings emphasize the importance of cochlear processing in brainstem potentials. This framework may provide a valuable tool for assessing human AN models and simulating AEPs for various subtypes of peripheral pathologies, offering opportunities for research and clinical applications.

Disentangling the effects of hearing loss and age on amplitude modulation frequency selectivity.

The processing and perception of amplitude modulation (AM) in the auditory system reflect a frequency-selective process, often described as a modulation filterbank. Previous studies on perceptual AM masking reported similar results for older listeners with hearing impairment (HI listeners) and young listeners with normal hearing (NH listeners), suggesting no effects of age or hearing loss on AM frequency selectivity. However, recent evidence has shown that age, independently of hearing loss, adversely affects AM frequency selectivity. Hence, this study aimed to disentangle the effects of hearing loss and age. A simultaneous AM masking paradigm was employed, using a sinusoidal carrier at 2.8 kHz, narrowband noise modulation maskers, and target modulation frequencies of 4, 16, 64, and 128 Hz. The results obtained from young (n = 3, 24-30 years of age) and older (n = 10, 63-77 years of age) HI listeners were compared to previously obtained data from young and older NH listeners. Notably, the HI listeners generally exhibited lower (unmasked) AM detection thresholds and greater AM frequency selectivity than their NH counterparts in both age groups. Overall, the results suggest that age negatively affects AM frequency selectivity for both NH and HI listeners, whereas hearing loss improves AM detection and AM selectivity, likely due to the loss of peripheral compression.

The effect of auditory training on listening effort in hearing-aid users: insights from a pupillometry study.

OBJECTIVE: The study investigated how auditory training affects effort exerted by hearing-impaired listeners in speech-in-noise task. DESIGN: Pupillometry was used to characterise listening effort during a hearing in noise test (HINT) before and after phoneme-in-noise identification training. Half of the study participants completed the training, while the other half formed an active control group. STUDY SAMPLE: Twenty 63-to-79 years old experienced hearing-aid users. RESULTS: Higher peak pupil dilations (PPDs) were obtained at the end of the study compared to the beginning in both groups of the participants. The analysis of pupil dilation in an extended time window revealed, however, that the magnitude of pupillary response increased more in the training than in the control group. The effect of training on effort was observed in pupil responses even when no improvement in HINT was found. CONCLUSION: The results demonstrate that using a listening effort metric adds additional insights into the effectiveness of auditory training compared to the situation when only speech-in-noise performance is considered. Trends observed in pupil responses suggested increased effort-both after the training and the placebo intervention-most likely reflecting the effect of the individual's motivation.

Modulation transfer functions for audiovisual speech.

Temporal synchrony between facial motion and acoustic modulations is a hallmark feature of audiovisual speech. The moving face and mouth during natural speech is known to be correlated with low-frequency acoustic envelope fluctuations (below 10 Hz), but the precise rates at which envelope information is synchronized with motion in different parts of the face are less clear. Here, we used regularized canonical correlation analysis (rCCA) to learn speech envelope filters whose outputs correlate with motion in different parts of the speakers face. We leveraged recent advances in video-based 3D facial landmark estimation allowing us to examine statistical envelope-face correlations across a large number of speakers (∼4000). Specifically, rCCA was used to learn modulation transfer functions (MTFs) for the speech envelope that significantly predict correlation with facial motion across different speakers. The AV analysis revealed bandpass speech envelope filters at distinct temporal scales. A first set of MTFs showed peaks around 3-4 Hz and were correlated with mouth movements. A second set of MTFs captured envelope fluctuations in the 1-2 Hz range correlated with more global face and head motion. These two distinctive timescales emerged only as a property of natural AV speech statistics across many speakers. A similar analysis of fewer speakers performing a controlled speech task highlighted only the well-known temporal modulations around 4 Hz correlated with orofacial motion. The different bandpass ranges of AV correlation align notably with the average rates at which syllables (3-4 Hz) and phrases (1-2 Hz) are produced in natural speech. Whereas periodicities at the syllable rate are evident in the envelope spectrum of the speech signal itself, slower 1-2 Hz regularities thus only become prominent when considering crossmodal signal statistics. This may indicate a motor origin of temporal regularities at the timescales of syllables and phrases in natural speech.

Age-related reduction of amplitude modulation frequency selectivity.

The perception of amplitude modulations (AMs) has been characterized by a frequency-selective process in the temporal envelope domain and simulated in computational auditory processing and perception models using a modulation filterbank. Such AM frequency-selective processing has been argued to be critical for the perception of complex sounds, including speech. This study aimed at investigating the effects of age on behavioral AM frequency selectivity in young (n = 11, 22-29 years) versus older (n = 10, 57-77 years) listeners with normal hearing, using a simultaneous AM masking paradigm with a sinusoidal carrier (2.8 kHz), target modulation frequencies of 4, 16, 64, and 128 Hz, and narrowband-noise modulation maskers. A reduction of AM frequency selectivity by a factor of up to 2 was found in the older listeners. While the observed AM selectivity co-varied with the unmasked AM detection sensitivity, the age-related broadening of the masked threshold patterns remained stable even when AM sensitivity was similar across groups for an extended stimulus duration. The results from the present study might provide a valuable basis for further investigations exploring the effects of age and reduced AM frequency selectivity on complex sound perception as well as the interaction of age and hearing impairment on AM processing and perception.

The effect of phoneme-based auditory training on speech intelligibility in hearing-aid users.

OBJECTIVE: Hearing loss commonly causes difficulties in understanding speech in the presence of background noise. The benefits of hearing-aids in terms of speech intelligibility in challenging listening scenarios remain limited. The present study investigated if phoneme-in-noise discrimination training improves phoneme identification and sentence intelligibility in noise in hearing-aid users. DESIGN: Two groups of participants received either a two-week training program or a control intervention. Three phoneme categories were trained: onset consonants (C1), vowels (V) and post-vowel consonants (C2) in C1-V-C2-/i/ logatomes from the Danish nonsense word corpus (DANOK). Phoneme identification test and hearing in noise test (HINT) were administered before and after the respective interventions and, for the training group only, after three months. STUDY SAMPLE: Twenty 63-to-79 years old individuals with a mild-to-moderate sensorineural hearing loss and at least one year of experience using hearing-aids. RESULTS: The training provided an improvement in phoneme identification scores for vowels and post-vowel consonants, which was retained over three months. No significant performance improvement in HINT was found. CONCLUSION: The study demonstrates that the training induced a robust refinement of auditory perception at a phoneme level but provides no evidence for the generalisation to an untrained sentence intelligibility task.

The effect of hearing aid dynamic range compression on speech intelligibility in a realistic virtual sound environment.

Measures of "aided" speech intelligibility (SI) for listeners wearing hearing aids (HAs) are commonly obtained using rather artificial acoustic stimuli and spatial configurations compared to those encountered in everyday complex listening scenarios. In the present study, the effect of hearing aid dynamic range compression (DRC) on SI was investigated in simulated real-world acoustic conditions. A spatialized version of the Danish Hearing In Noise Test was employed inside a loudspeaker-based virtual sound environment to present spatialized target speech in background noise consisting of either spatial recordings of two real-world sound scenarios or quadraphonic, artificial speech-shaped noise (SSN). Unaided performance was compared with results obtained with a basic HA simulator employing fast-acting DRC. Speech reception thresholds (SRTs) with and without DRC were found to be significantly higher in the conditions with real-world background noise than in the condition with artificial SSN. Improvements in SRTs caused by the HA were only significant in conditions with real-world background noise and were related to differences in the output signal-to-noise ratio of the HA signal processing between the real-world versus artificial conditions. The results may be valuable for the design, development, and evaluation of HA signal processing strategies in realistic, but controlled, acoustic settings.

Audiometric profiles and patterns of benefit: a data-driven analysis of subjective hearing difficulties and handicaps.

OBJECTIVE: Hearing rehabilitation attempts to compensate for auditory dysfunction, reduce hearing difficulties and minimise participation restrictions that can lead to social isolation. However, there is no systematic approach to assess the quality of the intervention at an individual level that might help to evaluate the need of further hearing rehabilitation in the hearing care clinic. DESIGN: A data-driven analysis on subjective data reflecting hearing disabilities and handicap was chosen to explore "benefit patterns" as a result of rehabilitation in different audiometric groups. The method was based on (1) dimensionality reduction; (2) stratification; (3) archetypal analysis; (4) clustering; (5) item importance estimation. STUDY SAMPLE: 572 hearing-aid users completed questionnaires of hearing difficulties (speech, spatial and qualities hearing scale; SSQ) and hearing handicap (HHQ). RESULTS: The data-driven approach revealed four benefit profiles that were different for each audiometric group. The groups with low degree of high-frequency hearing loss (HLHF) showed a priority for rehabilitating hearing handicaps, whereas the groups with HLHF > 50 dB HL showed a priority for improvements in speech understanding. CONCLUSIONS: The patterns of benefit and the stratification approach might guide the clinical intervention strategy and improve the efficacy and quality of service in the hearing care clinic.

Effects of Sensorineural Hearing Loss on Cortical Synchronization to Competing Speech during Selective Attention.

When selectively attending to a speech stream in multi-talker scenarios, low-frequency cortical activity is known to synchronize selectively to fluctuations in the attended speech signal. Older listeners with age-related sensorineural hearing loss (presbycusis) often struggle to understand speech in such situations, even when wearing a hearing aid. Yet, it is unclear whether a peripheral hearing loss degrades the attentional modulation of cortical speech tracking. Here, we used psychoacoustics and electroencephalography (EEG) in male and female human listeners to examine potential effects of hearing loss on EEG correlates of speech envelope synchronization in cortex. Behaviorally, older hearing-impaired (HI) listeners showed degraded speech-in-noise recognition and reduced temporal acuity compared with age-matched normal-hearing (NH) controls. During EEG recordings, we used a selective attention task with two spatially separated simultaneous speech streams where NH and HI listeners both showed high speech recognition performance. Low-frequency (<10 Hz) envelope-entrained EEG responses were enhanced in the HI listeners, both for the attended speech, but also for tone sequences modulated at slow rates (4 Hz) during passive listening. Compared with the attended speech, responses to the ignored stream were found to be reduced in both HI and NH listeners, allowing for the attended target to be classified from single-trial EEG data with similar high accuracy in the two groups. However, despite robust attention-modulated speech entrainment, the HI listeners rated the competing speech task to be more difficult. These results suggest that speech-in-noise problems experienced by older HI listeners are not necessarily associated with degraded attentional selection.SIGNIFICANCE STATEMENT People with age-related sensorineural hearing loss often struggle to follow speech in the presence of competing talkers. It is currently unclear whether hearing impairment may impair the ability to use selective attention to suppress distracting speech in situations when the distractor is well segregated from the target. Here, we report amplified envelope-entrained cortical EEG responses to attended speech and to simple tones modulated at speech rates (4 Hz) in listeners with age-related hearing loss. Critically, despite increased self-reported listening difficulties, cortical synchronization to speech mixtures was robustly modulated by selective attention in listeners with hearing loss. This allowed the attended talker to be classified from single-trial EEG responses with high accuracy in both older hearing-impaired listeners and age-matched normal-hearing controls.

Speech intelligibility in a realistic virtual sound environment.

In the present study, speech intelligibility was evaluated in realistic, controlled conditions. "Critical sound scenarios" were defined as acoustic scenes that hearing aid users considered important, difficult, and common through ecological momentary assessment. These sound scenarios were acquired in the real world using a spherical microphone array and reproduced inside a loudspeaker-based virtual sound environment (VSE) using Ambisonics. Speech reception thresholds (SRT) were measured for normal-hearing (NH) and hearing-impaired (HI) listeners, using sentences from the Danish hearing in noise test, spatially embedded in the acoustic background of an office meeting sound scenario. In addition, speech recognition scores (SRS) were obtained at a fixed signal-to-noise ratio (SNR) of -2.5 dB, corresponding to the median conversational SNR in the office meeting. SRTs measured in the realistic VSE-reproduced background were significantly higher for NH and HI listeners than those obtained with artificial noise presented over headphones, presumably due to an increased amount of modulation masking and a larger cognitive effort required to separate the target speech from the intelligible interferers in the realistic background. SRSs obtained at the fixed SNR in the realistic background could be used to relate the listeners' SI to the potential challenges they experience in the real world.

Guided ecological momentary assessment in real and virtual sound environments.

Ecological momentary assessment (EMA) outcome measures can relate people's subjective auditory experience to their objective acoustical reality. While highly realistic, EMA data often contain considerable variability, such that it can be difficult to interpret the results with respect to differences in people's hearing ability. To address this challenge, a method for "guided" EMA is proposed and evaluated. Accompanied and instructed by a guide, normal-hearing participants carried out specific passive and active listening tasks inside a real-world public lunch scenario and answered EMA questionnaires related to aspects of spatial hearing, listening ability, quality, and effort. In situ speech and background noise levels were tracked, allowing the guided EMA task to be repeated inside two acoustically matched, loudspeaker-based laboratory environments: a 64-channel virtual sound environment (VSE) and a three-channel audiology clinic setup. Results showed that guided EMA provided consistent passive listening assessments across participants and conditions. During active listening, the clinic setup was found to be less challenging than the real-world and the VSE conditions. The proposed guided EMA approach may provide more focused real-world assessments and can be applied in realistic laboratory settings to aid the development of ecologically valid hearing testing.

A method for realistic, conversational signal-to-noise ratio estimation.

The analysis of real-world conversational signal-to-noise ratios (SNRs) can provide insight into people's communicative strategies and difficulties and guide the development of hearing devices. However, measuring SNRs accurately is challenging in everyday recording conditions in which only a mixture of sound sources can be captured. This study introduces a method for accurate in situ SNR estimation where the speech signal of a target talker in natural conversation is captured by a cheek-mounted microphone, adjusted for free-field conditions and convolved with a measured impulse response to estimate its power at the receiving talker. A microphone near the receiver provides the noise-only component through voice activity detection. The method is applied to in situ recordings of conversations in two real-world sound scenarios. It is shown that the broadband speech level and SNR distributions are estimated more accurately by the proposed method compared to a typical single-channel method, especially in challenging, low-SNR environments. The application of the proposed two-channel method may render more realistic estimates of conversational SNRs and provide valuable input to hearing instrument processing strategies whose operating points are determined by accurate SNR estimates.

Cortical oscillations and entrainment in speech processing during working memory load.

Neuronal oscillations are thought to play an important role in working memory (WM) and speech processing. Listening to speech in real-life situations is often cognitively demanding but it is unknown whether WM load influences how auditory cortical activity synchronizes to speech features. Here, we developed an auditory n-back paradigm to investigate cortical entrainment to speech envelope fluctuations under different degrees of WM load. We measured the electroencephalogram, pupil dilations and behavioural performance from 22 subjects listening to continuous speech with an embedded n-back task. The speech stimuli consisted of long spoken number sequences created to match natural speech in terms of sentence intonation, syllabic rate and phonetic content. To burden different WM functions during speech processing, listeners performed an n-back task on the speech sequences in different levels of background noise. Increasing WM load at higher n-back levels was associated with a decrease in posterior alpha power as well as increased pupil dilations. Frontal theta power increased at the start of the trial and increased additionally with higher n-back level. The observed alpha-theta power changes are consistent with visual n-back paradigms suggesting general oscillatory correlates of WM processing load. Speech entrainment was measured as a linear mapping between the envelope of the speech signal and low-frequency cortical activity (< 13 Hz). We found that increases in both types of WM load (background noise and n-back level) decreased cortical speech envelope entrainment. Although entrainment persisted under high load, our results suggest a top-down influence of WM processing on cortical speech entrainment.

The effect of spatial energy spread on sound image size and speech intelligibility.

This study explored the relationship between perceived sound image size and speech intelligibility for sound sources reproduced over loudspeakers. Sources with varying degrees of spatial energy spread were generated using ambisonics processing. Young normal-hearing listeners estimated sound image size as well as performed two spatial release from masking (SRM) tasks with two symmetrically arranged interfering talkers. Either the target-to-masker ratio or the separation angle was varied adaptively. Results showed that the sound image size did not change systematically with the energy spread. However, a larger energy spread did result in a decreased SRM. Furthermore, the listeners needed a greater angular separation angle between the target and the interfering sources for sources with a larger energy spread. Further analysis revealed that the method employed to vary the energy spread did not lead to systematic changes in the interaural cross correlations. Future experiments with competing talkers using ambisonics or similar methods may consider the resulting energy spread in relation to the minimum separation angle between sound sources in order to avoid degradations in speech intelligibility.

Effects of Hearing Loss and Fast-Acting Compression on Amplitude Modulation Perception and Speech Intelligibility.

OBJECTIVE: The purpose was to investigate the effects of hearing-loss and fast-acting compression on speech intelligibility and two measures of temporal modulation sensitivity. DESIGN: Twelve adults with normal hearing (NH) and 16 adults with mild to moderately severe sensorineural hearing loss were tested. Amplitude modulation detection and modulation-depth discrimination (MDD) thresholds with sinusoidal carriers of 1 or 5 kHz and modulators in the range from 8 to 256 Hz were used as measures of temporal modulation sensitivity. Speech intelligibility was assessed by obtaining speech reception thresholds in stationary and fluctuating background noise. All thresholds were obtained with and without compression (using a fixed compression ratio of 2:1). RESULTS: For modulation detection, the thresholds were similar or lower for the group with hearing loss than for the group with NH. In contrast, the MDD thresholds were higher for the group with hearing loss than for the group with NH. Fast-acting compression increased the modulation detection thresholds, while no effect of compression on the MDD thresholds was observed. The speech reception thresholds obtained in stationary noise were slightly increased in the compression condition relative to the linear processing condition, whereas no difference in the speech reception thresholds obtained in fluctuating noise was observed. For the group with NH, individual differences in the MDD thresholds could account for 72% of the variability in the speech reception thresholds obtained in stationary noise, whereas the correlation was insignificant for the hearing-loss group. CONCLUSIONS: Fast-acting compression can restore modulation detection thresholds for listeners with hearing loss to the values observed for listeners with NH. Despite this normalization of the modulation detection thresholds, compression does not seem to provide a benefit for speech intelligibility. Furthermore, fast-acting compression may not be able to restore MDD thresholds to the values observed for listeners with NH, suggesting that the two measures of amplitude modulation sensitivity represent different aspects of temporal processing. For listeners with NH, the ability to discriminate modulation depth was highly correlated with speech intelligibility in stationary noise.

A speech-based computational auditory signal processing and perception model.

A new speech intelligibility prediction model is presented which is based on the Computational Auditory Signal Processing and Perception model (CASP) of Jepsen, Ewert, and Dau [(2008). J. Acoust. Soc. Am. 124(1), 422-438]. The model combines a non-linear auditory-inspired preprocessing with a backend based on the cross-correlation between the clean and the degraded speech representations in the modulation envelope domain. Several speech degradation and speech enhancement algorithms were considered to study the ability of the model to predict data from normal-hearing listeners. Degradations of speech intelligibility due to additive noise, phase-jitter distortion, and single-channel noise reduction as well as improved speech intelligibility due to ideal binary mask processing are shown to be successfully accounted for by the model. Furthermore, the model reflects stimulus-level dependent effects of auditory perception, including audibility limitations at low levels and degraded speech intelligibility at high levels. Given its realistic non-linear auditory processing frontend, the speech-based computational auditory signal processing and perception model may provide a valuable computational framework for studying the effects of sensorineural hearing impairment on speech intelligibility.

Predicting the effects of periodicity on the intelligibility of masked speech: An evaluation of different modelling approaches and their limitations.

Four existing speech intelligibility models with different theoretical assumptions were used to predict previously published behavioural data. Those data showed that complex tones with pitch-related periodicity are far less effective maskers of speech than aperiodic noise. This so-called masker-periodicity benefit (MPB) far exceeded the fluctuating-masker benefit (FMB) obtained from slow masker envelope fluctuations. In contrast, the normal-hearing listeners hardly benefitted from periodicity in the target speech. All tested models consistently underestimated MPB and FMB, while most of them also overestimated the intelligibility of vocoded speech. To understand these shortcomings, the internal signal representations of the models were analysed in detail. The best-performing model, the correlation-based version of the speech-based envelope power spectrum model (sEPSMcorr), combined an auditory processing front end with a modulation filterbank and a correlation-based back end. This model was then modified to further improve the predictions. The resulting second version of the sEPSMcorr outperformed the original model with all tested maskers and accounted for about half the MPB, which can be attributed to reduced modulation masking caused by the periodic maskers. However, as the sEPSMcorr2 failed to account for the other half of the MPB, the results also indicate that future models should consider the contribution of pitch-related effects, such as enhanced stream segregation, to further improve their predictive power.

The impact of noise power estimation on speech intelligibility in cochlear-implant speech coding strategies.

The advanced combination encoder (ACE™) is an established speech-coding strategy in cochlear-implant processing that selects a number of frequency channels based on amplitudes. However, speech intelligibility outcomes with this strategy are limited in noisy conditions. To improve speech intelligibility, either noise-dominant channels can be attenuated prior to ACE™ with noise reduction or, alternatively, channels can be selected based on estimated signal-to-noise ratios. A noise power estimation stage is, therefore, required. This study investigated the impact of noise power estimation in noise-reduction and channel-selection strategies. Results imply that estimation with improved noise-tracking capabilities does not necessarily translate into increased speech intelligibility.

Influence of talker discontinuity on cortical dynamics of auditory spatial attention.

In everyday acoustic scenes, listeners face the challenge of selectively attending to a sound source and maintaining attention on that source long enough to extract meaning. This task is made more daunting by frequent perceptual discontinuities in the acoustic scene: talkers move in space and conversations switch from one speaker to another in a background of many other sources. The inherent dynamics of such switches directly impact our ability to sustain attention. Here we asked how discontinuity in talker voice affects the ability to focus auditory attention to sounds from a particular location as well as neural correlates of underlying processes. During electroencephalography recordings, listeners attended to a stream of spoken syllables from one direction while ignoring distracting syllables from a different talker from the opposite hemifield. On some trials, the talker switched locations in the middle of the streams, creating a discontinuity. This switch disrupted attentional modulation of cortical responses; specifically, event-related potentials evoked by syllables in the to-be-attended direction were suppressed and power in alpha oscillations (8-12 Hz) were reduced following the discontinuity. Importantly, at an individual level, the ability to maintain attention to a target stream and report its content, despite the discontinuity, correlates with the magnitude of the disruption of these cortical responses. These results have implications for understanding cortical mechanisms supporting attention. The changes in the cortical responses may serve as a predictor of how well individuals can communicate in complex acoustic scenes and may help in the development of assistive devices and interventions to aid clinical populations.

Localization of broadband sounds carrying interaural time differences: Effects of frequency, reference location, and interaural coherence.

The auditory processes involved in the localization of sounds in rooms are still poorly understood. The present study investigated the auditory system's across-frequency processing of interaural time differences (ITDs) and the impact of the interaural coherence (IC) of the stimuli in ITD discrimination and localization. First, ITD discrimination thresholds were measured as a function of signal frequency, reference ITD, and IC using critical-band wide noises. The resulting data were fitted with a set of analytical functions and ITD weights were derived using concepts from signal detection theory. Inspired by the weighted-image model [Stern, Zeiberg, and Trahiotis. (1988). J. Acoust. Soc. Am. 84, 156-165], the derived ITD weights were then integrated in a simplified localization model using an optimal combination of ITD information across frequency. To verify this model, a series of localization experiments were conducted using broadband noise in which ITD and IC were varied across frequency. The model predictions were in good agreement with the experimental data, supporting the assumption that the auditory system performs a weighted integration of ITD information across frequency to localize a sound source. The results could be valuable for the design of new paradigms to measure localization in more complex acoustic conditions and may provide constraints for future localization models.