Pitch discrimination in electric hearing with inconsistent and consistent amplitude-modulation and inter-pulse rate cues.

Users of cochlear implants (CIs) struggle in situations that require selective hearing to focus on a target source while ignoring other sources. One major reason for that is the limited access to timing cues such as temporal pitch or interaural time differences (ITDs). Various approaches to improve timing-cue sensitivity while maintaining speech understanding have been proposed, among them inserting extra pulses with short inter-pulse intervals (SIPIs) into amplitude-modulated (AM) high-rate pulse trains. Indeed, SIPI rates matching the naturally occurring AM rates improve pitch discrimination. For ITD, however, low SIPI rates are required, potentially mismatching the naturally occurring AM rates and thus creating unknown pitch effects. In this study, we investigated the perceptual contribution of AM and SIPI rate to pitch discrimination in five CI listeners and with two AM depths (0.1 and 0.5). Our results show that the SIPI-rate cue generally dominated the percept for both consistent and inconsistent cues. When tested with inconsistent cues, also the AM rate contributed, however, at the large AM depth only. These findings have implications when aiming at jointly improving temporal-pitch and ITD sensitivity in a future mixed-rate stimulation approach.

Dynamic spectral cues do not affect human sound localization during small head movements.

Natural listening involves a constant deployment of small head movement. Spatial listening is facilitated by head movements, especially when resolving front-back confusions, an otherwise common issue during sound localization under head-still conditions. The present study investigated which acoustic cues are utilized by human listeners to localize sounds using small head movements (below ±10° around the center). Seven normal-hearing subjects participated in a sound localization experiment in a virtual reality environment. Four acoustic cue stimulus conditions were presented (full spectrum, flattened spectrum, frozen spectrum, free-field) under three movement conditions (no movement, head rotations over the yaw axis and over the pitch axis). Localization performance was assessed using three metrics: lateral and polar precision error and front-back confusion rate. Analysis through mixed-effects models showed that even small yaw rotations provide a remarkable decrease in front-back confusion rate, whereas pitch rotations did not show much of an effect. Furthermore, MSS cues improved localization performance even in the presence of dITD cues. However, performance was similar between stimuli with and without dMSS cues. This indicates that human listeners utilize the MSS cues before the head moves, but do not rely on dMSS cues to localize sounds when utilizing small head movements.

A comparative study of eight human auditory models of monaural processing.

A number of auditory models have been developed using diverging approaches, either physiological or perceptual, but they share comparable stages of signal processing, as they are inspired by the same constitutive parts of the auditory system. We compare eight monaural models that are openly accessible in the Auditory Modelling Toolbox. We discuss the considerations required to make the model outputs comparable to each other, as well as the results for the following model processing stages or their equivalents: Outer and middle ear, cochlear filter bank, inner hair cell, auditory nerve synapse, cochlear nucleus, and inferior colliculus. The discussion includes a list of recommendations for future applications of auditory models.

Sound Externalization: A Review of Recent Research.

Sound externalization, or the perception that a sound source is outside of the head, is an intriguing phenomenon that has long interested psychoacousticians. While previous reviews are available, the past few decades have produced a substantial amount of new data.In this review, we aim to synthesize those data and to summarize advances in our understanding of the phenomenon. We also discuss issues related to the definition and measurement of sound externalization and describe quantitative approaches that have been taken to predict the outcomes of externalization experiments. Last, sound externalization is of practical importance for many kinds of hearing technologies. Here, we touch on two examples, discussing the role of sound externalization in augmented/virtual reality systems and bringing attention to the somewhat overlooked issue of sound externalization in wearers of hearing aids.

Temporal-pitch sensitivity in electric hearing with amplitude modulation and inserted pulses with short inter-pulse intervals.

Listeners with cochlear implants (CIs) typically show poor sensitivity to the temporal-envelope pitch of high-rate pulse trains. Sensitivity to interaural time differences improves when adding pulses with short inter-pulse intervals (SIPIs) to high-rate pulse trains. In the current study, monaural temporal-pitch sensitivity with SIPI pulses was investigated for six CI listeners. Amplitude-modulated single-electrode stimuli, representing the coding of the fundamental frequency (F0) in the envelope of a high-rate carrier, were used. Two SIPI-insertion approaches, five modulation depths, two typical speech-F0s, and two carrier rates were tested. SIPI pulses were inserted either in every amplitude-modulation period (full-rate SIPI) to support the F0 cue or in every other amplitude-modulation period (half-rate SIPI) to circumvent a potential rate limitation at higher F0s. The results demonstrate that full-rate SIPI pulses improve temporal-pitch sensitivity across F0s and particularly at low modulation depths where envelope-pitch cues are weak. The half-rate SIPI pulses did not circumvent the limitation and further increased variability across listeners. Further, no effect of the carrier rate was found. Thus, the SIPI approach appears to be a promising approach to enhance CI listeners' access to temporal-envelope pitch cues at pulse rates used clinically.

Improving Interaural Time Difference Sensitivity Using Short Inter-pulse Intervals with Amplitude-Modulated Pulse Trains in Bilateral Cochlear Implants.

Interaural time differences (ITDs) at low frequencies are important for sound localization and spatial speech unmasking. These ITD cues are not encoded in commonly used envelope-based stimulation strategies for cochlear implants (CIs) using high pulse rates. However, ITD sensitivity can be improved by adding extra pulses with short inter-pulse intervals (SIPIs) in unmodulated high-rate trains. Here, we investigated whether this improvement also applies to amplitude-modulated (AM) high-rate pulse trains. To this end, we systematically varied the temporal position of SIPI pulses within the envelope cycle (SIPI phase), the fundamental frequency (F0) of AM (125 Hz and 250 Hz), and AM depth (from 0.1 to 0.9). Stimuli were presented at an interaurally place-matched electrode pair at a reference pulse rate of 1000 pulses/s. Participants performed an ITD-based left/right discrimination task. SIPI insertion resulted in improved ITD sensitivity throughout the range of modulation depths and for both male and female F0s. The improvements were largest for insertion at and around the envelope peak. These results are promising for conveying salient ITD cues at high pulse rates commonly used to encode speech information.

Ventriloquist Illusion Produced With Virtual Acoustic Spatial Cues and Asynchronous Audiovisual Stimuli in Both Young and Older Individuals.

Ventriloquist illusion, the change in perceived location of an auditory stimulus when a synchronously presented but spatially discordant visual stimulus is added, has been previously shown in young healthy populations to be a robust paradigm that mainly relies on automatic processes. Here, we propose ventriloquist illusion as a potential simple test to assess audiovisual (AV) integration in young and older individuals. We used a modified version of the illusion paradigm that was adaptive, nearly bias-free, relied on binaural stimulus representation using generic head-related transfer functions (HRTFs) instead of multiple loudspeakers, and tested with synchronous and asynchronous presentation of AV stimuli (both tone and speech). The minimum audible angle (MAA), the smallest perceptible difference in angle between two sound sources, was compared with or without the visual stimuli in young and older adults with no or minimal sensory deficits. The illusion effect, measured by means of MAAs implemented with HRTFs, was observed with both synchronous and asynchronous visual stimulus, but only with tone and not speech stimulus. The patterns were similar between young and older individuals, indicating the versatility of the modified ventriloquist illusion paradigm.

Introducing Short Interpulse Intervals in High-Rate Pulse Trains Enhances Binaural Timing Sensitivity in Electric Hearing.

Common envelope-based stimulation strategies for cochlear implants (CIs) use relatively high carrier rates in order to properly encode the speech envelope. For such rates, CI listeners show poor sensitivity to interaural time differences (ITDs), which are important for horizontal-plane sound localization and spatial unmasking of speech. Based on the findings from previous studies, we predicted that ITD sensitivity can be enhanced by including pulses with short interpulse intervals (SIPIs), to a 1000-pulses-per-second (pps) reference pulse train. We measured the sensitivity of eight bilateral CI listeners to ITD while systematically varying both the rate at which SIPIs are introduced ("SIPI rate") and the time interval between the two pulses forming a SIPI ("SIPI fraction"). Results showed largely enhanced ITD sensitivity relative to the reference condition, with the size of the improvement increasing with decreasing SIPI rate and decreasing SIPI fraction. For the lowest SIPI fraction, insertion of extra pulses brought ITD sensitivity to the level measured for low-rate pulse trains with rates matching the SIPI rates. The results appear promising for the goal of enhancing ITD sensitivity with envelope-based CI strategies by inserting SIPI pulses at strategic times in speech stimuli.

A framework for testing and comparing binaural models.

Auditory research has a rich history of combining experimental evidence with computational simulations of auditory processing in order to deepen our theoretical understanding of how sound is processed in the ears and in the brain. Despite significant progress in the amount of detail and breadth covered by auditory models, for many components of the auditory pathway there are still different model approaches that are often not equivalent but rather in conflict with each other. Similarly, some experimental studies yield conflicting results which has led to controversies. This can be best resolved by a systematic comparison of multiple experimental data sets and model approaches. Binaural processing is a prominent example of how the development of quantitative theories can advance our understanding of the phenomena, but there remain several unresolved questions for which competing model approaches exist. This article discusses a number of current unresolved or disputed issues in binaural modelling, as well as some of the significant challenges in comparing binaural models with each other and with the experimental data. We introduce an auditory model framework, which we believe can become a useful infrastructure for resolving some of the current controversies. It operates models over the same paradigms that are used experimentally. The core of the proposed framework is an interface that connects three components irrespective of their underlying programming language: The experiment software, an auditory pathway model, and task-dependent decision stages called artificial observers that provide the same output format as the test subject.

Asymmetries in behavioral and neural responses to spectral cues demonstrate the generality of auditory looming bias.

Studies of auditory looming bias have shown that sources increasing in intensity are more salient than sources decreasing in intensity. Researchers have argued that listeners are more sensitive to approaching sounds compared with receding sounds, reflecting an evolutionary pressure. However, these studies only manipulated overall sound intensity; therefore, it is unclear whether looming bias is truly a perceptual bias for changes in source distance, or only in sound intensity. Here we demonstrate both behavioral and neural correlates of looming bias without manipulating overall sound intensity. In natural environments, the pinnae induce spectral cues that give rise to a sense of externalization; when spectral cues are unnatural, sounds are perceived as closer to the listener. We manipulated the contrast of individually tailored spectral cues to create sounds of similar intensity but different naturalness. We confirmed that sounds were perceived as approaching when spectral contrast decreased, and perceived as receding when spectral contrast increased. We measured behavior and electroencephalography while listeners judged motion direction. Behavioral responses showed a looming bias in that responses were more consistent for sounds perceived as approaching than for sounds perceived as receding. In a control experiment, looming bias disappeared when spectral contrast changes were discontinuous, suggesting that perceived motion in distance and not distance itself was driving the bias. Neurally, looming bias was reflected in an asymmetry of late event-related potentials associated with motion evaluation. Hence, both our behavioral and neural findings support a generalization of the auditory looming bias, representing a perceptual preference for approaching auditory objects.

The relation between psychoacoustical factors and annoyance under different noise reduction conditions for railway noise.

The A-weighted sound pressure level (SPL) is commonly used to assess the effect of noise reduction measures on noise-induced annoyance. While for road traffic noise loudness seems to be a better descriptor of annoyance, for railway noise a systematic investigation seems to be lacking. Thus, in this study, the relation between annoyance and perceptually motivated descriptors was investigated for various conditions of binaural recordings of pass-bys of cargo and passenger trains. The conditions included free field and spectral mitigations caused by a 4 m high noise barrier, a 1 m high noise barrier close to the track, and rail dampers. Forty listeners performed a free magnitude estimation of annoyance for different presentation levels and the ratings were fit to various models. Further, level changes required to evoke a noticeable change in annoyance (annoyance thresholds) were acquired. The models based on the A-weighted SPL explained the ratings and thresholds better when the reduction measure was explicitly provided as a parameter. However, the optimal models were loudness-level-based models, which were able to better describe the annoyance, even independently of the reduction measure. Both experiments underline the effectiveness of loudness when describing the annoyance in the area of railway noise reduction.

Binaural timing information in electric hearing at low rates: Effects of inaccurate encoding and loudness.

Stimulation strategies for cochlear implants potentially impose timing limitations that may hinder the correct encoding and representation of interaural time differences (ITDs) in realistic bilateral signals. This study aimed to specify the tolerable room for inaccurate encoding of ITDs at low rates by investigating the perceptual degradation due to the removal of individual pulses at various levels of loudness. Unmodulated, 100-pulses-per-second pulse trains were presented at a single, interaurally pitch-matched electrode pair. In experiment I, ITD thresholds were measured applying different degrees of bilateral, interaurally-uncorrelated pulse removal. The ITD sensitivity deteriorated with increasing degree of pulse removal, with significant deterioration for degrees of 16% or greater. In experiment II, the interaction between loudness and pulse removal was investigated. Louder stimuli yielded better ITD sensitivity, however, no further improvement was found for stimuli louder than "medium." When removing 8% of the pulses, the ITD sensitivity deteriorated significantly across the entire loudness range tested. A loudness-induced compensation for the deterioration of ITD sensitivity due to pulse removal seems to be feasible for soft stimuli but not for medium or loud stimuli. Overall, our findings suggest that the degree of pulse removal employed in low-rate channels within coding strategies should not exceed 8%.

The role of compression in the simultaneous masker phase effect.

Peripheral compression is believed to play a major role in the masker phase effect (MPE). While compression is almost instantaneous, activation of the efferent system reduces compression in a temporally evolving manner. To study the role of efferent-controlled compression in the MPE, in experiment 1, simultaneous masking of a 30-ms 4-kHz tone by 40-ms Schroeder-phase harmonic complexes was measured with on- and off-frequency precursors as a function of masker phase curvature for two masker levels (60 and 90 dB sound pressure level). The MPE was quantified by the threshold range [min/max difference (MMD)] across the phase curvatures. For the 60-dB condition, the presence of on-frequency precursor decreased the MMD from 10 to 5 dB. Experiment 2 studied the role of the precursor on the auditory filter's bandwidth. The on-frequency precursor was found to increase the bandwidth, an effect incorporated in the subsequent modeling. A model of the auditory periphery including cochlear filtering and basilar membrane compression generally underestimated the MMDs. A model based on two-step compression, including compression of inner hair cells, accounted for the MMDs across precursor and level conditions. Overall, the observed precursor effects and the model predictions suggest an important role of compression in the simultaneous MPE.

Erratum: Modeling sound-source localization in sagittal planes for human listeners [J. Acoust. Soc. Am. 136, 791-802 (2014)].

This erratum concerns Eq. (4) of the original article, which defines the distance metric of the comparison process of the sagittal-plane sound localization model. The distance metric was actually implemented as a mean absolute difference but was erroneously described as a L1-norm difference.

Modeling the Effects of Sensorineural Hearing Loss on Sound Localization in the Median Plane.

Listeners use monaural spectral cues to localize sound sources in sagittal planes (along the up-down and front-back directions). How sensorineural hearing loss affects the salience of monaural spectral cues is unclear. To simulate the effects of outer-hair-cell (OHC) dysfunction and the contribution of different auditory-nerve fiber types on localization performance, we incorporated a nonlinear model of the auditory periphery into a model of sagittal-plane sound localization for normal-hearing listeners. The localization model was first evaluated in its ability to predict the effects of spectral cue modifications for normal-hearing listeners. Then, we used it to simulate various degrees of OHC dysfunction applied to different types of auditory-nerve fibers. Predicted localization performance was hardly affected by mild OHC dysfunction but was strongly degraded in conditions involving severe and complete OHC dysfunction. These predictions resemble the usually observed degradation in localization performance induced by sensorineural hearing loss. Predicted localization performance was best when preserving fibers with medium spontaneous rates, which is particularly important in view of noise-induced hearing loss associated with degeneration of this fiber type. On average across listeners, predicted localization performance was strongly related to level discrimination sensitivity of auditory-nerve fibers, indicating an essential role of this coding property for localization accuracy in sagittal planes.

Channel Interaction and Current Level Affect Across-Electrode Integration of Interaural Time Differences in Bilateral Cochlear-Implant Listeners.

Sensitivity to interaural time differences (ITDs) is important for sound localization. Normal-hearing listeners benefit from across-frequency processing, as seen with improved ITD thresholds when consistent ITD cues are presented over a range of frequency channels compared with when ITD information is only presented in a single frequency channel. This study aimed to clarify whether cochlear-implant (CI) listeners can make use of similar processing when being stimulated with multiple interaural electrode pairs transmitting consistent ITD information. ITD thresholds for unmodulated, 100-pulse-per-second pulse trains were measured in seven bilateral CI listeners using research interfaces. Consistent ITDs were presented at either one or two electrode pairs at different current levels, allowing for comparisons at either constant level per component electrode or equal overall loudness. Different tonotopic distances between the pairs were tested in order to clarify the potential influence of channel interaction. Comparison of ITD thresholds between double pairs and the respective single pairs revealed systematic effects of tonotopic separation and current level. At constant levels, performance with double-pair stimulation improved compared with single-pair stimulation but only for large tonotopic separation. Comparisons at equal overall loudness revealed no benefit from presenting ITD information at two electrode pairs for any tonotopic spacing. Irrespective of electrode-pair configuration, ITD sensitivity improved with increasing current level. Hence, the improved ITD sensitivity for double pairs found for a large tonotopic separation and constant current levels seems to be due to increased loudness. The overall data suggest that CI listeners can benefit from combining consistent ITD information across multiple electrodes, provided sufficient stimulus levels and that stimulating electrode pairs are widely spaced.

A priori mesh grading for the numerical calculation of the head-related transfer functions.

Head-related transfer functions (HRTFs) describe the directional filtering of the incoming sound caused by the morphology of a listener's head and pinnae. When an accurate model of a listener's morphology exists, HRTFs can be calculated numerically with the boundary element method (BEM). However, the general recommendation to model the head and pinnae with at least six elements per wavelength renders the BEM as a time-consuming procedure when calculating HRTFs for the full audible frequency range. In this study, a mesh preprocessing algorithm is proposed, viz., a priori mesh grading, which reduces the computational costs in the HRTF calculation process significantly. The mesh grading algorithm deliberately violates the recommendation of at least six elements per wavelength in certain regions of the head and pinnae and varies the size of elements gradually according to an a priori defined grading function. The evaluation of the algorithm involved HRTFs calculated for various geometric objects including meshes of three human listeners and various grading functions. The numerical accuracy and the predicted sound-localization performance of calculated HRTFs were analyzed. A-priori mesh grading appeared to be suitable for the numerical calculation of HRTFs in the full audible frequency range and outperformed uniform meshes in terms of numerical errors, perception based predictions of sound-localization performance, and computational costs.

A framework for geometry acquisition, 3-D printing, simulation, and measurement of head-related transfer functions with a focus on hearing-assistive devices.

Individual head-related transfer functions (HRTFs) are essential in applications like fitting hearing-assistive devices (HADs) for providing accurate sound localization performance. Individual HRTFs are usually obtained through intricate acoustic measurements. This paper investigates the use of a three-dimensional (3D) head model for acquisition of individual HRTFs. Two aspects were investigated; whether a 3D-printed model can replace measurements on a human listener and whether numerical simulations can replace acoustic measurements. For this purpose, HRTFs were acoustically measured for four human listeners and for a 3D printed head model of one of these listeners. Further, HRTFs were simulated by applying the finite element method to the 3D head model. The monaural spectral features and spectral distortions were very similar between re-measurements and between human and printed measurements, however larger deviations were observed between measurement and simulation. The binaural cues were in agreement among all HRTFs of the same listener, indicating that the 3D model is able to provide localization cues potentially accessible to HAD users. Hence, the pipeline of geometry acquisition, printing, and acoustic measurements or simulations, seems to be a promising step forward towards in-silico design of HADs.

Efficient Approximation of Head-Related Transfer Functions in Subbands for Accurate Sound Localization.

Head-related transfer functions (HRTFs) describe the acoustic filtering of incoming sounds by the human morphology and are essential for listeners to localize sound sources in virtual auditory displays. Since rendering complex virtual scenes is computationally demanding, we propose four algorithms for efficiently representing HRTFs in subbands, i.e., as an analysis filterbank (FB) followed by a transfer matrix and a synthesis FB. All four algorithms use sparse approximation procedures to minimize the computational complexity while maintaining perceptually relevant HRTF properties. The first two algorithms separately optimize the complexity of the transfer matrix associated to each HRTF for fixed FBs. The other two algorithms jointly optimize the FBs and transfer matrices for complete HRTF sets by two variants. The first variant aims at minimizing the complexity of the transfer matrices, while the second one does it for the FBs. Numerical experiments investigate the latency-complexity trade-off and show that the proposed methods offer significant computational savings when compared with other available approaches. Psychoacoustic localization experiments were modeled and conducted to find a reasonable approximation tolerance so that no significant localization performance degradation was introduced by the subband representation.

Modeling Localization of Amplitude-Panned Virtual Sources in Sagittal Planes.

Vector-base amplitude panning (VBAP) aims at creating virtual sound sources at arbitrary directions within multichannel sound reproduction systems. However, VBAP does not consistently produce listener-specific monaural spectral cues that are essential for localization of sound sources in sagittal planes, including the front-back and up-down dimensions. In order to better understand the limitations of VBAP, a functional model approximating human processing of spectro-spatial information was applied to assess accuracy in sagittal-plane localization of virtual sources created by means of VBAP. First, we evaluated VBAP applied on two loudspeakers in the median plane, and then we investigated the directional dependence of the localization accuracy in several three-dimensional loudspeaker arrangements designed in layers of constant elevation. The model predicted a strong dependence on listeners' individual head-related transfer functions, on virtual source directions, and on loudspeaker arrangements. In general, the simulations showed a systematic degradation with increasing polar-angle span between neighboring loudspeakers. For the design of VBAP systems, predictions suggest that spans up to 40° polar angle yield a good trade-off between system complexity and localization accuracy. Special attention should be paid to the frontal region where listeners are most sensitive to deviating spectral cues.