Reverberation Degrades Pitch Perception but Not Mandarin Tone and Vowel Recognition of Cochlear Implant Users.

OBJECTIVES: The primary goal of this study was to investigate the effects of reverberation on Mandarin tone and vowel recognition of cochlear implant (CI) users and normal-hearing (NH) listeners. To understand the performance of Mandarin tone recognition, this study also measured participants' pitch perception and the availability of temporal envelope cues in reverberation. DESIGN: Fifteen CI users and nine NH listeners, all Mandarin speakers, were asked to recognize Mandarin single-vowels produced in four lexical tones and rank harmonic complex tones in pitch with different reverberation times (RTs) from 0 to 1 second. Virtual acoustic techniques were used to simulate rooms with different degrees of reverberation. Vowel duration and correlation between amplitude envelope and fundamental frequency (F0) contour were analyzed for different tones as a function of the RT. RESULTS: Vowel durations of different tones significantly increased with longer RTs. Amplitude-F0 correlation remained similar for the falling Tone 4 but greatly decreased for the other tones in reverberation. NH listeners had robust pitch-ranking, tone recognition, and vowel recognition performance as the RT increased. Reverberation significantly degraded CI users' pitch-ranking thresholds but did not significantly affect the overall scores of tone and vowel recognition with CIs. Detailed analyses of tone confusion matrices showed that CI users reduced the flat Tone-1 responses but increased the falling Tone-4 responses in reverberation, possibly due to the falling amplitude envelope of late reflections after the original vowel segment. CI users' tone recognition scores were not correlated with their pitch-ranking thresholds. CONCLUSIONS: NH listeners can reliably recognize Mandarin tones in reverberation using salient pitch cues from spectral and temporal fine structures. However, CI users have poorer pitch perception using F0-related amplitude modulations that are reduced in reverberation. Reverberation distorts speech amplitude envelopes, which affect the distribution of tone responses but not the accuracy of tone recognition with CIs. Recognition of vowels with stationary formant trajectories is not affected by reverberation for both NH listeners and CI users, regardless of the available spectral resolution. Future studies should test how the relatively stable vowel and tone recognition may contribute to sentence recognition in reverberation of Mandarin-speaking CI users.

The impact of head-related impulse response delay treatment strategy on psychoacoustic cue reconstruction errors from virtual loudspeaker arrays.

Known errors exist in loudspeaker array processing techniques, often degrading source localization and timbre. The goal of the present study was to use virtual loudspeaker arrays to investigate how treatment of the interaural time delay (ITD) cue from each loudspeaker impacts these errors. Virtual loudspeaker arrays rendered over headphones using head-related impulse responses (HRIRs) allow flexible control of array size. Here, three HRIR delay treatment strategies were evaluated using minimum-phase loudspeaker HRIRs: reapplying the original HRIR delays, applying the relative ITD to the contralateral ear, or separately applying the HRIR delays prior to virtual array processing. Seven array sizes were simulated, and panning techniques were used to estimate HRIRs from 3000 directions using higher-order Ambisonics, vector-base amplitude panning, and the closest loudspeaker technique. Compared to a traditional, physical array, the prior HRIR delay treatment strategy produced similar errors with a 95% reduction in the required array size. When compared to direct spherical harmonic (SH) fitting of head-related transfer functions (HRTFs), the prior delays strategy reduced errors in reconstruction accuracy of timbral and directional psychoacoustic cues. This result suggests that delay optimization can greatly reduce the number of virtual loudspeakers required for accurate rendering of acoustic scenes without SH-based HRTF representation.

Effects of Reverberation on the Relation Between Compression Speed and Working Memory for Speech-in-Noise Perception.

OBJECTIVES: Previous study has suggested that when listening in modulated noise, individuals benefit from different wide dynamic range compression (WDRC) speeds depending on their working memory ability. Reverberation reduces the modulation depth of signals and may impact the relation between WDRC speed and working memory. The purpose of this study was to examine this relation across a range of reverberant conditions. DESIGN: Twenty-eight older listeners with mild-to-moderate sensorineural hearing impairment were recruited in the present study. Individual working memory was measured using a Reading Span test. Sentences were combined with noise at two signal to noise ratios (2 and 5 dB SNR), and reverberation was simulated at a range of reverberation times (0.00, 0.75, 1.50, and 3.00 sec). Speech intelligibility was measured in listeners when listening to the sentences processed with simulated fast-acting and slow-acting WDRC conditions. RESULTS: There was a significant relation between WDRC speed and working memory with minimal or no reverberation. Consistent with previous research, this relation was such that individuals with high working memory had higher speech intelligibility with fast-acting WDRC, and individuals with low working memory performed better with slow-acting WDRC. However, at longer reverberation times, there was no relation between WDRC speed and working memory. CONCLUSIONS: Consistent with previous studies, results suggest that there is an advantage of tailoring WDRC speed based on an individual's working memory under anechoic conditions. However, the present results further suggest that there may not be such a benefit in reverberant listening environments due to reduction in signal modulation.

Speech intelligibility in rooms: Disrupting the effect of prior listening exposure.

It has been demonstrated that prior listening exposure to reverberant environments can improve speech understanding in that environment. Previous studies have shown that the buildup of this effect is brief (less than 1 s) and seems largely to be elicited by exposure to the temporal modulation characteristics of the room environment. Situations that might be expected to cause a disruption in this process have yet to be demonstrated. This study seeks to address this issue by showing what types of changes in the acoustic environment cause a breakdown of the room exposure phenomenon. Using speech carrier phrases featuring sudden changes in the acoustic environment, breakdown in the room exposure effect was observed when there was change in the late reverberation characteristics of the room that signaled a different room environment. Changes in patterns of early reflections within the same room environment did not elicit breakdown. Because the environmental situations that resulted in breakdown also resulted in substantial changes to the broadband temporal modulation characteristic of the signal reaching the ears, results from this study provide additional support for the hypothesis that the room exposure phenomenon is linked to the temporal modulation characteristics of the environment.

Effects of reverberation, background talker number, and compression release time on signal-to-noise ratio.

Wide dynamic range compression (WDRC) processing in hearing aids alters the signal-to-noise ratio (SNR) of a speech-in-noise signal. This effect depends on the modulations of the speech and noise, input SNR, and WDRC speed. The purpose of the present experiment was to examine the change in output SNR caused by the interaction between modulation characteristics and WDRC speed. Two modulation manipulations were examined: (1) reverberation and (2) variation in background talker number. Results indicated that fast-acting WDRC altered SNR more than slow-acting WDRC; however, reverberation reduced this difference. Additionally, less modulated maskers led to poorer output SNRs than modulated maskers.

Auditory distance coding in rabbit midbrain neurons and human perception: monaural amplitude modulation depth as a cue.

Mechanisms underlying sound source distance localization are not well understood. Here we tested the hypothesis that a novel mechanism can create monaural distance sensitivity: a combination of auditory midbrain neurons' sensitivity to amplitude modulation (AM) depth and distance-dependent loss of AM in reverberation. We used virtual auditory space (VAS) methods for sounds at various distances in anechoic and reverberant environments. Stimulus level was constant across distance. With increasing modulation depth, some rabbit inferior colliculus neurons increased firing rates whereas others decreased. These neurons exhibited monotonic relationships between firing rates and distance for monaurally presented noise when two conditions were met: (1) the sound had AM, and (2) the environment was reverberant. The firing rates as a function of distance remained approximately constant without AM in either environment and, in an anechoic condition, even with AM. We corroborated this finding by reproducing the distance sensitivity using a neural model. We also conducted a human psychophysical study using similar methods. Normal-hearing listeners reported perceived distance in response to monaural 1 octave 4 kHz noise source sounds presented at distances of 35-200 cm. We found parallels between the rabbit neural and human responses. In both, sound distance could be discriminated only if the monaural sound in reverberation had AM. These observations support the hypothesis. When other cues are available (e.g., in binaural hearing), how much the auditory system actually uses the AM as a distance cue remains to be determined.

Speech intelligibility in rooms: Effect of prior listening exposure interacts with room acoustics.

There is now converging evidence that a brief period of prior listening exposure to a reverberant room can influence speech understanding in that environment. Although the effect appears to depend critically on the amplitude modulation characteristic of the speech signal reaching the ear, the extent to which the effect may be influenced by room acoustics has not been thoroughly evaluated. This study seeks to fill this gap in knowledge by testing the effect of prior listening exposure or listening context on speech understanding in five different simulated sound fields, ranging from anechoic space to a room with broadband reverberation time (T60) of approximately 3 s. Although substantial individual variability in the effect was observed and quantified, the context effect was, on average, strongly room dependent. At threshold, the effect was minimal in anechoic space, increased to a maximum of 3 dB on average in moderate reverberation (T60 = 1 s), and returned to minimal levels again in high reverberation. This interaction suggests that the functional effects of prior listening exposure may be limited to sound fields with moderate reverberation (0.4 ≤ T60 ≤ 1 s).

Enhancement of speech intelligibility in reverberant rooms: role of amplitude envelope and temporal fine structure.

The temporal envelope and fine structure of speech make distinct contributions to the perception of speech in normal-hearing listeners, and are differentially affected by room reverberation. Previous work has demonstrated enhanced speech intelligibility in reverberant rooms when prior exposure to the room was provided. Here, the relative contributions of envelope and fine structure cues to this intelligibility enhancement were tested using an open-set speech corpus and virtual auditory space techniques to independently manipulate the speech cues within a simulated room. Intelligibility enhancement was observed only when the envelope was reverberant, indicating that the enhancement is envelope-based.

Time course of a perceptual enhancement effect for noise-masked speech in reverberant environments.

Speech intelligibility has been shown to improve with prior exposure to a reverberant room environment [Brandewie and Zahorik (2010). J. Acoust. Soc. Am. 128, 291-299] with a spatially separated noise masker. Here, this speech enhancement effect was examined in multiple room environments using carrier phrases of varying lengths in order to control the amount of exposure. Speech intelligibility enhancement of between 5% and 18% was observed with as little as 850 ms of exposure, although the effect's time course varied considerably with reverberation and signal-to-noise ratio. In agreement with previous work, greater speech enhancement was found for reverberant environments compared to anechoic space.

Prior listening exposure to a reverberant room improves open-set intelligibility of high-variability sentences.

Previous studies have demonstrated that speech understanding in reverberant rooms improves when listeners are given prior exposure to the room. Results from these room-adaptation studies are limited, however, because they were conducted with materials that are not representative of the high acoustic variability observed in speech signals during everyday communication. Here, room adaptation effects were measured using an open-set speech corpus with high lexical and indexical variability and virtual auditory space techniques to simulate binaural listening in rooms. Room adaptation effects of comparable magnitude to previous studies were observed, suggesting general importance for facilitating speech intelligibility in reverberation.

Phonemic restoration effect reversed in a reverberant room.

Classic demonstrations of the phonemic restoration effect show increased intelligibility of interrupted speech when the interruptions are caused by a plausible masking sound rather than by silent periods. Previous studies of this effect have been conducted exclusively under anechoic or nearly anechoic listening conditions. This study demonstrates that the effect is reversed when sounds are presented in a realistically simulated reverberant room (broadband T(60) = 1.1 s): intelligibility is greater for silent interruptions than for interruptions by unmodulated noise. Additional results suggest that the reversal is primarily due to filling silent intervals with reverberant energy from the speech signal.

Asymmetric visual capture of virtual sound sources in the distance dimension.

Visual capture describes the tendency of a sound to be mislocalized to the location of a plausible visual target. This effect, also known as the ventriloquist effect, has been extensively studied in humans, but primarily for mismatches in the angular direction between auditory and visual targets. Here, visual capture was examined in the distance dimension using a single visual target (an un-energized loudspeaker) and invisible virtual sound sources presented over headphones. The sound sources were synthesized from binaural impulse-response measurements at distances ranging from 1 to 5 m (0.25 m steps) in the semi-reverberant room (7.7 × 4.2 × 2.7 m3) in which the experiment was conducted. Listeners (n = 11) were asked whether or not the auditory target appeared to be at the same distance as the visual target. Within a block of trials, the visual target was placed at a fixed distance of 1.5, 3, or 4.5 m, and the auditory target varied randomly from trial-to-trial over the sample of measurement distances. The resulting psychometric functions were generally consistent with visual capture in distance, but the capture was asymmetric: Sound sources behind the visual target were more strongly captured than sources in front of the visual target. This asymmetry is consistent with previous reports in the literature, and is shown here to be well predicted by a simple model of sensory integration and decision in which perceived auditory space is compressed logarithmically in distance and has lower resolution than perceived visual space.

Decoding the direction of auditory motion in blind humans.

Accurate processing of nonvisual stimuli is fundamental to humans with visual impairments. In this population, moving sounds activate an occipito-temporal region thought to encompass the equivalent of monkey area MT+, but it remains unclear whether the signal carries information beyond the mere presence of motion. To address this important question, we tested whether the processing in this region retains functional properties that are critical for accurate motion processing and that are well established in the visual modality. Specifically, we focussed on the property of 'directional selectivity', because MT+ neurons in non-human primates fire preferentially to specific directions of visual motion. Recent neuroimaging studies have revealed similar properties in sighted humans by successfully decoding different directions of visual motion from fMRI activation patterns. Here we used fMRI and multivariate pattern classification to demonstrate that the direction in which a sound is moving can be reliably decoded from dorsal occipito-temporal activation in the blind. We also show that classification performance is at chance (i) in a control region in posterior parietal cortex and (ii) when motion information is removed and subjects only hear a sequence of static sounds presented at the same start and end positions. These findings reveal that information about the direction of auditory motion is present in dorsal occipito-temporal responses of blind humans. As such, this area, which appears consistent with the hMT+ complex in the sighted, provides crucial information for the generation of a veridical percept of moving non-visual stimuli.

Reverberation strength perceived by normal-hearing listeners predictable based on time-varying binaural loudness.

Previous work has explored novel binaural combinations of reverberation and the resulting perceived reverberation strength (reverberance). The present study examines the perceptual effects of additional binaural combinations of reverberation with the goal of explaining reverberance in terms of basic psychoacoustic principles. Stimuli were generated using virtual space techniques simulating a speech source 3 m to the listener's right in a moderately reverberant environment. Reverberant energy at the ears was varied systematically relative to the natural level for the environment (0-dB gain). The method of magnitude estimation was used to estimate reverberance. Four experiments were conducted. Experiment 1 tested monaural listening conditions for both left and right ears at reverberation gains from -21 dB to 0 dB. Experiment 2 tested a binaural listening condition where only reverberant energy at the ear farther from the source was manipulated (-21 dB to 0 dB). Experiment 3 tested two binaural conditions over a wider range of reverberation gains (-18 dB to +24 dB). In one condition, reverberant energy was manipulated for both ears equally. In the other condition, reverberant energy was manipulated only for the ear nearer the source. In Experiment 4, reverberant tails of the stimuli were removed to test whether listeners were able to use ongoing reverberant information to judge reverberance. The results from all experiments were found to be well predicted by a model of time-varying binaural loudness that focused on "glimpses" in time with relatively high reverberant sound energy and low direct sound energy. These findings suggest that the mechanisms underlying reverberance and loudness may be similar.

Prior listening in rooms improves speech intelligibility.

Although results from previous studies have demonstrated that the acoustic effects of a single reflection are perceptually suppressed after repeated exposure to a particular configuration of source and reflection, the extent to which this dynamic echo suppression might generalize to speech understanding in room environments with multiple reflections and reverberation is largely unknown. Here speech intelligibility was measured using the coordinate response measure corpus both with and without prior listening exposure to a reverberant room environment, which was simulated using virtual auditory space techniques. Prior room listening exposure was manipulated by presenting either a two-sentence carrier phrase that preceded the target speech, or no carrier phrase within the room environment. Results from 14 listeners indicate that with prior room exposure, masked speech reception thresholds were on average 2.7 dB lower than thresholds without exposure, an improvement in intelligibility of over 18 percentage points on average. This effect, which is shown to be absent in anechoic space and greatly reduced under monaural listening conditions, demonstrates that prior binaural exposure to reverberant rooms can improve speech intelligibility, perhaps due to a process of perceptual adaptation to the acoustics of the listening room.

Associations between speech recognition at high levels, the middle ear muscle reflex and noise exposure in individuals with normal audiograms.

It has been hypothesized that noise-induced cochlear synaptopathy in humans may result in functional deficits such as a weakened middle ear muscle reflex (MEMR) and degraded speech perception in complex environments. Although relationships between noise-induced synaptic loss and the MEMR have been demonstrated in animals, effects of noise exposure on the MEMR have not been observed in humans. The hypothesized relationship between noise exposure and speech perception has also been difficult to demonstrate conclusively. Given that the MEMR is engaged at high sound levels, relationships between speech recognition in complex listening environments and noise exposure might be more evident at high speech presentation levels. In this exploratory study with 41 audiometrically normal listeners, a combination of behavioral and physiologic measures thought to be sensitive to synaptopathy were used to determine potential links with speech recognition at high presentation levels. We found decreasing speech recognition as a function of presentation level (from 74 to 104 dBA), which was associated with reduced MEMR magnitude. We also found that reduced MEMR magnitude was associated with higher estimated lifetime noise exposure. Together, these results suggest that the MEMR may be sensitive to noise-induced synaptopathy in humans, and this may underlie functional speech recognition deficits at high sound levels.

Interactions Between Digital Noise Reduction and Reverberation: Acoustic and Behavioral Effects.

BACKGROUND: Digital noise reduction (DNR) processing is used in hearing aids to enhance perception in noise by classifying and suppressing the noise acoustics. However, the efficacy of DNR processing is not known under reverberant conditions where the speech-in-noise acoustics are further degraded by reverberation. PURPOSE: The purpose of this study was to investigate acoustic and perceptual effects of DNR processing across a range of reverberant conditions for individuals with hearing impairment. RESEARCH DESIGN: This study used an experimental design to investigate the effects of varying reverberation on speech-in-noise processed with DNR. STUDY SAMPLE: Twenty-six listeners with mild-to-moderate sensorineural hearing impairment participated in the study. DATA COLLECTION AND ANALYSIS: Speech stimuli were combined with unmodulated broadband noise at several signal-to-noise ratios (SNRs). A range of reverberant conditions with realistic parameters were simulated, as well as an anechoic control condition without reverberation. Reverberant speech-in-noise signals were processed using a spectral subtraction DNR simulation. Signals were acoustically analyzed using a phase inversion technique to quantify improvement in SNR as a result of DNR processing. Sentence intelligibility and subjective ratings of listening effort, speech naturalness, and background noise comfort were examined with and without DNR processing across the conditions. RESULTS: Improvement in SNR was greatest in the anechoic control condition and decreased as the ratio of direct to reverberant energy decreased. There was no significant effect of DNR processing on speech intelligibility in the anechoic control condition, but there was a significant decrease in speech intelligibility with DNR processing in all of the reverberant conditions. Subjectively, listeners reported greater listening effort and lower speech naturalness with DNR processing in some of the reverberant conditions. Listeners reported higher background noise comfort with DNR processing only in the anechoic control condition. CONCLUSIONS: Results suggest that reverberation affects DNR processing using a spectral subtraction algorithm in such a way that decreases the ability of DNR to reduce noise without distorting the speech acoustics. Overall, DNR processing may be most beneficial in environments with little reverberation and that the use of DNR processing in highly reverberant environments may actually produce adverse perceptual effects. Further research is warranted using commercial hearing aids in realistic reverberant environments.

Perceptually relevant parameters for virtual listening simulation of small room acoustics.

Various physical aspects of room-acoustic simulation techniques have been extensively studied and refined, yet the perceptual attributes of the simulations have received relatively little attention. Here a method of evaluating the perceptual similarity between rooms is described and tested using 15 small-room simulations based on binaural room impulse responses (BRIRs) either measured from a real room or estimated using simple geometrical acoustic modeling techniques. Room size and surface absorption properties were varied, along with aspects of the virtual simulation including the use of individualized head-related transfer function (HRTF) measurements for spatial rendering. Although differences between BRIRs were evident in a variety of physical parameters, a multidimensional scaling analysis revealed that when at-the-ear signal levels were held constant, the rooms differed along just two perceptual dimensions: one related to reverberation time (T(60)) and one related to interaural coherence (IACC). Modeled rooms were found to differ from measured rooms in this perceptual space, but the differences were relatively small and should be easily correctable through adjustment of T(60) and IACC in the model outputs. Results further suggest that spatial rendering using individualized HRTFs offers little benefit over nonindividualized HRTF rendering for room simulation applications where source direction is fixed.

Time-forward speech intelligibility in time-reversed rooms.

The effects of time-reversed room acoustics on word recognition abilities were examined using virtual auditory space techniques, which allowed for temporal manipulation of the room acoustics independent of the speech source signals. Two acoustical conditions were tested: one in which room acoustics were simulated in a realistic time-forward fashion and one in which the room acoustics were reversed in time, causing reverberation and acoustic reflections to precede the direct-path energy. Significant decreases in speech intelligibility--from 89% on average to less than 25%--were observed between the time-forward and time-reversed rooms. This result is not predictable using standard methods for estimating speech intelligibility based on the modulation transfer function of the room. It may instead be due to increased degradation of onset information in the speech signals when room acoustics are time-reversed.

A dissociation between speech understanding and perceived reverberation.

As direct-to-reverberant energy ratio (DRR) decreases or decay time increases, speech intelligibility tends to decrease for both normal-hearing and hearing-impaired listeners. Given this relationship, it is easy to assume that perceived reverberation (reverberance) would act as an intermediary-as physical reverberation increases, so does reverberance, and speech intelligibility decreases as a result. This assumption has not been tested explicitly. Two experiments were conducted to test this hypothesis. In Experiment 1, listeners performed a magnitude estimation task, reporting reverberance for speech stimuli that were convolved with impulse responses whose reverberant properties were manipulated. Listeners reported a decrease in reverberance when the DRR was increased at both ears (Natural Room condition), but not when it was increased at only the ear nearest the source (Hybrid condition). In Experiment 2, listeners performed a speech intelligibility task wherein noise-masked speech was convolved with a subset of the impulse responses from Experiment 1. As predicted by the speech transmission index (STI), speech intelligibility was good in cases where at least one ear received non-reverberant speech, including the Hybrid listening condition in Experiment 1. Thus, the Hybrid listening condition resulted simultaneously in high reverberance (Exp. 1) and high speech intelligibility (Exp. 2), demonstrating that reverberance and speech intelligibility can be dissociated.