Shifting Fundamental Frequency in Simulated Electric-Acoustic Listening: Effects of F0 Variation.

OBJECTIVE: Shifting the mean fundamental frequency (F0) of target speech down in frequency may be a way to provide the benefits of electric-acoustic stimulation (EAS) to cochlear implant (CI) users whose limited residual hearing precludes a benefit typically, even with amplification. However, previous study showed a decline in the amount of benefit at the greatest downward frequency shifts, and the authors hypothesized that this might be related to F0 variation. Thus, in the present study, the authors sought to determine the relationship between mean F0, F0 variation, and the benefits of combining electric stimulation from a CI with low-frequency residual acoustic hearing. DESIGN: The authors measured speech intelligibility in normal-hearing listeners using an EAS simulation consisting of a sine vocoder combined either with speech low-pass filtered at 500 Hz, or with a pure tone representing target F0. The authors used extracted target voice pitch information to modulate the tone, and manipulated both the frequency of the carrier (mean F0), as well as the standard deviation of the voice pitch information (F0 variation). RESULTS: A decline in EAS benefit was observed at the lowest mean F0 tested, but this decline disappeared when F0 variation was reduced to be proportional to the amount of the shift in frequency (i.e., when F0 was shifted logarithmically instead of linearly). CONCLUSION: Lowering mean F0 by shifting the frequency of a pure tone carrying target voice pitch information can provide as much EAS benefit as an unshifted tone, at least in the current simulation of EAS. These results may have implications for CI users with extremely limited residual acoustic hearing.

Comparing the effects of reverberation and of noise on speech recognition in simulated electric-acoustic listening.

Cochlear implant users report difficulty understanding speech in both noisy and reverberant environments. Electric-acoustic stimulation (EAS) is known to improve speech intelligibility in noise. However, little is known about the potential benefits of EAS in reverberation, or about how such benefits relate to those observed in noise. The present study used EAS simulations to examine these questions. Sentences were convolved with impulse responses from a model of a room whose estimated reverberation times were varied from 0 to 1 sec. These reverberated stimuli were then vocoded to simulate electric stimulation, or presented as a combination of vocoder plus low-pass filtered speech to simulate EAS. Monaural sentence recognition scores were measured in two conditions: reverberated speech and speech in a reverberated noise. The long-term spectrum and amplitude modulations of the noise were equated to the reverberant energy, allowing a comparison of the effects of the interferer (speech vs noise). Results indicate that, at least in simulation, (1) EAS provides significant benefit in reverberation; (2) the benefits of EAS in reverberation may be underestimated by those in a comparable noise; and (3) the EAS benefit in reverberation likely arises from partially preserved cues in this background accessible via the low-frequency acoustic component.

On the mechanisms involved in the recovery of envelope information from temporal fine structure.

Three experiments were designed to provide psychophysical evidence for the existence of envelope information in the temporal fine structure (TFS) of stimuli that were originally amplitude modulated (AM). The original stimuli typically consisted of the sum of a sinusoidally AM tone and two unmodulated tones so that the envelope and TFS could be determined a priori. Experiment 1 showed that normal-hearing listeners not only perceive AM when presented with the Hilbert fine structure alone but AM detection thresholds are lower than those observed when presenting the original stimuli. Based on our analysis, envelope recovery resulted from the failure of the decomposition process to remove the spectral components related to the original envelope from the TFS and the introduction of spectral components related to the original envelope, suggesting that frequency- to amplitude-modulation conversion is not necessary to recover envelope information from TFS. Experiment 2 suggested that these spectral components interact in such a way that envelope fluctuations are minimized in the broadband TFS. Experiment 3 demonstrated that the modulation depth at the original carrier frequency is only slightly reduced compared to the depth of the original modulator. It also indicated that envelope recovery is not specific to the Hilbert decomposition.

Shifting fundamental frequency in simulated electric-acoustic listening.

Previous experiments have shown significant improvement in speech intelligibility under both simulated [Brown, C. A., and Bacon, S. P. (2009a). J. Acoust. Soc. Am. 125, 1658-1665; Brown, C. A., and Bacon, S. P. (2010). Hear. Res. 266, 52-59] and real [Brown, C. A., and Bacon, S. P. (2009b). Ear Hear. 30, 489-493] electric-acoustic stimulation when the target speech in the low-frequency region was replaced with a tone modulated in frequency to track the changes in the target talker's fundamental frequency (F0), and in amplitude with the amplitude envelope of the target speech. The present study examined the effects in simulation of applying these cues to a tone lower in frequency than the mean F0 of the target talker. Results showed that shifting the frequency of the tonal carrier downward by as much as 75 Hz had no negative impact on the benefit to intelligibility due to the tone, and that even a shift of 100 Hz resulted in a significant benefit over simulated electric-only stimulation when the sensation level of the tone was comparable to that of the tones shifted by lesser amounts.

Fundamental frequency and speech intelligibility in background noise.

Speech reception in noise is an especially difficult problem for listeners with hearing impairment as well as for users of cochlear implants (CIs). One likely cause of this is an inability to 'glimpse' a target talker in a fluctuating background, which has been linked to deficits in temporal fine-structure processing. A fine-structure cue that has the potential to be beneficial for speech reception in noise is fundamental frequency (F0). A challenging problem, however, is delivering the cue to these individuals. The benefits to speech intelligibility of F0 for both listeners with hearing impairment and users of CIs are reviewed, as well as various methods of delivering F0 to these listeners.

Achieving electric-acoustic benefit with a modulated tone.

OBJECTIVE: When either real or simulated electric stimulation from a cochlear implant (CI) is combined with low-frequency acoustic stimulation (electric-acoustic stimulation [EAS]), speech intelligibility in noise can improve dramatically. We recently showed that a similar benefit to intelligibility can be observed in simulation when the low-frequency acoustic stimulation (low-pass target speech) is replaced with a tone that is modulated both in frequency with the fundamental frequency (F0) of the target talker and in amplitude with the amplitude envelope of the low-pass target speech (). The goal of the current experiment was to examine the benefit of the modulated tone to intelligibility in CI patients. DESIGN: Eight CI users who had some residual acoustic hearing either in the implanted ear, the unimplanted ear, or both ears participated in this study. Target speech was combined with either multitalker babble or a single competing talker and presented to the implant. Stimulation to the acoustic region consisted of no signal, target speech, or a tone that was modulated in frequency to track the changes in the target talker's F0 and in amplitude to track the amplitude envelope of target speech low-pass filtered at 500 Hz. RESULTS: All patients showed improvements in intelligibility over electric-only stimulation when either the tone or target speech was presented acoustically. The average improvement in intelligibility was 46 percentage points due to the tone and 55 percentage points due to target speech. CONCLUSIONS: The results demonstrate that a tone carrying F0 and amplitude envelope cues of target speech can provide significant benefit to CI users and may lead to new technologies that could offer EAS benefit to many patients who would not benefit from current EAS approaches.

Low-frequency speech cues and simulated electric-acoustic hearing.

The addition of low-frequency acoustic information to real or simulated electric stimulation (so-called electric-acoustic stimulation or EAS) often results in large improvements in intelligibility, particularly in competing backgrounds. This may reflect the availability of fundamental frequency (F0) information in the acoustic region. The contributions of F0 and the amplitude envelope (as well as voicing) of speech to simulated EAS was examined by replacing the low-frequency speech with a tone that was modulated in frequency to track the F0 of the speech, in amplitude with the envelope of the low-frequency speech, or both. A four-channel vocoder simulated electric hearing. Significant benefit over vocoder alone was observed with the addition of a tone carrying F0 or envelope cues, and both cues combined typically provided significantly more benefit than either alone. The intelligibility improvement over vocoder was between 24 and 57 percentage points, and was unaffected by the presence of a tone carrying these cues from a background talker. These results confirm the importance of the F0 of target speech for EAS (in simulation). They indicate that significant benefit can be provided by a tone carrying F0 and amplitude envelope cues. The results support a glimpsing account of EAS and argue against segregation.

Hearing preservation surgery: psychophysical estimates of cochlear damage in recipients of a short electrode array.

In the newest implementation of cochlear implant surgery, electrode arrays of 10 or 20 mm are inserted into the cochlea with the aim of preserving hearing in the region apical to the tip of the electrode array. In the current study two measures were used to assess hearing preservation: changes in audiometric threshold and changes in psychophysical estimates of nonlinear cochlear processing. Nonlinear cochlear processing was evaluated at signal frequencies of 250 and 500 Hz using Schroeder phase maskers with various indices of masker phase curvature. A total of 15 normal-hearing listeners and 13 cochlear implant patients (7 with a 10 mm insertion and 6 with a 20 mm insertion) were tested. Following surgery the mean low-frequency threshold elevation was 12.7 dB (125-750 Hz). Nine patients had postimplant thresholds within 5-10 dB of preimplant thresholds. Only one patient, however, demonstrated a completely normal nonlinear cochlear function following surgery--although most retained some degree of residual nonlinear processing. This result indicates (i) that Schroeder phase masking functions are a more sensitive index of surgical trauma than audiometric threshold and (ii) that preservation of a normal cochlear function in the apex of the cochlea is relatively uncommon but possible.

Differential contribution of envelope fluctuations across frequency to consonant identification in quiet.

Two experiments investigated the effects of critical bandwidth and frequency region on the use of temporal envelope cues for speech. In both experiments, spectral details were reduced using vocoder processing. In experiment 1, consonant identification scores were measured in a condition for which the cutoff frequency of the envelope extractor was half the critical bandwidth (HCB) of the auditory filters centered on each analysis band. Results showed that performance is similar to those obtained in conditions for which the envelope cutoff was set to 160 Hz or above. Experiment 2 evaluated the impact of setting the cutoff frequency of the envelope extractor to values of 4, 8, and 16 Hz or to HCB in one or two contiguous bands for an eight-band vocoder. The cutoff was set to 16 Hz for all the other bands. Overall, consonant identification was not affected by removing envelope fluctuations above 4 Hz in the low- and high-frequency bands. In contrast, speech intelligibility decreased as the cutoff frequency was decreased in the midfrequency region from 16 to 4 Hz. The behavioral results were fairly consistent with a physical analysis of the stimuli, suggesting that clearly measurable envelope fluctuations cannot be attenuated without affecting speech intelligibility.

Selectivity of modulation interference for consonant identification in normal-hearing listeners.

The present study sought to establish whether speech recognition can be disrupted by the presence of amplitude modulation (AM) at a remote spectral region, and whether that disruption depends upon the rate of AM. The goal was to determine whether this paradigm could be used to examine which modulation frequencies in the speech envelope are most important for speech recognition. Consonant identification for a band of speech located in either the low- or high-frequency region was measured in the presence of a band of noise located in the opposite frequency region. The noise was either unmodulated or amplitude modulated by a sinusoid, a band of noise with a fixed absolute bandwidth, or a band of noise with a fixed relative bandwidth. The frequency of the modulator was 4, 16, 32, or 64 Hz. Small amounts of modulation interference were observed for all modulator types, irrespective of the location of the speech band. More important, the interference depended on modulation frequency, clearly supporting the existence of selectivity of modulation interference with speech stimuli. Overall, the results suggest a primary role of envelope fluctuations around 4 and 16 Hz without excluding the possibility of a contribution by faster rates.

The influence of spread of excitation on the detection of amplitude modulation imposed on sinusoidal carriers at high levels.

The improvement in amplitude modulation (AM) detection thresholds with increasing level of a sinusoidal carrier has been attributed to listening on the high-frequency side of the excitation pattern, where the growth of excitation is more linear, or to an increase in the number of "channels" via spread of excitation. In the present study, AM detection thresholds were measured using a 1000-Hz sinusoidal carrier. Thresholds for modulation frequencies of 4-64 Hz improved by about 10-20 dB as the carrier level increased from 10 dB SL (14.5 dB SPL on average) to 80 dB SPL. To minimize the use of spread of excitation with an 80-dB carrier, tonal "restrictors" with frequencies of 501, 801, 1210, and 1510 Hz were used alone and in combination. High-frequency restrictors elevated AM detection thresholds, whereas low-frequency restrictors did not, indicating that excitation on the high side is more important for detecting AM. Results of modeling suggest that the improvement in AM detection thresholds at high levels is likely due to the use of a relatively linear growth of response on the high-frequency side of the excitation pattern.

An examination of speech recognition in a modulated background and of forward masking in younger and older listeners.

PURPOSE: To compare speech intelligibility in the presence of a 10-Hz square-wave noise masker in younger and older listeners and to relate performance to recovery from forward masking. METHOD: The signal-to-noise ratio required to achieve 50% sentence identification in the presence of a 10-Hz square-wave noise masker was obtained for each of the 8 younger/older listener pairs. Listeners were matched according to their quiet thresholds for frequencies from 600 to 4800 Hz in octave steps. Forward masking was also measured in 2 younger/older threshold-matched groups for signal delays of 2-40 ms. RESULTS: Older listeners typically required a significantly higher signal-to-noise ratio than younger listeners to achieve 50% correct sentence recognition. This effect may be understood in terms of increased forward-masked thresholds throughout the range of signal delays corresponding to the silent intervals in the modulated noise (e.g., <50 ms). CONCLUSIONS: Significant differences were observed between older and younger listeners on measures of both speech intelligibility in a modulated background and forward masking over a range of signal delays (0-40 ms). Age-related susceptibility to forward masking at relatively short delays may reflect a deficit in processing at a fairly central level (e.g., broader temporal windows or less efficient processing).

Auditory function and speech understanding in listeners who qualify for EAS surgery.

OBJECTIVE: For patients with relatively good low-frequency hearing and relatively poor high-frequency hearing, who met the pre-implant criteria for combined electric and acoustic stimulation (EAS), our aims were to i) assess deficits in low-frequency auditory function, ii) to identify measures which might be sensitive to changes resulting from the insertion of an intracochlear electrode array, and iii) to quantify the relationship between measures of auditory function and performance on tasks of speech and melody recognition. DESIGN: Measures of frequency selectivity, temporal resolution, and nonlinear cochlear function, along with measures of word, sentence, consonant, vowel, and melody recognition, were obtained from 5 normal-hearing and 17 hearing-impaired listeners. The hearing-impaired listeners had auditory thresholds at 500 Hz, ranging from 20 to 60 dB HL, and thresholds at 1 kHz, ranging from 60 to 100 dB HL. RESULTS: Nonlinear cochlear function was either reduced or absent. Frequency selectivity at 500 Hz was significantly reduced but still present in most patients. Temporal resolution, when measured at low modulation frequencies, was normal. Speech recognition in a modulated background revealed significantly poorer performance than normal. Speech and melody recognition varied over a large range. No measure of auditory function was correlated significantly with speech recognition. However, frequency selectivity was related to melody recognition. CONCLUSIONS: (1) Patients who qualify for EAS surgery have a wide range of speech and melody recognition abilities. (2) A number of the psychophysical measures tested may prove more sensitive than the audiogram in determining the degree of damage inflicted by the intracochlear electrode array. (3) Speech recognition was not correlated with any of the measures of auditory function.

Effect of spectral frequency range and separation on the perception of asynchronous speech.

The use of across-frequency timing cues and the effect of disrupting these cues were examined across the frequency spectrum by introducing between-band asynchronies to pairs of narrowband temporal speech patterns. Sentence intelligibility by normal-hearing listeners fell when as little as 12.5 ms of asynchrony was introduced and was reduced to floor values by 100 ms. Disruptions to across-frequency timing had similar effects in the low-, mid-, and high-frequency regions, but band pairs having wider frequency separation were less disrupted by small amounts of asynchrony. In experiment 2, it was found that the disruptive influence of asynchrony on adjacent band pairs did not result from disruptions to the complex patterns present in overlapping excitation. The results of experiment 3 suggest that the processing of speech patterns may take place using mechanisms having different sensitivities to exact timing, similar to the dual mechanisms proposed for within- and across-channel gap detection. Preservation of relative timing can be critical to intelligibility. While the use of across-frequency timing cues appears similar across the spectrum, it may differ based on frequency separation. This difference appears to involve a greater reliance on exact timing during the processing of speech energy at proximate frequencies.

Measuring the critical band for speech.

The current experiments were designed to measure the frequency resolution employed by listeners during the perception of everyday sentences. Speech bands having nearly vertical filter slopes and narrow bandwidths were sharply partitioned into various numbers of equal log- or ERBN-width subbands. The temporal envelope from each partition was used to amplitude modulate a corresponding band of low-noise noise, and the modulated carriers were combined and presented to normal-hearing listeners. Intelligibility increased and reached asymptote as the number of partitions increased. In the mid- and high-frequency regions of the speech spectrum, the partition bandwidth corresponding to asymptotic performance matched current estimates of psychophysical tuning across a number of conditions. These results indicate that, in these regions, the critical band for speech matches the critical band measured using traditional psychoacoustic methods and nonspeech stimuli. However, in the low-frequency region, partition bandwidths at asymptote were somewhat narrower than would be predicted based upon psychophysical tuning. It is concluded that, overall, current estimates of psychophysical tuning represent reasonably well the ability of listeners to extract spectral detail from running speech.

The effect of a steep high-frequency hearing loss on growth-of-masking functions in simultaneous masking for f(m)

In normal-hearing subjects, the slope of the growth-of-masking (GOM) function obtained in simultaneous masking when the masker frequency (f(m)) is much less than the signal frequency (f(s)) often changes from a value near 2.0 to a value near 1.0 at high levels. The purpose of the present study was to evaluate whether this change in slope reflects a basal shift in the peak of the signal's basilar-membrane vibration pattern. To discourage the use of basally shifted peak excitation, GOM functions were obtained in seven subjects with a precipitously sloping high-frequency hearing loss. The signal was located at the normal-hearing edge of the loss, and the masker was located 3 equivalent rectangular bandwidths below f(s). In addition, GOM functions for an f(s) of 2000 Hz were obtained in four subjects with normal hearing, either "in quiet" or in the presence of a restrictor tone with a frequency of 2400 or 2600 Hz and a level of 90 dB SPL. Overall, the results generally are not consistent with the change in slope at high levels being due to a basal shift in the peak of the signal's basilar-membrane vibration pattern. Instead, the results are consistent with a decrease in compression at high input levels at the place corresponding to f(s).

Compression estimates using behavioral and otoacoustic emission measures.

Cochlear compression in normal-hearing listeners was estimated at octave frequencies from 250 to 4000 Hz using a forward-masking paradigm. Temporal masking curves (TMCs) for a 10-dB SL signal were obtained with two maskers -- one equal in frequency to the signal and another an octave below the signal. The ratio of the slope of the off-frequency function to that of the mid-level portion of the on-frequency function was computed as an estimate of the amount of compression at each frequency. Compression was less frequency selective at low frequencies, so an average of the off-frequency slopes at high frequencies (1000, 2000, and 4000 Hz) was used in computing the ratio for each signal frequency. Results indicated strong compression (approximately 0.15-0.30) at all frequencies using the averaged off-frequency slopes, indicating little difference in compression across frequencies. Distortion product otoacoustic emission (DPOAE) input-output (I-O) functions were obtained for each subject at 1000, 2000, and 4000 Hz. The slopes of the DPOAE I-O functions and the psychophysical growth rates were similar to one another, reinforcing the assumption that the forward-masking procedure is providing an estimate of cochlear compression, at least at frequencies from 1000 to 4000 Hz.

An across-frequency processing deficit in listeners with hearing impairment is supported by acoustic correlation.

It has been recently suggested that listeners having a sensorineural hearing impairment (HI) may possess a deficit in their ability to integrate speech information across different frequencies. When presented with a task that required across-frequency integration of speech patterns, listeners with HI performed more poorly than their normal-hearing (NH) counterparts (E. W. Healy & S. P. Bacon, 2002; C. W. Turner, S.-L. Chi, & S. Flock, 1999). E. W. Healy and S. P. Bacon (2002) also showed that performance of the listeners with HI fell more steeply when increasing amounts of temporal asynchrony were introduced to the pair of widely separated patterns. In the current study, the correlations between the fluctuating envelopes of the acoustic stimuli were calculated, both when the patterns were aligned and at various between-band asynchronies. It was found that the rate at which acoustic correlation fell as a function of asynchrony closely matched the rate at which intelligibility fell for the NH listeners. However, the intelligibility scores produced by the listeners with HI fell more steeply than the acoustic analysis would suggest. Thus, these data provide additional support for the hypothesis that individuals having sensorineural HI may have a deficit in their ability to integrate speech information present at different frequencies.

Psychophysical estimates of nonlinear cochlear processing in younger and older listeners.

The primary goal of this project was to compare the performance of younger and older listeners on a number of psychophysical measures thought to be influenced by nonlinear cochlear processing. Younger (mean of 25.6 years) and older (mean of 63.8 years) listeners with normal hearing were matched (within 5 dB) according to their quiet thresholds at the two test frequencies of 1200 and 2400 Hz. They were similarly matched at the adjacent octave frequencies of 600 and 4800 Hz (within 5 dB at one and 9 dB at the other). Performance was compared on measures of auditory filter shape, psychophysical suppression, and growth of forward masking. There was no difference between the two age groups on these psychophysical estimates reflecting nonlinear processing, suggesting that aging per se does not affect the cochlear nonlinearity, at least for the ages sampled here. The results did, however, consistently demonstrate an age-related increase in the susceptibility to forward masking.