Shifting Fundamental Frequency in Simulated Electric-Acoustic Listening: Effects of F0 Variation.

OBJECTIVE: Shifting the mean fundamental frequency (F0) of target speech down in frequency may be a way to provide the benefits of electric-acoustic stimulation (EAS) to cochlear implant (CI) users whose limited residual hearing precludes a benefit typically, even with amplification. However, previous study showed a decline in the amount of benefit at the greatest downward frequency shifts, and the authors hypothesized that this might be related to F0 variation. Thus, in the present study, the authors sought to determine the relationship between mean F0, F0 variation, and the benefits of combining electric stimulation from a CI with low-frequency residual acoustic hearing. DESIGN: The authors measured speech intelligibility in normal-hearing listeners using an EAS simulation consisting of a sine vocoder combined either with speech low-pass filtered at 500 Hz, or with a pure tone representing target F0. The authors used extracted target voice pitch information to modulate the tone, and manipulated both the frequency of the carrier (mean F0), as well as the standard deviation of the voice pitch information (F0 variation). RESULTS: A decline in EAS benefit was observed at the lowest mean F0 tested, but this decline disappeared when F0 variation was reduced to be proportional to the amount of the shift in frequency (i.e., when F0 was shifted logarithmically instead of linearly). CONCLUSION: Lowering mean F0 by shifting the frequency of a pure tone carrying target voice pitch information can provide as much EAS benefit as an unshifted tone, at least in the current simulation of EAS. These results may have implications for CI users with extremely limited residual acoustic hearing.

Comparing the effects of reverberation and of noise on speech recognition in simulated electric-acoustic listening.

Cochlear implant users report difficulty understanding speech in both noisy and reverberant environments. Electric-acoustic stimulation (EAS) is known to improve speech intelligibility in noise. However, little is known about the potential benefits of EAS in reverberation, or about how such benefits relate to those observed in noise. The present study used EAS simulations to examine these questions. Sentences were convolved with impulse responses from a model of a room whose estimated reverberation times were varied from 0 to 1 sec. These reverberated stimuli were then vocoded to simulate electric stimulation, or presented as a combination of vocoder plus low-pass filtered speech to simulate EAS. Monaural sentence recognition scores were measured in two conditions: reverberated speech and speech in a reverberated noise. The long-term spectrum and amplitude modulations of the noise were equated to the reverberant energy, allowing a comparison of the effects of the interferer (speech vs noise). Results indicate that, at least in simulation, (1) EAS provides significant benefit in reverberation; (2) the benefits of EAS in reverberation may be underestimated by those in a comparable noise; and (3) the EAS benefit in reverberation likely arises from partially preserved cues in this background accessible via the low-frequency acoustic component.

On the mechanisms involved in the recovery of envelope information from temporal fine structure.

Three experiments were designed to provide psychophysical evidence for the existence of envelope information in the temporal fine structure (TFS) of stimuli that were originally amplitude modulated (AM). The original stimuli typically consisted of the sum of a sinusoidally AM tone and two unmodulated tones so that the envelope and TFS could be determined a priori. Experiment 1 showed that normal-hearing listeners not only perceive AM when presented with the Hilbert fine structure alone but AM detection thresholds are lower than those observed when presenting the original stimuli. Based on our analysis, envelope recovery resulted from the failure of the decomposition process to remove the spectral components related to the original envelope from the TFS and the introduction of spectral components related to the original envelope, suggesting that frequency- to amplitude-modulation conversion is not necessary to recover envelope information from TFS. Experiment 2 suggested that these spectral components interact in such a way that envelope fluctuations are minimized in the broadband TFS. Experiment 3 demonstrated that the modulation depth at the original carrier frequency is only slightly reduced compared to the depth of the original modulator. It also indicated that envelope recovery is not specific to the Hilbert decomposition.

Shifting fundamental frequency in simulated electric-acoustic listening.

Previous experiments have shown significant improvement in speech intelligibility under both simulated [Brown, C. A., and Bacon, S. P. (2009a). J. Acoust. Soc. Am. 125, 1658-1665; Brown, C. A., and Bacon, S. P. (2010). Hear. Res. 266, 52-59] and real [Brown, C. A., and Bacon, S. P. (2009b). Ear Hear. 30, 489-493] electric-acoustic stimulation when the target speech in the low-frequency region was replaced with a tone modulated in frequency to track the changes in the target talker's fundamental frequency (F0), and in amplitude with the amplitude envelope of the target speech. The present study examined the effects in simulation of applying these cues to a tone lower in frequency than the mean F0 of the target talker. Results showed that shifting the frequency of the tonal carrier downward by as much as 75 Hz had no negative impact on the benefit to intelligibility due to the tone, and that even a shift of 100 Hz resulted in a significant benefit over simulated electric-only stimulation when the sensation level of the tone was comparable to that of the tones shifted by lesser amounts.

Achieving electric-acoustic benefit with a modulated tone.

OBJECTIVE: When either real or simulated electric stimulation from a cochlear implant (CI) is combined with low-frequency acoustic stimulation (electric-acoustic stimulation [EAS]), speech intelligibility in noise can improve dramatically. We recently showed that a similar benefit to intelligibility can be observed in simulation when the low-frequency acoustic stimulation (low-pass target speech) is replaced with a tone that is modulated both in frequency with the fundamental frequency (F0) of the target talker and in amplitude with the amplitude envelope of the low-pass target speech (). The goal of the current experiment was to examine the benefit of the modulated tone to intelligibility in CI patients. DESIGN: Eight CI users who had some residual acoustic hearing either in the implanted ear, the unimplanted ear, or both ears participated in this study. Target speech was combined with either multitalker babble or a single competing talker and presented to the implant. Stimulation to the acoustic region consisted of no signal, target speech, or a tone that was modulated in frequency to track the changes in the target talker's F0 and in amplitude to track the amplitude envelope of target speech low-pass filtered at 500 Hz. RESULTS: All patients showed improvements in intelligibility over electric-only stimulation when either the tone or target speech was presented acoustically. The average improvement in intelligibility was 46 percentage points due to the tone and 55 percentage points due to target speech. CONCLUSIONS: The results demonstrate that a tone carrying F0 and amplitude envelope cues of target speech can provide significant benefit to CI users and may lead to new technologies that could offer EAS benefit to many patients who would not benefit from current EAS approaches.

Low-frequency speech cues and simulated electric-acoustic hearing.

The addition of low-frequency acoustic information to real or simulated electric stimulation (so-called electric-acoustic stimulation or EAS) often results in large improvements in intelligibility, particularly in competing backgrounds. This may reflect the availability of fundamental frequency (F0) information in the acoustic region. The contributions of F0 and the amplitude envelope (as well as voicing) of speech to simulated EAS was examined by replacing the low-frequency speech with a tone that was modulated in frequency to track the F0 of the speech, in amplitude with the envelope of the low-frequency speech, or both. A four-channel vocoder simulated electric hearing. Significant benefit over vocoder alone was observed with the addition of a tone carrying F0 or envelope cues, and both cues combined typically provided significantly more benefit than either alone. The intelligibility improvement over vocoder was between 24 and 57 percentage points, and was unaffected by the presence of a tone carrying these cues from a background talker. These results confirm the importance of the F0 of target speech for EAS (in simulation). They indicate that significant benefit can be provided by a tone carrying F0 and amplitude envelope cues. The results support a glimpsing account of EAS and argue against segregation.

The influence of spread of excitation on the detection of amplitude modulation imposed on sinusoidal carriers at high levels.

The improvement in amplitude modulation (AM) detection thresholds with increasing level of a sinusoidal carrier has been attributed to listening on the high-frequency side of the excitation pattern, where the growth of excitation is more linear, or to an increase in the number of "channels" via spread of excitation. In the present study, AM detection thresholds were measured using a 1000-Hz sinusoidal carrier. Thresholds for modulation frequencies of 4-64 Hz improved by about 10-20 dB as the carrier level increased from 10 dB SL (14.5 dB SPL on average) to 80 dB SPL. To minimize the use of spread of excitation with an 80-dB carrier, tonal "restrictors" with frequencies of 501, 801, 1210, and 1510 Hz were used alone and in combination. High-frequency restrictors elevated AM detection thresholds, whereas low-frequency restrictors did not, indicating that excitation on the high side is more important for detecting AM. Results of modeling suggest that the improvement in AM detection thresholds at high levels is likely due to the use of a relatively linear growth of response on the high-frequency side of the excitation pattern.

Selectivity of modulation interference for consonant identification in normal-hearing listeners.

The present study sought to establish whether speech recognition can be disrupted by the presence of amplitude modulation (AM) at a remote spectral region, and whether that disruption depends upon the rate of AM. The goal was to determine whether this paradigm could be used to examine which modulation frequencies in the speech envelope are most important for speech recognition. Consonant identification for a band of speech located in either the low- or high-frequency region was measured in the presence of a band of noise located in the opposite frequency region. The noise was either unmodulated or amplitude modulated by a sinusoid, a band of noise with a fixed absolute bandwidth, or a band of noise with a fixed relative bandwidth. The frequency of the modulator was 4, 16, 32, or 64 Hz. Small amounts of modulation interference were observed for all modulator types, irrespective of the location of the speech band. More important, the interference depended on modulation frequency, clearly supporting the existence of selectivity of modulation interference with speech stimuli. Overall, the results suggest a primary role of envelope fluctuations around 4 and 16 Hz without excluding the possibility of a contribution by faster rates.

Differential contribution of envelope fluctuations across frequency to consonant identification in quiet.

Two experiments investigated the effects of critical bandwidth and frequency region on the use of temporal envelope cues for speech. In both experiments, spectral details were reduced using vocoder processing. In experiment 1, consonant identification scores were measured in a condition for which the cutoff frequency of the envelope extractor was half the critical bandwidth (HCB) of the auditory filters centered on each analysis band. Results showed that performance is similar to those obtained in conditions for which the envelope cutoff was set to 160 Hz or above. Experiment 2 evaluated the impact of setting the cutoff frequency of the envelope extractor to values of 4, 8, and 16 Hz or to HCB in one or two contiguous bands for an eight-band vocoder. The cutoff was set to 16 Hz for all the other bands. Overall, consonant identification was not affected by removing envelope fluctuations above 4 Hz in the low- and high-frequency bands. In contrast, speech intelligibility decreased as the cutoff frequency was decreased in the midfrequency region from 16 to 4 Hz. The behavioral results were fairly consistent with a physical analysis of the stimuli, suggesting that clearly measurable envelope fluctuations cannot be attenuated without affecting speech intelligibility.

Hearing preservation surgery: psychophysical estimates of cochlear damage in recipients of a short electrode array.

In the newest implementation of cochlear implant surgery, electrode arrays of 10 or 20 mm are inserted into the cochlea with the aim of preserving hearing in the region apical to the tip of the electrode array. In the current study two measures were used to assess hearing preservation: changes in audiometric threshold and changes in psychophysical estimates of nonlinear cochlear processing. Nonlinear cochlear processing was evaluated at signal frequencies of 250 and 500 Hz using Schroeder phase maskers with various indices of masker phase curvature. A total of 15 normal-hearing listeners and 13 cochlear implant patients (7 with a 10 mm insertion and 6 with a 20 mm insertion) were tested. Following surgery the mean low-frequency threshold elevation was 12.7 dB (125-750 Hz). Nine patients had postimplant thresholds within 5-10 dB of preimplant thresholds. Only one patient, however, demonstrated a completely normal nonlinear cochlear function following surgery--although most retained some degree of residual nonlinear processing. This result indicates (i) that Schroeder phase masking functions are a more sensitive index of surgical trauma than audiometric threshold and (ii) that preservation of a normal cochlear function in the apex of the cochlea is relatively uncommon but possible.

An examination of speech recognition in a modulated background and of forward masking in younger and older listeners.

PURPOSE: To compare speech intelligibility in the presence of a 10-Hz square-wave noise masker in younger and older listeners and to relate performance to recovery from forward masking. METHOD: The signal-to-noise ratio required to achieve 50% sentence identification in the presence of a 10-Hz square-wave noise masker was obtained for each of the 8 younger/older listener pairs. Listeners were matched according to their quiet thresholds for frequencies from 600 to 4800 Hz in octave steps. Forward masking was also measured in 2 younger/older threshold-matched groups for signal delays of 2-40 ms. RESULTS: Older listeners typically required a significantly higher signal-to-noise ratio than younger listeners to achieve 50% correct sentence recognition. This effect may be understood in terms of increased forward-masked thresholds throughout the range of signal delays corresponding to the silent intervals in the modulated noise (e.g., <50 ms). CONCLUSIONS: Significant differences were observed between older and younger listeners on measures of both speech intelligibility in a modulated background and forward masking over a range of signal delays (0-40 ms). Age-related susceptibility to forward masking at relatively short delays may reflect a deficit in processing at a fairly central level (e.g., broader temporal windows or less efficient processing).

Compression estimates using behavioral and otoacoustic emission measures.

Cochlear compression in normal-hearing listeners was estimated at octave frequencies from 250 to 4000 Hz using a forward-masking paradigm. Temporal masking curves (TMCs) for a 10-dB SL signal were obtained with two maskers -- one equal in frequency to the signal and another an octave below the signal. The ratio of the slope of the off-frequency function to that of the mid-level portion of the on-frequency function was computed as an estimate of the amount of compression at each frequency. Compression was less frequency selective at low frequencies, so an average of the off-frequency slopes at high frequencies (1000, 2000, and 4000 Hz) was used in computing the ratio for each signal frequency. Results indicated strong compression (approximately 0.15-0.30) at all frequencies using the averaged off-frequency slopes, indicating little difference in compression across frequencies. Distortion product otoacoustic emission (DPOAE) input-output (I-O) functions were obtained for each subject at 1000, 2000, and 4000 Hz. The slopes of the DPOAE I-O functions and the psychophysical growth rates were similar to one another, reinforcing the assumption that the forward-masking procedure is providing an estimate of cochlear compression, at least at frequencies from 1000 to 4000 Hz.

The effect of a steep high-frequency hearing loss on growth-of-masking functions in simultaneous masking for f(m)

In normal-hearing subjects, the slope of the growth-of-masking (GOM) function obtained in simultaneous masking when the masker frequency (f(m)) is much less than the signal frequency (f(s)) often changes from a value near 2.0 to a value near 1.0 at high levels. The purpose of the present study was to evaluate whether this change in slope reflects a basal shift in the peak of the signal's basilar-membrane vibration pattern. To discourage the use of basally shifted peak excitation, GOM functions were obtained in seven subjects with a precipitously sloping high-frequency hearing loss. The signal was located at the normal-hearing edge of the loss, and the masker was located 3 equivalent rectangular bandwidths below f(s). In addition, GOM functions for an f(s) of 2000 Hz were obtained in four subjects with normal hearing, either "in quiet" or in the presence of a restrictor tone with a frequency of 2400 or 2600 Hz and a level of 90 dB SPL. Overall, the results generally are not consistent with the change in slope at high levels being due to a basal shift in the peak of the signal's basilar-membrane vibration pattern. Instead, the results are consistent with a decrease in compression at high input levels at the place corresponding to f(s).

An across-frequency processing deficit in listeners with hearing impairment is supported by acoustic correlation.

It has been recently suggested that listeners having a sensorineural hearing impairment (HI) may possess a deficit in their ability to integrate speech information across different frequencies. When presented with a task that required across-frequency integration of speech patterns, listeners with HI performed more poorly than their normal-hearing (NH) counterparts (E. W. Healy & S. P. Bacon, 2002; C. W. Turner, S.-L. Chi, & S. Flock, 1999). E. W. Healy and S. P. Bacon (2002) also showed that performance of the listeners with HI fell more steeply when increasing amounts of temporal asynchrony were introduced to the pair of widely separated patterns. In the current study, the correlations between the fluctuating envelopes of the acoustic stimuli were calculated, both when the patterns were aligned and at various between-band asynchronies. It was found that the rate at which acoustic correlation fell as a function of asynchrony closely matched the rate at which intelligibility fell for the NH listeners. However, the intelligibility scores produced by the listeners with HI fell more steeply than the acoustic analysis would suggest. Thus, these data provide additional support for the hypothesis that individuals having sensorineural HI may have a deficit in their ability to integrate speech information present at different frequencies.

Learning in discrimination of frequency or modulation rate: generalization to fundamental frequency discrimination.

Fifteen initially inexperienced subjects were trained for 4 weeks (12 2-h sessions) in frequency discrimination with pure tones around 88, 250, or 1605 Hz, or amplitude modulation rate discrimination of noise bands, using modulation rates around 88 or 250 Hz. Before, in the middle of, and after this training period, pure-tone frequency discrimination thresholds (DLFs), harmonic complex tone fundamental frequency discrimination thresholds (DLF0s), and amplitude modulation rate discrimination thresholds (DLFMs) were measured in several conditions including the trained one. Training in pure-tone frequency discrimination resulted in significantly larger improvements in DLF0s when the test complexes contained resolved harmonics than when they were composed of unresolved harmonics. This result supports the hypothesis that the discrimination of the F0 of resolved harmonics shares common underlying mechanisms with the frequency discrimination of pure tones. Training in rate discrimination did not result in larger DLF0 improvements for unresolved than for resolved harmonics.

Modulation detection interference in listeners with normal and impaired hearing.

Listeners were asked to detect amplitude modulation (AM) of a target (or signal) carrier that was presented in isolation or in the presence of an additional (masker) carrier. The signal was modulated at a rate of 10 Hz, and the masker was unmodulated or was modulated at a rate of 2, 10, or 40 Hz. Nine listeners had normal hearing, 4 had a bilateral hearing loss, and 4 had a unilateral hearing loss; those with a unilateral loss were tested in both ears. The listeners with a hearing loss had normal hearing at 1 kHz and a 30- to 40-dB loss at 4 kHz. The carrier frequencies were 984 and 3952 Hz. In one set of conditions, the lower frequency carrier was the signal and the higher frequency carrier was the masker. In the other set, the reverse was true. For the impaired ears, the carriers were presented at 70 dB SPL. For the normal ears, either the carriers were both presented at 70 dB SPL or the higher frequency carrier was reduced to 40 dB SPL to simulate the lower sensation level experienced by the impaired ears. There was considerable individual variability in the results, and there was no clear effect of hearing loss. These results suggest that a mild, presumably cochlear hearing loss does not affect the ability to process AM in one frequency region in the presence of competing AM from another region.

Across-frequency comparison of temporal speech information by listeners with normal and impaired hearing.

Listeners with normal hearing (NH) and with sensorineural hearing impairment (HI) were tested on a speech-recognition task requiring across-frequency integration of temporal speech information. Listeners with NH correctly identified a majority of key words in everyday sentences when presented with a synchronous pair of speech-modulated tones at 750 and 3,000 Hz. They could tolerate small amounts (12.5 ms) of across-frequency asynchrony, but performance fell as the delay between bands was increased to 100 ms. Listeners with HI performed more poorly than those with NH when presented with synchronous across-frequency information. Further, performance of listeners with HI fell as a function of asynchrony more steeply than that of their NH counterparts. These results suggest that listeners with HI have particular difficulty comparing and effectively processing temporal speech information at different frequencies. The increased influence of asynchrony indicates that these listeners are especially hindered by slight disruptions in across-frequency information, which implies a less robust comparison mechanism. The results could not be attributed to differences in signal or sensation level, or in listener age, but instead appear to be related to the degree of hearing loss. This across-frequency deficit is unlikely to be attributed to known processing difficulties and may exist in addition to other known disruptions.

Fundamental frequency and speech intelligibility in background noise.

Speech reception in noise is an especially difficult problem for listeners with hearing impairment as well as for users of cochlear implants (CIs). One likely cause of this is an inability to 'glimpse' a target talker in a fluctuating background, which has been linked to deficits in temporal fine-structure processing. A fine-structure cue that has the potential to be beneficial for speech reception in noise is fundamental frequency (F0). A challenging problem, however, is delivering the cue to these individuals. The benefits to speech intelligibility of F0 for both listeners with hearing impairment and users of CIs are reviewed, as well as various methods of delivering F0 to these listeners.

Effect of spectral frequency range and separation on the perception of asynchronous speech.

The use of across-frequency timing cues and the effect of disrupting these cues were examined across the frequency spectrum by introducing between-band asynchronies to pairs of narrowband temporal speech patterns. Sentence intelligibility by normal-hearing listeners fell when as little as 12.5 ms of asynchrony was introduced and was reduced to floor values by 100 ms. Disruptions to across-frequency timing had similar effects in the low-, mid-, and high-frequency regions, but band pairs having wider frequency separation were less disrupted by small amounts of asynchrony. In experiment 2, it was found that the disruptive influence of asynchrony on adjacent band pairs did not result from disruptions to the complex patterns present in overlapping excitation. The results of experiment 3 suggest that the processing of speech patterns may take place using mechanisms having different sensitivities to exact timing, similar to the dual mechanisms proposed for within- and across-channel gap detection. Preservation of relative timing can be critical to intelligibility. While the use of across-frequency timing cues appears similar across the spectrum, it may differ based on frequency separation. This difference appears to involve a greater reliance on exact timing during the processing of speech energy at proximate frequencies.