On mistuning detection and beat perception for harmonic complex tones at low and very high frequencies.

This study assessed the detection of mistuning of a single harmonic in complex tones (CTs) containing either low-frequency harmonics or very high-frequency harmonics, for which phase locking to the temporal fine structure is weak or absent. CTs had F0s of either 280 or 1400 Hz and contained harmonics 6-10, the 8th of which could be mistuned. Harmonics were presented either diotically or dichotically (odd and even harmonics to different ears). In the diotic condition, mistuning-detection thresholds were very low for both F0s and consistent with detection of temporal interactions (beats) produced by peripheral interactions of components. In the dichotic condition, for which the components in each ear were more widely spaced and beats were not reported, the mistuned component was perceptually segregated from the complex for the low F0, but subjects reported no "popping out" for the high F0 and performance was close to chance. This is consistent with the idea that phase locking is required for perceptual segregation to occur. For diotic presentation, the perceived beat rate corresponded to the amount of mistuning (in Hz). It is argued that the beat percept cannot be explained solely by interactions between the mistuned component and its two closest harmonic neighbours.

Temporal Pitch Sensitivity in an Animal Model: Psychophysics and Scalp Recordings : Temporal Pitch Sensitivity in Cat.

Cochlear implant (CI) users show limited sensitivity to the temporal pitch conveyed by electric stimulation, contributing to impaired perception of music and of speech in noise. Neurophysiological studies in cats suggest that this limitation is due, in part, to poor transmission of the temporal fine structure (TFS) by the brainstem pathways that are activated by electrical cochlear stimulation. It remains unknown, however, how that neural limit might influence perception in the same animal model. For that reason, we developed non-invasive psychophysical and electrophysiological measures of temporal (i.e., non-spectral) pitch processing in the cat. Normal-hearing (NH) cats were presented with acoustic pulse trains consisting of band-limited harmonic complexes that simulated CI stimulation of the basal cochlea while removing cochlear place-of-excitation cues. In the psychophysical procedure, trained cats detected changes from a base pulse rate to a higher pulse rate. In the scalp-recording procedure, the cortical-evoked acoustic change complex (ACC) and brainstem-generated frequency following response (FFR) were recorded simultaneously in sedated cats for pulse trains that alternated between the base and higher rates. The range of perceptual sensitivity to temporal pitch broadly resembled that of humans but was shifted to somewhat higher rates. The ACC largely paralleled these perceptual patterns, validating its use as an objective measure of temporal pitch sensitivity. The phase-locked FFR, in contrast, showed strong brainstem encoding for all tested pulse rates. These measures demonstrate the cat's perceptual sensitivity to pitch in the absence of cochlear-place cues and may be valuable for evaluating neural mechanisms of temporal pitch perception in the feline animal model of stimulation by a CI or novel auditory prostheses.

The effect of a coding strategy that removes temporally masked pulses on speech perception by cochlear implant users.

Speech recognition in noisy environments remains a challenge for cochlear implant (CI) recipients. Unwanted charge interactions between current pulses, both within and between electrode channels, are likely to impair performance. Here we investigate the effect of reducing the number of current pulses on speech perception. This was achieved by implementing a psychoacoustic temporal-masking model where current pulses in each channel were passed through a temporal integrator to identify and remove pulses that were less likely to be perceived by the recipient. The decision criterion of the temporal integrator was varied to control the percentage of pulses removed in each condition. In experiment 1, speech in quiet was processed with a standard Continuous Interleaved Sampling (CIS) strategy and with 25, 50 and 75% of pulses removed. In experiment 2, performance was measured for speech in noise with the CIS reference and with 50 and 75% of pulses removed. Speech intelligibility in quiet revealed no significant difference between reference and test conditions. For speech in noise, results showed a significant improvement of 2.4 dB when removing 50% of pulses and performance was not significantly different between the reference and when 75% of pulses were removed. Further, by reducing the overall amount of current pulses by 25, 50, and 75% but accounting for the increase in charge necessary to compensate for the decrease in loudness, estimated average power savings of 21.15, 40.95, and 63.45%, respectively, could be possible for this set of listeners. In conclusion, removing temporally masked pulses may improve speech perception in noise and result in substantial power savings.

Effects of the relative timing of opposite-polarity pulses on loudness for cochlear implant listeners.

The symmetric biphasic pulses used in contemporary cochlear implants (CIs) consist of both cathodic and anodic currents, which may stimulate different sites on spiral ganglion neurons and, potentially, interact with each other. The effect on the order of anodic and cathodic stimulation on loudness at short inter-pulse intervals (IPIs; 0-800 µs) is investigated. Pairs of opposite-polarity pseudomonophasic (PS) pulses were used and the amplitude of each pulse was manipulated independently. In experiment 1 the two PS pulses differed in their current level in order to elicit the same loudness when presented separately. Six users of the Advanced Bionics CI (Valencia, CA) loudness-ranked trains of the pulse pairs using a midpoint-comparison procedure. Stimuli with anodic-leading polarity were louder than those with cathodic-leading polarity for IPIs shorter than 400 µs. This effect was small-about 0.3 dB-but consistent across listeners. When the same procedure was repeated with both PS pulses having the same current level (experiment 2), anodic-leading stimuli were still louder than cathodic-leading stimuli at very short intervals. However, when using symmetric biphasic pulses (experiment 3) the effect disappeared at short intervals and reversed at long intervals. Possible peripheral sources of such polarity interactions are discussed.

Development and validation of a spectro-temporal processing test for cochlear-implant listeners.

Psychophysical tests of spectro-temporal resolution may aid the evaluation of methods for improving hearing by cochlear implant (CI) listeners. Here the STRIPES (Spectro-Temporal Ripple for Investigating Processor EffectivenesS) test is described and validated. Like speech, the test requires both spectral and temporal processing to perform well. Listeners discriminate between complexes of sine sweeps which increase or decrease in frequency; difficulty is controlled by changing the stimulus spectro-temporal density. Care was taken to minimize extraneous cues, forcing listeners to perform the task only on the direction of the sweeps. Vocoder simulations with normal hearing listeners showed that the STRIPES test was sensitive to the number of channels and temporal information fidelity. An evaluation with CI listeners compared a standard processing strategy with one having very wide filters, thereby spectrally blurring the stimulus. Psychometric functions were monotonic for both strategies and five of six participants performed better with the standard strategy. An adaptive procedure revealed significant differences, all in favour of the standard strategy, at the individual listener level for six of eight CI listeners. Subsequent measures validated a faster version of the test, and showed that STRIPES could be performed by recently implanted listeners having no experience of psychophysical testing.

Neural Decoding of Bistable Sounds Reveals an Effect of Intention on Perceptual Organization.

Auditory signals arrive at the ear as a mixture that the brain must decompose into distinct sources based to a large extent on acoustic properties of the sounds. An important question concerns whether listeners have voluntary control over how many sources they perceive. This has been studied using pure high (H) and low (L) tones presented in the repeating pattern HLH-HLH-, which can form a bistable percept heard either as an integrated whole (HLH-) or as segregated into high (H-H-) and low (-L-) sequences. Although instructing listeners to try to integrate or segregate sounds affects reports of what they hear, this could reflect a response bias rather than a perceptual effect. We had human listeners (15 males, 12 females) continuously report their perception of such sequences and recorded neural activity using MEG. During neutral listening, a classifier trained on patterns of neural activity distinguished between periods of integrated and segregated perception. In other conditions, participants tried to influence their perception by allocating attention either to the whole sequence or to a subset of the sounds. They reported hearing the desired percept for a greater proportion of time than when listening neutrally. Critically, neural activity supported these reports; stimulus-locked brain responses in auditory cortex were more likely to resemble the signature of segregation when participants tried to hear segregation than when attempting to perceive integration. These results indicate that listeners can influence how many sound sources they perceive, as reflected in neural responses that track both the input and its perceptual organization.SIGNIFICANCE STATEMENT Can we consciously influence our perception of the external world? We address this question using sound sequences that can be heard either as coming from a single source or as two distinct auditory streams. Listeners reported spontaneous changes in their perception between these two interpretations while we recorded neural activity to identify signatures of such integration and segregation. They also indicated that they could, to some extent, choose between these alternatives. This claim was supported by corresponding changes in responses in auditory cortex. By linking neural and behavioral correlates of perception, we demonstrate that the number of objects that we perceive can depend not only on the physical attributes of our environment, but also on how we intend to experience it.

Attentional Modulation of Envelope-Following Responses at Lower (93-109 Hz) but Not Higher (217-233 Hz) Modulation Rates.

Directing attention to sounds of different frequencies allows listeners to perceive a sound of interest, like a talker, in a mixture. Whether cortically generated frequency-specific attention affects responses as low as the auditory brainstem is currently unclear. Participants attended to either a high- or low-frequency tone stream, which was presented simultaneously and tagged with different amplitude modulation (AM) rates. In a replication design, we showed that envelope-following responses (EFRs) were modulated by attention only when the stimulus AM rate was slow enough for the auditory cortex to track-and not for stimuli with faster AM rates, which are thought to reflect 'purer' brainstem sources. Thus, we found no evidence of frequency-specific attentional modulation that can be confidently attributed to brainstem generators. The results demonstrate that different neural populations contribute to EFRs at higher and lower rates, compatible with cortical contributions at lower rates. The results further demonstrate that stimulus AM rate can alter conclusions of EFR studies.

Perception of stochastic envelopes by normal-hearing and cochlear-implant listeners.

We assessed auditory sensitivity to three classes of temporal-envelope statistics (modulation depth, modulation rate, and comodulation) that are important for the perception of 'sound textures'. The textures were generated by a probabilistic model that prescribes the temporal statistics of a selected number of modulation envelopes, superimposed onto noise carriers. Discrimination thresholds were measured for normal-hearing (NH) listeners and users of a MED-EL pulsar cochlear implant (CI), for separate manipulations of the average rate and modulation depth of the envelope in each frequency band of the stimulus, and of the co-modulation between bands. Normal-hearing (NH) listeners' discrimination of envelope rate was similar for baseline modulation rates of 5 and 34 Hz, and much poorer than previously reported for sinusoidally amplitude-modulated sounds. In contrast, discrimination of model parameters that controlled modulation depth was poorer at the lower baseline rate, consistent with the idea that, at the lower rate, subjects get fewer 'looks' at the relevant information when comparing stimuli differing in modulation depth. NH listeners could discriminate differences in co-modulation across bands; a multidimensional scaling study revealed that this was likely due to genuine across-frequency processing, rather than within-channel cues. CI users' discrimination performance was worse overall than for NH listeners, but showed a similar dependence on stimulus parameters.

The role of excitation-pattern cues in the detection of frequency shifts in bandpass-filtered complex tones.

One task intended to measure sensitivity to temporal fine structure (TFS) involves the discrimination of a harmonic complex tone from a tone in which all harmonics are shifted upwards by the same amount in hertz. Both tones are passed through a fixed bandpass filter centered on the high harmonics to reduce the availability of excitation-pattern cues and a background noise is used to mask combination tones. The role of frequency selectivity in this "TFS1" task was investigated by varying level. Experiment 1 showed that listeners performed more poorly at a high level than at a low level. Experiment 2 included intermediate levels and showed that performance deteriorated for levels above about 57 dB sound pressure level. Experiment 3 estimated the magnitude of excitation-pattern cues from the variation in forward masking of a pure tone as a function of frequency shift in the complex tones. There was negligible variation, except for the lowest level used. The results indicate that the changes in excitation level at threshold for the TFS1 task would be too small to be usable. The results are consistent with the TFS1 task being performed using TFS cues, and with frequency selectivity having an indirect effect on performance via its influence on TFS cues.

Re-examining the upper limit of temporal pitch.

Five normally hearing listeners pitch-ranked harmonic complexes of different fundamental frequencies (F0s) filtered in three different frequency regions. Harmonics were summed either in sine, alternating sine-cosine (ALT), or pulse-spreading (PSHC) phase. The envelopes of ALT and PSHC complexes repeated at rates of 2F0 and 4F0. Pitch corresponded to those rates at low F0s, but, as F0 increased, there was a range of F0s over which pitch remained constant or dropped. Gammatone-filterbank simulations showed that, as F0 increased and the number of harmonics interacting in a filter dropped, the output of that filter switched from repeating at 2F0 or 4F0 to repeating at F0. A model incorporating this phenomenon accounted well for the data, except for complexes filtered into the highest frequency region (7800-10 800 Hz). To account for the data in that region it was necessary to assume either that auditory filters at very high frequencies are sharper than traditionally believed, and/or that the auditory system applies smaller weights to filters whose outputs repeat at high rates. The results also provide evidence on the highest pitch that can be derived from purely temporal cues, and corroborate recent reports that a complex pitch can be derived from very-high-frequency resolved harmonics.

Evaluation of a cochlear-implant processing strategy incorporating phantom stimulation and asymmetric pulses.

OBJECTIVE: To evaluate a speech-processing strategy in which the lowest frequency channel is conveyed using an asymmetric pulse shape and "phantom stimulation", where current is injected into one intra-cochlear electrode and where the return current is shared between an intra-cochlear and an extra-cochlear electrode. This strategy is expected to provide more selective excitation of the cochlear apex, compared to a standard strategy where the lowest-frequency channel is conveyed by symmetric pulses in monopolar mode. In both strategies all other channels were conveyed by monopolar stimulation. DESIGN: Within-subjects comparison between the two strategies. Four experiments: (1) discrimination between the strategies, controlling for loudness differences, (2) consonant identification, (3) recognition of lowpass-filtered sentences in quiet, (4) sentence recognition in the presence of a competing speaker. STUDY SAMPLE: Eight users of the Advanced Bionics CII/Hi-Res 90k cochlear implant. RESULTS: Listeners could easily discriminate between the two strategies but no consistent differences in performance were observed. CONCLUSIONS: The proposed method does not improve speech perception, at least in the short term.

No evidence for ITD-specific adaptation in the frequency following response.

Neurons sensitive to interaural time differences (ITDs) in the fine structure of low-frequency signals have been found in binaurally responsive auditory nuclei in a wide range of species. The present study investigated whether the frequency following response (FFR) would show evidence for neurons "tuned" to ITD in humans. The FFR is a scalp-recorded measure of sustained phase-locked brainstem activity that has been shown to follow the frequency of low-frequency tones. The magnitude of the FFR often decreases over time for tones of long duration. The present study investigated whether this adaptation effect is ITD specific.The FFR to a 100-ms, 80-dB SPL, 504-Hz target tone was measured for ten subjects. The target was preceded by a 200-ms, 80-dB SPL, 504-Hz adaptor. The target always led by 0.5 ms in the left ear. The adaptor led either in the left ear or in the right ear by 0.5 ms. Stimuli (adaptor + target = pair) were presented in alternating polarity at a rate of 1.81 Hz. We used a "vertical" montage (+Fz, ­ C7, ground = Fpz) for which the FFR is assumed to reflect phase-locked neural activity from rostral generators in the brainstem. The averaged FFR waveforms for each polarity were subtracted, to enhance temporal fine structure responses. The results showed significant adaptation effects in the spectral magnitude of the FFR. However, adaptation was not larger when the adaptor had the same ITD as the target than when the ITD of the adaptor differed from that of the target. Thus, the current data provide no evidence that the spectral magnitude of the scalp-recorded FFR provides a non-invasive indicator of ITD-specific neural activation.

Further examination of complex pitch perception in the absence of a place-rate match.

Oxenham et al. [Proc. Nat. Acad. Sci. 101, 1421-1425 (2004)] reported that listeners cannot derive a "missing fundamental" from three transposed tones having high carrier frequencies and harmonically related low-frequency modulators. This finding was attributed to complex pitch perception requiring correct tonotopic representation but could have been due to the very high modulator rate difference limens (DLs) observed for individual transposed tones. Experiments 1 and 2 showed that much lower DLs could be obtained for bandpass-filtered pulse trains than for transposed tones with repetition rates of 100 or 300 pps; however, DLs were still larger than for low-frequency pure tones. Experiment 3 presented three pulse trains filtered between 1375 and 1875, 3900 and 5400, and 7800 and 10 800 Hz simultaneously with a pink-noise background. Listeners could not compare the "missing fundamental" of a stimulus in which the pulse rates were, respectively, 150, 225, and 300 pps, to one where all pulse trains had a rate of 75 pps, even though they could compare a 150 + 225 + 300 Hz complex tone to a 75-Hz pure tone. Hence although filtered pulse trains can produce fairly good pitch perception of simple stimuli having low repetition rates and high-frequency spectral content, no evidence that such stimuli enable complex pitch perception in the absence of a place-rate match was found.

Using Zebra-speech to study sequential and simultaneous speech segregation in a cochlear-implant simulation.

Previous studies have suggested that cochlear implant users may have particular difficulties exploiting opportunities to glimpse clear segments of a target speech signal in the presence of a fluctuating masker. Although it has been proposed that this difficulty is associated with a deficit in linking the glimpsed segments across time, the details of this mechanism are yet to be explained. The present study introduces a method called Zebra-speech developed to investigate the relative contribution of simultaneous and sequential segregation mechanisms in concurrent speech perception, using a noise-band vocoder to simulate cochlear implants. One experiment showed that the saliency of the difference between the target and the masker is a key factor for Zebra-speech perception, as it is for sequential segregation. Furthermore, forward masking played little or no role, confirming that intelligibility was not limited by energetic masking but by across-time linkage abilities. In another experiment, a binaural cue was used to distinguish the target and the masker. It showed that the relative contribution of simultaneous and sequential segregation depended on the spectral resolution, with listeners relying more on sequential segregation when the spectral resolution was reduced. The potential of Zebra-speech as a segregation enhancement strategy for cochlear implants is discussed.

Differences between psychoacoustic and frequency following response measures of distortion tone level and masking.

The scalp-recorded frequency following response (FFR) in humans was measured for a 244-Hz pure tone at a range of input levels and for complex tones containing harmonics 2-4 of a 300-Hz fundamental, but shifted by ±56 Hz. The effective magnitude of the cubic difference tone (CDT) and the quadratic difference tone (QDT, at F(2)-F(1)) in the FFR for the complex was estimated by comparing the magnitude spectrum of the FFR at the distortion product (DP) frequency with that for the pure tone. The effective DP levels in the FFR were higher than those commonly estimated in psychophysical experiments, indicating contributions to the DP in the FFR in addition to the audible propagated component. A low-frequency narrowband noise masker reduced the magnitude of FFR responses to the CDT but also to primary components over a wide range of frequencies. The results indicate that audible DPs may contribute very little to the DPs observed in the FFR and that using a narrowband noise for the purpose of masking audible DPs can have undesired effects on the FFR over a wide frequency range. The results are consistent with the notion that broadly tuned mechanisms central to the auditory nerve strongly influence the FFR.

The frequency following response (FFR) may reflect pitch-bearing information but is not a direct representation of pitch.

The frequency following response (FFR), a scalp-recorded measure of phase-locked brainstem activity, is often assumed to reflect the pitch of sounds as perceived by humans. In two experiments, we investigated the characteristics of the FFR evoked by complex tones. FFR waveforms to alternating-polarity stimuli were averaged for each polarity and added, to enhance envelope, or subtracted, to enhance temporal fine structure information. In experiment 1, frequency-shifted complex tones, with all harmonics shifted by the same amount in Hertz, were presented diotically. Only the autocorrelation functions (ACFs) of the subtraction-FFR waveforms showed a peak at a delay shifted in the direction of the expected pitch shifts. This expected pitch shift was also present in the ACFs of the output of an auditory nerve model. In experiment 2, the components of a harmonic complex with harmonic numbers 2, 3, and 4 were presented either to the same ear ("mono") or the third harmonic was presented contralaterally to the ear receiving the even harmonics ("dichotic"). In the latter case, a pitch corresponding to the missing fundamental was still perceived. Monaural control conditions presenting only the even harmonics ("2 + 4") or only the third harmonic ("3") were also tested. Both the subtraction and the addition waveforms showed that (1) the FFR magnitude spectra for "dichotic" were similar to the sum of the spectra for the two monaural control conditions and lacked peaks at the fundamental frequency and other distortion products visible for "mono" and (2) ACFs for "dichotic" were similar to those for "2 + 4" and dissimilar to those for "mono." The results indicate that the neural responses reflected in the FFR preserve monaural temporal information that may be important for pitch, but provide no evidence for any additional processing over and above that already present in the auditory periphery, and do not directly represent the pitch of dichotic stimuli.

Frequency discrimination duration effects for Huggins pitch and narrowband noise (L).

Frequency difference limens (FDLs) were measured for Huggins pitch (HP) stimuli, consisting of a 30-Hz wide band of interaurally decorrelated noise in a diotic low-pass noise and for 30-Hz wide bands of diotic narrowband noise presented in a diotic low-pass noise background. FDLs at a 400-ms duration for the two stimulus types were equated by adjusting the level of the narrowband noise relative to the background. The effects of duration on the FDLs were then measured for center frequencies of 300, 600, and 900 Hz. Although the results were compromised by floor effects at 900 Hz, at 300 and 600 Hz, the duration effects were very similar for the HP and narrowband noise stimuli, with a large improvement in performance between 100 and 400 ms. In contrast to previous results for pure tones, the effect of duration was independent of frequency. The results suggest that: (1) Binaural and monaural pitches may be processed using a common mechanism; (2) discrimination performance for HP and low-sensation-level narrowband noise stimuli is not determined by the number of waveform periods.

The continuity illusion does not depend on attentional state: FMRI evidence from illusory vowels.

We investigate whether the neural correlates of the continuity illusion, as measured using fMRI, are modulated by attention. As we have shown previously, when two formants of a synthetic vowel are presented in an alternating pattern, the vowel can be identified if the gaps in each formant are filled with bursts of plausible masking noise, causing the illusory percept of a continuous vowel ("Illusion" condition). When the formant-to-noise ratio is increased so that noise no longer plausibly masks the formants, the formants are heard as interrupted ("Illusion Break" condition) and vowels are not identifiable. A region of the left middle temporal gyrus (MTG) is sensitive both to intact synthetic vowels (two formants present simultaneously) and to Illusion stimuli, compared to Illusion Break stimuli. Here, we compared these conditions in the presence and absence of attention. We examined fMRI signal for different sound types under three attentional conditions: full attention to the vowels; attention to a visual distracter; or attention to an auditory distracter. Crucially, although a robust main effect of attentional state was observed in many regions, the effect of attention did not differ systematically for the illusory vowels compared to either intact vowels or to the Illusion Break stimuli in the left STG/MTG vowel-sensitive region. This result suggests that illusory continuity of vowels is an obligatory perceptual process, and operates independently of attentional state. An additional finding was that the sensitivity of primary auditory cortex to the number of sound onsets in the stimulus was modulated by attention.

Temporal pitch perception at high rates in cochlear implants.

A recent study reported that a group of Med-El COMBI 40+CI (cochlear implant) users could, in a forced-choice task, detect changes in the rate of a pulse train for rates higher than the 300 pps "upper limit" commonly reported in the literature [Kong, Y.-Y., et al. (2009). J. Acoust. Soc. Am. 125, 1649-1657]. The present study further investigated the upper limit of temporal pitch in the same group of CI users on three tasks [pitch ranking, rate discrimination, and multidimensional scaling (MDS)]. The patterns of results were consistent across the three tasks and all subjects could follow rate changes above 300 pps. Two subjects showed exceptional ability to follow temporal pitch change up to about 900 pps. Results from the MDS study indicated that, for the two listeners tested, changes in pulse rate over the range of 500-840 pps were perceived along a perceptual dimension that was orthogonal to the place of excitation. Some subjects showed a temporal pitch reversal at rates beyond their upper limit of pitch and some showed a reversal within a small range of rates below the upper limit. These results are discussed in relation to the possible neural bases for temporal pitch processing at high rates.

Simulations of cochlear-implant speech perception in modulated and unmodulated noise.

Experiment 1 replicated the finding that normal-hearing listeners identify speech better in modulated than in unmodulated noise. This modulated-unmodulated difference ("MUD") has been previously shown to be reduced or absent for cochlear-implant listeners and for normal-hearing listeners presented with noise-vocoded speech. Experiments 2-3 presented normal-hearing listeners with noise-vocoded speech in unmodulated or 16-Hz-square-wave modulated noise, and investigated whether the introduction of simple binaural differences between target and masker could restore the masking release. Stimuli were presented over headphones. When the target and masker were presented to one ear, adding a copy of the masker to the other ear ("diotic configuration") aided performance but did so to a similar degree for modulated and unmodulated maskers, thereby failing to improve the modulation masking release. Presenting an uncorrelated noise to the opposite ear ("dichotic configuration") had no effect, either for modulated or unmodulated maskers, consistent with the improved performance in the diotic configuration being due to interaural decorrelation processing. For noise-vocoded speech, the provision of simple spatial differences did not allow listeners to take greater advantage of the dips present in a modulated masker.