Peripheral Neural Synchrony in Postlingually Deafened Adult Cochlear Implant Users.

OBJECTIVES: This paper reports a noninvasive method for quantifying neural synchrony in the cochlear nerve (i.e., peripheral neural synchrony) in cochlear implant (CI) users, which allows for evaluating this physiological phenomenon in human CI users for the first time in the literature. In addition, this study assessed how peripheral neural synchrony was correlated with temporal resolution acuity and speech perception outcomes measured in quiet and in noise in postlingually deafened adult CI users. It tested the hypothesis that peripheral neural synchrony was an important factor for temporal resolution acuity and speech perception outcomes in noise in postlingually deafened adult CI users. DESIGN: Study participants included 24 postlingually deafened adult CI users with a Cochlear™ Nucleus® device. Three study participants were implanted bilaterally, and each ear was tested separately. For each of the 27 implanted ears tested in this study, 400 sweeps of the electrically evoked compound action potential (eCAP) were measured at four electrode locations across the electrode array. Peripheral neural synchrony was quantified at each electrode location using the phase-locking value (PLV), which is a measure of trial-by-trial phase coherence among eCAP sweeps/trials. Temporal resolution acuity was evaluated by measuring the within-channel gap detection threshold (GDT) using a three-alternative, forced-choice procedure in a subgroup of 20 participants (23 implanted ears). For each ear tested in these participants, GDTs were measured at two electrode locations with a large difference in PLVs. For 26 implanted ears tested in 23 participants, speech perception performance was evaluated using consonant-nucleus-consonant (CNC) word lists presented in quiet and in noise at signal to noise ratios (SNRs) of +10 and +5 dB. Linear Mixed effect Models were used to evaluate the effect of electrode location on the PLV and the effect of the PLV on GDT after controlling for the stimulation level effects. Pearson product-moment correlation tests were used to assess the correlations between PLVs, CNC word scores measured in different conditions, and the degree of noise effect on CNC word scores. RESULTS: There was a significant effect of electrode location on the PLV after controlling for the effect of stimulation level. There was a significant effect of the PLV on GDT after controlling for the effects of stimulation level, where higher PLVs (greater synchrony) led to lower GDTs (better temporal resolution acuity). PLVs were not significantly correlated with CNC word scores measured in any listening condition or the effect of competing background noise presented at an SNR of +10 dB on CNC word scores. In contrast, there was a significant negative correlation between the PLV and the degree of noise effect on CNC word scores for a competing background noise presented at an SNR of +5 dB, where higher PLVs (greater synchrony) correlated with smaller noise effects on CNC word scores. CONCLUSIONS: This newly developed method can be used to assess peripheral neural synchrony in CI users, a physiological phenomenon that has not been systematically evaluated in electrical hearing. Poorer peripheral neural synchrony leads to lower temporal resolution acuity and is correlated with a larger detrimental effect of competing background noise presented at an SNR of 5 dB on speech perception performance in postlingually deafened adult CI users.

WaveNet-based approximation of a cochlear filtering and hair cell transduction model.

Computational auditory models are important tools for gaining new insights into hearing mechanisms, and they can provide a foundation for bio-inspired speech and audio processing algorithms. However, accurate models often entail an immense computational effort, rendering their application unfeasible if quick execution is required. This paper presents a WaveNet-based approximation of the normal-hearing cochlear filtering and inner hair cell (IHC) transduction stages of a widely used auditory model [Zilany and Bruce (2006). J. Acoust. Soc. Am. 120(3), 1446-1466]. The WaveNet model was trained and optimized using a large dataset of clean speech, noisy speech, and music for a wide range of sound pressure levels (SPLs) and characteristic frequencies between 125 Hz and 8 kHz. The model was evaluated with unseen (noisy) speech, music signals, sine tones, and click signals at SPLs between 30 and 100 dB. It provides accurate predictions of the IHC receptor potentials for a given input stimulus and allows an efficient execution with processing times up to 250 times lower compared to an already optimized reference implementation of the original auditory model. The WaveNet model is fully differentiable, thus, allowing its application in the context of deep-learning-based speech and audio enhancement algorithms.

Envelope following responses, noise exposure, and evidence of cochlear synaptopathy in humans: Correction and comment.

A correction and comment are provided for a recent article by Paul, Waheed, Bruce, and Roberts [(2017). J. Acoust. Soc. Am. 142(5), EL434-EL440].

Superior time perception for lower musical pitch explains why bass-ranged instruments lay down musical rhythms.

The auditory environment typically contains several sound sources that overlap in time, and the auditory system parses the complex sound wave into streams or voices that represent the various sound sources. Music is also often polyphonic. Interestingly, the main melody (spectral/pitch information) is most often carried by the highest-pitched voice, and the rhythm (temporal foundation) is most often laid down by the lowest-pitched voice. Previous work using electroencephalography (EEG) demonstrated that the auditory cortex encodes pitch more robustly in the higher of two simultaneous tones or melodies, and modeling work indicated that this high-voice superiority for pitch originates in the sensory periphery. Here, we investigated the neural basis of carrying rhythmic timing information in lower-pitched voices. We presented simultaneous high-pitched and low-pitched tones in an isochronous stream and occasionally presented either the higher or the lower tone 50 ms earlier than expected, while leaving the other tone at the expected time. EEG recordings revealed that mismatch negativity responses were larger for timing deviants of the lower tones, indicating better timing encoding for lower-pitched compared with higher-pitch tones at the level of auditory cortex. A behavioral motor task revealed that tapping synchronization was more influenced by the lower-pitched stream. Results from a biologically plausible model of the auditory periphery suggest that nonlinear cochlear dynamics contribute to the observed effect. The low-voice superiority effect for encoding timing explains the widespread musical practice of carrying rhythm in bass-ranged instruments and complements previously established high-voice superiority effects for pitch and melody.

Subcortical amplitude modulation encoding deficits suggest evidence of cochlear synaptopathy in normal-hearing 18-19 year olds with higher lifetime noise exposure.

Noise exposure and aging can damage cochlear synapses required for suprathreshold listening, even when cochlear structures needed for hearing at threshold remain unaffected. To control for effects of aging, behavioral amplitude modulation (AM) detection and subcortical envelope following responses (EFRs) to AM tones in 25 age-restricted (18-19 years) participants with normal thresholds, but different self-reported noise exposure histories were studied. Participants with more noise exposure had smaller EFRs and tended to have poorer AM detection than less-exposed individuals. Simulations of the EFR using a well-established cochlear model were consistent with more synaptopathy in participants reporting greater noise exposure.

Predicting the quality of enhanced wideband speech with a cochlear model.

Objective measures are commonly used in the development of speech coding algorithms as an adjunct to human subjective evaluation. Predictors of speech quality based on models of physiological or perceptual processing tend to perform better than measures based on simple acoustical properties. Here, a modeling method based on a detailed physiological model and a neurogram similarity measure is developed and optimized to predict the quality of an enhanced wideband speech dataset. A model capturing temporal modulations in neural activity up to 267 Hz was found to perform as well as or better than several existing objective quality measures.

Predicting phoneme and word recognition in noise using a computational model of the auditory periphery.

Several filterbank-based metrics have been proposed to predict speech intelligibility (SI). However, these metrics incorporate little knowledge of the auditory periphery. Neurogram-based metrics provide an alternative, incorporating knowledge of the physiology of hearing by using a mathematical model of the auditory nerve response. In this work, SI was assessed utilizing different filterbank-based metrics (the speech intelligibility index and the speech-based envelope power spectrum model) and neurogram-based metrics, using the biologically inspired model of the auditory nerve proposed by Zilany, Bruce, Nelson, and Carney [(2009), J. Acoust. Soc. Am. 126(5), 2390-2412] as a front-end and the neurogram similarity metric and spectro temporal modulation index as a back-end. Then, the correlations with behavioural scores were computed. Results showed that neurogram-based metrics representing the speech envelope showed higher correlations with the behavioural scores at a word level. At a per-phoneme level, it was found that phoneme transitions contribute to higher correlations between objective measures that use speech envelope information at the auditory periphery level and behavioural data. The presented framework could function as a useful tool for the validation and tuning of speech materials, as well as a benchmark for the development of speech processing algorithms.

The history and future of neural modeling for cochlear implants.

This special issue of Network: Computation in Neural Systems on the topic of "Computational models of the electrically stimulated auditory system" incorporates review articles spanning a wide range of approaches to modeling cochlear implant stimulation of the auditory system. The purpose of this overview paper is to provide a historical context for the different modeling endeavors and to point toward how computational modeling could play a key role in the understanding, evaluation, and improvement of cochlear implants in the future.

Phenomenological modelling of electrically stimulated auditory nerve fibers: A review.

Auditory nerve fibers (ANFs) play a crucial role in hearing by encoding and transporting the synaptic input from inner hair cells into afferent spiking information for higher stages of the auditory system. If the inner hair cells are degenerated, cochlear implants may restore hearing by directly stimulating the ANFs. The response of an ANF is affected by several characteristics of the electrical stimulus and of the ANF, and neurophysiological measurements are needed to know how the ANF responds to a particular stimulus. However, recording from individual nerve fibers in humans is not feasible and obtaining compound neural or psychophysical responses is often time-consuming. This motivates the design and use of models to estimate the ANF response to the electrical stimulation. Phenomenological models reproduce the ANF response based on a simplified description of ANF functionality and on a limited parameter space by not directly describing detailed biophysical mechanisms. Here, we give an overview of phenomenological models published to date and demonstrate how different modeling approaches can account for the diverse phenomena affecting the ANF response. To highlight the success achieved in designing such models, we also describe a number of applications of phenomenological models to predict percepts of cochlear implant listeners.

Analysis of spatiotemporal pattern correction using a computational model of the auditory periphery.

OBJECTIVES: The purpose of this study was to determine the cause of poor experimental performance of a spatiotemporal pattern correction (SPC) scheme that has been proposed as a hearing aid algorithm and to determine contexts in which it may provide benefit. The SPC scheme is intended to compensate for altered phase response and group delay differences in the auditory nerve spiking patterns in impaired ears. Based on theoretical models of loudness and the hypothesized importance of temporal fine structure for intelligibility, the compensations of the SPC scheme are expected to provide benefit; however, preliminary experiments revealed that listeners preferred unprocessed or minimally processed speech as opposed to complete SPC processed speech. DESIGN: An improved version of the SPC scheme was evaluated with a computational auditory model in response to a synthesized vowel at multiple SPLs. The impaired model auditory nerve response to SPC-aided stimuli was compared to the unaided stimuli for spectrotemporal response similarity to the healthy auditory model. This comparison included analysis of synchronized rate across auditory nerve characteristic frequencies and a measure of relative phase response of auditory nerve fibers to complex stimuli derived from cross-correlations. RESULTS: Analysis indicates that SPC can improve a metric of relative phase response at low SPLs, but may do so at the cost of decreased spectrotemporal response similarity to the healthy auditory model and degraded synchrony to vowel formants. In-depth analysis identifies several technical and conceptual problems associated with SPC that need to be addressed. These include the following: (1) a nonflat frequency response through the analysis-synthesis filterbank that results from time-varying changes in the relative temporal alignment of filterbank channels, (2) group delay corrections that are based on incorrect frequencies because of spread of synchrony in auditory nerve responses, and (3) frequency modulations in the processed signal created by the insertion of delays. CONCLUSIONS: Despite these issues, SPC provided benefit to an error metric derived from auditory nerve response cross-correlations at low SPLs, which may mean phase adjustment is achieved at the expense of other metrics, but could be beneficial for low-level speech.

Modulation of electrocortical brain activity by attention in individuals with and without tinnitus.

Age and hearing-level matched tinnitus and control groups were presented with a 40 Hz AM sound using a carrier frequency of either 5 kHz (in the tinnitus frequency region of the tinnitus subjects) or 500 Hz (below this region). On attended blocks subjects pressed a button after each sound indicating whether a single 40 Hz AM pulse of variable increased amplitude (target, probability 0.67) had or had not occurred. On passive blocks subjects rested and ignored the sounds. The amplitude of the 40 Hz auditory steady-state response (ASSR) localizing to primary auditory cortex (A1) increased with attention in control groups probed at 500 Hz and 5 kHz and in the tinnitus group probed at 500 Hz, but not in the tinnitus group probed at 5 kHz (128 channel EEG). N1 amplitude (this response localizing to nonprimary cortex, A2) increased with attention at both sound frequencies in controls but at neither frequency in tinnitus. We suggest that tinnitus-related neural activity occurring in the 5 kHz but not the 500 Hz region of tonotopic A1 disrupted attentional modulation of the 5 kHz ASSR in tinnitus subjects, while tinnitus-related activity in A1 distributing nontonotopically in A2 impaired modulation of N1 at both sound frequencies.

Updated parameters and expanded simulation options for a model of the auditory periphery.

A phenomenological model of the auditory periphery in cats was previously developed by Zilany and colleagues [J. Acoust. Soc. Am. 126, 2390-2412 (2009)] to examine the detailed transformation of acoustic signals into the auditory-nerve representation. In this paper, a few issues arising from the responses of the previous version have been addressed. The parameters of the synapse model have been readjusted to better simulate reported physiological discharge rates at saturation for higher characteristic frequencies [Liberman, J. Acoust. Soc. Am. 63, 442-455 (1978)]. This modification also corrects the responses of higher-characteristic frequency (CF) model fibers to low-frequency tones that were erroneously much higher than the responses of low-CF model fibers in the previous version. In addition, an analytical method has been implemented to compute the mean discharge rate and variance from the model's synapse output that takes into account the effects of absolute refractoriness.

Peripheral neural synchrony in post-lingually deafened adult cochlear implant users.

Objective: This paper reports a noninvasive method for quantifying neural synchrony in the cochlear nerve (i.e., peripheral neural synchrony) in cochlear implant (CI) users, which allows for evaluating this physiological phenomenon in human CI users for the first time in the literature. In addition, this study assessed how peripheral neural synchrony was correlated with temporal resolution acuity and speech perception outcomes measured in quiet and in noise in post-lingually deafened adult CI users. It tested the hypothesis that peripheral neural synchrony was an important factor for temporal resolution acuity and speech perception outcomes in noise in post-lingually deafened adult CI users. Design: Study participants included 24 post-lingually deafened adult CI users with a Cochlear™ Nucleus® device. Three study participants were implanted bilaterally, and each ear was tested separately. For each of the 27 implanted ears tested in this study, 400 sweeps of the electrically evoked compound action potential (eCAP) were measured at four electrode locations across the electrode array. Peripheral neural synchrony was quantified at each electrode location using the phase locking value (PLV), which is a measure of trial-by-trial phase coherence among eCAP sweeps/trials. Temporal resolution acuity was evaluated by measuring the within-channel gap detection threshold (GDT) using a three-alternative, forced-choice procedure in a subgroup of 20 participants (23 implanted ears). For each ear tested in these participants, GDTs were measured at two electrode locations with a large difference in PLVs. For 26 implanted ears tested in 23 participants, speech perception performance was evaluated using Consonant-Nucleus-Consonant (CNC) word lists presented in quiet and in noise at signal-to-noise ratios (SNRs) of +10 and +5 dB. Linear Mixed effect Models were used to evaluate the effect of electrode location on the PLV and the effect of the PLV on GDT after controlling for the stimulation level effects. Pearson product-moment correlation tests were used to assess the correlations between PLVs, CNC word scores measured in different conditions, and the degree of noise effect on CNC word scores. Results: There was a significant effect of electrode location on the PLV after controlling for the effect of stimulation level. There was a significant effect of the PLV on GDT after controlling for the effects of stimulation level, where higher PLVs (greater synchrony) led to lower GDTs (better temporal resolution acuity). PLVs were not significantly correlated with CNC word scores measured in any listening condition or the effect of competing background noise presented at a SNR of +10 dB on CNC word scores. In contrast, there was a significant negative correlation between the PLV and the degree of noise effect on CNC word scores for a competing background noise presented at a SNR of +5 dB, where higher PLVs (greater synchrony) correlated with smaller noise effects on CNC word scores. Conclusions: This newly developed method can be used to assess peripheral neural synchrony in CI users, a physiological phenomenon that has not been systematically evaluated in electrical hearing. Poorer peripheral neural synchrony leads to lower temporal resolution acuity and is correlated with a larger detrimental effect of competing background noise presented at a SNR of 5 dB on speech perception performance in post-lingually deafened adult CI users.

Comparison of response properties of the electrically stimulated auditory nerve reported in human listeners and in animal models.

Cochlear implants (CIs) provide acoustic information to implanted patients by electrically stimulating nearby auditory nerve fibers (ANFs) which then transmit the information to higher-level neural structures for further processing and interpretation. Computational models that simulate ANF responses to CI stimuli enable the exploration of the mechanisms underlying CI performance beyond the capacity of in vivo experimentation alone. However, all ANF models developed to date utilize to some extent anatomical/morphometric data, biophysical properties and/or physiological data measured in non-human animal models. This review compares response properties of the electrically stimulated auditory nerve (AN) in human listeners and different mammalian models. Properties of AN responses to single pulse stimulation, paired-pulse stimulation, and pulse-train stimulation are presented. While some AN response properties are similar between human listeners and animal models (e.g., increased AN sensitivity to single pulse stimuli with long interphase gaps), there are some significant differences. For example, the AN of most animal models is typically more sensitive to cathodic stimulation while the AN of human listeners is generally more sensitive to anodic stimulation. Additionally, there are substantial differences in the speed of recovery from neural adaptation between animal models and human listeners. Therefore, results from animal models cannot be simply translated to human listeners. Recognizing the differences in responses of the AN to electrical stimulation between humans and other mammals is an important step for creating ANF models that are more applicable to various human CI patient populations.

A comparative study of eight human auditory models of monaural processing.

A number of auditory models have been developed using diverging approaches, either physiological or perceptual, but they share comparable stages of signal processing, as they are inspired by the same constitutive parts of the auditory system. We compare eight monaural models that are openly accessible in the Auditory Modelling Toolbox. We discuss the considerations required to make the model outputs comparable to each other, as well as the results for the following model processing stages or their equivalents: Outer and middle ear, cochlear filter bank, inner hair cell, auditory nerve synapse, cochlear nucleus, and inferior colliculus. The discussion includes a list of recommendations for future applications of auditory models.

Can homeostatic plasticity in deafferented primary auditory cortex lead to travelling waves of excitation?

Travelling waves of activity in neural circuits have been proposed as a mechanism underlying a variety of neurological disorders, including epileptic seizures, migraine auras and brain injury. The highly influential Wilson-Cowan cortical model describes the dynamics of a network of excitatory and inhibitory neurons. The Wilson-Cowan equations predict travelling waves of activity in rate-based models that have sufficiently reduced levels of lateral inhibition. Travelling waves of excitation may play a role in functional changes in the auditory cortex after hearing loss. We propose that down-regulation of lateral inhibition may be induced in deafferented cortex via homeostatic plasticity mechanisms. We use the Wilson-Cowan equations to construct a spiking model of the primary auditory cortex that includes a novel, mathematically formalized description of homeostatic plasticity. In our model, the homeostatic mechanisms respond to hearing loss by reducing inhibition and increasing excitation, producing conditions under which travelling waves of excitation can emerge. However, our model predicts that the presence of spontaneous activity prevents the development of long-range travelling waves of excitation. Rather, our simulations show short-duration excitatory waves that cancel each other out. We also describe changes in spontaneous firing, synchrony and tuning after simulated hearing loss. With the exception of shifts in characteristic frequency, changes after hearing loss were qualitatively the same as empirical findings. Finally, we discuss possible applications to tinnitus, the perception of sound without an external stimulus.

Phenomenological model of auditory nerve population responses to cochlear implant stimulation.

BACKGROUND: Models of auditory nerve fiber (ANF) responses to electrical stimulation are helpful to develop advanced coding for cochlear implants (CIs). A phenomenological model of ANF population responses to CI electrical stimulation with a lower computational complexity compared to a biophysical model would be beneficial to evaluate new CI coding strategies. NEW METHOD: This study presents a phenomenological model which combines four temporal characteristics of ANFs (refractoriness, facilitation, accommodation and spike rate adaptation) in addition to a spatial spread of the electric field. RESULTS: The model predicts the performances of CI subjects in the melodic contour identification (MCI) experiment. The simulations for the MCI experiment were consistent with CI recipients' experimental outcomes that were not predictable from the electrical stimulation patterns themselves. COMPARISON WITH EXISTING METHODS: Previously, no phenomenological population model of ANFs has combined all four aforementioned temporal phenomena. CONCLUSIONS: The proposed model would help the further investigations of ANFs responses to different electrical stimulation patterns and comparison of different sound coding strategies in CIs.

A phenomenological model of the synapse between the inner hair cell and auditory nerve: long-term adaptation with power-law dynamics.

There is growing evidence that the dynamics of biological systems that appear to be exponential over short time courses are in some cases better described over the long-term by power-law dynamics. A model of rate adaptation at the synapse between inner hair cells and auditory-nerve (AN) fibers that includes both exponential and power-law dynamics is presented here. Exponentially adapting components with rapid and short-term time constants, which are mainly responsible for shaping onset responses, are followed by two parallel paths with power-law adaptation that provide slowly and rapidly adapting responses. The slowly adapting power-law component significantly improves predictions of the recovery of the AN response after stimulus offset. The faster power-law adaptation is necessary to account for the "additivity" of rate in response to stimuli with amplitude increments. The proposed model is capable of accurately predicting several sets of AN data, including amplitude-modulation transfer functions, long-term adaptation, forward masking, and adaptation to increments and decrements in the amplitude of an ongoing stimulus.

Measuring temporal response properties of auditory nerve fibers in cochlear implant recipients.

Auditory nerve fibers' (ANFs) refractoriness and facilitation can be quantified in electrically evoked compound action potentials (ECAPs) recorded via neural response telemetry (NRT). Although facilitation has been observed in animals and human cochlear implant (CI) recipients, no study has modeled this in human CI users until now. In this study, recovery and facilitation effects at different masker and probe levels for three test electrodes (E6, E12 and E18) in 11 CI subjects were recorded. The ECAP recovery and facilitation were modeled by exponential functions and the same function used for +10 CL masker offset condition can be applied to all other masker offsets measurements. Goodness of fit was evaluated for the exponential functions. A significant effect of probe level was observed on a recovery time constant which highlights the importance of recording the recovery function at the maximum acceptable stimulus level. Facilitation time constant and amplitude showed no dependency on the probe level. However, facilitation was stronger for masker level at or around the threshold of the ECAP (T-ECAP). There was a positive correlation between facilitation magnitude and amplitude growth function (AGF) slope, which indicates that CI subjects with better peripheral neural survival have stronger facilitation.

Perceptual and Model-Based Evaluation of Ideal Time-Frequency Noise Reduction in Hearing-Impaired Listeners.

State-of-the-art hearing aids (HAs) try to overcome the deficit of poor speech intelligibility (SI) in noisy listening environments using digital noise reduction (NR) techniques. The application of time-frequency masks to the noisy sound input is a common NR technique to increase SI. The binary mask with its binary weights and the Wiener filter with continuous weights are representatives of a hard- and a soft-decision approach for time-frequency masking. In normal-hearing listeners, the ideal Wiener filter (IWF) outperforms the ideal binary mask (IBM) in terms of SI and speech quality with perfect SI even at very low signal-to-noise ratios. In this paper, both approaches were investigated for hearing-impaired (HI) listeners. Perceptual and auditory model-based measures were used for the evaluation. The IWF outperformed the IBM in terms of SI. Quality-wise, there was no overall difference between the NR algorithms perceived. Additionally, the processed signals were evaluated based on an auditory nerve model using the neurogram similarity metric (NSIM). The mean NSIM values were significantly different for intelligible and unintelligible sentences. The results suggest that a soft-mask seems to be promising for application in HAs.