WaveNet-based approximation of a cochlear filtering and hair cell transduction model.

Computational auditory models are important tools for gaining new insights into hearing mechanisms, and they can provide a foundation for bio-inspired speech and audio processing algorithms. However, accurate models often entail an immense computational effort, rendering their application unfeasible if quick execution is required. This paper presents a WaveNet-based approximation of the normal-hearing cochlear filtering and inner hair cell (IHC) transduction stages of a widely used auditory model [Zilany and Bruce (2006). J. Acoust. Soc. Am. 120(3), 1446-1466]. The WaveNet model was trained and optimized using a large dataset of clean speech, noisy speech, and music for a wide range of sound pressure levels (SPLs) and characteristic frequencies between 125 Hz and 8 kHz. The model was evaluated with unseen (noisy) speech, music signals, sine tones, and click signals at SPLs between 30 and 100 dB. It provides accurate predictions of the IHC receptor potentials for a given input stimulus and allows an efficient execution with processing times up to 250 times lower compared to an already optimized reference implementation of the original auditory model. The WaveNet model is fully differentiable, thus, allowing its application in the context of deep-learning-based speech and audio enhancement algorithms.

A subjective evaluation of different music preprocessing approaches in cochlear implant listeners.

Cochlear implants (CIs) can partially restore speech perception to relatively high levels in listeners with moderate to profound hearing loss. However, for most CI listeners, the perception and enjoyment of music remains notably poor. Since a number of technical and physiological restrictions of current implant designs cannot be easily overcome, a number of preprocessing methods for music signals have been proposed recently. They aim to emphasize the leading voice and rhythmic elements and to reduce their spectral complexity. In this study, CI listeners evaluated five remixing approaches in comparison to unprocessed signals. To identify potential explaining factors of CI preference ratings, different signal quality criteria of the processed signals were additionally assessed by normal-hearing listeners. Additional factors were investigated based on instrumental signal-level features. For three preprocessing methods, a significant improvement over the unprocessed reference was found. Especially, two deep neural network-based remix strategies proved to enhance music perception in CI listeners. These strategies provide remixes of the respective harmonic and percussive signal components of the four source stems "vocals," "bass," "drums," and "other accompaniment." Moreover, the results demonstrate that CI listeners prefer an attenuation of sustained components of drum source signals.

A versatile deep-neural-network-based music preprocessing and remixing scheme for cochlear implant listeners.

While cochlear implants (CIs) have proven to restore speech perception to a remarkable extent, access to music remains difficult for most CI users. In this work, a methodology for the design of deep learning-based signal preprocessing strategies that simplify music signals and emphasize rhythmic information is proposed. It combines harmonic/percussive source separation and deep neural network (DNN) based source separation in a versatile source mixture model. Two different neural network architectures were assessed with regard to their applicability for this task. The method was evaluated with instrumental measures and in two listening experiments for both network architectures and six mixing presets. Normal-hearing subjects rated the signal quality of the processed signals compared to the original both with and without a vocoder which provides an approximation of the auditory perception in CI listeners. Four combinations of remix models and DNNs have been selected for an evaluation with vocoded signals and were all rated significantly better in comparison to the unprocessed signal. In particular, the two best-performing remix networks are promising candidates for further evaluation in CI listeners.

Interactive Evaluation of a Music Preprocessing Scheme for Cochlear Implants Based on Spectral Complexity Reduction.

Music is difficult to access for the majority of CI users as the reduced dynamic range and poor spectral resolution in cochlear implants (CI), amongst others constraints, severely impair their auditory perception. The reduction of spectral complexity is therefore a promising means to facilitate music enjoyment for CI listeners. We evaluate a spectral complexity reduction method for music signals based on principal component analysis that enforces spectral sparsity, emphasizes the melody contour and attenuates interfering accompanying voices. To cover a wide range of spectral complexity reduction levels a new experimental design for listening experiments was introduced. It allows CI users to select the preferred level of spectral complexity reduction interactively and in real-time. Ten adult CI recipients with post-lingual bilateral profound sensorineural hearing loss and CI experience of at least 6 months were enrolled in the study. In eight consecutive sessions over a period of 4 weeks they were asked to choose their preferred version out of 10 different complexity settings for a total number of 16 recordings of classical western chamber music. As the experiments were performed in consecutive sessions we also studied a potential long term effect. Therefore, we investigated the hypothesis that repeated engagement with music signals of reduced spectral complexity leads to a habituation effect which allows CI users to deal with music signals of increasing complexity. Questionnaires and tests about music listening habits and musical abilities complemented these experiments. The participants significantly preferred signals with high spectral complexity reduction levels over the unprocessed versions. While the results of earlier studies comprising only two preselected complexity levels were generally confirmed, this study revealed a tendency toward a selection of even higher spectral complexity reduction levels. Therefore, spectral complexity reduction for music signals is a useful strategy to enhance music enjoyment for CI users. Although there is evidence for a habituation effect in some subjects, such an effect has not been significant in general.

Music complexity prediction for cochlear implant listeners based on a feature-based linear regression model.

This paper presents a model for predicting music complexity as perceived by cochlear implant (CI) users. To this end, 10 CI users and 19 normal-hearing (NH) listeners rated 12 selected music pieces on a bipolar music complexity scale and 5 other perception-related scales. The results indicate statistically significant differences in the ratings between CI and NH listeners. In particular, the ratings among different scales were significantly correlated for CI users, which hints at a common, hidden scale. The median complexity ratings by CI listeners and features accounting for high-frequency energy, spectral center of gravity, spectral bandwidth, and roughness were used to train a linear principal component regression model for an average CI user. The model was evaluated by means of cross-validation and using an independent database of processed chamber music signals for which music preferences scores by CI users were available. The predictions indicate a clear linear relationship with the preference scores, confirming the negative correlation between music complexity and music preference for CI users found in previous studies. The proposed model is a first step toward an instrumental evaluation procedure in the emerging field of music processing for CIs.

Spectral complexity reduction of music signals based on frequency-domain reduced-rank approximations: An evaluation with cochlear implant listeners.

Methods for spectral complexity reduction of music signals were evaluated in a listening test with cochlear implant (CI) listeners. To this end, reduced-rank approximations were computed in the constant-Q spectral domain using blind and score-informed dimensionality reduction techniques, which were compared to a procedure using a supervised source separation and remixing scheme. Previous works have shown that timbre and pitch cues are transmitted inaccurately through CIs and thus cause perceptual distortions in CI listeners. Hence, the scope of this evaluation was narrowed down to classical chamber music, which is mainly characterized by timbre and pitch and less by rhythmic cues. Suitable music pieces were selected in accordance to a statistical experimental design, which took musically relevant influential factors into account. In a blind two-alternative forced choice task, 14 CI listeners were asked to indicate a preference either for the original signals or a specific processed variant. The results exhibit a statistically significant preference rate of up to 74% for the reduced-rank approximations, whereas the source separation and remixing scheme did not provide any improvement.