Inference of the distortion component of hearing impairment from speech recognition by predicting the effect of the attenuation component.

OBJECTIVE: A model-based determination of the average supra-threshold ("distortion") component of hearing impairment which limits the benefit of hearing aid amplification. DESIGN: Published speech recognition thresholds (SRTs) were predicted with the framework for auditory discrimination experiments (FADE), which simulates recognition processes, the speech intelligibility index (SII), which exploits frequency-dependent signal-to-noise ratios (SNR), and a modified SII with a hearing-loss-dependent band importance function (PAV). Their attenuation-component-based prediction errors were interpreted as estimates of the distortion component. STUDY SAMPLE: Unaided SRTs of 315 hearing-impaired ears measured with the German matrix sentence test in stationary noise. RESULTS: Overall, the models showed root-mean-square errors (RMSEs) of 7 dB, but for steeply sloping hearing loss FADE and PAV were more accurate (RMSE = 9 dB) than the SII (RMSE = 23 dB). Prediction errors of FADE and PAV increased linearly with the average hearing loss. The consideration of the distortion component estimate significantly improved the accuracy of FADE's and PAV's predictions. CONCLUSIONS: The supra-threshold distortion component-estimated by prediction errors of FADE and PAV-seems to increase with the average hearing loss. Accounting for a distortion component improves the model predictions and implies a need for effective compensation strategies for supra-threshold processing deficits with increasing audibility loss.

Thoughts on the potential to compensate a hearing loss in noise.

Background: The effect of hearing impairment on speech perception was described by Plomp (1978) as a sum of a loss of class A, due to signal attenuation, and a loss of class D, due to signal distortion. While a loss of class A can be compensated by linear amplification, a loss of class D, which severely limits the benefit of hearing aids in noisy listening conditions, cannot. The hearing loss of class D is assumed to be the main reason why not few users of hearing aids keep complaining about the limited benefit of their devices in noisy environments. Working compensation strategies against it are unknown. Methods: Recently, in an approach to model human speech recognition by means of a re-purposed automatic speech recognition (ASR) system, the loss of class D was explained by introducing a level uncertainty which reduces the individual accuracy of spectro-temporal signal levels. Based on this finding, an implementation of a patented dynamic range manipulation scheme (PLATT) is proposed which aims to mitigate the effect of increased level uncertainty on speech recognition in noise by expanding spectral modulation patterns in the range of 2 to 4 ERB. This compensation approach is objectively evaluated regarding the benefit in speech recognition thresholds in noise using the ASR-based speech recognition model. Recommendations for an evaluation with human listeners are derived. Results: The objective evaluation suggests that approximately half of the class D loss due to an increased level uncertainty might be compensable. To measure the effect with human listeners, an experiment needs to be carefully designed to prevent the confusion class A and D loss compensations. Conclusions: A working compensation strategy for the class D loss could provide previously unexploited potential for relief. Evidence has to be provided in experiments with human listeners.

DARF: A data-reduced FADE version for simulations of speech recognition thresholds with real hearing aids.

Developing and selecting hearing aids is a time consuming process which is simplified by using objective models. Previously, the framework for auditory discrimination experiments (FADE) accurately simulated benefits of hearing aid algorithms with root mean squared prediction errors below 3 dB. One FADE simulation requires several hours of (un)processed signals, which is obstructive when the signals have to be recorded. We propose and evaluate a data-reduced FADE version (DARF) which facilitates simulations with signals that cannot be processed digitally, but that can only be recorded in real-time. DARF simulates one speech recognition threshold (SRT) with about 30 min of recorded and processed signals of the (German) matrix sentence test. Benchmark experiments were carried out to compare DARF and standard FADE exhibiting small differences for stationary maskers (1 dB), but larger differences with strongly fluctuating maskers (5 dB). Hearing impairment and hearing aid algorithms seemed to reduce the differences. Hearing aid benefits were simulated in terms of speech recognition with three pairs of real hearing aids in silence (≥8 dB), in stationary and fluctuating maskers in co-located (stat. 2 dB; fluct. 6 dB), and spatially separated speech and noise signals (stat. ≥8 dB; fluct. 8 dB). The simulations were plausible in comparison to data from literature, but a comparison with empirical data is still open. DARF facilitates objective SRT simulations with real devices with unknown signal processing in real environments. Yet, a validation of DARF for devices with unknown signal processing is still pending since it was only tested with three similar devices. Nonetheless, DARF could be used for improving as well as for developing or model-based fitting of hearing aids.

Sensitivity and specificity of automatic audiological classification using expert-labelled audiological data and Common Audiological Functional Parameters.

OBJECTIVE: As a step towards the development of an audiological diagnostic supporting tool employing machine learning methods, this article aims at evaluating the classification performance of different audiological measures as well as Common Audiological Functional Parameters (CAFPAs). CAFPAs are designed to integrate different clinical databases and provide abstract representations of measures. DESIGN: Classification and evaluation of classification performance in terms of sensitivity and specificity are performed on a data set from a previous study, where statistical models of diagnostic cases were estimated from expert-labelled data. STUDY SAMPLE: The data set contains 287 cases. RESULTS: The classification performance in clinically relevant comparison sets of two competing categories was analysed for audiological measures and CAFPAs. It was found that for different audiological diagnostic questions a combination of measures using different weights of the parameters is useful. A set of four to six measures was already sufficient to achieve maximum classification performance which indicates that the measures contain redundant information. CONCLUSIONS: The current set of CAFPAs was confirmed to yield in most cases approximately the same classification performance as the respective optimum set of audiological measures. Overall, the concept of CAFPAs as compact, abstract representation of auditory deficiencies is confirmed.

Simulations with FADE of the effect of impaired hearing on speech recognition performance cast doubt on the role of spectral resolution.

Listeners with hearing impairment show sub-optimal processing of acoustic signals which affects their ability to recognize speech. In this contribution, three effective signal processing deficits are proposed to simulate sensorineural hearing impairment and their effect on simulated speech recognition performance is studied. Psychoacoustic and speech recognition experiments were simulated with the framework for auditory discrimination experiments (FADE). Loss in absolute hearing threshold was modeled as lower level limit, supra-threshold loss in envelope amplitude resolution as multiplicative noise, and reduced spectral resolution was simulated with an increase of the analysis bandwidth. Their effects on the speech recognition performance with the German matrix test in quiet and noise, the audiogram, and tone in (notched) noise detection experiments were systematically examined. The simulations indicate that each psychoacoustic experiment relates to at least one signal processing deficit. This indicates the possibility to determine individual model parameters from the outcome of psychoacoustic experiments. Moreover, absolute hearing thresholds yield the highest effects on simulated speech recognition thresholds, followed by supra-threshold loss in envelope amplitude resolution, and-to a much smaller degree-spectral resolution. A reduced spectral resolution only affected recognition performance in fluctuating masker for normal hearing thresholds, indicating its potential relevance for more complex listening conditions. In contrast to popular interpretations in the literature, the simulations reveal that reduced spectral resolution plays a minor role compared to a reduced envelope amplitude resolution in characterizing supra-threshold hearing loss at least in stationary noise.

Common Audiological Functional Parameters (CAFPAs) for single patient cases: deriving statistical models from an expert-labelled data set.

Objective: Statistical knowledge about many patients could be exploited using machine learning to provide supporting information to otolaryngologists and other hearing health care professionals, but needs to be made accessible. The Common Audiological Functional Parameters (CAFPAs) were recently introduced for the purpose of integrating data from different databases by providing an abstract representation of audiological measurements. This paper aims at collecting expert labels for a sample database and to determine statistical models from the labelled data set.Design: By an expert survey, CAFPAs as well as labels for audiological findings and treatment recommendations were collected for patients from the database of Hörzentrum Oldenburg.Study sample: A total of 287 single patient cases were assessed by twelve highly experienced audiological experts.Results: The labelled data set was used to derive probability density functions for categories given by the expert labels. The collected data set is suitable for estimating training distributions due to realistic variability contained in data for different, distinct categories. Suitable distribution functions were determined. The derived training distributions were compared regarding different audiological questions.Conclusions: The method-expert survey, sorting data into categories, and determining training distributions - could be extended to other data sets, which could then be integrated via the CAFPAs and used in a classification task.

Common Audiological Functional Parameters (CAFPAs): statistical and compact representation of rehabilitative audiological classification based on expert knowledge.

OBJECTIVE: As a step towards objectifying audiological rehabilitation and providing comparability between different test batteries and clinics, the Common Audiological Functional Parameters (CAFPAs) were introduced as a common and abstract representation of audiological knowledge obtained from diagnostic tests. DESIGN: Relationships between CAFPAs as an intermediate representation between diagnostic tests and audiological findings, diagnoses and treatment recommendations (summarised as "diagnostic cases") were established by means of an expert survey. Expert knowledge was collected for 14 given categories covering different diagnostic cases. For each case, the experts were asked to indicate expected ranges of diagnostic test outcomes, as well as traffic light-encoded CAFPAs. STUDY SAMPLE: Eleven German experts in the field of audiological rehabilitation from Hanover and Oldenburg participated in the survey. RESULTS: Audiological findings or treatment recommendations could be distinguished by a statistical model derived from the experts' answers for CAFPAs as well as audiological tests. CONCLUSIONS: The CAFPAs serve as an abstract, comprehensive representation of audiological knowledge. If more detailed information on certain functional aspects of the auditory system is required, the CAFPAs indicate which information is missing. The statistical graphical representations for CAFPAs and audiological tests are suitable for audiological teaching material; they are universally applicable for real clinical databases.

Modeling the onset advantage in musical instrument recognition.

Sound onsets provide particularly valuable cues for musical instrument identification by human listeners. It has remained unclear whether this onset advantage is due to enhanced perceptual encoding or the richness of acoustical information during onsets. Here this issue was approached by modeling a recent study on instrument identification from tone excerpts [Siedenburg. (2019). J. Acoust. Soc. Am. 145(2), 1078-1087]. A simple Hidden Markov Model classifier with separable Gabor filterbank features simulated human performance and replicated the onset advantage observed previously for human listeners. These results provide evidence that the onset advantage may be driven by the distinct acoustic qualities of onsets.

A physiologically-inspired model reproducing the speech intelligibility benefit in cochlear implant listeners with residual acoustic hearing.

This study introduces a speech intelligibility model for cochlear implant users with ipsilateral preserved acoustic hearing that aims at simulating the observed speech-in-noise intelligibility benefit when receiving simultaneous electric and acoustic stimulation (EA-benefit). The model simulates the auditory nerve spiking in response to electric and/or acoustic stimulation. The temporally and spatially integrated spiking patterns were used as the final internal representation of noisy speech. Speech reception thresholds (SRTs) in stationary noise were predicted for a sentence test using an automatic speech recognition framework. The model was employed to systematically investigate the effect of three physiologically relevant model factors on simulated SRTs: (1) the spatial spread of the electric field which co-varies with the number of electrically stimulated auditory nerves, (2) the "internal" noise simulating the deprivation of auditory system, and (3) the upper bound frequency limit of acoustic hearing. The model results show that the simulated SRTs increase monotonically with increasing spatial spread for fixed internal noise, and also increase with increasing the internal noise strength for a fixed spatial spread. The predicted EA-benefit does not follow such a systematic trend and depends on the specific combination of the model parameters. Beyond 300 Hz, the upper bound limit for preserved acoustic hearing is less influential on speech intelligibility of EA-listeners in stationary noise. The proposed model-predicted EA-benefits are within the range of EA-benefits shown by 18 out of 21 actual cochlear implant listeners with preserved acoustic hearing.

Sentence Recognition Prediction for Hearing-impaired Listeners in Stationary and Fluctuation Noise With FADE: Empowering the Attenuation and Distortion Concept by Plomp With a Quantitative Processing Model.

To characterize the individual patient's hearing impairment as obtained with the matrix sentence recognition test, a simulation Framework for Auditory Discrimination Experiments (FADE) is extended here using the Attenuation and Distortion (A+D) approach by Plomp as a blueprint for setting the individual processing parameters. FADE has been shown to predict the outcome of both speech recognition tests and psychoacoustic experiments based on simulations using an automatic speech recognition system requiring only few assumptions. It builds on the closed-set matrix sentence recognition test which is advantageous for testing individual speech recognition in a way comparable across languages. Individual predictions of speech recognition thresholds in stationary and in fluctuating noise were derived using the audiogram and an estimate of the internal level uncertainty for modeling the individual Plomp curves fitted to the data with the Attenuation (A-) and Distortion (D-) parameters of the Plomp approach. The "typical" audiogram shapes from Bisgaard et al with or without a "typical" level uncertainty and the individual data were used for individual predictions. As a result, the individualization of the level uncertainty was found to be more important than the exact shape of the individual audiogram to accurately model the outcome of the German Matrix test in stationary or fluctuating noise for listeners with hearing impairment. The prediction accuracy of the individualized approach also outperforms the (modified) Speech Intelligibility Index approach which is based on the individual threshold data only.

A simulation framework for auditory discrimination experiments: Revealing the importance of across-frequency processing in speech perception.

A framework for simulating auditory discrimination experiments, based on an approach from Schädler, Warzybok, Hochmuth, and Kollmeier [(2015). Int. J. Audiol. 54, 100-107] which was originally designed to predict speech recognition thresholds, is extended to also predict psychoacoustic thresholds. The proposed framework is used to assess the suitability of different auditory-inspired feature sets for a range of auditory discrimination experiments that included psychoacoustic as well as speech recognition experiments in noise. The considered experiments were 2 kHz tone-in-broadband-noise simultaneous masking depending on the tone length, spectral masking with simultaneously presented tone signals and narrow-band noise maskers, and German Matrix sentence test reception threshold in stationary and modulated noise. The employed feature sets included spectro-temporal Gabor filter bank features, Mel-frequency cepstral coefficients, logarithmically scaled Mel-spectrograms, and the internal representation of the Perception Model from Dau, Kollmeier, and Kohlrausch [(1997). J. Acoust. Soc. Am. 102(5), 2892-2905]. The proposed framework was successfully employed to simulate all experiments with a common parameter set and obtain objective thresholds with less assumptions compared to traditional modeling approaches. Depending on the feature set, the simulated reference-free thresholds were found to agree with-and hence to predict-empirical data from the literature. Across-frequency processing was found to be crucial to accurately model the lower speech reception threshold in modulated noise conditions than in stationary noise conditions.

Matrix sentence intelligibility prediction using an automatic speech recognition system.

OBJECTIVE: The feasibility of predicting the outcome of the German matrix sentence test for different types of stationary background noise using an automatic speech recognition (ASR) system was studied. DESIGN: Speech reception thresholds (SRT) of 50% intelligibility were predicted in seven noise conditions. The ASR system used Mel-frequency cepstral coefficients as a front-end and employed whole-word Hidden Markov models on the back-end side. The ASR system was trained and tested with noisy matrix sentences on a broad range of signal-to-noise ratios. STUDY SAMPLE: The ASR-based predictions were compared to data from the literature ( Hochmuth et al, 2015 ) obtained with 10 native German listeners with normal hearing and predictions of the speech intelligibility index (SII). RESULTS: The ASR-based predictions showed a high and significant correlation (R² = 0.95, p < 0.001) with the empirical data across different noise conditions, outperforming the SII-based predictions which showed no correlation with the empirical data (R² = 0.00, p = 0.987). CONCLUSIONS: The SRTs for the German matrix test for listeners with normal hearing in different stationary noise conditions could well be predicted based on the acoustical properties of the speech and noise signals. Minimum assumptions were made about human speech processing already incorporated in a reference-free ordinary ASR system.

Separable spectro-temporal Gabor filter bank features: Reducing the complexity of robust features for automatic speech recognition.

To test if simultaneous spectral and temporal processing is required to extract robust features for automatic speech recognition (ASR), the robust spectro-temporal two-dimensional-Gabor filter bank (GBFB) front-end from Schädler, Meyer, and Kollmeier [J. Acoust. Soc. Am. 131, 4134-4151 (2012)] was de-composed into a spectral one-dimensional-Gabor filter bank and a temporal one-dimensional-Gabor filter bank. A feature set that is extracted with these separate spectral and temporal modulation filter banks was introduced, the separate Gabor filter bank (SGBFB) features, and evaluated on the CHiME (Computational Hearing in Multisource Environments) keywords-in-noise recognition task. From the perspective of robust ASR, the results showed that spectral and temporal processing can be performed independently and are not required to interact with each other. Using SGBFB features permitted the signal-to-noise ratio (SNR) to be lowered by 1.2 dB while still performing as well as the GBFB-based reference system, which corresponds to a relative improvement of the word error rate by 12.8%. Additionally, the real time factor of the spectro-temporal processing could be reduced by more than an order of magnitude. Compared to human listeners, the SNR needed to be 13 dB higher when using Mel-frequency cepstral coefficient features, 11 dB higher when using GBFB features, and 9 dB higher when using SGBFB features to achieve the same recognition performance.

Spectro-temporal modulation subspace-spanning filter bank features for robust automatic speech recognition.

In an attempt to increase the robustness of automatic speech recognition (ASR) systems, a feature extraction scheme is proposed that takes spectro-temporal modulation frequencies (MF) into account. This physiologically inspired approach uses a two-dimensional filter bank based on Gabor filters, which limits the redundant information between feature components, and also results in physically interpretable features. Robustness against extrinsic variation (different types of additive noise) and intrinsic variability (arising from changes in speaking rate, effort, and style) is quantified in a series of recognition experiments. The results are compared to reference ASR systems using Mel-frequency cepstral coefficients (MFCCs), MFCCs with cepstral mean subtraction (CMS) and RASTA-PLP features, respectively. Gabor features are shown to be more robust against extrinsic variation than the baseline systems without CMS, with relative improvements of 28% and 16% for two training conditions (using only clean training samples or a mixture of noisy and clean utterances, respectively). When used in a state-of-the-art system, improvements of 14% are observed when spectro-temporal features are concatenated with MFCCs, indicating the complementarity of those feature types. An analysis of the importance of specific MF shows that temporal MF up to 25 Hz and spectral MF up to 0.25 cycles/channel are beneficial for ASR.