Deep Hybrid Multimodal Biometric Recognition System Based on Features-Level Deep Fusion of Five Biometric Traits.

The need for information security and the adoption of the relevant regulations is becoming an overwhelming demand worldwide. As an efficient solution, hybrid multimodal biometric systems utilize fusion to combine multiple biometric traits and sources with improving recognition accuracy, higher security assurance, and to cope with the limitations of the uni-biometric system. In this paper, three strategies for dealing with a feature-level deep fusion of five biometric traits (face, both irises, and two fingerprints) derived from three sources of evidence are proposed and compared. In the first two proposed methodologies, each feature vector is mapped from the feature space into the reproducing kernel Hilbert space (RKHS) separately by selecting the appropriate reproducing kernel. In this higher space, where the result is the conversion of nonlinear relations to linear ones, dimensionality reduction algorithms (KPCA, KLDA) and quaternion-based algorithms (KQPCA, KQPCA) are used for the fusion of the feature vectors. In the third methodology, the fusion of feature spaces based on deep learning is administered by combining feature vectors in in-depth and fully connected layers. The experimental results on 6 databases in the proposed hybrid multibiometric system clearly show the multimodal template obtained from the deep fusion of feature spaces; while being secure against spoof attacks and making the system robust, they can use the low dimensionality of the fused vector to increase the accuracy of a hybrid multimodal biometric system to 100%, showing a significant improvement compared with uni-biometric and other multimodal systems.

A Hybrid Approach to Multimodal Biometric Recognition Based on Feature-level Fusion of Face, Two Irises, and Both Thumbprints.

Background: The most significant motivations for designing multi-biometric systems are high-accuracy recognition, high-security assurances as well as overcoming the limitations like non-universality, noisy sensor data, and large intra-user variations. Therefore, choosing data for fusion is of high significance for the design of a multimodal biometric system. The feature vectors contain richer information than the scores, decisions and even raw data, thereby making feature-level fusion more effective than other levels. Method: In the proposed method, kernel is used for fusion in feature space. First, the face features are extracted using kernel-based methods, the features of both right and left irises are extracted using Hough Transform and Daugman algorithm methods, and the features of both thumb prints are extracted using the Gabor filter bank. Second, after normalization operations, we use kernel methods to map the feature vectors to a kernel Hilbert space where non-linear relations are shown as linear for the purpose of compatibility of feature spaces. Then, dimensionality reduction algorithms are used to the fusion of the feature vectors extracted from fingerprints, irises and the face. since the proposed system uses face, both right 7and left irises and right and left thumbprints, it is hybrid multi-biometric system. We c8arried out the tests on seven databases. Results: Our results show that the hybrid multimodal template, while being secure against spoof attacks and making the system robust, can use the dimensionality of only 15 features to increase the accuracy of a hybrid multimodal biometric system to 100%, which shows a significant improvement compared with uni-biometric and other multimodal systems. Conclusion: The proposed method can be used to search large databases. Consequently, a large database of a secure multimodal template could be correctly differentiated based on the corresponding class of a test sample without any consistency error.

A Fast Approximate Method for Predicting the Behavior of Auditory Nerve Fibers and the Evoked Compound Action Potential (ECAP) Signal.

BACKGROUND: The goal of the current research is to develop a model based on computer simulations which describes both the behavior of the auditory nerve fibers and the cochlear implant system as a rehabilitation device. METHODS: The approximate method was proposed as a low error and fast tool for predicting the behavior of auditory nerve fibers as well as the evoked compound action potential (ECAP) signal. In accurate methods every fiber is simulated; whereas, in approximate method information related to the response of every fiber and its characteristics such as the activation threshold of cochlear fibers are saved and interpolated to predict the behavior of a set of nerve fibers. RESULTS: The approximate model can predict and analyze different stimulation techniques. Although precision is reduced to <1.66% of the accurate method, the required execution time for simulation is reduced by more than 98%. CONCLUSION: The amplitudes of the ECAP signal and the growth function were investigated by changing the parameters of the approximate model including geometrical parameters, electrical, and temporal parameters. In practice, an audiologist can tune the stimulation parameters to reach an effective restoration of the acoustic signal.

Trial-Specific Feature Performance on Single-Channel Auditory Mismatch Negativity Detection.

Successful detection of uncommon events is vital in the survival of an organism. Specifically, the study of neuro-sensory detection lends itself widely to understanding the human brain. Mismatch Negativity (MMN) is an important Event-Related Potential (ERP) response to an oddball stimulus which is preceded by repeated homogeneous stimulation. MMN is associated with perceptual learning and medical diagnostics among other applications. Currently, MMN detection relies on visual inspection of ERPs by skilled clinicians which makes for a costly, slow and subjective tool. In this paper, we use MMN to quantify the discriminative abilities of healthy or diagnosed subjects. We introduce a novel algorithmic method to extract and select important trial-specific features for discriminating standard from deviant responses. We utilize machine learning and classification approaches to evaluate our novel model using single-subject trial data while minimizing the number of necessary selection features provided by statistical test parameters and Genetic Algorithm (GA). In this work, a large variety of methods with 27 subjects, hundreds of trials and electrode counts compete for the definitive discrimination of MMN events. Our model requires only one EEG channel, a single subject and as low as five deviant tones. The results show statistically significant detection improvement over the traditional methods while maximizing resource economy.

Spectro-temporal modulation detection and its relation to speech perception in children with auditory processing disorder.

OBJECTIVES: Poor speech perception in noise is one of the most common complaints reported for children with auditory processing disorder (APD). APD is defined as a deficit in perceptual processing of acoustic information in the auditory system in which decreased spectro-temporal resolution may also contribute. Since the recognition of spoken message in the context of other sounds, is based on the processing of auditory spectro-temporal modulations, the assessment of spectro-temporal modulations sensitivity can evaluate the listener's ability to retrieve and integrate speech segments covered by noise. Therefore, the purpose of this study was to examine spectro-temporal modulation (STM) detection and its relation to speech perception in children with APD and to compare the results with aged-matched normally developed children. METHODS: 35 children with APD and 32 normal hearing children (8-12 years old) were enrolled. In order to examine STM detection performance, six different STM stimulus conditions were employed using three different temporal modulation rates (4, 12 and 32 Hz) and two different spectral modulation densities (0.5 and 2.0 cycles/octave). Initially, the STM detection thresholds at these six STM stimulus conditions were measured in both groups and the results were compared. Thereafter, the relation between STM detection thresholds and speech perception tests, including consonant-vowel in noise and word in noise tests were assessed. RESULTS: The STM sensitivity was poorer than normal for APD children at all STM stimulus conditions. Children with APD displayed significantly poorer STM detection thresholds than those of normally developed children (p < 0.05). Significant correlations were found between STM detection thresholds and speech perception in noise in both groups (p < 0.05). CONCLUSION: The results suggest that the altered encoding of spectro-temporal acoustic cues in the auditory nervous system may be one of the underlying factors of reduced STM detection performance in children with APD. The present study may suggest that poor ability to extract STM cues in children with APD, can be an underlying factor for their listening problems in noise and poor speech perception in challenging situations.

Low variable rate stimulation strategy for cochlear implants using temporal cues and electrophysiological factors.

BACKGROUND: The performance of a cochlear implant (CI), especially in conveying pitch depends on its electrical stimulation strategy. OBJECTIVE: The present study proposes a variable-rate stimulation algorithm which improves speech emotion perception by using temporal fine-structure cues and electrophysiological parameters of the patient. METHODS: This method is based on the coding of the phase information at the peak time intervals of the band-passed signals. The stimulation pulse is generated at the time of peak occurrence, which is able to excite the number of fibers with a discharge probability above a threshold. Calculating the discharge probability is based on the excitable fiber model and taking into account the biological characteristics of the patient, such as the fiber threshold and the distribution of remaining intact fibers. RESULTS: The results of the emotion detection test on selective reconstructed sentences from the Persian emotional speech database (Persian ESD) indicated that the listeners have been able to detect the emotion by an average of 83.82% using the proposed stimulation algorithm while it was 75% and 48.03% for the zero-crossing and the continuous interleaved sampling (CIS), respectively. Furthermore, the number of pulses compared to the zero-crossing and the CIS has decreased by 76.3% and 75.4%, respectively. CONCLUSIONS: In this paper, a stimulation method was proposed for cochlear implants by considering the patient's biological parameters. It has been successful in transmitting speech emotion despite the reduction of stimulating pulses. This has some advantages such as reducing the interaction of current fields between electrodes during stimulation and reducing battery usage.

A new approach for speech synthesis in cochlear implant systems based on electrophysiological factors.

BACKGROUND: Speech synthesis models have been considered as viable tools for performance evaluation of cochlear stimulation algorithms, due to the difficulties of clinical tests. OBJECTIVE: The present study has developed a tool that can be used before any audio signal reconstruction algorithm, which shows more conformity with the electrophysiological parameters of the patient in evaluation of the cochlear implant stimulation algorithms. METHODS: In this method, excitable nerve fiber characteristics such as stimulation threshold and effective refractory period have been considered in the signal pre-reconstruction process. This algorithm subsumes the user's biological parameters (e.g., the manner of distribution of the remaining intact nerve fibers) as well as the stimulation signal parameters (e.g., stimulation rate, pulse width, amplitude of stimulation, the distance between stimulation electrode and fibers) in the signal pre-reconstruction. RESULTS: Effect of changes in these parameters can be observed by the number of excited fibers, which is directly related to the signal intensity and pitch frequency perceived by the user. The obtained results from simulations are in accordance with previous clinical findings. Also, the ability of the proposed tool can be seen by the correspondence between the results obtained from the proposed model and the amplitude growth functions of the cochlear implant users. CONCLUSIONS: This paper has introduced a tool for signal reconstruction from electrical stimulation so that a more comprehensive criterion for examination of the stimulating algorithms in cochlear implant can be achieved.

Effects of an Auditory Lateralization Training in Children Suspected to Central Auditory Processing Disorder.

BACKGROUND AND OBJECTIVES: Central auditory processing disorder [(C)APD] refers to a deficit in auditory stimuli processing in nervous system that is not due to higher-order language or cognitive factors. One of the problems in children with (C)APD is spatial difficulties which have been overlooked despite their significance. Localization is an auditory ability to detect sound sources in space and can help to differentiate between the desired speech from other simultaneous sound sources. Aim of this research was investigating effects of an auditory lateralization training on speech perception in presence of noise/competing signals in children suspected to (C)APD. SUBJECTS AND METHODS: In this analytical interventional study, 60 children suspected to (C)APD were selected based on multiple auditory processing assessment subtests. They were randomly divided into two groups: control (mean age 9.07) and training groups (mean age 9.00). Training program consisted of detection and pointing to sound sources delivered with interaural time differences under headphones for 12 formal sessions (6 weeks). Spatial word recognition score (WRS) and monaural selective auditory attention test (mSAAT) were used to follow the auditory lateralization training effects. RESULTS: This study showed that in the training group, mSAAT score and spatial WRS in noise (p value≤0.001) improved significantly after the auditory lateralization training. CONCLUSIONS: We used auditory lateralization training for 6 weeks and showed that auditory lateralization can improve speech understanding in noise significantly. The generalization of this results needs further researches.

Relation between Working Memory Capacity and Auditory Stream Segregation in Children with Auditory Processing Disorder.

BACKGROUND: This study assessed the relationship between working memory capacity and auditory stream segregation by using the concurrent minimum audible angle in children with a diagnosed auditory processing disorder (APD). METHODS: The participants in this cross-sectional, comparative study were 20 typically developing children and 15 children with a diagnosed APD (age, 9-11 years) according to the subtests of multiple-processing auditory assessment. Auditory stream segregation was investigated using the concurrent minimum audible angle. Working memory capacity was evaluated using the non-word repetition and forward and backward digit span tasks. Nonparametric statistics were utilized to compare the between-group differences. The Pearson correlation was employed to measure the degree of association between working memory capacity and the localization tests between the 2 groups. RESULTS: The group with APD had significantly lower scores than did the typically developing subjects in auditory stream segregation and working memory capacity. There were significant negative correlations between working memory capacity and the concurrent minimum audible angle in the most frontal reference location (0° azimuth) and lower negative correlations in the most lateral reference location (60° azimuth) in the children with APD. CONCLUSION: The study revealed a relationship between working memory capacity and auditory stream segregation in children with APD. The research suggests that lower working memory capacity in children with APD may be the possible cause of the inability to segregate and group incoming information.

Nonlinear feature extraction for objective classification of complex auditory brainstem responses to diotic perceptually critical consonant-vowel syllables.

OBJECTIVE: To examine if nonlinear feature extraction method yields appropriate results in complex brainstem response classification of three different consonant vowels diotically presented in normal Persian speaking adults. METHODS: Speech-evoked auditory brainstem responses were obtained in 27 normal hearing young adults by using G.tec EEG recording system. 170ms synthetic consonant-vowel stimuli /ba/, /da/, /ga/ were presented binaurally and the recurrence quantification analysis was performed on the responses. The recurrence time of second type was proposed as a suitable feature. ANOVA was also used for testing the significance of extracted feature. Post-comparison statistical method was used for showing which means are significantly different from each other. RESULTS: Dimension embedding and state space reconstruction were helpful for visualizing nonlinearity in auditory system. The proposed feature was successful in the objective classification of responses in window time 20.1-35.3ms, which belonged to formant transition period of stimuli. Also the p value behavior of recurrence time of second type feature as a discriminant feature was close to the nature of the response that includes transient and sustained parts. On the other hand, the /ba/ and /ga/ classification period was wider than the others. CONCLUSION: The extracted feature shown in this paper is helpful for the objective of distinguishing individuals with auditory processing disorders in the structurally similar voices. On the other hand, differing nonlinear feature is meaningful in a special region of response, equal to formant transition period, and this feature is related to the state space changes of brainstem response. It can be assumed that more information is within this region of signal and it is a sign of processing role of brainstem. The state changes of system are dependent on input stimuli, so the existence of top down feedback from cortex to brainstem forces the system to act differently.

Waveform efficiency analysis of auditory nerve fiber stimulation for cochlear implants.

Evaluation of the electrical stimulation efficiency of various stimulating waveforms is an important issue for efficient neural stimulator design. Concerning the implantable micro devices design, it is also necessary to consider the feasibility of hardware implementation of the desired waveforms. In this paper, the charge, power and energy efficiency of four waveforms (i.e. square, rising ramp, triangular and rising ramp-decaying exponential) in various durations have been simulated and evaluated based on the computational model of the auditory nerve fibers. Moreover, for a fair comparison of their feasibility, a fully integrated current generator circuit has been developed so that the desired stimulating waveforms can be generated. The simulation results show that stimulation with the square waveforms is a proper choice in short and intermediate durations while the rising ramp-decaying exponential or triangular waveforms can be employed for long durations.

Central auditory processing during chronic tinnitus as indexed by topographical maps of the mismatch negativity obtained with the multi-feature paradigm.

This study aimed to compare the neural correlates of acoustic stimulus representation in the auditory sensory memory on an automatic basis between tinnitus subjects and normal hearing (NH) controls, using topographical maps of the MMNs obtained with the multi-feature paradigm. A new and faster paradigm was adopted to look for differences between 2 groups of subjects. Twenty-eight subjects with chronic subjective idiopathic tinnitus and 33 matched healthy controls were included in the study. Brain electrical activity mapping of multi-feature MMN paradigm was recorded from 32 surface scalp electrodes. Three MMN parameters for five deviants consisting frequency, intensity, duration, location and silent gap were compared between the two groups. The MMN amplitude, latency and area under the curve over a region of interest comprising: F3, F4, Fz, FC3, FC4, FCz, and Cz were computed to provide better signal to noise ratio. These three measures could differentiate the cognitive processing disturbances in tinnitus sufferers. The MMN topographic maps revealed significant differences in amplitude and area under the curve for frequency, duration and silent gap deviants in tinnitus subjects compared to NH controls. The current study provides electrophysiological evidence supporting the theory that the pre-attentive and automatic central auditory processing is impaired in individuals with chronic tinnitus. Considering the advantages offered by the MMN paradigm used here, these data might be a useful reference point for the assessment of sensory memory in tinnitus patients and it can be applied with reliability and success in treatment monitoring.

A comparison of auditory perception in hearing-impaired and normal-hearing listeners: an auditory scene analysis study.

BACKGROUND: Auditory scene analysis (ASA) is the process by which the auditory system separates individual sounds in natural-world situations. ASA is a key function of auditory system, and contributes to speech discrimination in noisy backgrounds. It is known that sensorineural hearing loss (SNHL) detrimentally affects auditory function in complex environments, but relatively few studies have focused on the influence of SNHL on higher level processes which are likely involved in auditory perception in different situations. OBJECTIVES: The purpose of the current study was to compare the auditory system ability of normally hearing and SNHL subjects using the ASA examination. MATERIALS AND METHODS: A total of 40 right-handed adults (age range: 18 - 45 years) participated in this study. The listeners were divided equally into control and mild to moderate SNHL groups. ASA ability was measured using an ABA-ABA sequence. The frequency of the "A" was kept constant at 500, 1000, 2000 or 4000 Hz, while the frequency of the "B" was set at 3 to 80 percent above the" A" tone. For ASA threshold detection, the frequency of the B stimulus was decreased until listeners reported that they could no longer hear two separate sounds. RESULTS: The ASA performance was significantly better for controls than the SNHL group; these differences were more obvious at higher frequencies. We found no significant differences between ASA ability as a function of tone durations in both groups. CONCLUSIONS: The present study indicated that SNHL may cause a reduction in perceptual separation of the incoming acoustic information to form accurate representations of our acoustic world.