Smartphone-based single-channel speech enhancement application for hearing aids.

This work presents a single-channel speech enhancement (SE) framework based on the super-Gaussian extension of the joint maximum a posteriori (SGJMAP) estimation rule. The developed SE algorithm is an open-source research smartphone-based application for hearing improvement studies. In this algorithm, the SGJMAP-based estimation for noisy speech mixture is smoothed along the frequency axis by a Mel filter-bank, resulting in a Mel-warped frequency-domain SGJMAP estimation. The impulse response of this Mel-warped estimation is obtained by applying a Mel-warped inverse discrete cosine transform (Mel-IDCT). This helps in filtering out the background noise and enhancing the speech signal. The proposed application is implemented on an iPhone (Apple, Cupertino, CA) to operate in real time and tested with normal-hearing (NH) and hearing-impaired (HI) listeners with different types of hearing aids through wireless connectivity. The objective speech quality and intelligibility test results are used to compare the performance of the proposed algorithm to existing conventional single-channel SE methods. Additionally, test results from NH and HI listeners show substantial improvement in speech recognition with the developed method in simulated real-world noisy conditions at different signal-to-noise ratio levels.

Behavioral Validation of the Smartphone for Remote Microphone Technology.

As part of a National Institutes of Health-National Institute on Deafness and Other communication Disorders (NIH-NIDCD)-supported project to develop open-source research and smartphone-based apps for enhancing speech recognition in noise, an app called Smartphone Hearing Aid Research Project Version 2 (SHARP-2) was tested with persons with normal and impaired hearing when using three sets of hearing aids (HAs) with wireless connectivity to an iPhone. Participants were asked to type sentences presented from a speaker in front of them while hearing noise from behind in two conditions, HA alone and HA + SHARP-2 app running on the iPhone. The signal was presented at a constant level of 65 dBA and the signal-to-noise ratio varied from -10 to +10, so that the task was difficult when listening through the bilateral HAs alone. This was important to allow for improvement to be measured when the HAs were connected to the SHARP-2 app on the smartphone. Benefit was achieved for most listeners with all three manufacturer HAs with the greatest improvements recorded for persons with normal (33.56%) and impaired hearing (22.21%) when using the SHARP-2 app with one manufacturer's made-for-all phones HAs. These results support the continued development of smartphone-based apps as an economical solution for enhancing speech recognition in noise for both persons with normal and impaired hearing.

Real-time dual-channel speech enhancement by VAD assisted MVDR beamformer for hearing aid applications using smartphone.

In this paper, a dual-channel speech enhancement (SE) method is proposed. The proposed method is a combination of minimum variance distortionless response (MVDR) beamformer and a super-Gaussian joint maximum a posteriori (SGJMAP) based SE gain function. The proposed SE method runs on a smartphone in real-time, providing a portable device for hearing aid (HA) applications. Spectral Flux based voice activity detector (VAD) is used to improve the accuracy of the beamformer output. The efficiency of the proposed SE method is evaluated using speech quality and intelligibility measures and compared with that of other SE techniques. The objective and subjective test results show the capability of the proposed SE method in three different noisy conditions at low signal to noise ratios (SNRs) of -5, 0, and +5 dB.

Automated machine learning based speech classification for hearing aid applications and its real-time implementation on smartphone.

Deep neural networks (DNNs) have been useful in solving benchmark problems in various domains including audio. DNNs have been used to improve several speech processing algorithms that improve speech perception for hearing impaired listeners. To make use of DNNs to their full potential and to configure models easily, automated machine learning (AutoML) systems are developed, focusing on model optimization. As an application of AutoML to audio and hearing aids, this work presents an AutoML based voice activity detector (VAD) that is implemented on a smartphone as a real-time application. The developed VAD can be used to elevate the performance of speech processing applications like speech enhancement that are widely used in hearing aid devices. The classification model generated by AutoML is computationally fast and has minimal processing delay, which enables an efficient, real-time operation on a smartphone. The steps involved in real-time implementation are discussed in detail. The key contribution of this work include the utilization of AutoML platform for hearing aid applications and the realization of AutoML model on smartphone. The experimental analysis and results demonstrate the significance and importance of using the AutoML for the current approach. The evaluations also show improvements over the state of art techniques and reflect the practical usability of the developed smartphone app in different noisy environments.

Design and Integration of Alert Signal Detector and Separator for Hearing Aid Applications.

Alert signals like sirens and home alarms are important as they warn people of precarious situations. This work presents the detection and separation of these acoustically important alert signals, not to be attenuated as noise, to assist the hearing impaired listeners. The proposed method is based on convolutional neural network (CNN) and convolutional-recurrent neural network (CRNN). The developed method consists of two blocks, the detector block, and the separator block. The entire setup is integrated with speech enhancement (SE) algorithms, and before the compression stage, used in a hearing aid device (HAD) signal processing pipeline. The detector recognizes the presence of alert signal in various noisy environments. The separator block separates the alert signal from the mixture of noisy signals before passing it through SE to ensure minimal or no attenuation of the alert signal. It is implemented on a smartphone as an application that seamlessly works with HADs in real-time. This smartphone assistive setup allows the hearing aid users to know the presence of the alert sounds even when these are out of sight. The algorithm is computationally efficient with a low processing delay. The key contribution of this paper includes the development and integration of alert signal separator block with SE and the realization of the entire setup on a smartphone in real-time. The proposed method is compared with several state-of-the-art techniques through objective measures in various noisy conditions. The experimental analysis demonstrates the effectiveness and practical usefulness of the developed setup in real-world noisy scenarios.

Efficient two-microphone speech enhancement using basic recurrent neural network cell for hearing and hearing aids.

This work presents a two-microphone speech enhancement (SE) framework based on basic recurrent neural network (RNN) cell. The proposed method operates in real-time, improving the speech quality and intelligibility in noisy environments. The RNN model trained using a simple feature set-real and imaginary parts of the short-time Fourier transform (STFT) are computationally efficient with a minimal input-output processing delay. The proposed algorithm can be used in any stand-alone platform such as a smartphone using its two inbuilt microphones. The detailed operation of the real-time implementation on the smartphone is presented. The developed application works as an assistive tool for hearing aid devices (HADs). Speech quality and intelligibility test results are used to compare the proposed algorithm to existing conventional and neural network-based SE methods. Subjective and objective scores show the superior performance of the developed method over several conventional methods in different noise conditions and low signal to noise ratios (SNRs).

High-resolution light-activated electrochemistry on amorphous silicon-based photoelectrodes.

Light-activated electrochemistry (LAE) consists of employing a focused light beam to illuminate a semiconducting area and make it electrochemically active. Here, we show how to reduce the electrochemical spatial resolution to submicron by exploiting the short lateral diffusion of charge carriers in amorphous silicon to improve the resolution of LAE by 60 times.

Functionalized Silicon Electrodes in Electrochemistry.

Avoiding the growth of SiOx has been an enduring task for the use of silicon as an electrode material in dynamic electrochemistry. This is because electrochemical assays become unstable when the SiOx levels change during measurements. Moreover, the silicon electrode can be completely passivated for electron transfer if a thick layer of insulating SiOx grows on the surface. As such, the field of silicon electrochemistry was mainly developed by electron-transfer studies in nonaqueous electrolytes and by applications employing SiOx-passivated silicon-electrodes where no DC currents are required to cross the electrode/electrolyte interface. A solution to this challenge began by functionalizing Si-H electrodes with monolayers based on Si-O-Si linkages. These monolayers have proven very efficient to avoid SiOx formation but are not stable for a long-term operation in aqueous electrolytes due to hydrolysis. It was only with the development of self-assembled monolayers based on Si-C linkages that a reliable protection against SiOx formation was achieved, particularly with monolayers based on α,ω-dialkynes. This review discusses in detail how this surface chemistry achieves such protection, the electron-transfer behavior of these monolayer-modified silicon surfaces, and the new opportunities for electrochemical applications in aqueous solution.

Spatially localized electrodeposition of multiple metals via light-activated electrochemistry for surface enhanced Raman spectroscopy applications.

Light can be used to address electrochemical reactions on a monolithic semiconducting electrode with spatial and temporal resolution. Herein, such principle was used for the electrodeposition of Au, Ag and Cu nanoparticles on a unique silicon-based electrode. The parallel nature of the process granted manufacturing speed and platforms were applied to discriminate molecules via multi-wavelength and multivariate Raman analysis.

Real-time single-channel deep neural network-based speech enhancement on edge devices.

In this paper, we present a deep neural network architecture comprising of both convolutional neural network (CNN) and recurrent neural network (RNN) layers for real-time single-channel speech enhancement (SE). The proposed neural network model focuses on enhancing the noisy speech magnitude spectrum on a frame-by-frame process. The developed model is implemented on the smartphone (edge device), to demonstrate the real-time usability of the proposed method. Perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) test results are used to compare the proposed algorithm to previously published conventional and deep learning-based SE methods. Subjective ratings show the performance improvement of the proposed model over the other baseline SE methods.

Universal method for direct bioconjugation of electrode surfaces by fast enzymatic polymerization.

We report HRP-catalyzed polymerization of Tannic acid (TA) and application of the poly (Tannic acid) (p(TA)) as a versatile platform for covalent immobilization of biomolecules on various electrode surfaces based on electrochemical oxidation of the p(TA) and subsequent oxidative coupling reactions with the biomolecules. We also used this method for capturing cancer cells through a linker molecule, folic acid (FA). Furthermore, we have demonstrated that enhanced electrocatalytic activity of the p(TA)-modified surface could be used for simultaneous electrochemical determination of biologically important electroactive molecules such as ascorbic acid (AA), dopamine (DA), and uric acid (UA). This HRP-catalyzed polymerization of TA and p(TA)-mediated surface modification method can provide a simple and new framework to construct multifunctional platforms for covalent attachment of biomolecules and development of sensitive electrochemical sensing devices.

Smartphone-based portable wireless optical system for the detection of target analytes.

Rapid and accurate on-site wireless measurement of hazardous molecules or biomarkers is one of the biggest challenges in nanobiotechnology. A novel smartphone-based Portable and Wireless Optical System (PAWS) for rapid, quantitative, and on-site analysis of target analytes is described. As a proof-of-concept, we employed gold nanoparticles (GNP) and an enzyme, horse radish peroxidase (HRP), to generate colorimetric signals in response to two model target molecules, melamine and hydrogen peroxide, respectively. The colorimetric signal produced by the presence of the target molecules is converted to an electrical signal by the inbuilt electronic circuit of the device. The converted electrical signal is then measured wirelessly via multimeter in the smartphone which processes the data and displays the results, including the concentration of analytes and its significance. This handheld device has great potential as a programmable and miniaturized platform to achieve rapid and on-site detection of various analytes in a point-of-care testing (POCT) manner.