Utilizing Deep Learning on Limited Mobile Speech Recordings for Detection of Obstructive Pulmonary Disease.

Passive assessment of obstructive pulmonary disease has gained substantial interest over the past few years in the mobile and wearable computing communities. One of the promising approaches is speech-based pulmonary assessment wherein spontaneous or scripted speech is used to evaluate an individual's pulmonary condition. Recent approaches in this regard heavily rely on accurate speech activity segmentation and specific, hand-crafted features. In this paper, we present an end-to-end deep learning approach for detecting obstructive pulmonary disease. We leveraged transfer learning using a network pre-trained for a different audio-based task, and employed our own additional shallow network on top as a binary classifier to indicate if a given speech recording belongs to an asthma or COPD patient. The additional network was a fully connected neural net with 2 hidden layers, and this was evaluated on two real-world datasets. We demonstrated that the system can identify subjects with obtructive pulmonary disease using their speech with 88.3 % precision, 88.8 % recall and 88.3% F-1 score using 10-fold cross-validation. The system showed improved performance in identifying the most severely affected subgroup of patients in the dataset, with an average 93.6 % accuracy.

Atrial Fibrillation Detection and Atrial Fibrillation Burden Estimation via Wearables.

Atrial Fibrillation (AF) is an important cardiac rhythm disorder, which if left untreated can lead to serious complications such as a stroke. AF can remain asymptomatic, and it can progressively worsen over time; it is thus a disorder that would benefit from detection and continuous monitoring with a wearable sensor. We develop an AF detection algorithm, deploy it on a smartwatch, and prospectively and comprehensively validate its performance on a real-world population that included patients diagnosed with AF. The algorithm showed a sensitivity of 87.8% and a specificity of 97.4% over every 5-minute segment of PPG evaluated. Furthermore, we introduce novel algorithm blocks and system designs to increase the time of coverage and monitor for AF even during periods of motion noise and other artifacts that would be encountered in daily-living scenarios. An average of 67.8% of the entire duration the patients wore the smartwatch produced a valid decision. Finally, we present the ability of our algorithm to function throughout the day and estimate the AF burden, a first-of-this-kind measure using a wearable sensor, showing 98% correlation with the ground truth and an average error of 6.2%.

Detecting Physiological Responses Using Multimodal Earbud Sensors.

Continuous stress exposure negatively impacts mental and physical well-being. Physiological arousal due to stress affects heartbeat frequency, changes breathing pattern and peripheral temperature, among several other bodily responses. Traditionally stress detection is performed by collecting signals such as electrocardiogram (ECG), respiration, and skin conductance response using uncomfortable sensors such as a chestband. In this study, we use earbuds that passively measure photoplethysmography (PPG), core body temperature, and inertial measurements. We have conducted a lab study exposing 18 participants to an evaluated speech task and additional tasks aimed at increasing stress or promoting relaxation. We simultaneously collected PPG, ECG, impedance cardiography (ICG), and blood pressure using laboratory grade equipment as reference measurements. We show that the earbud PPG sensor can reliably capture heart rate and heart rate variability. We further show that earbud signals can be used to classify the physiological responses associated with stress with 91.30% recall, 80.52% precision, and 85.12% F1-score using a random forest classifier with leave-one-subject-out cross-validation. The accuracy can further be improved through multi-modal sensing. These findings demonstrate the feasibility of using earbuds for passively monitoring users' physiological responses.

SpeechSpiro: Lung Function Assessment from Speech Pattern as an Alternative to Spirometry for Mobile Health Tracking.

Respiratory illnesses are common in the United States and globally; people deal with these illnesses in various forms, such as asthma, chronic obstructive pulmonary diseases, or infectious respiratory diseases (e.g., coronavirus). The lung function of subjects affected by these illnesses degrades due to infection or inflammation in their respiratory airways. Typically, lung function is assessed using in-clinic medical equipment, and quite recently, via portable spirometry devices. Research has shown that the obstruction and restriction in the respiratory airways affect individuals' voice characteristics. Hence, audio features could play a role in predicting the lung function and severity of the obstruction. In this paper, we go beyond well-known voice audio features and create a hybrid deep learning model using CNN-LSTM to discover spatiotemporal patterns in speech and predict the lung function parameters with accuracy comparable to conventional devices. We validate the performance and generalizability of our method using the data collected from 201 subjects enrolled in two studies internally and in collaboration with a pulmonary hospital. SpeechSpiro measures lung function parameters (e.g., forced vital capacity) with a mean normalized RMSE of 12% and R2 score of up to 76% using 60-second phone audio recordings of individuals reading a passage.Clinical relevance - Speech-based spirometry has the potential to eliminate the need for an additional device to carry out the lung function assessment outside clinical settings; hence, it can enable continuous and mobile track of the individual's condition, healthy or with a respiratory illness, using a smartphone.

Validation of an algorithm for continuous monitoring of atrial fibrillation using a consumer smartwatch.

BACKGROUND: Consumer devices with broad reach may be useful in screening for atrial fibrillation (AF) in appropriate populations. However, currently no consumer devices are capable of continuous monitoring for AF. OBJECTIVE: The purpose of this study was to estimate the sensitivity and specificity of a smartwatch algorithm for continuous detection of AF from sinus rhythm in a free-living setting. METHODS: We studied a commercially available smartwatch with photoplethysmography (W-PPG) and electrocardiogram (W-ECG) capabilities. We validated a novel W-PPG algorithm combined with a W-ECG algorithm in a free-living setting, and compared the results to those of a 28-day continuous ECG patch (P-ECG). RESULTS: A total of 204 participants completed the free-living study, recording 81,944 hours with both P-ECG and smartwatch measurements. We found sensitivity of 87.8% (95% confidence interval [CI] 83.6%-91.0%) and specificity of 97.4% (95% CI 97.1%-97.7%) for the W-PPG algorithm (every 5-minute classification); sensitivity of 98.9% (95% CI 98.1%-99.4%) and specificity of 99.3% (95% CI 99.1%-99.5%) for the W-ECG algorithm; and sensitivity of 96.9% (95% CI 93.7%-98.5%) and specificity of 99.3% (95% CI 98.4%-99.7%) for W-PPG triggered W-ECG with a single W-ECG required for confirmation of AF. We found a very strong correlation of W-PPG in quantifying AF burden compared to P-ECG (r = 0.98). CONCLUSION: Our findings demonstrate that a novel algorithm using a commercially available smartwatch can continuously detect AF with excellent performance and that confirmation with W-ECG further enhances specificity. In addition, our W-PPG algorithm can estimate AF burden. Further research is needed to determine whether this algorithm is useful in screening for AF in select at-risk patients.

A Comprehensive Approach for Classification of the Cough Type.

Identifying the presence of sputum in the lung is essential in detection of diseases such as lung infection, pneumonia and cancer. Cough type classification (dry/wet) is an effective way of examining presence of lung sputum. This is traditionally done through physical exam in a clinical visit which is subjective and inaccurate. This work proposes an objective approach relying on the acoustic features of the cough sound. A total number of 5971 coughs (5242 dry and 729 wet) were collected from 131 subjects using Smartphone. The data was reviewed and annotated by a novel multi-layer labeling platform. The annotation kappa inter-rater agreement score is measured to be 0.81 and 0.37 for 1st and 2nd layer respectively. Sensitivity and specificity values of 88% and 86% are measured for classification between wet and dry coughs (highest across the literature).

CoughGAN: Generating Synthetic Coughs that Improve Respiratory Disease Classification.

Despite the prevalence of respiratory diseases, their diagnosis by clinicians is challenging. Accurately assessing airway sounds requires extensive clinical training and equipment that may not be easily available. Current methods that automate this diagnosis are hindered by their use of features that require pulmonary function tests. We leverage the audio characteristics of coughs to create classifiers that can distinguish common respiratory diseases in adults. Moreover, we build on recent advances in generative adversarial networks to augment our dataset with cleverly engineered synthetic cough samples for each class of major respiratory disease, to balance and increase our dataset size. We experimented on cough samples collected with a smartphone from 45 subjects in a clinic. Our CoughGAN-improved Support Vector Machine and Random Forest models show up to 76% test accuracy and 83% F1 score in classifying subjects' conditions between healthy and three major respiratory diseases. Adding our synthetic coughs improves the performance we can obtain from a relatively small unbalanced healthcare dataset by boosting the accuracy over 30%. Our data augmentation reduces overfitting and discourages the prediction of a single, dominant class. These results highlight the feasibility of automatic, cough-based respiratory disease diagnosis using smartphones or wearables in the wild.

CoughMatch - Subject Verification Using Cough for Personal Passive Health Monitoring.

Automatic cough detection using audio has advanced passive health monitoring on devices such as smart phones and wearables; it enables capturing longitudinal health data by eliminating user interaction and effort. One major issue arises when coughs from surrounding people are also detected; capturing false coughs leads to significant false alarms, excessive cough frequency, and thereby misdiagnosis of user condition. To address this limitation, in this paper, a method is proposed that creates a personal cough model of the primary subject using limited number of cough samples; the model is used by the automatic cough detection to verify whether the identified coughs match the personal pattern and belong to the primary subject. A Gaussian mixture model is trained using audio features from cough to implement the subject verification method; novel cough embeddings are learned using neural networks and integrated into the model to further improve the prediction accuracy. We analyze the performance of the method using our cough dataset collected by a smart phone in a clinical study. Population in the dataset involves subjects categorized of healthy or patients with COPD or Asthma, with the purpose of covering a wider range of pulmonary conditions. Cross-subject validation on a diverse dataset shows that the method achieves an average error rate of less than 10%, using a personal cough model generated by only 5 coughs from the primary subject.

Adapting to Noise in Speech Obfuscation by Audio Profiling Using Generative Models for Passive Health Monitoring.

Passive health monitoring has been introduced as a solution for continuous diagnosis and tracking of subjects' condition with minimal effort. This is partially achieved by the technology of passive audio recording although it poses major audio privacy issues for subjects. Existing methods are limited to controlled recording environments and their prediction is significantly influenced by background noises. Meanwhile, they are too compute-intensive to be continuously running on smart phones. In this paper, we implement an efficient and robust audio privacy preserving method that profiles the background audio to focus only on audio activities detected during recording for performance improvement, and to adapt to the noise for more accurate speech segmentation. We analyze the performance of our method using audio data collected by a smart watch in lab noisy settings. Our obfuscation results show a low false positive rate of 20% with a 92% true positive rate by adapting to the recording noise level. We also reduced model memory footprint and execution time of the method on a smart phone by 75% and 62% to enable continuous speech obfuscation.

Toward Early Severity Assessment of Obstructive Lung Disease Using Multi-Modal Wearable Sensor Data Fusion During Walking.

Early detection of chronic diseases helps to minimize the disease impact on patient's health and reduce the economic burden. Continuous monitoring of such diseases helps in the evaluation of rehabilitation program effectiveness as well as in the detection of exacerbation. The use of everyday wearables i.e. chest band, smartwatch and smart band equipped with good quality sensor and light weight machine learning algorithm for the early detection of diseases is very promising and holds tremendous potential as they are widely used. In this study, we have investigated the use of acceleration, electrocardiogram, and respiration sensor data from a chest band for the evaluation of obstructive lung disease severity. Recursive feature elimination technique has been used to identity top 15 features from a set of 62 features including gait characteristics, respiration pattern and heart rate variability. A precision of 0.93, recall of 0.91 and F-1 score of 0.92 have been achieved with a support vector machine for the classification of severe patients from the non-severe patients in a data set of 60 patients. In addition, the selected features showed significant correlation with the percentage of predicted FEV1.Clinical Relevance- The study result indicates that wearable sensor data collected during natural walk can be used in the early evaluation of pulmonary patients thus enabling them to seek medical attention and avoid exacerbation. In addition, it may serve as a complementary tool for pulmonary patient evaluation during a 6-minute walk test.

Estimation of the Lung Function Using Acoustic Features of the Voluntary Cough.

Spirometry test, a measure of the patient's lung function, is the gold standard for diagnosis and monitoring of chronic pulmonary diseases. Spirometry is currently being done in hospital settings by having the patients blow the air out of their lungs forcefully and into the spirometer's tubes under the supervision and constant guidance of clinicians. This test is expensive, cumbersome and not easily applicable to every-day monitoring of these patients. The lung mechanism when performing a cough is very similar to when spirometry test is done. That includes a big inhalation, air compression and forceful exhalation. Therefore, it is reasonable to assume that obstruction of lung airways should have a similar effect on both cough features and spirometry measures. This paper explores the estimation of lung obstruction using cough acoustic features. A total number of 3695 coughs were collected from patients from 4 different conditions and 4 different severity categories along with their lung function measures in a clinical setting using a smartphone's microphone and a hospital-grade spirometry lab. After feature-set optimization and model hyperparameter tuning, the lung obstruction was estimated with MAE (Mean Absolute Error) of 8% for COPD and 9% for asthma populations. In addition to lung obstruction estimation, we were able to classify patients' disease state with 91% accuracy and patients' severity within each disease state with 95% accuracy.Clinical Relevance- This enables effort-independent estimation of lung function spirometry parameters which could potentially lead to passive monitoring of pulmonary patients.

Particle Filtering and Sensor Fusion for Robust Heart Rate Monitoring Using Wearable Sensors.

This paper describes a novel methodology leveraging particle filters for the application of robust heart rate monitoring in the presence of motion artifacts. Motion is a key source of noise that confounds traditional heart rate estimation algorithms for wearable sensors due to the introduction of spurious artifacts in the signals. In contrast to previous particle filtering approaches, we formulate the heart rate itself as the only state to be estimated, and do not rely on multiple specific signal features. Instead, we design observation mechanisms to leverage the known steady, consistent nature of heart rate variations to meet the objective of continuous monitoring of heart rate using wearable sensors. Furthermore, this independence from specific signal features also allows us to fuse information from multiple sensors and signal modalities to further improve estimation accuracy. The signal processing methods described in this work were tested on real motion artifact affected electrocardiogram and photoplethysmogram data with concurrent accelerometer readings. Results show promising average error rates less than 2 beats/min for data collected during intense running activities. Furthermore, a comparison with contemporary signal processing techniques for the same objective shows how the proposed implementation is also computationally more efficient for comparable performance.

Exploration and validation of alternate sensing methods for wearable continuous pulse transit time measurement using optical and bioimpedance modalities.

In this work we explore the viability of a multimodal sensing device that can be integrated in a wearable form factor for daily, non-invasive ambulatory blood pressure (BP) monitoring. A common approach in previous research has been to rely on measuring the pulse transit time (PTT), which has been shown to be correlated with the BP. In this work, we look into the feasibility of measuring PTT using sensors separated by a small distance on one arm so that any eventual realization of the system is convenient to wear and use over long periods of time. Moreover, we investigate the combined use of two different modalities for cardiovascular measurement: the optical photoplethysmogram (PPG) as well as the bio-potential based impedance (Bio-Z) measurement. These two modalities have been previously only studied on their own or in conjunction with the electrocardiogram (ECG) for the purpose of estimating PTT. We measure the PTT from the wrist to the finger using Bio-Z and PPG sensors, and compare it to the conventional PTT measured from the ECG to PPG at the finger, in order to prove that it can be an effective replacement for existing PTT measurement strategies. Moreover, successful measurement of PTT with two different modalities of sensors at close proximity will allow designs with multiple heterogeneous sensors on a more versatile wearable sensing platform that is optimized for power and is more robust to environmental or skin contact changes. This will enable the next generation of smart watches that capture PTT and BP. Experiments were conducted in vivo with simultaneous ECG, Bio-Z and PPG sensors, and results indicate that the PTT calculated from the Bio-Z and PPG sensors placed at a close distance correlates well with the more established PTT measurement using the ECG in conjunction with PPG, with correlation coefficient as high as 0.92.

Automatic Identification of Artifact-Related Independent Components for Artifact Removal in EEG Recordings.

Electroencephalography (EEG) is the recording of electrical activity produced by the firing of neurons within the brain. These activities can be decoded by signal processing techniques. However, EEG recordings are always contaminated with artifacts which hinder the decoding process. Therefore, identifying and removing artifacts is an important step. Researchers often clean EEG recordings with assistance from independent component analysis (ICA), since it can decompose EEG recordings into a number of artifact-related and event-related potential (ERP)-related independent components. However, existing ICA-based artifact identification strategies mostly restrict themselves to a subset of artifacts, e.g., identifying eye movement artifacts only, and have not been shown to reliably identify artifacts caused by nonbiological origins like high-impedance electrodes. In this paper, we propose an automatic algorithm for the identification of general artifacts. The proposed algorithm consists of two parts: 1) an event-related feature-based clustering algorithm used to identify artifacts which have physiological origins; and 2) the electrode-scalp impedance information employed for identifying nonbiological artifacts. The results on EEG data collected from ten subjects show that our algorithm can effectively detect, separate, and remove both physiological and nonbiological artifacts. Qualitative evaluation of the reconstructed EEG signals demonstrates that our proposed method can effectively enhance the signal quality, especially the quality of ERPs, even for those that barely display ERPs in the raw EEG. The performance results also show that our proposed method can effectively identify artifacts and subsequently enhance the classification accuracies compared to four commonly used automatic artifact removal methods.

BioWatch: A Noninvasive Wrist-Based Blood Pressure Monitor That Incorporates Training Techniques for Posture and Subject Variability.

Noninvasive continuous blood pressure (BP) monitoring is not yet practically available for daily use. Challenges include making the system easily wearable, reducing noise level and improving accuracy. Variations in each person's physical characteristics, as well as the possibility of different postures, increase the complexity of continuous BP monitoring, especially outside the hospital. This study attempts to provide an easily wearable solution and proposes training to specific posture and individual for further improving accuracy. The wrist watch-based system we developed can measure electrocardiogram and photoplethysmogram. From these two signals, we measure pulse transit time through which we can obtain systolic and diastolic blood pressure through regression techniques. In this study, we investigate various functions to perform the training to obtain blood pressure. We validate measurements on different postures and subjects, and show the value of training the device to each posture and each subject. We observed that the average RMSE between the measured actual systolic BP and calculated systolic BP is between 7.83 to 9.37 mmHg across 11 subjects. The corresponding range of error for diastolic BP is 5.77 to 6.90 mmHg. The system can also automatically detect the arm position of the user using an accelerometer with an average accuracy of 98%, to make sure that the sensor is kept at the proper height. This system, called BioWatch, can potentially be a unified solution for heart rate, SPO2 and continuous BP monitoring.

Design Principles and Dynamic Front End Reconfiguration for Low Noise EEG Acquisition With Finger Based Dry Electrodes.

Dry electrodes are a convenient alternative to wet electrodes for electroencephalography (EEG) acquisition systems. Dry electrodes are subject to a higher amount of noise at the electrode scalp interface and these effects can be worsened due to non-ideal design or improper placement on the head. In this work, we investigate a popular dry electrode design based on a number of resistive 'finger' shaped contacts. We conduct experiments comparing designs with varying numbers of fingers using two impedance measurement methods and show that sparser arrangements of fingers are more robust to varying use cases and are more effective at penetrating through hair on the scalp. We then show that these impedance measurement metrics could be used to sort individual fingers within one electrode according to quality of electrical contact. We show that the signals from individual fingers can differ from each other significantly due to differing local effects of impedance and noise, and demonstrate through experimental results that dynamically selecting only a subset of fingers with good contact impedance can improve the overall signal-to-noise ratio of the EEG signal from that electrode.

A particle filter framework for the estimation of heart rate from ECG signals corrupted by motion artifacts.

In this work, we describe a methodology to probabilistically estimate the R-peak locations of an electrocardiogram (ECG) signal using a particle filter. This is useful for heart rate estimation, which is an important metric for medical diagnostics. Some scenarios require constant in-home monitoring using a wearable device. This poses a particularly challenging environment for heart rate detection, due to the susceptibility of ECG signals to motion artifacts. In this work, we show how the particle filter can effectively track the true R-peak locations amidst the motion artifacts, given appropriate heart rate and R-peak observation models. A particle filter based framework has several advantages due to its freedom from strict assumptions on signal and noise models, as well as its ability to simultaneously track multiple possible heart rate hypotheses. Moreover, the proposed framework is not exclusive to ECG signals and could easily be leveraged for tracking other physiological parameters. We describe the implementation of the particle filter and validate our approach on real ECG data affected by motion artifacts from the MIT-BIH noise stress test database. The average heart rate estimation error is about 5 beats per minute for signal streams contaminated with noisy segments with SNR as low as -6 dB.

Characterizing contact impedance, signal quality and robustness as a function of the cardinality and arrangement of fingers on dry contact EEG electrodes.

Continuous monitoring of patients' electroencephalography (EEG) outside of clinical settings will be valuable for detecting the onset of medical conditions such as epilepsy, as well as for enabling patients with physically disabling conditions like amyotrophic lateral sclerosis (ALS) to communicate using a brain-computer interface (BCI). This requires the development of a wearable dry-contact EEG system that takes into account not only the signal quality but also the robustness of the system for everyday use. To this end, we investigate whether certain designs of dry electrodes lend themselves to better characteristics overall with respect to these factors. Five different metallic finger-based dry electrodes were designed and scalp electrode impedance was used to compare them under varying capping conditions, followed by an evaluation of how well they captured steady state visually evoked potentials (SSVEP). Our findings indicate that configurations with a relatively low density of fingers can more effectively penetrate through hair on the scalp and are more robust to varying conditions. This was confirmed to be a statistically significant observation through a one-sided paired t-test that resulted in a p-value <; 0.004.

A novel stimulation for multi-class SSVEP-based brain-computer interface using patterns of time-varying frequencies.

Steady-state visual evoked potential (SSVEP) has become one of the most widely employed modalities in online brain computer interface (BCI) because of its high signal-to-noise ratio. However, due to the limitations of brain physiology and the refresh rate of the display devices, the available stimulation frequencies that evoke strong SSVEPs are generally limited for practical applications. In this paper, we introduce a novel stimulation method using patterns of time-varying frequencies that can increase the number of visual stimuli with a fixed number of stimulation frequencies for use in multi-class SSVEP-based BCI systems. We then propose a probabilistic framework and investigate three approaches to detect different patterns of time-varying frequencies. The results confirmed that our proposed stimulation is a promising method for multi-class SSVEP-based BCI tasks. Our pattern detection approaches improved the detection performance significantly by extracting higher quality discriminative information from the input signal.

BioWatch - a wrist watch based signal acquisition system for physiological signals including blood pressure.

A wrist watch based system, which can measure electrocardiogram (ECG) and photoplethysmogram (PPG), is presented in this work. By using both ECG and PPG we also measure pulse transit time (PTT), which studies show to correlate well with blood pressure (BP). The system is also capable of monitoring heart rate using either ECG or PPG and can monitor blood oxygenation by easily replacing the PPG sensors with a different set. In this work, we investigate methods to train a fitting function to convert a PTT measurement to its corresponding systolic BP. We also validate measurements on different postures and show the value of calibrating the device for each posture. This system, called BioWatch, can potentially facilitate continuous and ubiquitous monitoring of ECG, PPG, heart rate, blood oxygenation and BP.