The use of deep learning for smartphone-based human activity recognition.

The emerging field of digital phenotyping leverages the numerous sensors embedded in a smartphone to better understand its user's current psychological state and behavior, enabling improved health support systems for patients. As part of this work, a common task is to use the smartphone accelerometer to automatically recognize or classify the behavior of the user, known as human activity recognition (HAR). In this article, we present a deep learning method using the Resnet architecture to implement HAR using the popular UniMiB-SHAR public dataset, containing 11,771 measurement segments from 30 users ranging in age between 18 and 60 years. We present a unified deep learning approach based on a Resnet architecture that consistently exceeds the state-of-the-art accuracy and F1-score across all classification tasks and evaluation methods mentioned in the literature. The most notable increase we disclose regards the leave-one-subject-out evaluation, known as the most rigorous evaluation method, where we push the state-of-the-art accuracy from 78.24 to 80.09% and the F1-score from 78.40 to 79.36%. For such results, we resorted to deep learning techniques, such as hyper-parameter tuning, label smoothing, and dropout, which helped regularize the Resnet training and reduced overfitting. We discuss how our approach could easily be adapted to perform HAR in real-time and discuss future research directions.

Detection of heart rate using smartphone gyroscope data: a scoping review.

Heart rate (HR) is closely related to heart rhythm patterns, and its irregularity can imply serious health problems. Therefore, HR is used in the diagnosis of many health conditions. Traditionally, HR has been measured through an electrocardiograph (ECG), which is subject to several practical limitations when applied in everyday settings. In recent years, the emergence of smartphones and microelectromechanical systems has allowed innovative solutions for conveniently measuring HR, such as smartphone ECG, smartphone photoplethysmography (PPG), and seismocardiography (SCG). However, these measurements generally rely on external sensor hardware or are highly susceptible to inaccuracies due to the presence of significant levels of motion artifact. Data from gyrocardiography (GCG), however, while largely overlooked for this application, has the potential to overcome the limitations of other forms of measurements. For this scoping review, we performed a literature search on HR measurement using smartphone gyroscope data. In this review, from among the 114 articles that we identified, we include seven relevant articles from the last decade (December 2012 to January 2023) for further analysis of their respective methods for data collection, signal pre-processing, and HR estimation. The seven selected articles' sample sizes varied from 11 to 435 participants. Two articles used a sample size of less than 40, and three articles used a sample size of 300 or more. We provide elaborations about the algorithms used in the studies and discuss the advantages and disadvantages of these methods. Across the articles, we noticed an inconsistency in the algorithms used and a lack of established standardization for performance evaluation for HR estimation using smartphone GCG data. Among the seven articles included, five did not perform any performance evaluation, while the other two used different reference signals (HR and PPG respectively) and metrics for accuracy evaluation. We conclude the review with a discussion of challenges and future directions for the application of GCG technology.

Fall detection using accelerometer-based smartphones: Where do we go from here?

According to World Health Organization statistics, falls are the second leading cause of unintentional injury deaths worldwide. With older people being particularly vulnerable, detecting, and reporting falls have been the focus of numerous health technology studies. We screened 267 studies and selected 15 that detailed pervasive fall detection and alerting apps that used smartphone accelerometers. The fall datasets used for the analyses included between 4 and 38 participants and contained data from young and old subjects, with the recorded falls performed exclusively by young subjects. Threshold-based detection was implemented in six cases, while machine learning approaches were implemented in the other nine, including decision trees, k-nearest neighbors, boosting, and neural networks. All methods could ultimately achieve real-time detection, with reported sensitivities ranging from 60.4 to 99.3% and specificities from 74.6 to 100.0%. However, the studies had limitations in their experimental set-ups or considered a restricted scope of daily activities-not always representative of daily life-with which to define falls during the development of their algorithms. Finally, the studies omitted some aspects of data science methodology, such as proper test sets for results evaluation, putting into question whether reported results would correspond to real-world performance. The two primary outcomes of our review are: a ranking of selected articles based on bias risk and a set of 12 impactful and actionable recommendations for future work in fall detection.

Ethnic disparities in publicly-available pulse oximetry databases.

Inaccuracies have been reported in pulse oximetry measurements taken from people who identified as Black. Here, we identify substantial ethnic disparities in the population numbers within 12 pulse oximetry databases, which may affect the testing of new oximetry devices and impact patient outcomes.

Use of Convolutional Neural Nets and Transfer Learning for Prediction of Surgical Site Infection from Color Images.

One of the greatest concerns in post-operative care is the infection of the surgical wound. Such infections are a particular concern in global health and low-resource areas, where microbial antibiotic resistance is often common. In order to help address this problem, there is a great interest in developing simple tools for early detection of surgical wounds. Motivated by this need, we describe the development of two Convolutional Neural Net (CNN) models designed to detect an infection in a surgical wound using a color image taken from a mobile device. These models were developed using image data collected from a clinical study with 572 women in Rural Rwanda, who underwent Cesarean section surgery and had photos taken approximately 10 days after surgery. Infected wounds (N=62) were diagnosed by a trained doctor through a physical exam. In our model development, we observed a trade-off between AUC accuracy and sensitivity, and we chose to optimize for sensitivity, to match its use as a screening tool. Our naïve CNN model, with a limited number of convolutions and parameters, achieved median AUC = 0.655, true positive rate sensitivity = 0.75, specificity = 0.58, classification accuracy = 0.86. The second CNN model, developed with transfer learning using the Resnet50 architecture, produced a median AUC = 0.639 sensitivity = 0.92, specificity = 0.18, and classification accuracy 0.82. We discuss the specific training and optimization methods used to compensate for significant class imbalance and maximize sensitivity.

The Use of Mobile Thermal Imaging and Deep Learning for Prediction of Surgical Site Infection.

The ability to detect surgical site infections (SSI) is a critical need for healthcare worldwide, but is especially important in low-income countries, where there is limited access to health facilities and trained clinical staff. In this paper, we present a new method of predicting SSI using a thermal image collected with a smart phone. Machine learning algorithms were developed using images collected as part of a clinical study that included 530 women in rural Rwanda who underwent cesarean section surgery. Thermal images were collected approximately 10 days after surgery, in conjunction with an examination by a trained doctor to determine the status of the wound (infected or not). Of the 530 women, 30 were found to have infected wounds. The data were used to develop two Convolutional Neural Net (CNN) models, with special care taken to avoid overfitting and address the problem of class imbalance in binary classification. The first model, a 6-layer naïve CNN model, demonstrated a median accuracy of AUC=0.84 with sensitivity=71% and specificity=87%. The transfer learning CNN model demonstrated a median accuracy of AUC=0.90 with sensitivity =95% and specificity=84%. To our knowledge, this is the first successful demonstration of a machine learning algorithm to predict surgical infection using thermal images alone.Clinical Relevance- This work establishes a promising new method for automated detection of surgical site infection.

The use of multi-site photoplethysmography (PPG) as a screening tool for coronary arterial disease and atherosclerosis.

Objective. We present the design and validation of a non-invasive smart-phone based screening tool for atherosclerosis and coronary arterial disease (CAD), which is the leading cause of mortality worldwide.Approach. We designed a three-channel photoplethysmography (PPG) device that connects to a smart phone application for measuring pulse transit time (PTT) and pulse wave velocity (PWV) using PPG probes that are simultaneously clipped onto to the ear, index finger, and big toe, respectively. Validation was performed through a clinical study with 100 participants (age 20 to 77) at a research hospital in Nagpur, India. Study subjects were stratified by age and divided into three groups corresponding to the disease severity: CAD, hypertensive ('Pre-CAD'), and Healthy.Main results. PWV measurements derived from the Ear-Toe probe measurements yielded the best performance, with median PWV values increasing monotonically as a function of disease severity and age, as follows: 14.2 m s-1for the older-patient CAD group, 12.2 m s-1for the younger-patient CAD group, 11.6 m s-1for the older-patient Pre-CAD group, 10.2 m s-1for the younger-patient Pre-CAD group, 9.7 m s-1for the older healthy controls, and 8.4 m s-1for the younger healthy controls. Using just two simple features, the PTT and patient height, we demonstrate a machine learning prediction model for CAD with a median accuracy of 0.83 (AUC).Significance. This work demonstrates the ability to predict atherosclerosis and CAD using a single simple physiological measurement with a multi-site PPG tool that is electrically powered by a mobile phone and does not require any electrocardiogram reference. Furthermore, this method only requires a single anthropometric measurement, which is the patient's height.

Assessment of Hypertension Using Clinical Electrocardiogram Features: A First-Ever Review.

Hypertension affects an estimated 1.4 billion people and is a major cause of morbidity and mortality worldwide. Early diagnosis and intervention can potentially decrease cardiovascular events later in life. However, blood pressure (BP) measurements take time and require training for health care professionals. The measurements are also inconvenient for patients to access, numerous daily variables affect BP values, and only a few BP readings can be collected per session. This leads to an unmet need for an accurate, 24-h continuous, and portable BP measurement system. Electrocardiograms (ECGs) have been considered as an alternative way to measure BP and may meet this need. This review summarizes the literature published from January 1, 2010, to January 1, 2020, on the use of only ECG wave morphology to monitor BP or identify hypertension. From 35 articles analyzed (9 of those with no listed comorbidities and confounders), the P wave, QTc intervals and TpTe intervals may be promising for this purpose. Unfortunately, with the limited number of articles and the variety of participant populations, we are unable to make conclusions about the effectiveness of ECG-only BP monitoring. We provide 13 recommendations for future ECG-only BP monitoring studies and highlight the limited findings in pregnant and pediatric populations. With the advent of convenient and portable ECG signal recording in smart devices and wearables such as watches, understanding how to apply ECG-only findings to identify hypertension early is crucial to improving health outcomes worldwide.

The Performance of Deep Neural Networks in Differentiating Chest X-Rays of COVID-19 Patients From Other Bacterial and Viral Pneumonias.

Chest radiography is a critical tool in the early detection, management planning, and follow-up evaluation of COVID-19 pneumonia; however, in smaller clinics around the world, there is a shortage of radiologists to analyze large number of examinations especially performed during a pandemic. Limited availability of high-resolution computed tomography and real-time polymerase chain reaction in developing countries and regions of high patient turnover also emphasizes the importance of chest radiography as both a screening and diagnostic tool. In this paper, we compare the performance of 17 available deep learning algorithms to help identify imaging features of COVID19 pneumonia. We utilize an existing diagnostic technology (chest radiography) and preexisting neural networks (DarkNet-19) to detect imaging features of COVID-19 pneumonia. Our approach eliminates the extra time and resources needed to develop new technology and associated algorithms, thus aiding the front-line healthcare workers in the race against the COVID-19 pandemic. Our results show that DarkNet-19 is the optimal pre-trained neural network for the detection of radiographic features of COVID-19 pneumonia, scoring an overall accuracy of 94.28% over 5,854 X-ray images. We also present a custom visualization of the results that can be used to highlight important visual biomarkers of the disease and disease progression.

Addressing Fairness, Bias, and Appropriate Use of Artificial Intelligence and Machine Learning in Global Health.

In Low- and Middle- Income Countries (LMICs), machine learning (ML) and artificial intelligence (AI) offer attractive solutions to address the shortage of health care resources and improve the capacity of the local health care infrastructure. However, AI and ML should also be used cautiously, due to potential issues of fairness and algorithmic bias that may arise if not applied properly. Furthermore, populations in LMICs can be particularly vulnerable to bias and fairness in AI algorithms, due to a lack of technical capacity, existing social bias against minority groups, and a lack of legal protections. In order to address the need for better guidance within the context of global health, we describe three basic criteria (Appropriateness, Fairness, and Bias) that can be used to help evaluate the use of machine learning and AI systems: 1) APPROPRIATENESS is the process of deciding how the algorithm should be used in the local context, and properly matching the machine learning model to the target population; 2) BIAS is a systematic tendency in a model to favor one demographic group vs another, which can be mitigated but can lead to unfairness; and 3) FAIRNESS involves examining the impact on various demographic groups and choosing one of several mathematical definitions of group fairness that will adequately satisfy the desired set of legal, cultural, and ethical requirements. Finally, we illustrate how these principles can be applied using a case study of machine learning applied to the diagnosis and screening of pulmonary disease in Pune, India. We hope that these methods and principles can help guide researchers and organizations working in global health who are considering the use of machine learning and artificial intelligence.

Application of Machine Learning to Prediction of Surgical Site Infection.

Surgical site infections are an important health concern, particularly in low-resource areas, where there is poor access to clinical facilities or trained clinical staff. As an application of machine learning, we present results from a study conducted in rural Rwanda for the purpose of predicting infection in Cesarean section wounds, which is a leading cause of maternal mortality. Questionnaire and image data were collected from 572 mothers approximately 10 days after surgery at a district hospital. Of the 572 women, 61 surgical wounds were determined to be infected as determined by a physical exam conducted by trained doctors. Machine learning models, logistic regression and Support Vector Machines (SVM), were developed independently for the questionnaire data and the image data. For the questionnaire data, the best results were achieved by the Logistic regression model, with an AUC Accuracy = 96.50% (93.0%-99.3%), Sensitivity = 0.71 (0.33 - 0.92), and Specificity = 0.99 (0.98 - 1.00). The features with the greatest predictive value were the presence of malcolored drainage from the wound and the presence of an odorous discharge from the wound. Using the image data alone, the SVM model performed best, with an AUC Accuracy = 99.5% (99.2%-100%), Sensitivity = 0.99 (0.99 - 1.00), and Specificity = 0.99 (0.99 - 1.00). Combining both questionnaire data and image data, the SVM model achieved an AUC Accuracy = 99.9% (99.7%-100%), Sensitivity = 0.99 (0.99 -1.00), and Specificity = 0.99 (0.99 - 1.00). Results from this initial study are very encouraging and demonstrate that good objective prediction of surgical infection for women in rural Rwanda is feasible using machine learning, even when using image data alone.

Low-Cost Mobile Device for Screening of Atherosclerosis and Coronary Arterial Disease.

In the context of global health, telemedicine, and low-resource settings, we present a non-invasive smart-phone based device that can be used to screen for atherosclerosis,which is the leading factor for ischemic heart attacks and strokes. Using acustom Android mobile application, our device computes Pulse Wave Velocity(PWV) using the pulse signals from photo-plethysmographic (PPG) probes, which are simultaneously clipped onto the ear, index finger, and big toe of a human subject. Unlike other designs which require the use of an ECG reference, our mobile device uses only PPG signals and is entirely powered by the mobile phone via the USB port. Using the ear signal as a reference, we derived PWV values from two locations: the right index finger, and the right big toe.We present data from a recent clinical study with 78 participants (age 26 to 74) who were divided into three groups: Coronary Arterial Disease ("CAD"), hypertensive group ("PreCAD"), and Healthy controls. The CAD group was clinically diagnosed and confirmed with a CT-scan and calcium scoring. PWV values derived from the finger was found to have too much variance to be clinically useful. However, PWV values derived from the toe location showed significant differences between the groups, even after accounting for age. Measured PWV values were: 10.07 (8.51-12.01) for the older CAD group, 9.39 (7.44-9.75) for the younger CAD group, 8.26 (7.26-9.22) for the older Pre-CAD group, 10.57 m/s (8.5-11.2) for the younger Pre-CAD group, 7.13 m/s (5.97-7.69) for older healthy controls, and 6.71 m/s (4.86-7.26) for the younger healthy control subjects. These results demonstrate good potential value of this mobile PWV device as a simple low-cost screening tool for atherosclerosis and coronary arterial disease.

Classification of voluntary coughs applied to the screening of respiratory disease.

Pulmonary and respiratory diseases (e.g. asthma, COPD, allergies, pneumonia, tuberculosis, etc.) represent a large proportion of the global disease burden, mortality, and disability. In this context of creating automated diagnostic tools, we explore how the analysis of voluntary cough sounds may be used to screen for pulmonary disease. As a clinical study, voluntary coughs were recorded using a custom mobile phone stethoscope from 54 patients, of which 7 had COPD, 15 had asthma, 11 had allergic rhinitis, 17 had both asthma and allergic rhinitis, and four had both COPD and allergic rhinitis. Data were also collected from 33 healthy subjects. These patients also received full auscultation at 11 sites, given a clinical questionnaire, and underwent full pulmonary function testing (spirometer, body plethysmograph, DLCO) which culminated in a diagnosis provided by an experienced pulmonologist. From machine learning analysis of these data, we show that it is possible to achieve good classification of cough sounds in terms of Wet vs Dry, yielding an ROC curve with AUC of 0.94, and show that voluntary coughs can serve as an effective test for determining Healthy vs Unhealthy (sensitivity=35.7% specificity=100%). We also show that the use of cough sounds can enhance the performance of other diagnostic tools such as a patient questionnaire and peak flow meter; however voluntary coughs alone provide relatively little value in determining specific disease diagnosis.

A mobile platform for automated screening of asthma and chronic obstructive pulmonary disease.

Chronic Obstructive Pulmonary Disease (COPD) and asthma each represent a large proportion of the global disease burden; COPD is the third leading cause of death worldwide and asthma is one of the most prevalent chronic diseases, afflicting over 300 million people. Much of this burden is concentrated in the developing world, where patients lack access to physicians trained in the diagnosis of pulmonary disease. As a result, these patients experience high rates of underdiagnosis and misdiagnosis. To address this need, we present a mobile platform capable of screening for Asthma and COPD. Our solution is based on a mobile smart phone and consists of an electronic stethoscope, a peak flow meter application, and a patient questionnaire. This data is combined with a machine learning algorithm to identify patients with asthma and COPD. To test and validate the design, we collected data from 119 healthy and sick participants using our custom mobile application and ran the analysis on a PC computer. For comparison, all subjects were examined by an experienced pulmonologist using a full pulmonary testing laboratory. Employing a two-stage logistic regression model, our algorithms were first able to identify patients with either asthma or COPD from the general population, yielding an ROC curve with an AUC of 0.95. Then, after identifying these patients, our algorithm was able to distinguish between patients with asthma and patients with COPD, yielding an ROC curve with AUC of 0.97. This work represents an important milestone towards creating a self-contained mobile phone-based platform that can be used for screening and diagnosis of pulmonary disease in many parts of the world.

Application of semi-supervised deep learning to lung sound analysis.

The analysis of lung sounds, collected through auscultation, is a fundamental component of pulmonary disease diagnostics for primary care and general patient monitoring for telemedicine. Despite advances in computation and algorithms, the goal of automated lung sound identification and classification has remained elusive. Over the past 40 years, published work in this field has demonstrated only limited success in identifying lung sounds, with most published studies using only a small numbers of patients (typically N<;20) and usually limited to a single type of lung sound. Larger research studies have also been impeded by the challenge of labeling large volumes of data, which is extremely labor-intensive. In this paper, we present the development of a semi-supervised deep learning algorithm for automatically classify lung sounds from a relatively large number of patients (N=284). Focusing on the two most common lung sounds, wheeze and crackle, we present results from 11,627 sound files recorded from 11 different auscultation locations on these 284 patients with pulmonary disease. 890 of these sound files were labeled to evaluate the model, which is significantly larger than previously published studies. Data was collected with a custom mobile phone application and a low-cost (US$30) electronic stethoscope. On this data set, our algorithm achieves ROC curves with AUCs of 0.86 for wheeze and 0.74 for crackle. Most importantly, this study demonstrates how semi-supervised deep learning can be used with larger data sets without requiring extensive labeling of data.

Implementation of smart phone video plethysmography and dependence on lighting parameters.

The remote measurement of heart rate (HR) and heart rate variability (HRV) via a digital camera (video plethysmography) has emerged as an area of great interest for biomedical and health applications. While a few implementations of video plethysmography have been demonstrated on smart phones under controlled lighting conditions, it has been challenging to create a general scalable solution due to the large variability in smart phone hardware performance, software architecture, and the variable response to lighting parameters. In this context, we present a selfcontained smart phone implementation of video plethysmography for Android OS, which employs both stochastic and deterministic algorithms, and we use this to study the effect of lighting parameters (illuminance, color spectrum) on the accuracy of the remote HR measurement. Using two different phone models, we present the median HR error for five different video plethysmography algorithms under three different types of lighting (natural sunlight, compact fluorescent, and halogen incandescent) and variations in brightness. For most algorithms, we found the optimum light brightness to be in the range 1000-4000 lux and the optimum lighting types to be compact fluorescent and natural light. Moderate errors were found for most algorithms with some devices under conditions of low-brightness (<;500 lux) and highbrightness (>4000 lux). Our analysis also identified camera frame rate jitter as a major source of variability and error across different phone models, but this can be largely corrected through non-linear resampling. Based on testing with six human subjects, our real-time Android implementation successfully predicted the measured HR with a median error of -0.31 bpm, and an inter-quartile range of 2.1bpm.

Design and clinical feasibility of personal wearable monitor for measurement of activity and environmental exposure.

Human exposure to specific environmental factors (e.g. air quality, lighting, and sound) is known to play an important role in the pathogenesis of many chronic diseases (e.g. asthma) and mental health disorders (e.g. anxiety). However, conventional fixed environmental monitoring stations are sparsely located and, despite environmental models, cannot adequately assess individual exposure levels. New forms of low-cost portable monitors have begun to emerge that enable the collection of higher spatial density "crowd sourced" data; however, the first generation of these low-cost environmental monitors have generally not been suitable for clinical environmental health studies due to practical challenges such as calibration, reproducibility, form factor, and battery life. In this paper, we present a wearable environmental monitor that overcomes these challenges and can be used in clinical studies The new device, called "Eco-Mini," can be used without a smart phone and is capable of locally sampling and recording a variety of environmental parameters (Ozone, Sulfur Dioxide, Volatile Organic Compounds, humidity, temperature, ambient light color balance, and sound level) as well as individual activity (3-axis accelerometer) and location (GPS). In this paper, we also report findings and discuss lessons learned from a feasibility study conducted for one week with pediatric patients as part of an ongoing asthma research study.

Wearable wireless sensor platform for studying autonomic activity and social behavior in non-human primates.

A portable system has been designed to enable remote monitoring of autonomic nervous system output in non-human primates for the purpose of studying neural function related to social behavior over extended periods of time in an ambulatory setting. In contrast to prior systems which only measure heart activity, are restricted to a constrained laboratory setting, or require surgical attachment, our system is comprised of a multi-sensor self-contained wearable vest that can easily be transferred from one subject to another. The vest contains a small detachable low-power electronic sensor module for measuring electrodermal activity (EDA), electrocardiography (ECG), 3-axis acceleration, and temperature. The wireless transmission is implemented using a standard Bluetooth protocol and a mobile phone, which enables freedom of movement for the researcher as well as for the test subject. A custom Android software application was created on the mobile phone for viewing and recording live data as well as creating annotations. Data from up to seven monkeys can be recorded simultaneously using the mobile phone, with the option of real-time upload to a remote web server. Sample data are presented from two rhesus macaque monkeys showing stimulus-induced response in the laboratory as well as long-term ambulatory data collected in a large monkey cage. This system enables new possibilities for studying underlying mechanisms between autonomic brain function and social behavior with connection to human research in areas such as autism, substance abuse, and mood disorders.