Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data.

Activity recognition is one of the significant technologies accompanying the development of the Internet of Things (IoT). It can help in recording daily life activities or reporting emergencies, thus improving the user's quality of life and safety, and even easing the workload of caregivers. This study proposes a human activity recognition (HAR) system based on activity data obtained via the micro-Doppler effect, combining a two-stream one-dimensional convolutional neural network (1D-CNN) with a bidirectional gated recurrent unit (BiGRU). Initially, radar sensor data are used to generate information related to time and frequency responses using short-time Fourier transform (STFT). Subsequently, the magnitudes and phase values are calculated and fed into the 1D-CNN and Bi-GRU models to extract spatial and temporal features for subsequent model training and activity recognition. Additionally, we propose a simple cross-channel operation (CCO) to facilitate the exchange of magnitude and phase features between parallel convolutional layers. An open dataset collected through radar, named Rad-HAR, is employed for model training and performance evaluation. Experimental results demonstrate that the proposed 1D-CNN+CCO-BiGRU model demonstrated superior performance, achieving an impressive accuracy rate of 98.2%. This outperformance of existing systems with the radar sensor underscores the proposed model's potential applicability in real-world scenarios, marking a significant advancement in the field of HAR within the IoT framework.

High Accuracy and Cost-Effective Fiber Optic Liquid Level Sensing System Based on Deep Neural Network.

In this paper, a novel liquid level sensing system is proposed to enhance the capacity of the sensing system, as well as reduce the cost and increase the sensing accuracy. The proposed sensing system can monitor the liquid level of several points at the same time in the sensing unit. Additionally, for cost efficiency, the proposed system employs only one sensor at each spot and all the sensors are multiplexed. In multiplexed systems, when changing the liquid level inside the container, the float position is changed and leads to an overlap or cross-talk between two sensors. To solve this overlap problem and to accurately predict the liquid level of each container, we proposed a deep neural network (DNN) approach to properly identify the water level. The performance of the proposed DNN model is evaluated via two different scenarios and the result proves that the proposed DNN model can accurately predict the liquid level of each point. Furthermore, when comparing the DNN model with the conventional machine learning schemes, including random forest (RF) and support vector machines (SVM), the DNN model exhibits the best performance.

Using a Hybrid Neural Network and a Regularized Extreme Learning Machine for Human Activity Recognition with Smartphone and Smartwatch.

Mobile health (mHealth) utilizes mobile devices, mobile communication techniques, and the Internet of Things (IoT) to improve not only traditional telemedicine and monitoring and alerting systems, but also fitness and medical information awareness in daily life. In the last decade, human activity recognition (HAR) has been extensively studied because of the strong correlation between people's activities and their physical and mental health. HAR can also be used to care for elderly people in their daily lives. This study proposes an HAR system for classifying 18 types of physical activity using data from sensors embedded in smartphones and smartwatches. The recognition process consists of two parts: feature extraction and HAR. To extract features, a hybrid structure consisting of a convolutional neural network (CNN) and a bidirectional gated recurrent unit GRU (BiGRU) was used. For activity recognition, a single-hidden-layer feedforward neural network (SLFN) with a regularized extreme machine learning (RELM) algorithm was used. The experimental results show an average precision of 98.3%, recall of 98.4%, an F1-score of 98.4%, and accuracy of 98.3%, which results are superior to those of existing schemes.

Developing Low-Cost Mobile Device and Apps for Accurate Skin Spectrum Measurement via Low-Cost Spectrum Sensors and Deep Neural Network Technology.

In recent years, skin spectral information has been gradually applied in various fields, such as the cosmetics industry and clinical medicine. However, the high price and the huge size of the skin spectrum measurement device make the related applications of the skin spectrum unable to be widely used in practical applications. We used convolutional neural network (CNN) to achieve a satisfying accuracy of the Fitzpatrick skin-type classification by using a simple self-developed device in 2018. Leveraging on the hardware, firmware, and software app-developing experience, a low-cost miniature skin spectrum measurement system (LMSSMS) using deep neural network (DNN) technology was further studied, and the feasibility of the system is verified in this paper. The developed LMSSMS is divided into three parts: (1) miniature skin spectrum measurement device (MSSMD), (2) DNN model, and (3) mobile app. The MSSMD was developed with innovative low-cost MSSC, 3D printing, and a simple LED light source. The DNN model is designed to enhance measurement accuracy. Finally, the mobile app is used to control and show the measurement results. The developed app also includes a variety of skin-spectrum-related applications, such as erythema index and melanin index (EI/MI) measurement, Fitzpatrick skin-type classification, Pantone SkinTone classification, sun-exposure estimation, and body-fat measurement. In order to verify the feasibility of LMSSMS, we used the standard instrumentation device as a reference. The results show that the accuracy of the LMSSMS can reach 94.7%, which also confirms that this development idea has much potential for further development.

SHEL5K: An Extended Dataset and Benchmarking for Safety Helmet Detection.

Wearing a safety helmet is important in construction and manufacturing industrial activities to avoid unpleasant situations. This safety compliance can be ensured by developing an automatic helmet detection system using various computer vision and deep learning approaches. Developing a deep-learning-based helmet detection model usually requires an enormous amount of training data. However, there are very few public safety helmet datasets available in the literature, in which most of them are not entirely labeled, and the labeled one contains fewer classes. This paper presents the Safety HELmet dataset with 5K images (SHEL5K) dataset, an enhanced version of the SHD dataset. The proposed dataset consists of six completely labeled classes (helmet, head, head with helmet, person with helmet, person without helmet, and face). The proposed dataset was tested on multiple state-of-the-art object detection models, i.e., YOLOv3 (YOLOv3, YOLOv3-tiny, and YOLOv3-SPP), YOLOv4 (YOLOv4 and YOLOv4pacsp-x-mish), YOLOv5-P5 (YOLOv5s, YOLOv5m, and YOLOv5x), the Faster Region-based Convolutional Neural Network (Faster-RCNN) with the Inception V2 architecture, and YOLOR. The experimental results from the various models on the proposed dataset were compared and showed improvement in the mean Average Precision (mAP). The SHEL5K dataset had an advantage over other safety helmet datasets as it contains fewer images with better labels and more classes, making helmet detection more accurate.

Sound Source Localization Using a Convolutional Neural Network and Regression Model.

In this research, a novel sound source localization model is introduced that integrates a convolutional neural network with a regression model (CNN-R) to estimate the sound source angle and distance based on the acoustic characteristics of the interaural phase difference (IPD). The IPD features of the sound signal are firstly extracted from time-frequency domain by short-time Fourier transform (STFT). Then, the IPD features map is fed to the CNN-R model as an image for sound source localization. The Pyroomacoustics platform and the multichannel impulse response database (MIRD) are used to generate both simulated and real room impulse response (RIR) datasets. The experimental results show that an average accuracy of 98.96% and 98.31% are achieved by the proposed CNN-R for angle and distance estimations in the simulation scenario at SNR = 30 dB and RT60 = 0.16 s, respectively. Moreover, in the real environment, the average accuracies of the angle and distance estimations are 99.85% and 99.38% at SNR = 30 dB and RT60 = 0.16 s, respectively. The performance obtained in both scenarios is superior to that of existing models, indicating the potential of the proposed CNN-R model for real-life applications.

Towards Privacy-Preserved Aging in Place: A Systematic Review.

Owing to progressive population aging, elderly people (aged 65 and above) face challenges in carrying out activities of daily living, while placement of the elderly in a care facility is expensive and mentally taxing for them. Thus, there is a need to develop their own homes into smart homes using new technologies. However, this raises concerns of privacy and data security for users since it can be handled remotely. Hence, with advancing technologies it is important to overcome this challenge using privacy-preserving and non-intrusive models. For this review, 235 articles were scanned from databases, out of which 31 articles pertaining to in-home technologies that assist the elderly in living independently were shortlisted for inclusion. They described the adoption of various methodologies like different sensor-based mechanisms, wearables, camera-based techniques, robots, and machine learning strategies to provide a safe and comfortable environment to the elderly. Recent innovations have rendered these technologies more unobtrusive and privacy-preserving with increasing use of environmental sensors and less use of cameras and other devices that may compromise the privacy of individuals. There is a need to develop a comprehensive system for smart homes which ensures patient safety, privacy, and data security; in addition, robots should be integrated with the existing sensor-based platforms to assist in carrying out daily activities and therapies as required.

Binary Sensors-Based Privacy-Preserved Activity Recognition of Elderly Living Alone Using an RNN.

The recent growth of the elderly population has led to the requirement for constant home monitoring as solitary living becomes popular. This protects older people who live alone from unwanted instances such as falling or deterioration caused by some diseases. However, although wearable devices and camera-based systems can provide relatively precise information about human motion, they invade the privacy of the elderly. One way to detect the abnormal behavior of elderly residents under the condition of maintaining privacy is to equip the resident's house with an Internet of Things system based on a non-invasive binary motion sensor array. We propose to concatenate external features (previous activity and begin time-stamp) along with extracted features with a bi-directional long short-term memory (Bi-LSTM) neural network to recognize the activities of daily living with a higher accuracy. The concatenated features are classified by a fully connected neural network (FCNN). The proposed model was evaluated on open dataset from the Center for Advanced Studies in Adaptive Systems (CASAS) at Washington State University. The experimental results show that the proposed method outperformed state-of-the-art models with a margin of more than 6.25% of the F1 score on the same dataset.

Using Direct Acyclic Graphs to Enhance Skeleton-Based Action Recognition with a Linear-Map Convolution Neural Network.

Research on the human activity recognition could be utilized for the monitoring of elderly people living alone to reduce the cost of home care. Video sensors can be easily deployed in the different zones of houses to achieve monitoring. The goal of this study is to employ a linear-map convolutional neural network (CNN) to perform action recognition with RGB videos. To reduce the amount of the training data, the posture information is represented by skeleton data extracted from the 300 frames of one film. The two-stream method was applied to increase the accuracy of recognition by using the spatial and motion features of skeleton sequences. The relations of adjacent skeletal joints were employed to build the direct acyclic graph (DAG) matrices, source matrix, and target matrix. Two features were transferred by DAG matrices and expanded as color texture images. The linear-map CNN had a two-dimensional linear map at the beginning of each layer to adjust the number of channels. A two-dimensional CNN was used to recognize the actions. We applied the RGB videos from the action recognition datasets of the NTU RGB+D database, which was established by the Rapid-Rich Object Search Lab, to execute model training and performance evaluation. The experimental results show that the obtained precision, recall, specificity, F1-score, and accuracy were 86.9%, 86.1%, 99.9%, 86.3%, and 99.5%, respectively, in the cross-subject source, and 94.8%, 94.7%, 99.9%, 94.7%, and 99.9%, respectively, in the cross-view source. An important contribution of this work is that by using the skeleton sequences to produce the spatial and motion features and the DAG matrix to enhance the relation of adjacent skeletal joints, the computation speed was faster than the traditional schemes that utilize single frame image convolution. Therefore, this work exhibits the practical potential of real-life action recognition.

A Portable and Wireless Multi-Channel Acquisition System for Physiological Signal Measurements.

We propose a portable and wireless acquisition system to help consumers or users register important physiological signals. The acquisition system mainly consists of a portable device, a graphic user interface (GUI), and an application program for displaying the signals on a notebook (NB) computer or a smart device. Essential characteristics of the portable device include eight measuring channels, a powerful microcontroller unit, a lithium battery, Bluetooth 3.0 data transmission, and a built-in 2 GB flash memory. In addition, the signals that are measured can be displayed on a tablet, a smart phone, or a notebook computer concurrently. Additionally, the proposed system provides extra power supply sources of ±3 V for the usage of external circuits. On the other hand, consumers or users can design their own sensing circuits and combine them with this system to carry out ubiquitous physiological studies. Four major advantages in the proposed system are the capability of combining it with a NB computer or a smart phone to display the signals being measured in real time, the superior mobility due to its own independent power system, flash memory, and good expandability. Briefly, this acquisition system offers consumers or users a convenient and portable studying tool to measure dynamic vital signals of interest in psychological and physiological research fields.

Intelligent Lecturer Tracking and Capturing System Based on Face Detection and Wireless Sensing Technology.

In this paper, we propose an intelligent lecturer tracking and capturing (ILTC) system to automatically record course videos. Real-time and stable lecturer localization is realized by combining face detection with infrared (IR) thermal sensors, preventing detection failure caused by abrupt and rapid movements in face detection and solving the non-real-time sensing problem for IR thermal sensors. Further, the camera is panned automatically by a servo motor controlled with a microcontroller to keep the lecturer in the center of the screen. Experiments were conducted in a classroom and a laboratory. Experimental results demonstrated that the accuracy of the proposed system is much higher than that of the system without IR thermal sensors. The survey of 32 teachers from two universities showed that the proposed system is a more practical utility and meets the demand of increasing online courses.

ECG Noise Cancellation Based on Grey Spectral Noise Estimation.

In recent years, wearable devices have been popularly applied in the health care field. The electrocardiogram (ECG) is the most used signal. However, the ECG is measured under a body-motion condition, which is easily coupled with some noise, like as power line noise (PLn) and electromyogram (EMG). This paper presents a grey spectral noise cancellation (GSNC) scheme for electrocardiogram (ECG) signals where two-stage discrimination is employed with the empirical mode decomposition (EMD), the ensemble empirical mode decomposition (EEMD) and the grey spectral noise estimation (GSNE). In the first stage of the proposed GSNC scheme, the input ECG signal is decomposed by the EMD to obtain a set of intrinsic mode functions (IMFs). Then, the noise energies of IMFs are estimated by the GSNE. When an IMF is considered as noisy one, it is forwarded to the second stage for further check. In the second stage, the suspicious IMFs are reconstructed and decomposed by the EEMD. Then the IMFs are discriminated with a threshold. If the IMF is considered as noisy, it is discarded in the reconstruction process of the ECG signal. The proposed GSNC scheme is justified by forty-three ECG signal datasets from the MIT-BIH cardiac arrhythmia database where the PLn and EMG noise are under consideration. The results indicate that the proposed GSNC scheme outperforms the traditional EMD and EEMD based noise cancellation schemes in the given datasets.

Using the Pulse Contour Method to Measure the Changes in Stroke Volume during a Passive Leg Raising Test.

The pulse contour method is often used with the Windkessel model to measure stroke volume. We used a digital pressure and flow sensors to detect the parameters of the Windkessel model from the pulse waveform. The objective of this study was to assess the stability and accuracy of this method by making use of the passive leg raising test. We studied 24 healthy subjects (40 ± 9.3 years), and used the Medis® CS 1000, an impedance cardiography, as the comparing reference. The pulse contour method measured the waveform of the brachial artery by using a cuff. The compliance and resistance of the peripheral artery was detected from the cuff characteristics and the blood pressure waveform. Then, according to the method proposed by Romano et al., the stroke volume could be measured. This method was implemented in our designed blood pressure monitor. A passive leg raising test, which could immediately change the preloading of the heart, was done to certify the performance of our method. The pulse contour method and impedance cardiography simultaneously measured the stroke volume. The measurement of the changes in stroke volume using the pulse contour method had a very high correlation with the Medis® CS 1000 measurement, the correlation coefficient of the changed ratio and changed differences in stroke volume were r² = 0.712 and r² = 0.709, respectively. It was shown that the stroke volume measured by using the pulse contour method was not accurate enough. But, the changes in the stroke volume could be accurately measured with this pulse contour method. Changes in stroke volume are often used to understand the conditions of cardiac preloading in the clinical field. Moreover, the operation of the pulse contour method is easier than using impedance cardiography and echocardiography. Thus, this method is suitable to use in different healthcare fields.

Ubiquitous Emergency Medical Service System Based on Wireless Biosensors, Traffic Information, and Wireless Communication Technologies: Development and Evaluation.

This study presents a new ubiquitous emergency medical service system (UEMS) that consists of a ubiquitous tele-diagnosis interface and a traffic guiding subsystem. The UEMS addresses unresolved issues of emergency medical services by managing the sensor wires for eliminating inconvenience for both patients and paramedics in an ambulance, providing ubiquitous accessibility of patients' biosignals in remote areas where the ambulance cannot arrive directly, and offering availability of real-time traffic information which can make the ambulance reach the destination within the shortest time. In the proposed system, patient's biosignals and real-time video, acquired by wireless biosensors and a webcam, can be simultaneously transmitted to an emergency room for pre-hospital treatment via WiMax/3.5 G networks. Performances of WiMax and 3.5 G, in terms of initialization time, data rate, and average end-to-end delay are evaluated and compared. A driver can choose the route of the shortest time among the suggested routes by Google Maps after inspecting the current traffic conditions based on real-time CCTV camera streams and traffic information. The destination address can be inputted vocally for easiness and safety in driving. A series of field test results validates the feasibility of the proposed system for application in real-life scenarios.

Using K-Nearest Neighbor Classification to Diagnose Abnormal Lung Sounds.

A reported 30% of people worldwide have abnormal lung sounds, including crackles, rhonchi, and wheezes. To date, the traditional stethoscope remains the most popular tool used by physicians to diagnose such abnormal lung sounds, however, many problems arise with the use of a stethoscope, including the effects of environmental noise, the inability to record and store lung sounds for follow-up or tracking, and the physician's subjective diagnostic experience. This study has developed a digital stethoscope to help physicians overcome these problems when diagnosing abnormal lung sounds. In this digital system, mel-frequency cepstral coefficients (MFCCs) were used to extract the features of lung sounds, and then the K-means algorithm was used for feature clustering, to reduce the amount of data for computation. Finally, the K-nearest neighbor method was used to classify the lung sounds. The proposed system can also be used for home care: if the percentage of abnormal lung sound frames is > 30% of the whole test signal, the system can automatically warn the user to visit a physician for diagnosis. We also used bend sensors together with an amplification circuit, Bluetooth, and a microcontroller to implement a respiration detector. The respiratory signal extracted by the bend sensors can be transmitted to the computer via Bluetooth to calculate the respiratory cycle, for real-time assessment. If an abnormal status is detected, the device will warn the user automatically. Experimental results indicated that the error in respiratory cycles between measured and actual values was only 6.8%, illustrating the potential of our detector for home care applications.

Development and evaluation of an E-health system to care for patients with bladder pain syndrome/interstitial cystitis.

OBJECTIVES: Bladder pain syndrome/interstitial cystitis (BPS/IC) is a chronic disease that highly degrades the quality of life for patients. In the present study, Internet intervention was used to care for bladder pain syndrome/interstitial cystitis patients to alleviate their pain and bothering symptoms. METHODS: Healthcare education was carried out through the Internet by asking the patients, who were randomly divided into study (40 patients) and control (40 patients) groups, to check possible sensitive foods, habits, and behaviors weekly to remind and consolidate important rules for promoting quality of life. The symptom flares consultation through short message service with the Internet used to elevate healthcare efficiency was undertaken. Questionnaires, including Short Form 36 health survey, O'Leary-Sant symptom and problem indices, as well as visual analog scales pain and urgency scales, were used to evaluate quality of life and disease severity improvements before and after information and communication technology intervention. The outcome was evaluated at week 8. RESULTS: The quality of life of both the control and study groups was significantly improved. The quality of life and visual analog scales for the patients in the study group with information and communication technology intervention showed a much greater improvement compared with the patients in the control group (P < 0.05). CONCLUSIONS: The E-health system was shown to be effective in improving quality of life of bladder pain syndrome/interstitial cystitis patients through intervention of Internet healthcare education and short message service for the consolidation of healthy behavior and lifestyle in the 8-week follow up.

Precision Non-Alcoholic Fatty Liver Disease (NAFLD) Diagnosis: Leveraging Ensemble Machine Learning and Gender Insights for Cost-Effective Detection.

Non-Alcoholic Fatty Liver Disease (NAFLD) is characterized by the accumulation of excess fat in the liver. If left undiagnosed and untreated during the early stages, NAFLD can progress to more severe conditions such as inflammation, liver fibrosis, cirrhosis, and even liver failure. In this study, machine learning techniques were employed to predict NAFLD using affordable and accessible laboratory test data, while the conventional technique hepatic steatosis index (HSI)was calculated for comparison. Six algorithms (random forest, K-nearest Neighbors, Logistic Regression, Support Vector Machine, extreme gradient boosting, decision tree), along with an ensemble model, were utilized for dataset analysis. The objective was to develop a cost-effective tool for enabling early diagnosis, leading to better management of the condition. The issue of imbalanced data was addressed using the Synthetic Minority Oversampling Technique Edited Nearest Neighbors (SMOTEENN). Various evaluation metrics including the F1 score, precision, accuracy, recall, confusion matrix, the mean absolute error (MAE), receiver operating characteristics (ROC), and area under the curve (AUC) were employed to assess the suitability of each technique for disease prediction. Experimental results using the National Health and Nutrition Examination Survey (NHANES) dataset demonstrated that the ensemble model achieved the highest accuracy (0.99) and AUC (1.00) compared to the machine learning techniques that we used and HSI. These findings indicate that the ensemble model holds potential as a beneficial tool for healthcare professionals to predict NAFLD, leveraging accessible and cost-effective laboratory test data.

Clinical verification of a clinical decision support system for ventilator weaning.

BACKGROUND: Weaning is typically regarded as a process of discontinuing mechanical ventilation in the daily practice of an intensive care unit (ICU). Among the ICU patients, 39%-40% need mechanical ventilator for sustaining their lives. The predictive rate of successful weaning achieved only 35-60% for decisions made by physicians. Clinical decision support systems (CDSSs) are promising in enhancing diagnostic performance and improve healthcare quality in clinical setting. To our knowledge, a prospective study has never been conducted to verify the effectiveness of the CDSS in ventilator weaning before. In this study, the CDSS capable of predicting weaning outcome and reducing duration of ventilator support for patients has been verified. METHODS: A total of 380 patients admitted to the respiratory care center of the hospital were randomly assigned to either control or study group. In the control group, patients were weaned with traditional weaning method, while in the study group, patients were weaned with CDSS monitored by physicians. After excluding the patients who transferred to other hospitals, refused further treatments, or expired the admission period, data of 168 and 144 patients in the study and control groups, respectively, were used for analysis. RESULTS: The results show that a sensitivity of 87.7% has been achieved, which is significantly higher (p<0.01) than the weaning determined by physicians (sensitivity: 61.4%). Furthermore, the days using mechanical ventilator for the study group (38.41 ± 3.35) is significantly (p<0.001) shorter than the control group (43.69 ± 14.89), with a decrease of 5.2 days in average, resulting in a saving of healthcare cost of NT$45,000 (US$1,500) per patient in the current Taiwanese National Health Insurance setting. CONCLUSIONS: The CDSS is demonstrated to be effective in identifying the earliest time of ventilator weaning for patients to resume and sustain spontaneous breathing, thereby avoiding unnecessary prolonged ventilator use and decreasing healthcare cost.

Performance Comparison of Machine Learning Approaches on Hepatitis C Prediction Employing Data Mining Techniques.

Hepatitis C is a liver infection caused by the hepatitis C virus (HCV). Due to the late onset of symptoms, early diagnosis is difficult in this disease. Efficient prediction can save patients before permeant liver damage. The main objective of this study is to employ various machine learning techniques to predict this disease based on common and affordable blood test data to diagnose and treat patients in the early stages. In this study, six machine learning algorithms (Support Vector Machine (SVM), K-nearest Neighbors (KNN), Logistic Regression, decision tree, extreme gradient boosting (XGBoost), artificial neural networks (ANN)) were utilized on two datasets. The performances of these techniques were compared in terms of confusion matrix, precision, recall, F1 score, accuracy, receiver operating characteristics (ROC), and the area under the curve (AUC) to identify a method that is appropriate for predicting this disease. The analysis, on NHANES and UCI datasets, revealed that SVM and XGBoost (with the highest accuracy and AUC among the test models, >80%) can be effective tools for medical professionals using routine and affordable blood test data to predict hepatitis C.

Correction: Alkhaleefah et al. Connected-SegNets: A Deep Learning Model for Breast Tumor Segmentation from X-ray Images. Cancers 2022, 14, 4030.

In the original publication [...].