An Unobtrusive Human Activity Recognition System Using Low Resolution Thermal Sensors, Machine and Deep Learning.

Given the aging population, healthcare systems need to be established to deal with health issues such as injurious falls. Wearable devices can be used to detect falls. However, most wearable devices are obtrusive, and patients generally do not like or may forget to wear them. In this study, we developed an unobtrusive monitoring system using infrared technology to unobtrusively detect locations and recognize human activities such as sitting, standing, walking, lying, and falling. We prototyped a system consisting of two 24×32 thermal array sensors and collected data from healthy young volunteers performing ten different scenarios. A supervised deep learning (DL)-based approach classified activities and detected locations from images. The performance of the DL approach was also compared with the machine learning (ML)-based methods. In addition, we fused the data of two sensors and formed a stereo system, which resulted in better performance compared to a single sensor. Furthermore, to detect critical activities such as falling and lying on floor, we performed a binary classification in which one class was falling plus lying on floor and another class was all the remaining activities. Using the DL-based algorithm on the stereo dataset to recognize activities, overall average accuracy and F1-score were achieved as 97.6%, and 0.935, respectively. These scores for location detection were 97.3%, and 0.927, respectively. These scores for binary classification were 97.9%, and 0.945, respectively. Our results suggest the proposed system recognized human activities, detected locations, and detected critical activities namely falling and lying on floor accurately.

An Unobtrusive Fall Detection System Using Low Resolution Thermal Sensors and Convolutional Neural Networks.

Human activity recognition has many potential applications. In an aged care facility, it is crucial to monitor elderly patients and assist them in the case of falls or other needs. Wearable devices can be used for such a purpose. However, most of them have been proven to be obtrusive, and patients reluctate or forget to wear them. In this study, we used infrared technology to recognize certain human activities including sitting, standing, walking, laying in bed, laying down, and falling. We evaluated a system consisting of two 24×32 thermal array sensors. One infrared sensor was installed on side and another one was installed on the ceiling of an experimental room capturing the same scene. We chose side and overhead mounts to compare the performance of classifiers. We used our prototypes to collect data from healthy young volunteers while performing eight different scenarios. After that, we converted data coming from the sensors into images and applied a supervised deep learning approach. The scene was captured by a visible camera and the video from the visible camera was used as the ground truth. The deep learning network consisted of a convolutional neural network which automatically extracted features from infrared images. Overall average F1-score of all classes for the side mount was 0.9044 and for the overhead mount was 0.8893. Overall average accuracy of all classes for the side mount was 96.65% and for the overhead mount was 95.77%. Our results suggested that our infrared-based method not only could unobtrusively recognize human activities but also was reasonably accurate.