An Unobtrusive Human Activity Recognition System Using Low Resolution Thermal Sensors, Machine and Deep Learning.

Given the aging population, healthcare systems need to be established to deal with health issues such as injurious falls. Wearable devices can be used to detect falls. However, most wearable devices are obtrusive, and patients generally do not like or may forget to wear them. In this study, we developed an unobtrusive monitoring system using infrared technology to unobtrusively detect locations and recognize human activities such as sitting, standing, walking, lying, and falling. We prototyped a system consisting of two 24×32 thermal array sensors and collected data from healthy young volunteers performing ten different scenarios. A supervised deep learning (DL)-based approach classified activities and detected locations from images. The performance of the DL approach was also compared with the machine learning (ML)-based methods. In addition, we fused the data of two sensors and formed a stereo system, which resulted in better performance compared to a single sensor. Furthermore, to detect critical activities such as falling and lying on floor, we performed a binary classification in which one class was falling plus lying on floor and another class was all the remaining activities. Using the DL-based algorithm on the stereo dataset to recognize activities, overall average accuracy and F1-score were achieved as 97.6%, and 0.935, respectively. These scores for location detection were 97.3%, and 0.927, respectively. These scores for binary classification were 97.9%, and 0.945, respectively. Our results suggest the proposed system recognized human activities, detected locations, and detected critical activities namely falling and lying on floor accurately.

An Unobtrusive Fall Detection System Using Low Resolution Thermal Sensors and Convolutional Neural Networks.

Human activity recognition has many potential applications. In an aged care facility, it is crucial to monitor elderly patients and assist them in the case of falls or other needs. Wearable devices can be used for such a purpose. However, most of them have been proven to be obtrusive, and patients reluctate or forget to wear them. In this study, we used infrared technology to recognize certain human activities including sitting, standing, walking, laying in bed, laying down, and falling. We evaluated a system consisting of two 24×32 thermal array sensors. One infrared sensor was installed on side and another one was installed on the ceiling of an experimental room capturing the same scene. We chose side and overhead mounts to compare the performance of classifiers. We used our prototypes to collect data from healthy young volunteers while performing eight different scenarios. After that, we converted data coming from the sensors into images and applied a supervised deep learning approach. The scene was captured by a visible camera and the video from the visible camera was used as the ground truth. The deep learning network consisted of a convolutional neural network which automatically extracted features from infrared images. Overall average F1-score of all classes for the side mount was 0.9044 and for the overhead mount was 0.8893. Overall average accuracy of all classes for the side mount was 96.65% and for the overhead mount was 95.77%. Our results suggested that our infrared-based method not only could unobtrusively recognize human activities but also was reasonably accurate.

The SleepFit Tablet Application for Home-Based Clinical Data Collection in Parkinson Disease: User-Centric Development and Usability Study.

BACKGROUND: Parkinson disease (PD) is a common, multifaceted neurodegenerative disorder profoundly impacting patients' autonomy and quality of life. Assessment in real-life conditions of subjective symptoms and objective metrics of mobility and nonmotor symptoms such as sleep disturbance is strongly advocated. This information would critically guide the adaptation of antiparkinsonian medications and nonpharmacological interventions. Moreover, since the spread of the COVID-19 pandemic, health care practices are being reshaped toward a more home-based care. New technologies could play a pivotal role in this new approach to clinical care. Nevertheless, devices and information technology tools might be unhandy for PD patients, thus dramatically limiting their widespread employment. OBJECTIVE: The goals of the research were development and usability evaluation of an application, SleepFit, for ecological momentary assessment of objective and subjective clinical metrics at PD patients' homes, and as a remote tool for researchers to monitor patients and integrate and manage data. METHODS: An iterative and user-centric strategy was employed for the development of SleepFit. The core structure of SleepFit consists of (1) an electronic finger-tapping test; (2) motor, sleepiness, and emotional subjective scales; and (3) a sleep diary. Applicable design, ergonomic, and navigation principles have been applied while tailoring the application to the specific patient population. Three progressively enhanced versions of the application (alpha, v1.0, v2.0) were tested by a total of 56 patients with PD who were asked to perform multiple home assessments 4 times per day for 2 weeks. Patient compliance was calculated as the proportion of completed tasks out of the total number of expected tasks. Satisfaction on the latest version (v2.0) was evaluated as potential willingness to use SleepFit again after the end of the study. RESULTS: From alpha to v1.0, SleepFit was improved in graphics, ergonomics, and navigation, with automated flows guiding the patients in performing tasks throughout the 24 hours, and real-time data collection and consultation were made possible thanks to a remote web portal. In v2.0, the kiosk-mode feature restricts the use of the tablet to the SleepFit application only, thus preventing users from accidentally exiting the application. A total of 52 (4 dropouts) patients were included in the analyses. Overall compliance (all versions) was 88.89% (5707/6420). SleepFit was progressively enhanced and compliance increased from 87.86% (2070/2356) to 89.92% (2899/3224; P=.04). Among the patients who used v2.0, 96% (25/26) declared they would use SleepFit again. CONCLUSIONS: SleepFit can be considered a state-of-the-art home-based system that increases compliance in PD patients, ensures high-quality data collection, and works as a handy tool for remote monitoring and data management in clinical research. Thanks to its user-friendliness and modular structure, it could be employed in other clinical studies with minimum adaptation efforts. TRIAL REGISTRATION: ClinicalTrials.gov NCT02723396; https://clinicaltrials.gov/ct2/show/NCT02723396.

A New Prospective, Home-Based Monitoring of Motor Symptoms in Parkinson's Disease.

BACKGROUND: Subjective symptoms, which are retrospectively assessed during clinical interviews in the office, may be influenced by patient recall in Parkinson's disease (PD). Prospective collection of subjective data might be an effective tool to overcome this bias. OBJECTIVE: We investigated the correspondence between prospectively and retrospectively assessed motor symptoms in PD. METHODS: Forty-two consecutive patients (9 females, 67±9.8 years old) with mild to moderate PD reported their symptoms four times a day for two weeks, using the "SleepFit" application (app) for tablets. This app incorporates a new Visual Analogue Scale assessing global mobility (m-VAS), and the Scales for Outcome in Parkinson Assessment Diary Card (SCOPA-DC). At day 14, the Movement Disorders Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) parts II and IV questionnaires were completed at the hospital. Agreement (root mean square difference) and the tendency to under- or overestimate their symptoms by patients (relative difference after normalization) were calculated to compare prospectively vs. retrospectively collected information. RESULTS: Although agreement was good for overall scores (m-VAS: 10.0%; SCOPA-DC: 18.3%), and for single motor symptoms (involuntary movements, hand dexterity, walking, changing position; each <20%), some individuals with more advanced disease, higher fatigue or worse sleep quality showed poor symptom recall in retrospect. Moreover, a subgroup of patients (16.7%) either over- or underestimated symptom severity. CONCLUSIONS: Regular, prospective monitoring of motor symptoms is suitable in PD patients. SleepFit might be a useful tool in routine practice to identify patients tending to under- or overestimate their symptoms, and for their follow-up.