Sensing technologies and machine learning methods for emotion recognition in autism: Systematic review.

BACKGROUND: Human Emotion Recognition (HER) has been a popular field of study in the past years. Despite the great progresses made so far, relatively little attention has been paid to the use of HER in autism. People with autism are known to face problems with daily social communication and the prototypical interpretation of emotional responses, which are most frequently exerted via facial expressions. This poses significant practical challenges to the application of regular HER systems, which are normally developed for and by neurotypical people. OBJECTIVE: This study reviews the literature on the use of HER systems in autism, particularly with respect to sensing technologies and machine learning methods, as to identify existing barriers and possible future directions. METHODS: We conducted a systematic review of articles published between January 2011 and June 2023 according to the 2020 PRISMA guidelines. Manuscripts were identified through searching Web of Science and Scopus databases. Manuscripts were included when related to emotion recognition, used sensors and machine learning techniques, and involved children with autism, young, or adults. RESULTS: The search yielded 346 articles. A total of 65 publications met the eligibility criteria and were included in the review. CONCLUSIONS: Studies predominantly used facial expression techniques as the emotion recognition method. Consequently, video cameras were the most widely used devices across studies, although a growing trend in the use of physiological sensors was observed lately. Happiness, sadness, anger, fear, disgust, and surprise were most frequently addressed. Classical supervised machine learning techniques were primarily used at the expense of unsupervised approaches or more recent deep learning models. Studies focused on autism in a broad sense but limited efforts have been directed towards more specific disorders of the spectrum. Privacy or security issues were seldom addressed, and if so, at a rather insufficient level of detail.

T1DiabetesGranada: a longitudinal multi-modal dataset of type 1 diabetes mellitus.

Type 1 diabetes mellitus (T1D) patients face daily difficulties in keeping their blood glucose levels within appropriate ranges. Several techniques and devices, such as flash glucose meters, have been developed to help T1D patients improve their quality of life. Most recently, the data collected via these devices is being used to train advanced artificial intelligence models to characterize the evolution of the disease and support its management. Data scarcity is the main challenge for generating these models, as most works use private or artificially generated datasets. For this reason, this work presents T1DiabetesGranada, an open under specific permission longitudinal dataset that not only provides continuous glucose levels, but also patient demographic and clinical information. The dataset includes 257 780 days of measurements spanning four years from 736 T1D patients from the province of Granada, Spain. This dataset advances beyond the state of the art as one the longest and largest open datasets of continuous glucose measurements, thus boosting the development of new artificial intelligence models for glucose level characterization and prediction.

HIGEA: An Intelligent Conversational Agent to Detect Caregiver Burden.

Mental health disorders increasingly affect people worldwide. As a consequence, more families and relatives find themselves acting as caregivers. Most often, these are untrained people who experience loneliness, abandonment, and often develop signs of depression (i.e., caregiver burden syndrome). In this work, we present HIGEA, a digital system based on a conversational agent to help to detect caregiver burden. The conversational agent naturally embeds psychological test questions into informal conversations, which aim at increasing the adherence of use and avoiding user bias. A proof-of-concept is developed based on the popular Zarit Test, which is widely used to assess caregiver burden. Preliminary results show the system is useful and effective.

Mobile and Wearable Technology for the Monitoring of Diabetes-Related Parameters: Systematic Review.

BACKGROUND: Diabetes mellitus is a metabolic disorder that affects hundreds of millions of people worldwide and causes several million deaths every year. Such a dramatic scenario puts some pressure on administrations, care services, and the scientific community to seek novel solutions that may help control and deal effectively with this condition and its consequences. OBJECTIVE: This study aims to review the literature on the use of modern mobile and wearable technology for monitoring parameters that condition the development or evolution of diabetes mellitus. METHODS: A systematic review of articles published between January 2010 and July 2020 was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Manuscripts were identified through searching the databases Web of Science, Scopus, and PubMed as well as through hand searching. Manuscripts were included if they involved the measurement of diabetes-related parameters such as blood glucose level, performed physical activity, or feet condition via wearable or mobile devices. The quality of the included studies was assessed using the Newcastle-Ottawa Scale. RESULTS: The search yielded 1981 articles. A total of 26 publications met the eligibility criteria and were included in the review. Studies predominantly used wearable devices to monitor diabetes-related parameters. The accelerometer was by far the most used sensor, followed by the glucose monitor and heart rate monitor. Most studies applied some type of processing to the collected data, mainly consisting of statistical analysis or machine learning for activity recognition, finding associations among health outcomes, and diagnosing conditions related to diabetes. Few studies have focused on type 2 diabetes, even when this is the most prevalent type and the only preventable one. None of the studies focused on common diabetes complications. Clinical trials were fairly limited or nonexistent in most of the studies, with a common lack of detail about cohorts and case selection, comparability, and outcomes. Explicit endorsement by ethics committees or review boards was missing in most studies. Privacy or security issues were seldom addressed, and even if they were addressed, they were addressed at a rather insufficient level. CONCLUSIONS: The use of mobile and wearable devices for the monitoring of diabetes-related parameters shows early promise. Its development can benefit patients with diabetes, health care professionals, and researchers. However, this field is still in its early stages. Future work must pay special attention to privacy and security issues, the use of new emerging sensor technologies, the combination of mobile and clinical data, and the development of validated clinical trials.

mlCAF: Multi-Level Cross-Domain Semantic Context Fusioning for Behavior Identification.

The emerging research on automatic identification of user's contexts from the cross-domain environment in ubiquitous and pervasive computing systems has proved to be successful. Monitoring the diversified user's contexts and behaviors can help in controlling lifestyle associated to chronic diseases using context-aware applications. However, availability of cross-domain heterogeneous contexts provides a challenging opportunity for their fusion to obtain abstract information for further analysis. This work demonstrates extension of our previous work from a single domain (i.e., physical activity) to multiple domains (physical activity, nutrition and clinical) for context-awareness. We propose multi-level Context-aware Framework (mlCAF), which fuses the multi-level cross-domain contexts in order to arbitrate richer behavioral contexts. This work explicitly focuses on key challenges linked to multi-level context modeling, reasoning and fusioning based on the mlCAF open-source ontology. More specifically, it addresses the interpretation of contexts from three different domains, their fusioning conforming to richer contextual information. This paper contributes in terms of ontology evolution with additional domains, context definitions, rules and inclusion of semantic queries. For the framework evaluation, multi-level cross-domain contexts collected from 20 users were used to ascertain abstract contexts, which served as basis for behavior modeling and lifestyle identification. The experimental results indicate a context recognition average accuracy of around 92.65% for the collected cross-domain contexts.

Ontology-Based High-Level Context Inference for Human Behavior Identification.

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is required to scale to long-term scenarios with a large number of users.

Human Behavior Analysis by Means of Multimodal Context Mining.

There is sufficient evidence proving the impact that negative lifestyle choices have on people's health and wellness. Changing unhealthy behaviours requires raising people's self-awareness and also providing healthcare experts with a thorough and continuous description of the user's conduct. Several monitoring techniques have been proposed in the past to track users' behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous sensory data through machine learning techniques. Low-level contexts are combined using ontological mechanisms to derive a more abstract representation of the user's context, here referred to as high-level context. An initial implementation of the proposed framework supporting real-time context identification is also presented. The developed system is evaluated for various realistic scenarios making use of a novel multimodal context open dataset and data on-the-go, demonstrating prominent context-aware capabilities at both low and high levels.

Design, implementation and validation of a novel open framework for agile development of mobile health applications.

The delivery of healthcare services has experienced tremendous changes during the last years. Mobile health or mHealth is a key engine of advance in the forefront of this revolution. Although there exists a growing development of mobile health applications, there is a lack of tools specifically devised for their implementation. This work presents mHealthDroid, an open source Android implementation of a mHealth Framework designed to facilitate the rapid and easy development of mHealth and biomedical apps. The framework is particularly planned to leverage the potential of mobile devices such as smartphones or tablets, wearable sensors and portable biomedical systems. These devices are increasingly used for the monitoring and delivery of personal health care and wellbeing. The framework implements several functionalities to support resource and communication abstraction, biomedical data acquisition, health knowledge extraction, persistent data storage, adaptive visualization, system management and value-added services such as intelligent alerts, recommendations and guidelines. An exemplary application is also presented along this work to demonstrate the potential of mHealthDroid. This app is used to investigate on the analysis of human behavior, which is considered to be one of the most prominent areas in mHealth. An accurate activity recognition model is developed and successfully validated in both offline and online conditions.

mDurance: A Novel Mobile Health System to Support Trunk Endurance Assessment.

Low back pain is the most prevalent musculoskeletal condition. This disorder constitutes one of the most common causes of disability worldwide, and as a result, it has a severe socioeconomic impact. Endurance tests are normally considered in low back pain rehabilitation practice to assess the muscle status. However, traditional procedures to evaluate these tests suffer from practical limitations, which potentially lead to inaccurate diagnoses. The use of digital technologies is considered here to facilitate the task of the expert and to increase the reliability and interpretability of the endurance tests. This work presents mDurance, a novel mobile health system aimed at supporting specialists in the functional assessment of trunk endurance by using wearable and mobile devices. The system employs a wearable inertial sensor to track the patient trunk posture, while portable electromyography sensors are used to seamlessly measure the electrical activity produced by the trunk muscles. The information registered by the sensors is processed and managed by a mobile application that facilitates the expert's normal routine, while reducing the impact of human errors and expediting the analysis of the test results. In order to show the potential of the mDurance system, a case study has been conducted. The results of this study prove the reliability of mDurance and further demonstrate that practitioners are certainly interested in the regular use of a system of this nature.

Mining human behavior for health promotion.

The monitoring of human lifestyles has gained much attention in the recent years. This work presents a novel approach to combine multiple context-awareness technologies for the automatic analysis of people's conduct in a comprehensive and holistic manner. Activity recognition, emotion recognition, location detection, and social analysis techniques are integrated with ontological mechanisms as part of a framework to identify human behavior. Key architectural components, methods and evidences are described in this paper to illustrate the interest of the proposed approach.

PhysioDroid: combining wearable health sensors and mobile devices for a ubiquitous, continuous, and personal monitoring.

Technological advances on the development of mobile devices, medical sensors, and wireless communication systems support a new generation of unobtrusive, portable, and ubiquitous health monitoring systems for continuous patient assessment and more personalized health care. There exist a growing number of mobile apps in the health domain; however, little contribution has been specifically provided, so far, to operate this kind of apps with wearable physiological sensors. The PhysioDroid, presented in this paper, provides a personalized means to remotely monitor and evaluate users' conditions. The PhysioDroid system provides ubiquitous and continuous vital signs analysis, such as electrocardiogram, heart rate, respiration rate, skin temperature, and body motion, intended to help empower patients and improve clinical understanding. The PhysioDroid is composed of a wearable monitoring device and an Android app providing gathering, storage, and processing features for the physiological sensor data. The versatility of the developed app allows its use for both average users and specialists, and the reduced cost of the PhysioDroid puts it at the reach of most people. Two exemplary use cases for health assessment and sports training are presented to illustrate the capabilities of the PhysioDroid. Next technical steps include generalization to other mobile platforms and health monitoring devices.