Digital Phenotyping of Mental Health using multimodal sensing of multiple situations of interest: A Systematic Literature Review.

Many studies have used Digital Phenotyping of Mental Health (DPMH) to complement classic methods of mental health assessment and monitoring. This research area proposes innovative methods that perform multimodal sensing of multiple situations of interest (e.g., sleep, physical activity, mobility) to health professionals. In this paper, we present a Systematic Literature Review (SLR) to recognize, characterize and analyze the state of the art on DPMH using multimodal sensing of multiple situations of interest to professionals. We searched for studies in six digital libraries, which resulted in 1865 retrieved published papers. Next, we performed a systematic process of selecting studies based on inclusion and exclusion criteria, which selected 59 studies for the data extraction phase. First, based on the analysis of the extracted data, we describe an overview of this field, then presenting characteristics of the selected studies, the main mental health topics targeted, the physical and virtual sensors used, and the identified situations of interest. Next, we outline answers to research questions, describing the context data sources used to detect situations, the DPMH workflow used for multimodal sensing of situations, and the application of DPMH solutions in the mental health assessment and monitoring process. In addition, we recognize trends presented by DPMH studies, such as the design of solutions for high-level information recognition, association of features with mental states/disorders, classification of mental states/disorders, and prediction of mental states/disorders. We also recognize the main open issues in this research area. Based on the results of this SLR, we conclude that despite the potential and continuous evolution for using these solutions as medical decision support tools, this research area needs more work to overcome technology and methodological rigor issues to adopt proposed solutions in real clinical settings.

Feasibility and Acceptability of an Internet of Things-Enabled Sedentary Behavior Intervention: Mixed Methods Study.

BACKGROUND: Encouraging office workers to break up prolonged sedentary behavior (SB) at work with regular microbreaks can be beneficial yet challenging. The Internet of Things (IoT) offers great promise for delivering more subtle and hence acceptable behavior change interventions in the workplace. We previously developed an IoT-enabled SB intervention, called WorkMyWay, by applying a combination of theory-informed and human-centered design approaches. According to the Medical Research Council's framework for developing and evaluating complex interventions such as WorkMyWay, process evaluation in the feasibility phase can help establish the viability of novel modes of delivery and identify facilitators and barriers to successful delivery. OBJECTIVE: This study aims to evaluate the feasibility and acceptability of the WorkMyWay intervention and its technological delivery system. METHODS: A mixed methods approach was adopted. A sample of 15 office workers were recruited to use WorkMyWay during work hours for 6 weeks. Questionnaires were administered before and after the intervention period to assess self-report occupational sitting and physical activity (OSPA) and psychosocial variables theoretically aligned with prolonged occupational SB (eg, intention, perceived behavioral control, prospective memory and retrospective memory of breaks, and automaticity of regular break behaviors). Behavioral and interactional data were obtained through the system database to determine adherence, quality of delivery, compliance, and objective OSPA. Semistructured interviews were conducted at the end of the study, and a thematic analysis was performed on interview transcripts. RESULTS: All 15 participants completed the study (attrition=0%) and on average used the system for 25 tracking days (out of a possible 30 days; adherence=83%). Although no significant change was observed in either objective or self-report OSPA, postintervention improvements were significant in the automaticity of regular break behaviors (t14=2.606; P=.02), retrospective memory of breaks (t14=7.926; P<.001), and prospective memory of breaks (t14=-2.661; P=.02). The qualitative analysis identified 6 themes, which lent support to the high acceptability of WorkMyWay, though delivery was compromised by issues concerning Bluetooth connectivity and factors related to user behaviors. Fixing technical issues, tailoring to individual differences, soliciting organizational supports, and harnessing interpersonal influences could facilitate delivery and enhance acceptance. CONCLUSIONS: It is acceptable and feasible to deliver an SB intervention with an IoT system that involves a wearable activity tracking device, an app, and a digitally augmented everyday object (eg, cup). More industrial design and technological development work on WorkMyWay is warranted to improve delivery. Future research should seek to establish the broad acceptability of similar IoT-enabled interventions while expanding the range of digitally augmented objects as the modes of delivery to meet diverse needs.

Domain Adaptation Methods for Lab-to-Field Human Context Recognition.

Human context recognition (HCR) using sensor data is a crucial task in Context-Aware (CA) applications in domains such as healthcare and security. Supervised machine learning HCR models are trained using smartphone HCR datasets that are scripted or gathered in-the-wild. Scripted datasets are most accurate because of their consistent visit patterns. Supervised machine learning HCR models perform well on scripted datasets but poorly on realistic data. In-the-wild datasets are more realistic, but cause HCR models to perform worse due to data imbalance, missing or incorrect labels, and a wide variety of phone placements and device types. Lab-to-field approaches learn a robust data representation from a scripted, high-fidelity dataset, which is then used for enhancing performance on a noisy, in-the-wild dataset with similar labels. This research introduces Triplet-based Domain Adaptation for Context REcognition (Triple-DARE), a lab-to-field neural network method that combines three unique loss functions to enhance intra-class compactness and inter-class separation within the embedding space of multi-labeled datasets: (1) domain alignment loss in order to learn domain-invariant embeddings; (2) classification loss to preserve task-discriminative features; and (3) joint fusion triplet loss. Rigorous evaluations showed that Triple-DARE achieved 6.3% and 4.5% higher F1-score and classification, respectively, than state-of-the-art HCR baselines and outperformed non-adaptive HCR models by 44.6% and 10.7%, respectively.

ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation.

Although voice authentication is generally secure, voiceprint-based authentication methods have the drawback of being affected by environmental noise, long passphrases, and large registered samples. Therefore, we present a breakthrough idea for smartphone user authentication by analyzing articulation and integrating the physiology and behavior of the vocal tract, tongue position, and lip movement to expose the uniqueness of individuals while making utterances. The key idea is to leverage the smartphone speaker and microphone to simultaneously transmit and receive speech and ultrasonic signals, construct identity-related features, and determine whether a single utterance is a legitimate user or an attacker. Physiological authentication methods prevent other users from copying or reproducing passwords. Compared to other types of behavioral authentication, the system is more accurately able to recognize the user's identity and adapt accordingly to environmental variations. The proposed system requires a smaller number of samples because single utterances are utilized, resulting in a user-friendly system that resists mimicry attacks with an average accuracy of 99% and an equal error rate of 0.5% under the three different surroundings.

WiCHORD+: A Scalable, Sustainable, and P2P Chord-Based Ecosystem for Smart Agriculture Applications.

In the evolving landscape of Industry 4.0, the convergence of peer-to-peer (P2P) systems, LoRa-enabled wireless sensor networks (WSNs), and distributed hash tables (DHTs) represents a major advancement that enhances sustainability in the modern agriculture framework and its applications. In this study, we propose a P2P Chord-based ecosystem for sustainable and smart agriculture applications, inspired by the inner workings of the Chord protocol. The node-centric approach of WiCHORD+ is a standout feature, streamlining operations in WSNs and leading to more energy-efficient and straightforward system interactions. Instead of traditional key-centric methods, WiCHORD+ is a node-centric protocol that is compatible with the inherent characteristics of WSNs. This unique design integrates seamlessly with distributed hash tables (DHTs), providing an efficient mechanism to locate nodes and ensure robust data retrieval while reducing energy consumption. Additionally, by utilizing the MAC address of each node in data routing, WiCHORD+ offers a more direct and efficient data lookup mechanism, essential for the timely and energy-efficient operation of WSNs. While the increasing dependence of smart agriculture on cloud computing environments for data storage and machine learning techniques for real-time prediction and analytics continues, frameworks like the proposed WiCHORD+ appear promising for future IoT applications due to their compatibility with modern devices and peripherals. Ultimately, the proposed approach aims to effectively incorporate LoRa, WSNs, DHTs, cloud computing, and machine learning, by providing practical solutions to the ongoing challenges in the current smart agriculture landscape and IoT applications.

Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances.

Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human-computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.

A Comprehensive Review of Internet of Things: Technology Stack, Middlewares, and Fog/Edge Computing Interface.

The Internet of Things (IoT) is an extensive network of heterogeneous devices that provides an array of innovative applications and services. IoT networks enable the integration of data and services to seamlessly interconnect the cyber and physical systems. However, the heterogeneity of devices, underlying technologies and lack of standardization pose critical challenges in this domain. On account of these challenges, this research article aims to provide a comprehensive overview of the enabling technologies and standards that build up the IoT technology stack. First, a layered architecture approach is presented where the state-of-the-art research and open challenges are discussed at every layer. Next, this research article focuses on the role of middleware platforms in IoT application development and integration. Furthermore, this article addresses the open challenges and provides comprehensive steps towards IoT stack optimization. Finally, the interfacing of Fog/Edge Networks to IoT technology stack is thoroughly investigated by discussing the current research and open challenges in this domain. The main scope of this study is to provide a comprehensive review into IoT technology (the horizontal fabric), the associated middleware and networks required to build future proof applications (the vertical markets).

Mental health ubiquitous monitoring supported by social situation awareness: A systematic review.

Traditionally, the process of monitoring and evaluating social behavior related to mental health has based on self-reported information, which is limited by the subjective character of responses and various cognitive biases. Today, however, there is a growing amount of studies that have provided methods to objectively monitor social behavior through ubiquitous devices and have used this information to support mental health services. In this paper, we present a Systematic Literature Review (SLR) to identify, analyze and characterize the state of the art about the use of ubiquitous devices to monitor users' social behavior focused on mental health. For this purpose, we performed an exhaustive literature search on the six main digital libraries. A screening process was conducted on 160 peer-reviewed publications by applying suitable selection criteria to define the appropriate studies to the scope of this SLR. Next, 20 selected studies were forwarded to the data extraction phase. From an analysis of the selected studies, we recognized the types of social situations identified, the process of transforming contextual data into social situations, the use of social situation awareness to support mental health monitoring, and the methods used to evaluate proposed solutions. Additionally, we identified the main trends presented by this research area, as well as open questions and perspectives for future research. Results of this SLR showed that social situation-aware ubiquitous systems represent promising assistance tools for patients and mental health professionals. However, studies still present limitations in methodological rigor and restrictions in experiments, and solutions proposed by them have limitations to be overcome.

Evolving dynamic self-adaptation policies of mHealth systems for long-term monitoring.

Tele-rehabilitation can complement traditional rehabilitation therapies by providing valuable information that can help in the evaluation, monitoring, and treatment of patients. Many patient tele-monitoring systems that integrate wearable technology are emerging as an effective tool for the long-term surveillance of rehabilitation progression, enabling continuous sampling of patient real-time movement in a non-invasive way, without affecting the normal daily activity of the outpatient, who, therefore, will not need to make frequent clinic visits. One of the main challenges of tele-rehabilitation systems is to pay special attention to the diversity of dysfunctions in patients by offering devices with customized behaviours adaptable to the physical conditions of each patient at the different stages of the rehabilitation therapy. Long-term monitoring systems need an adaptation policy to autonomously reconfigure their behaviour according to vital signs read during the physical activity of the patient, the remaining battery level, or the required accuracy of collected data. However, it would alsobe desirable to adjust such adaptation policies over time, according to the patient's evolution. This work presents a wearable patient-monitoring system for tele-rehabilitation that is able to dynamically self-configure its internal behaviour to the current context of the outpatient according to a set of adaptation policies that optimize battery consumption, taking into account other QoS parameters at the same time. Our system is also able to self-adapt its internal adaptation policies as a patient's condition improves, while maintaining the system's efficiency. We illustrate our proposal with a real mHealth case study. The results of the experiments show that the system updates the adaptation policies, taking into account specific indicators of the disease. The validation results show that the evolution of the self-adaptation policies correlates with the progression of different patients.

Internet of Things for Smart Spaces: A University Campus Case Study.

Advances in technology and data analysis provide rich opportunities for developing intelligent environments assisting their inhabitants, so-called smart environments or smart spaces. Enhanced with technology, sensors, user interfaces, and various applications, such smart spaces are capable of recognizing users and situations they are in, react accordingly, e.g., by providing certain services or changes to the environment itself. Therefore, smart space solutions are gradually coming to different application domains, each with corresponding specific characteristics. In this article, we discuss our experiences and explore the challenges of a long-term real-world Internet of Things (IoT) deployment at a University campus. We demonstrate the technical implementation and data quality issues. We conduct several studies, from data analysis to interaction with space, utilizing the developed infrastructure, and we also share our actions to open the data for education purposes and discuss their outcomes. With this article, we aim to share our experience and provide real-world lessons learned when building an open, multipurpose, publicly used smart space at a University campus.

Understanding the complexities of Bluetooth for representing real-life social networks: A methodology for inferring and validating Bluetooth-based social network graphs.

Bluetooth (BT) data has been extensively used for recognizing social patterns and inferring social networks, as BT is widely present in everyday technological devices. However, even though collecting BT data is subject to random noise and may result in substantial measurement errors, there is an absence of rigorous procedures for validating the quality of the inferred BT social networks. This paper presents a methodology for inferring and validating BT-based social networks based on parameter optimization algorithm and social network analysis (SNA). The algorithm performs edge inference in a brute-force search over a given BT data set, for deriving optimal BT social networks by validating them with predefined ground truth (GT) networks. The algorithm seeks to optimize a set of parameters, predefined considering some reliability challenges associated to the BT technology itself. The outcomes show that optimizing the parameters can reduce the number of BT data false positives or generate BT networks with the minimum amount of BT data observations. The subsequent SNA shows that the inferred BT social networks are unable to reproduce some network characteristics present in the corresponding GT networks. Finally, the generalizability of the proposed methodology is demonstrated by applying the algorithm on external BT data sets, while obtaining comparable results.

Wearable activity trackers, accuracy, adoption, acceptance and health impact: A systematic literature review.

Wearable activity trackers (WAT) are electronic monitoring devices that enable users to track and monitor their health-related physical fitness metrics including steps taken, level of activity, walking distance, heart rate, and sleep patterns. Despite the proliferation of these devices in various contexts of use and rising research interests, there is limited understanding of the broad research landscape. The purpose of this systematic review is therefore to synthesize the existing wealth of research on WAT, and to provide a comprehensive summary based on common themes and approaches. This article includes academic work published between 2013 and 2017 in PubMed, Embase, Scopus, Web of Science, ACM Digital Library, and Google Scholar. A final list of 463 articles was analyzed for this review. Topic modeling methods were used to identify six key themes (topics) of WAT research, namely: (1) Technology Focus, (2) Patient Treatment and Medical Settings, (3) Behavior Change, (4) Acceptance and Adoption (Abandonment), (5) Self-monitoring Data Centered, and (6) Privacy. We take an interdisciplinary approach to wearable activity trackers to propose several new research questions. The most important research gap we identify is to attempt to understand the rich human-information interaction that is enabled by WAT adoption.

Mitigating the Impact on Users' Privacy Caused by over Specifications in the Design of IoT Applications.

Privacy has long been an important issue for IT systems that handle personal information, and is further aggravated as technology for collecting and analyzing massive amounts of data is becoming increasingly effective. There are methods to help practitioners analyze the privacy implications of a system during the design time. However, this is still a difficult task, especially when dealing with Internet of Things scenarios. The problem of privacy can become even more unmanageable with the introduction of overspecifications during the system development life cycle. In this paper, we carried out a controlled experiment with students performing an analysis of privacy implications using two different methods. One method aims at reducing the impact of overspecifications through the application of a goal-oriented analysis. The other method does not involve a goal-oriented analysis and is used as a control. Our initial findings show that conducting a goal-oriented analysis early during design time can have a positive impact over the privacy friendliness of the resulting system.

FeinPhone: Low-cost Smartphone Camera-based 2D Particulate Matter Sensor.

Precise, location-specific fine dust measurement is central for the assessment of urban air quality. Classic measurement approaches require dedicated hardware, of which professional equipment is still prohibitively expensive (>10k$) for dense measurements, and inexpensive sensors do not meet accuracy demands. As a step towards filling this gap, we propose FeinPhone, a phone-based fine dust measurement system that uses camera and flashlight functions that are readily available on today's off-the-shelf smart phones. We introduce a cost-effective passive hardware add-on together with a novel counting approach based on light-scattering particle sensors. Since our approach features a 2D sensor (the camera) instead of a single photodiode, we can employ it to capture the scatter traces from individual particles rather than just retaining a light intensity sum signal as in simple photometers. This is a more direct way of assessing the particle count, it is robust against side effects, e.g., from camera image compression, and enables gaining information on the size spectrum of the particles. Our proof-of-concept evaluation comparing several FeinPhone sensors with data from a high-quality APS/SMPS (Aerodynamic Particle Sizer/Scanning Mobility Particle Sizer) reference device at the World Calibration Center for Aerosol Physics shows that the collected data shows excellent correlation with the inhalable coarse fraction of fine dust particles (r > 0.9) and can successfully capture its levels under realistic conditions.

ADLAuth: Passive Authentication Based on Activity of Daily Living Using Heterogeneous Sensing in Smart Cities.

The Internet of Things is a rapidly growing paradigm for smart cities that provides a way of communication, identification, and sensing capabilities among physically distributed devices. With the evolution of the Internet of Things (IoTs), user dependence on smart systems and services, such as smart appliances, smartphone, security, and healthcare applications, has been increased. This demands secure authentication mechanisms to preserve the users' privacy when interacting with smart devices. This paper proposes a heterogeneous framework "ADLAuth" for passive and implicit authentication of the user using either a smartphone's built-in sensor or wearable sensors by analyzing the physical activity patterns of the users. Multiclass machine learning algorithms are applied to users' identity verification. Analyses are performed on three different datasets of heterogeneous sensors for a diverse number of activities. A series of experiments have been performed to test the effectiveness of the proposed framework. The results demonstrate the better performance of the proposed scheme compared to existing work for user authentication.

ProFUSO: Business process and ontology-based framework to develop ubiquitous computing support systems for chronic patients' management.

New advances in telemedicine, ubiquitous computing, and artificial intelligence have supported the emergence of more advanced applications and support systems for chronic patients. This trend addresses the important problem of chronic illnesses, highlighted by multiple international organizations as a core issue in future healthcare. Despite the myriad of exciting new developments, each application and system is designed and implemented for specific purposes and lacks the flexibility to support different healthcare concerns. Some of the known problems of such developments are the integration issues between applications and existing healthcare systems, the reusability of technical knowledge in the creation of new and more sophisticated systems and the usage of data gathered from multiple sources in the generation of new knowledge. This paper proposes a framework for the development of chronic disease support systems and applications as an answer to these shortcomings. Through this framework our pursuit is to create a common ground methodology upon which new developments can be created and easily integrated to provide better support to chronic patients, medical staff and other relevant participants. General requirements are inferred for any support system from the primary attention process of chronic patients by the Business Process Management Notation. Numerous technical approaches are proposed to design a general architecture that considers the medical organizational requirements in the treatment of a patient. A framework is presented for any application in support of chronic patients and evaluated by a case study to test the applicability and pertinence of the solution.

Using Mobile Phone Sensor Technology for Mental Health Research: Integrated Analysis to Identify Hidden Challenges and Potential Solutions.

BACKGROUND: Mobile phone sensor technology has great potential in providing behavioral markers of mental health. However, this promise has not yet been brought to fruition. OBJECTIVE: The objective of our study was to examine challenges involved in developing an app to extract behavioral markers of mental health from passive sensor data. METHODS: Both technical challenges and acceptability of passive data collection for mental health research were assessed based on literature review and results obtained from a feasibility study. Socialise, a mobile phone app developed at the Black Dog Institute, was used to collect sensor data (Bluetooth, location, and battery status) and investigate views and experiences of a group of people with lived experience of mental health challenges (N=32). RESULTS: On average, sensor data were obtained for 55% (Android) and 45% (iOS) of scheduled scans. Battery life was reduced from 21.3 hours to 18.8 hours when scanning every 5 minutes with a reduction of 2.5 hours or 12%. Despite this relatively small reduction, most participants reported that the app had a noticeable effect on their battery life. In addition to battery life, the purpose of data collection, trust in the organization that collects data, and perceived impact on privacy were identified as main factors for acceptability. CONCLUSIONS: Based on the findings of the feasibility study and literature review, we recommend a commitment to open science and transparent reporting and stronger partnerships and communication with users. Sensing technology has the potential to greatly enhance the delivery and impact of mental health care. Realizing this requires all aspects of mobile phone sensor technology to be rigorously assessed.

Smart Device-Based Notifications to Promote Healthy Behavior Related to Childhood Obesity and Overweight.

Obesity is one of the most serious public health challenges of the 21st century and it is a threat to the life of people according to World Health Organization. In this scenario, family environment is important to establish healthy habits which help to reduce levels of obesity and control overweight in children. However, little efforts have been focused on helping parents to promote and have healthy lifestyles. In this paper, we present two smart device-based notification prototypes to promote healthy behavior with the aim of avoiding childhood overweight and obesity. The first prototype helps parents to follow a healthy snack routine, based on a nutritionist suggestion. Using a fridge magnet, parents receive graphical reminders of which snacks they and their children should consume. The second prototype provides a graphical reminder that prevents parents from forgetting the required equipment to practice sports. Prototypes were evaluated by nine nutritionists from three countries (Costa Rica, Mexico and Spain). Evaluations were based on anticipation of use and the ergonomics of human-system interaction according to the ISO 9241-210. Results show that the system is considered useful. Even though they might not be willing to use the system, they would recommend it to their patients. Based on the ISO 9241-210 the best ranked features were the system's comprehensibility, the perceived effectiveness and clarity. The worst ranked features were the system's suitability for learning and its discriminability.

Energy Optimization Using a Case-Based Reasoning Strategy.

At present, the domotization of homes and public buildings is becoming increasingly popular. Domotization is most commonly applied to the field of energy management, since it gives the possibility of managing the consumption of the devices connected to the electric network, the way in which the users interact with these devices, as well as other external factors that influence consumption. In buildings, Heating, Ventilation and Air Conditioning (HVAC) systems have the highest consumption rates. The systems proposed so far have not succeeded in optimizing the energy consumption associated with a HVAC system because they do not monitor all the variables involved in electricity consumption. For this reason, this article presents an agent approach that benefits from the advantages provided by a Multi-Agent architecture (MAS) deployed in a Cloud environment with a wireless sensor network (WSN) in order to achieve energy savings. The agents of the MAS learn social behavior thanks to the collection of data and the use of an artificial neural network (ANN). The proposed system has been assessed in an office building achieving an average energy savings of 41% in the experimental group offices.

Strategies to Improve Activity Recognition Based on Skeletal Tracking: Applying Restrictions Regarding Body Parts and Similarity Boundaries.

This paper aims to improve activity recognition systems based on skeletal tracking through the study of two different strategies (and its combination): (a) specialized body parts analysis and (b) stricter restrictions for the most easily detectable activities. The study was performed using the Extended Body-Angles Algorithm, which is able to analyze activities using only a single key sample. This system allows to select, for each considered activity, which are its relevant joints, which makes it possible to monitor the body of the user selecting only a subset of the same. But this feature of the system has both advantages and disadvantages. As a consequence, in the past we had some difficulties with the recognition of activities that only have a small subset of the joints of the body as relevant. The goal of this work, therefore, is to analyze the effect produced by the application of several strategies on the results of an activity recognition system based on skeletal tracking joint oriented devices. Strategies that we applied with the purpose of improve the recognition rates of the activities with a small subset of relevant joints. Through the results of this work, we aim to give the scientific community some first indications about which considered strategy is better.