Validity and Feasibility of the Monitoring and Modeling Family Eating Dynamics System to Automatically Detect In-field Family Eating Behavior: Observational Study.

BACKGROUND: The field of dietary assessment has a long history, marked by both controversies and advances. Emerging technologies may be a potential solution to address the limitations of self-report dietary assessment methods. The Monitoring and Modeling Family Eating Dynamics (M2FED) study uses wrist-worn smartwatches to automatically detect real-time eating activity in the field. The ecological momentary assessment (EMA) methodology was also used to confirm whether eating occurred (ie, ground truth) and to measure other contextual information, including positive and negative affect, hunger, satiety, mindful eating, and social context. OBJECTIVE: This study aims to report on participant compliance (feasibility) to the 2 distinct EMA protocols of the M2FED study (hourly time-triggered and eating event-triggered assessments) and on the performance (validity) of the smartwatch algorithm in automatically detecting eating events in a family-based study. METHODS: In all, 20 families (58 participants) participated in the 2-week, observational, M2FED study. All participants wore a smartwatch on their dominant hand and responded to time-triggered and eating event-triggered mobile questionnaires via EMA while at home. Compliance to EMA was calculated overall, for hourly time-triggered mobile questionnaires, and for eating event-triggered mobile questionnaires. The predictors of compliance were determined using a logistic regression model. The number of true and false positive eating events was calculated, as well as the precision of the smartwatch algorithm. The Mann-Whitney U test, Kruskal-Wallis test, and Spearman rank correlation were used to determine whether there were differences in the detection of eating events by participant age, gender, family role, and height. RESULTS: The overall compliance rate across the 20 deployments was 89.26% (3723/4171) for all EMAs, 89.7% (3328/3710) for time-triggered EMAs, and 85.7% (395/461) for eating event-triggered EMAs. Time of day (afternoon odds ratio [OR] 0.60, 95% CI 0.42-0.85; evening OR 0.53, 95% CI 0.38-0.74) and whether other family members had also answered an EMA (OR 2.07, 95% CI 1.66-2.58) were significant predictors of compliance to time-triggered EMAs. Weekend status (OR 2.40, 95% CI 1.25-4.91) and deployment day (OR 0.92, 95% CI 0.86-0.97) were significant predictors of compliance to eating event-triggered EMAs. Participants confirmed that 76.5% (302/395) of the detected events were true eating events (ie, true positives), and the precision was 0.77. The proportion of correctly detected eating events did not significantly differ by participant age, gender, family role, or height (P>.05). CONCLUSIONS: This study demonstrates that EMA is a feasible tool to collect ground-truth eating activity and thus evaluate the performance of wearable sensors in the field. The combination of a wrist-worn smartwatch to automatically detect eating and a mobile device to capture ground-truth eating activity offers key advantages for the user and makes mobile health technologies more accessible to nonengineering behavioral researchers.

iWash: A smartwatch handwashing quality assessment and reminder system with real-time feedback in the context of infectious disease.

Washing hands properly and frequently is the simplest and most cost-effective interventions to prevent the spread of infectious diseases. People are often ignorant about proper handwashing in different situations and do not know if they wash hands properly. Smartwatches are found to be effective for assessing the quality of handwashing. However, the existing smartwatch based systems are not comprehensive enough in terms of achieving accuracy as well as reminding people to handwash and providing feedback to the user about the quality of handwashing. On-device processing is often required to provide real-time feedback to the user, and so it is important to develop a system that runs efficiently on low-resource devices like smartwatches. However, none of the existing systems for handwashing quality assessment are optimized for on-device processing. We present iWash, a comprehensive system for quality assessment and context-aware reminders for handwashing with real-time feedback using smartwatches. iWash is a hybrid deep neural network based system that is optimized for on-device processing to ensure high accuracy with minimal processing time and battery usage. Additionally, it is a context-aware system that detects when the user is entering home using a Bluetooth beacon and provides reminders to wash hands. iWash also offers touch-free interaction between the user and the smartwatch that minimizes the risk of germ transmission. We collected a real-life dataset and conducted extensive evaluations to demonstrate the performance of iWash. Compared to existing handwashing quality assessment systems, we achieve around 12% higher accuracy for quality assessment, as well as we reduce the processing time and battery usage by around 37% and 10%, respectively.

Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review.

Dietary intake, eating behaviors, and context are important in chronic disease development, yet our ability to accurately assess these in research settings can be limited by biased traditional self-reporting tools. Objective measurement tools, specifically, wearable sensors, present the opportunity to minimize the major limitations of self-reported eating measures by generating supplementary sensor data that can improve the validity of self-report data in naturalistic settings. This scoping review summarizes the current use of wearable devices/sensors that automatically detect eating-related activity in naturalistic research settings. Five databases were searched in December 2019, and 618 records were retrieved from the literature search. This scoping review included N = 40 studies (from 33 articles) that reported on one or more wearable sensors used to automatically detect eating activity in the field. The majority of studies (N = 26, 65%) used multi-sensor systems (incorporating > 1 wearable sensors), and accelerometers were the most commonly utilized sensor (N = 25, 62.5%). All studies (N = 40, 100.0%) used either self-report or objective ground-truth methods to validate the inferred eating activity detected by the sensor(s). The most frequently reported evaluation metrics were Accuracy (N = 12) and F1-score (N = 10). This scoping review highlights the current state of wearable sensors' ability to improve upon traditional eating assessment methods by passively detecting eating activity in naturalistic settings, over long periods of time, and with minimal user interaction. A key challenge in this field, wide variation in eating outcome measures and evaluation metrics, demonstrates the need for the development of a standardized form of comparability among sensors/multi-sensor systems and multidisciplinary collaboration.

Sensing eating mimicry among family members.

Family relationships influence eating behavior and health outcomes (e.g., obesity). Because eating is often habitual (i.e., automatically driven by external cues), unconscious behavioral mimicry may be a key interpersonal influence mechanism for eating within families. This pilot study extends existing literature on eating mimicry by examining whether multiple family members mimicked each other's bites during natural meals. Thirty-three participants from 10 families were videotaped while eating an unstructured family meal in a kitchen lab setting. Videotapes were coded for participants' bite occurrences and times. We tested whether the likelihood of a participant taking a bite increased when s/he was externally cued by a family eating partner who had recently taken a bite (i.e., bite mimicry). A paired-sample t-test indicated that participants had a significantly faster eating rate within the 5 s following a bite by their eating partner, compared to their bite rate at other times (t = 7.32, p < .0001). Nonparametric permutation testing identified five of 78 dyads in which there was significant evidence of eating mimicry; and 19 of 78 dyads that had p values < .1. This pilot study provides preliminary evidence that suggests eating mimicry may occur among a subset of family members, and that there may be types of family ties more prone to this type of interpersonal influence during meals.

Behavioral patterns of older-adults in assisted living.

In this paper, we examine at-home activity rhythms and present a dozen of behavioral patterns obtained from an activity monitoring pilot study of 22 residents in an assisted living setting with four case studies. Established behavioral patterns have been captured using custom software based on a statistical predictive algorithm that models circadian activity rhythms (CARs) and their deviations. The CAR was statistically estimated based on the average amount of time a resident spent in each room within their assisted living apartment, and also on the activity level given by the average number of motion events per room. A validated in-home monitoring system (IMS) recorded the monitored resident's movement data and established the occupancy period and activity level for each room. Using these data, residents' circadian behaviors were extracted, deviations indicating anomalies were detected, and the latter were correlated to activity reports generated by the IMS as well as notes of the facility's professional caregivers on the monitored residents. The system could be used to detect deviations in activity patterns and to warn caregivers of such deviations, which could reflect changes in health status, thus providing caregivers with the opportunity to apply standard of care diagnostics and to intervene in a timely manner.

A Weakly Supervised Learning Framework for Detecting Social Anxiety and Depression.

Although social anxiety and depression are common, they are often underdiagnosed and undertreated, in part due to difficulties identifying and accessing individuals in need of services. Current assessments rely on client self-report and clinician judgment, which are vulnerable to social desirability and other subjective biases. Identifying objective, nonburdensome markers of these mental health problems, such as features of speech, could help advance assessment, prevention, and treatment approaches. Prior research examining speech detection methods has focused on fully supervised learning approaches employing strongly labeled data. However, strong labeling of individuals high in symptoms or state affect in speech audio data is impractical, in part because it is not possible to identify with high confidence which regions of a long speech indicate the person's symptoms or affective state. We propose a weakly supervised learning framework for detecting social anxiety and depression from long audio clips. Specifically, we present a novel feature modeling technique named NN2Vec that identifies and exploits the inherent relationship between speakers' vocal states and symptoms/affective states. Detecting speakers high in social anxiety or depression symptoms using NN2Vec features achieves F-1 scores 17% and 13% higher than those of the best available baselines. In addition, we present a new multiple instance learning adaptation of a BLSTM classifier, named BLSTM-MIL. Our novel framework of using NN2Vec features with the BLSTM-MIL classifier achieves F-1 scores of 90.1% and 85.44% in detecting speakers high in social anxiety and depression symptoms.

Energy management in sensor networks.

This paper presents a holistic view of energy management in sensor networks. We first discuss hardware designs that support the life cycle of energy, namely: (i) energy harvesting, (ii) energy storage and (iii) energy consumption and control. Then, we discuss individual software designs that manage energy consumption in sensor networks. These energy-aware designs include media access control, routing, localization and time-synchronization. At the end of this paper, we present a case study of the VigilNet system to explain how to integrate various types of energy management techniques to achieve collaborative energy savings in a large-scale deployed military surveillance system.