Lifelong Adaptive Machine Learning for Sensor-Based Human Activity Recognition Using Prototypical Networks.

Continual learning (CL), also known as lifelong learning, is an emerging research topic that has been attracting increasing interest in the field of machine learning. With human activity recognition (HAR) playing a key role in enabling numerous real-world applications, an essential step towards the long-term deployment of such systems is to extend the activity model to dynamically adapt to changes in people's everyday behavior. Current research in CL applied to the HAR domain is still under-explored with researchers exploring existing methods developed for computer vision in HAR. Moreover, analysis has so far focused on task-incremental or class-incremental learning paradigms where task boundaries are known. This impedes the applicability of such methods for real-world systems. To push this field forward, we build on recent advances in the area of continual learning and design a lifelong adaptive learning framework using Prototypical Networks, LAPNet-HAR, that processes sensor-based data streams in a task-free data-incremental fashion and mitigates catastrophic forgetting using experience replay and continual prototype adaptation. Online learning is further facilitated using contrastive loss to enforce inter-class separation. LAPNet-HAR is evaluated on five publicly available activity datasets in terms of its ability to acquire new information while preserving previous knowledge. Our extensive empirical results demonstrate the effectiveness of LAPNet-HAR in task-free CL and uncover useful insights for future challenges.

Eating Episode Detection with Jawbone-Mounted Inertial Sensing.

Recent work in Automated Dietary Monitoring (ADM) has shown promising results in eating detection by tracking jawbone movements with a proximity sensor mounted on a necklace. A significant challenge with this approach, however, is that motion artifacts introduced by natural body movements cause the necklace to move freely and the sensor to become misaligned. In this paper, we propose a different but related approach: we developed a small wireless inertial sensing platform and perform eating detection by mounting the sensor directly on the underside of the jawbone. We implemented a data analysis pipeline to recognize eating episodes from the inertial sensor data, and evaluated our approach in two different conditions: in the laboratory and in naturalistic settings. We demonstrated that in the lab (n=9), the system can detect eating with 91.7% precision and 91.3% recall using the leave-one-participant-out cross-validation (LOPO-CV) performance metric. In naturalistic settings, we obtained an average precision of 92.3% and a recall of 89.0% (n=14). These results represent a significant improvement (>10% in F1 score) over state-of-the-art necklace-based approaches. Additionally, this work presents a wearable device that is more inconspicuous and thus more likely to be adopted in clinical applications.

Towards a Generalizable Method for Detecting Fluid Intake with Wrist-Mounted Sensors and Adaptive Segmentation.

Over the last decade, advances in mobile technologies have enabled the development of intelligent systems that attempt to recognize and model a variety of health-related human behaviors. While automated dietary monitoring based on passive sensors has been an area of increasing research activity for many years, much less attention has been given to tracking fluid intake. In this work, we apply an adaptive segmentation technique on a continuous stream of inertial data captured with a practical, off-the-shelf wrist-mounted device to detect fluid intake gestures passively. We evaluated our approach in a study with 30 participants where 561 drinking instances were recorded. Using a leave-one-participant-out (LOPO), we were able to detect drinking episodes with 90.3% precision and 91.0% recall, demonstrating the generalizability of our approach. In addition to our proposed method, we also contribute an anonymized and labeled dataset of drinking and non-drinking gestures to encourage further work in the field.