Contextual AI Journaling: Integrating LLM and Time Series Behavioral Sensing Technology to Promote Self-Reflection and Well-being using the MindScape App.

MindScape aims to study the benefits of integrating time series behavioral patterns (e.g., conversational engagement, sleep, location) with Large Language Models (LLMs) to create a new form of contextual AI journaling, promoting self-reflection and well-being. We argue that integrating behavioral sensing in LLMs will likely lead to a new frontier in AI. In this Late-Breaking Work paper, we discuss the MindScape contextual journal App design that uses LLMs and behavioral sensing to generate contextual and personalized journaling prompts crafted to encourage self-reflection and emotional development. We also discuss the MindScape study of college students based on a preliminary user study and our upcoming study to assess the effectiveness of contextual AI journaling in promoting better well-being on college campuses. MindScape represents a new application class that embeds behavioral intelligence in AI.

MeshID: Few-Shot Finger Gesture Based User Identification Using Orthogonal Signal Interference.

Radio frequency (RF) technology has been applied to enable advanced behavioral sensing in human-computer interaction. Due to its device-free sensing capability and wide availability on Internet of Things devices. Enabling finger gesture-based identification with high accuracy can be challenging due to low RF signal resolution and user heterogeneity. In this paper, we propose MeshID, a novel RF-based user identification scheme that enables identification through finger gestures with high accuracy. MeshID significantly improves the sensing sensitivity on RF signal interference, and hence is able to extract subtle individual biometrics through velocity distribution profiling (VDP) features from less-distinct finger motions such as drawing digits in the air. We design an efficient few-shot model retraining framework based on first component reverse module, achieving high model robustness and performance in a complex environment. We conduct comprehensive real-world experiments and the results show that MeshID achieves a user identification accuracy of 95.17% on average in three indoor environments. The results indicate that MeshID outperforms the state-of-the-art in identification performance with less cost.

Turning data into better mental health: Past, present, and future.

In this mini-review, we discuss the fundamentals of using technology in mental health diagnosis and tracking. We highlight those principles using two clinical concepts: (1) cravings and relapse in the context of addictive disorders and (2) anhedonia in the context of depression. This manuscript is useful for both clinicians wanting to understand the scope of technology use in psychiatry and for computer scientists and engineers wishing to assess psychiatric frameworks useful for diagnosis and treatment. The increase in smartphone ownership and internet connectivity, as well as the accelerated development of wearable devices, have made the observation and analysis of human behavior patterns possible. This has, in turn, paved the way to understand mental health conditions better. These technologies have immense potential in facilitating the diagnosis and tracking of mental health conditions; they also allow the implementation of existing behavioral treatments in new contexts (e.g., remotely, online, and in rural/underserved areas), and the possibility to develop new treatments based on new understanding of behavior patterns. The path to understand how to best use technology in mental health includes the need to match interdisciplinary frameworks from engineering/computer sciences and psychiatry. Thus, we start our review by introducing bio-behavioral sensing, the types of information available, and what behavioral patterns they may reflect and be related to in psychiatric diagnostic frameworks. This information is linked to the use of functional imaging, highlighting how imaging modalities can be considered "ground truth" for mental health/psychiatric dimensions, given the heterogeneity of clinical presentations, and the difficulty of determining what symptom corresponds to what disease. We then discuss how mental health/psychiatric dimensions overlap, yet differ from, psychiatric diagnoses. Using two clinical examples, we highlight the potential agreement areas in assessment/management of anhedonia and cravings. These two dimensions were chosen because of their link to two very prevalent diseases worldwide: depression and addiction. Anhedonia is a core symptom of depression, which is one of the leading causes of disability worldwide. Cravings, the urge to use a substance or perform an action (e.g., shopping, internet), is the leading step before relapse. Lastly, through the manuscript, we discuss potential mental health dimensions.

Smartphone sensing of social interactions in people with and without schizophrenia.

Social impairment is a cardinal feature of schizophrenia spectrum disorders (SZ). Smaller social network size, diminished social skills, and loneliness are highly prevalent. Existing, gold-standard assessments of social impairment in SZ often rely on self-reported information that depends on retrospective recall and detailed accounts of complex social behaviors. This is particularly problematic in people with SZ given characteristic cognitive impairments and reduced insight. Ecological Momentary Assessment (EMA; repeated self-reports completed in the context of daily life) allows for the measurement of social behavior as it occurs in vivo, yet still relies on participant input. Momentary characterization of behavior using smartphone sensors (e.g., GPS, microphone) may also provide ecologically valid indicators of social functioning. In the current study we tested associations between both active (e.g., EMA-reported number of interactions) and passive (GPS-based mobility, conversations captured by microphone) smartphone-based measures of social activity and measures of social functioning and loneliness to examine the promise of such measures for understanding social impairment in SZ. Our results indicate that passive markers of mobility were more consistently associated with EMA measures of social behavior in controls than in people with SZ. Furthermore, dispositional loneliness showed associations with mobility metrics in both groups, while general social functioning was less related to these metrics. Finally, interactions detected in the ambient audio were more tied to social functioning in SZ than in controls. Findings speak to the promise of smartphone-based digital phenotyping as an approach to understanding objective markers of social activity in people with and without schizophrenia.

Face and Gesture Analysis for Health Informatics.

The goal of Face and Gesture Analysis for Health Informatics's workshop is to share and discuss the achievements as well as the challenges in using computer vision and machine learning for automatic human behavior analysis and modeling for clinical research and healthcare applications. The workshop aims to promote current research and support growth of multidisciplinary collaborations to advance this groundbreaking research. The meeting gathers scientists working in related areas of computer vision and machine learning, multi-modal signal processing and fusion, human centered computing, behavioral sensing, assistive technologies, and medical tutoring systems for healthcare applications and medicine.

Development and Evaluation of a Smartphone-Based Measure of Social Rhythms for Bipolar Disorder.

Dynamic psychological processes are most often assessed using self-report instruments. This places a constraint on how often and for how long data can be collected due to the burden placed on human participants. Smartphones are ubiquitous and highly personal devices, equipped with sensors that offer an opportunity to measure and understand psychological processes in real-world contexts over the long term. In this article, we present a novel smartphone approach to address the limitations of self-report in bipolar disorder where mood and activity are key constructs. We describe the development of MoodRhythm, a smartphone application that incorporates existing self-report elements from interpersonal and social rhythm therapy, a clinically validated treatment, and combines them with novel inputs from smartphone sensors. We reflect on lessons learned in transitioning from an existing self-report instrument to one that involves smartphone sensors and discuss the potential impact of these changes on the future of psychological assessment.