Development of an interactive tool of early social responsiveness to track autism risk in infants and toddlers.

AIM: To evaluate the psychometric properties of a 4-minute assessment designed to identify early autism spectrum disorder (ASD) status through evaluation of early social responsiveness (ESR). METHOD: This retrospective, preliminary study included children between 13 and 24 months (78 males, 79 females mean age 19.4mo, SD 3.1) from two independent data sets (an experimental/training sample [n=120] and a validation/test sample [n=37]). The ESR assessment examined social behaviors (e.g. eye contact, smiling, ease-of-social-engagement) across five common play activities (e.g. rolling a ball, looking at a book). Data analyses examined reliability and accuracy of the assessment in identifying ESR abilities and in discriminating children with and without ASD. RESULTS: Results indicated adequate internal consistency and test-retest reliability of the ESR assessment. Receiver operator curve analysis identified a cutoff score that discriminated infants with ASD-risk from peers in the training sample. This score yielded moderate sensitivity and high specificity for best-estimate ASD diagnosis in the validation sample. INTERPRETATION: Preliminary findings indicated that brief, systematic observation of ESR may assist in discriminating infants with and without ASD, providing concrete evidence to validate or supplement parents', pediatricians', or clinicians' concerns. Future studies could examine the utility of ESR 'growth curves'.

Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables.

Supervised training of human activity recognition (HAR) systems based on body-worn inertial measurement units (IMUs) is often constrained by the typically rather small amounts of labeled sample data. Systems like IMUTube have been introduced that employ cross-modality transfer approaches to convert videos of activities of interest into virtual IMU data. We demonstrate for the first time how such large-scale virtual IMU datasets can be used to train HAR systems that are substantially more complex than the state-of-the-art. Complexity is thereby represented by the number of model parameters that can be trained robustly. Our models contain components that are dedicated to capture the essentials of IMU data as they are of relevance for activity recognition, which increased the number of trainable parameters by a factor of 1100 compared to state-of-the-art model architectures. We evaluate the new model architecture on the challenging task of analyzing free-weight gym exercises, specifically on classifying 13 dumbbell execises. We have collected around 41 h of virtual IMU data using IMUTube from exercise videos available from YouTube. The proposed model is trained with the large amount of virtual IMU data and calibrated with a mere 36 min of real IMU data. The trained model was evaluated on a real IMU dataset and we demonstrate the substantial performance improvements of 20% absolute F1 score compared to the state-of-the-art convolutional models in HAR.

A Pilot Randomized Trial of Text-Messaging for Symptom Awareness and Diabetes Knowledge in Adolescents With Type 1 Diabetes.

Adolescents with type 1 diabetes typically receive clinical care every 3 months. Between visits, diabetes-related issues may not be frequently reflected, learned, and documented by the patients, limiting their self-awareness and knowledge about their condition. We designed a text-messaging system to help resolve this problem. In a pilot, randomized controlled trial with 30 adolescents, we examined the effect of text messages about symptom awareness and diabetes knowledge on glucose control and quality of life. The intervention group that received more text messages between visits had significant improvements in quality of life.

IntiVisor: A Visual Analytics System for Interaction Log Analysis.

Application developers frequently augment their code to produce event logs of specific operations performed by their users. Subsequent analysis of these event logs can help provide insight about the users' behavior relative to its intended use. The analysis process typically includes both event organization and pattern discovery activities. However, most existing visual analytics systems for interaction log analysis excel at supporting pattern discovery and overlook the importance of flexible event organization. This omission limits the practical application of these systems. Therefore, we developed a novel visual analytics system called IntiVisor that implements the entire end-to-end interaction analysis approach. An evaluation of the system with interaction data from four visualization applications showed the value and importance of supporting event organization in interaction log analysis.

Leveraging Social Media to Predict COVID-19-Induced Disruptions to Mental Well-Being Among University Students: Modeling Study.

BACKGROUND: Large-scale crisis events such as COVID-19 often have secondary impacts on individuals' mental well-being. University students are particularly vulnerable to such impacts. Traditional survey-based methods to identify those in need of support do not scale over large populations and they do not provide timely insights. We pursue an alternative approach through social media data and machine learning. Our models aim to complement surveys and provide early, precise, and objective predictions of students disrupted by COVID-19. OBJECTIVE: This study aims to demonstrate the feasibility of language on private social media as an indicator of crisis-induced disruption to mental well-being. METHODS: We modeled 4124 Facebook posts provided by 43 undergraduate students, spanning over 2 years. We extracted temporal trends in the psycholinguistic attributes of their posts and comments. These trends were used as features to predict how COVID-19 disrupted their mental well-being. RESULTS: The social media-enabled model had an F1-score of 0.79, which was a 39% improvement over a model trained on the self-reported mental state of the participant. The features we used showed promise in predicting other mental states such as anxiety, depression, social, isolation, and suicidal behavior (F1-scores varied between 0.85 and 0.93). We also found that selecting the windows of time 7 months after the COVID-19-induced lockdown presented better results, therefore, paving the way for data minimization. CONCLUSIONS: We predicted COVID-19-induced disruptions to mental well-being by developing a machine learning model that leveraged language on private social media. The language in these posts described psycholinguistic trends in students' online behavior. These longitudinal trends helped predict mental well-being disruption better than models trained on correlated mental health questionnaires. Our work inspires further research into the potential applications of early, precise, and automatic warnings for individuals concerned about their mental health in times of crisis.

Patient Perspectives on AI-Driven Predictions of Schizophrenia Relapses: Understanding Concerns and Opportunities for Self-Care and Treatment.

Early detection and intervention for relapse is important in the treatment of schizophrenia spectrum disorders. Researchers have developed AI models to predict relapse from patient-contributed data like social media. However, these models face challenges, including misalignment with practice and ethical issues related to transparency, accountability, and potential harm. Furthermore, how patients who have recovered from schizophrenia view these AI models has been underexplored. To address this gap, we first conducted semi-structured interviews with 28 patients and reflexive thematic analysis, which revealed a disconnect between AI predictions and patient experience, and the importance of the social aspect of relapse detection. In response, we developed a prototype that used patients' Facebook data to predict relapse. Feedback from seven patients highlighted the potential for AI to foster collaboration between patients and their support systems, and to encourage self-reflection. Our work provides insights into human-AI interaction and suggests ways to empower people with schizophrenia.

Empirical networks for localized COVID-19 interventions using WiFi infrastructure at university campuses.

Infectious diseases, like COVID-19, pose serious challenges to university campuses, which typically adopt closure as a non-pharmaceutical intervention to control spread and ensure a gradual return to normalcy. Intervention policies, such as remote instruction (RI) where large classes are offered online, reduce potential contact but also have broad side-effects on campus by hampering the local economy, students' learning outcomes, and community wellbeing. In this paper, we demonstrate that university policymakers can mitigate these tradeoffs by leveraging anonymized data from their WiFi infrastructure to learn community mobility-a methodology we refer to as WiFi mobility models (WiMob). This approach enables policymakers to explore more granular policies like localized closures (LC). WiMob can construct contact networks that capture behavior in various spaces, highlighting new potential transmission pathways and temporal variation in contact behavior. Additionally, WiMob enables us to design LC policies that close super-spreader locations on campus. By simulating disease spread with contact networks from WiMob, we find that LC maintains the same reduction in cumulative infections as RI while showing greater reduction in peak infections and internal transmission. Moreover, LC reduces campus burden by closing fewer locations, forcing fewer students into completely online schedules, and requiring no additional isolation. WiMob can empower universities to conceive and assess a variety of closure policies to prevent future outbreaks.

Clinician Perspectives on Using Computational Mental Health Insights From Patients' Social Media Activities: Design and Qualitative Evaluation of a Prototype.

BACKGROUND: Previous studies have suggested that social media data, along with machine learning algorithms, can be used to generate computational mental health insights. These computational insights have the potential to support clinician-patient communication during psychotherapy consultations. However, how clinicians perceive and envision using computational insights during consultations has been underexplored. OBJECTIVE: The aim of this study is to understand clinician perspectives regarding computational mental health insights from patients' social media activities. We focus on the opportunities and challenges of using these insights during psychotherapy consultations. METHODS: We developed a prototype that can analyze consented patients' Facebook data and visually represent these computational insights. We incorporated the insights into existing clinician-facing assessment tools, the Hamilton Depression Rating Scale and Global Functioning: Social Scale. The design intent is that a clinician will verbally interview a patient (eg, How was your mood in the past week?) while they reviewed relevant insights from the patient's social media activities (eg, number of depression-indicative posts). Using the prototype, we conducted interviews (n=15) and 3 focus groups (n=13) with mental health clinicians: psychiatrists, clinical psychologists, and licensed clinical social workers. The transcribed qualitative data were analyzed using thematic analysis. RESULTS: Clinicians reported that the prototype can support clinician-patient collaboration in agenda-setting, communicating symptoms, and navigating patients' verbal reports. They suggested potential use scenarios, such as reviewing the prototype before consultations and using the prototype when patients missed their consultations. They also speculated potential negative consequences: patients may feel like they are being monitored, which may yield negative effects, and the use of the prototype may increase the workload of clinicians, which is already difficult to manage. Finally, our participants expressed concerns regarding the prototype: they were unsure whether patients' social media accounts represented their actual behaviors; they wanted to learn how and when the machine learning algorithm can fail to meet their expectations of trust; and they were worried about situations where they could not properly respond to the insights, especially emergency situations outside of clinical settings. CONCLUSIONS: Our findings support the touted potential of computational mental health insights from patients' social media account data, especially in the context of psychotherapy consultations. However, sociotechnical issues, such as transparent algorithmic information and institutional support, should be addressed in future endeavors to design implementable and sustainable technology.

Measuring Self-Esteem with Passive Sensing.

Self-esteem encompasses how individuals evaluate themselves and is an important contributor to their success. Self-esteem has been traditionally measured using survey-based methodologies. However, surveys suffer from limitations such as retrospective recall and reporting biases, leading to a need for proactive measurement approaches. Our work uses smartphone sensors to predict self-esteem and is situated in a multimodal sensing study on college students for five weeks. We use theory-driven features, such as phone communications and physical activity to predict three dimensions, performance, social, and appearance self-esteem. We conduct statistical modeling including linear, ensemble, and neural network regression to measure self-esteem. Our best model predicts self-esteem with a high correlation (r) of 0.60 and low SMAPE of 7.26% indicating high predictive accuracy. We inspect the top features finding theoretical alignment; for example, social interaction significantly contributes to performance and appearance-based self-esteem, whereas, and physical activity is the most significant contributor towards social self-esteem. Our work reveals the efficacy of passive sensors for predicting self-esteem, and we situate our observations with literature and discuss the implications of our work for tailored interventions and improving wellbeing.

Designing a Clinician-Facing Tool for Using Insights From Patients' Social Media Activity: Iterative Co-Design Approach.

BACKGROUND: Recent research has emphasized the need for accessing information about patients to augment mental health patients' verbal reports in clinical settings. Although it has not been introduced in clinical settings, computational linguistic analysis on social media has proved it can infer mental health attributes, implying a potential use as collateral information at the point of care. To realize this potential and make social media insights actionable to clinical decision making, the gaps between computational linguistic analysis on social media and the current work practices of mental health clinicians must be bridged. OBJECTIVE: This study aimed to identify information derived from patients' social media data that can benefit clinicians and to develop a set of design implications, via a series of low-fidelity (lo-fi) prototypes, on how to deliver the information at the point of care. METHODS: A team of clinical researchers and human-computer interaction (HCI) researchers conducted a long-term co-design activity for over 6 months. The needs-affordances analysis framework was used to refine the clinicians' potential needs, which can be supported by patients' social media data. On the basis of those identified needs, the HCI researchers iteratively created 3 different lo-fi prototypes. The prototypes were shared with both groups of researchers via a videoconferencing software for discussion and feedback. During the remote meetings, potential clinical utility, potential use of the different prototypes in a treatment setting, and areas of improvement were discussed. RESULTS: Our first prototype was a card-type interface that supported treatment goal tracking. Each card included attribute levels: depression, anxiety, social activities, alcohol, and drug use. This version confirmed what types of information are helpful but revealed the need for a glanceable dashboard that highlights the trends of these information. As a result, we then developed the second prototype, an interface that shows the clinical state and trend. We found that focusing more on the changes since the last visit without visual representation can be more compatible with clinicians' work practices. In addition, the second phase of needs-affordances analysis identified 3 categories of information relevant to patients with schizophrenia: symptoms related to psychosis, symptoms related to mood and anxiety, and social functioning. Finally, we developed the third prototype, a clinical summary dashboard that showed changes from the last visit in plain texts and contrasting colors. CONCLUSIONS: This exploratory co-design research confirmed that mental health attributes inferred from patients' social media data can be useful for clinicians, although it also revealed a gap between computational social media analyses and clinicians' expectations and conceptualizations of patients' mental health states. In summary, the iterative co-design process crystallized design directions for the future interface, including how we can organize and provide symptom-related information in a way that minimizes the clinicians' workloads.

A Real-Time Eating Detection System for Capturing Eating Moments and Triggering Ecological Momentary Assessments to Obtain Further Context: System Development and Validation Study.

BACKGROUND: Eating behavior has a high impact on the well-being of an individual. Such behavior involves not only when an individual is eating, but also various contextual factors such as with whom and where an individual is eating and what kind of food the individual is eating. Despite the relevance of such factors, most automated eating detection systems are not designed to capture contextual factors. OBJECTIVE: The aims of this study were to (1) design and build a smartwatch-based eating detection system that can detect meal episodes based on dominant hand movements, (2) design ecological momentary assessment (EMA) questions to capture meal contexts upon detection of a meal by the eating detection system, and (3) validate the meal detection system that triggers EMA questions upon passive detection of meal episodes. METHODS: The meal detection system was deployed among 28 college students at a US institution over a period of 3 weeks. The participants reported various contextual data through EMAs triggered when the eating detection system correctly detected a meal episode. The EMA questions were designed after conducting a survey study with 162 students from the same campus. Responses from EMAs were used to define exclusion criteria. RESULTS: Among the total consumed meals, 89.8% (264/294) of breakfast, 99.0% (406/410) of lunch, and 98.0% (589/601) of dinner episodes were detected by our novel meal detection system. The eating detection system showed a high accuracy by capturing 96.48% (1259/1305) of the meals consumed by the participants. The meal detection classifier showed a precision of 80%, recall of 96%, and F1 of 87.3%. We found that over 99% (1248/1259) of the detected meals were consumed with distractions. Such eating behavior is considered "unhealthy" and can lead to overeating and uncontrolled weight gain. A high proportion of meals was consumed alone (680/1259, 54.01%). Our participants self-reported 62.98% (793/1259) of their meals as healthy. Together, these results have implications for designing technologies to encourage healthy eating behavior. CONCLUSIONS: The presented eating detection system is the first of its kind to leverage EMAs to capture the eating context, which has strong implications for well-being research. We reflected on the contextual data gathered by our system and discussed how these insights can be used to design individual-specific interventions.

Viz-A-Vis: toward visualizing video through computer vision.

In the established procedural model of information visualization, the first operation is to transform raw data into data tables [1]. The transforms typically include abstractions that aggregate and segment relevant data and are usually defined by a human, user or programmer. The theme of this paper is that for video, data transforms should be supported by low level computer vision. High level reasoning still resides in the human analyst, while part of the low level perception is handled by the computer. To illustrate this approach, we present Viz-A-Vis, an overhead video capture and access system for activity analysis in natural settings over variable periods of time. Overhead video provides rich opportunities for long-term behavioral and occupancy analysis, but it poses considerable challenges. We present initial steps addressing two challenges. First, overhead video generates overwhelmingly large volumes of video impractical to analyze manually. Second, automatic video analysis remains an open problem for computer vision.

NotifiVR: Exploring Interruptions and Notifications in Virtual Reality.

The proliferation of high resolution and affordable virtual reality (VR) headsets is quickly making room-scale VR experiences available in our homes. Most VR experiences strive to achieve complete immersion by creating a disconnect from the real world. However, due to the lack of a standardized notification management system and minimal context awareness in VR, an immersed user may face certain situations such as missing an important phone call (digital scenario), tripping over wandering pets (physical scenario), or losing track of time (temporal scenario). In this paper, we present the results of 1) a survey across 61 VR users to understand common interruptions and scenarios that would benefit from some form of notifications; 2) a design exercise with VR professionals to explore possible notification methods; and 3) an empirical study on the noticeability and perception of 5 different VR interruption scenarios across 6 modality combinations (e.g., audio, visual, haptic, audio + haptic, visual + haptic, and audio + visual) implemented in Unity and presented using the HTC Vive headset. Finally, we combine key learnings from each of these steps along with participant feedback to present a set of observations and recommendations for notification design in VR.

Exploring Symmetric and Asymmetric Bimanual Eating Detection with Inertial Sensors on the Wrist.

Motivated by health applications, eating detection with off-the-shelf devices has been an active area of research. A common approach has been to recognize and model individual intake gestures with wrist-mounted inertial sensors. Despite promising results, this approach is limiting as it requires the sensing device to be worn on the hand performing the intake gesture, which cannot be guaranteed in practice. Through a study with 14 participants comparing eating detection performance when gestural data is recorded with a wrist-mounted device on (1) both hands, (2) only the dominant hand, and (3) only the non-dominant hand, we provide evidence that a larger set of arm and hand movement patterns beyond food intake gestures are predictive of eating activities when L1 or L2 normalization is applied to the data. Our results are supported by the theory of asymmetric bimanual action and contribute to the field of automated dietary monitoring. In particular, it shines light on a new direction for eating activity recognition with consumer wearables in realistic settings.

Investigating the accuracy of a novel telehealth diagnostic approach for autism spectrum disorder.

Research indicates that a substantial amount of time elapses between parents' first concerns about their child's development and a formal diagnosis of autism spectrum disorder (ASD). Telehealth presents an opportunity to expedite the diagnostic process. This project compared a novel telehealth diagnostic approach that utilizes clinically guided in-home video recordings to the gold standard in-person diagnostic assessment. Participants included 40 families seeking an ASD evaluation for their child and 11 families of typically developing children. Children were between the ages of 18 months and 6 years 11 months; mean adaptive behavior composite = 75.47 (SD = 15.94). All parent participants spoke English fluently. Families completed the Naturalistic Observation Diagnostic Assessment (NODA) for ASD, which was compared to an in-person assessment (IPA). Agreement between the 2 methods, as well as sensitivity, specificity, and interrater reliability, were calculated for the full sample and the subsample of families seeking an ASD evaluation. Diagnostic agreement between NODA and the IPA was 88.2% (κ = 0.75) in the full sample and 85% (κ = 0.58) in the subsample. Sensitivity was 84.9% in both, whereas specificity was 94.4% in the full sample and 85.7% in the subsample. Kappa coefficients for interrater reliability indicated 85% to 90% accuracy between raters. NODA utilizes telehealth technology for families to share information with professionals and provides a method to inform clinical judgment for a diagnosis of ASD. Due to the high level of agreement with the IPA in this sample, NODA has potential to improve the efficiency of the diagnostic process for ASD. (PsycINFO Database Record

EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments.

Chronic and widespread diseases such as obesity, diabetes, and hypercholesterolemia require patients to monitor their food intake, and food journaling is currently the most common method for doing so. However, food journaling is subject to self-bias and recall errors, and is poorly adhered to by patients. In this paper, we propose an alternative by introducing EarBit, a wearable system that detects eating moments. We evaluate the performance of inertial, optical, and acoustic sensing modalities and focus on inertial sensing, by virtue of its recognition and usability performance. Using data collected in a simulated home setting with minimum restrictions on participants' behavior, we build our models and evaluate them with an unconstrained outside-the-lab study. For both studies, we obtained video footage as ground truth for participants activities. Using leave-one-user-out validation, EarBit recognized all the eating episodes in the semi-controlled lab study, and achieved an accuracy of 90.1% and an F1-score of 90.9% in detecting chewing instances. In the unconstrained, outside-the-lab evaluation, EarBit obtained an accuracy of 93% and an F1-score of 80.1% in detecting chewing instances. It also accurately recognized all but one recorded eating episodes. These episodes ranged from a 2 minute snack to a 30 minute meal.

ewrapper: Operationalizing engagement strategies in mHealth.

The advancement of digital technologies particularly in the domain of mobile health (mHealth) holds great promise in the promotion of health behavior. However, keeping users engaged remains a central challenge. This paper proposes a novel approach to address this issue by supplementing existing and future mHealth applications with an engagement wrapper - a collection of engagement strategies integrated into a single, coherent model. The engagement wrapper is operationalized within the format of an ambient display on the lock screen of mobile devices.

A Practical Approach for Recognizing Eating Moments with Wrist-Mounted Inertial Sensing.

Recognizing when eating activities take place is one of the key challenges in automated food intake monitoring. Despite progress over the years, most proposed approaches have been largely impractical for everyday usage, requiring multiple on-body sensors or specialized devices such as neck collars for swallow detection. In this paper, we describe the implementation and evaluation of an approach for inferring eating moments based on 3-axis accelerometry collected with a popular off-the-shelf smartwatch. Trained with data collected in a semi-controlled laboratory setting with 20 subjects, our system recognized eating moments in two free-living condition studies (7 participants, 1 day; 1 participant, 31 days), with F-scores of 76.1% (66.7% Precision, 88.8% Recall), and 71.3% (65.2% Precision, 78.6% Recall). This work represents a contribution towards the implementation of a practical, automated system for everyday food intake monitoring, with applicability in areas ranging from health research and food journaling.

Inferring Meal Eating Activities in Real World Settings from Ambient Sounds: A Feasibility Study.

Dietary self-monitoring has been shown to be an effective method for weight-loss, but it remains an onerous task despite recent advances in food journaling systems. Semi-automated food journaling can reduce the effort of logging, but often requires that eating activities be detected automatically. In this work we describe results from a feasibility study conducted in-the-wild where eating activities were inferred from ambient sounds captured with a wrist-mounted device; twenty participants wore the device during one day for an average of 5 hours while performing normal everyday activities. Our system was able to identify meal eating with an F-score of 79.8% in a person-dependent evaluation, and with 86.6% accuracy in a person-independent evaluation. Our approach is intended to be practical, leveraging off-the-shelf devices with audio sensing capabilities in contrast to systems for automated dietary assessment based on specialized sensors.