Modeling inter-individual differences in ambulatory-based multimodal signals via metric learning: a case study of personalized well-being estimation of healthcare workers.

Introduction: Intelligent ambulatory tracking can assist in the automatic detection of psychological and emotional states relevant to the mental health changes of professionals with high-stakes job responsibilities, such as healthcare workers. However, well-known differences in the variability of ambulatory data across individuals challenge many existing automated approaches seeking to learn a generalizable means of well-being estimation. This paper proposes a novel metric learning technique that improves the accuracy and generalizability of automated well-being estimation by reducing inter-individual variability while preserving the variability pertaining to the behavioral construct. Methods: The metric learning technique implemented in this paper entails learning a transformed multimodal feature space from pairwise similarity information between (dis)similar samples per participant via a Siamese neural network. Improved accuracy via personalization is further achieved by considering the trait characteristics of each individual as additional input to the metric learning models, as well as individual trait base cluster criteria to group participants followed by training a metric learning model for each group. Results: The outcomes of the proposed models demonstrate significant improvement over the other inter-individual variability reduction and deep neural baseline methods for stress, anxiety, positive affect, and negative affect. Discussion: This study lays the foundation for accurate estimation of psychological and emotional states in realistic and ambulatory environments leading to early diagnosis of mental health changes and enabling just-in-time adaptive interventions.

Machine learning nonresponse adjustment of patient-reported opioid consumption data to enable consumption-informed postoperative opioid prescribing guidelines.

Background: Post-discharge opioid consumption is a crucial patient-reported outcome informing opioid prescribing guidelines, but its collection is resource-intensive and vulnerable to inaccuracy due to nonresponse bias. Methods: We developed a post-discharge text message-to-web survey system for efficient collection of patient-reported pain outcomes. We prospectively recruited surgical patients at Beth Israel Deaconess Medical Center in Boston, Massachusetts from March 2019 through October 2020, sending an SMS link to a secure web survey to quantify opioids consumed after discharge from hospitalization. Patient factors extracted from the electronic health record were tested for nonresponse bias and observable confounding. Following targeted learning-based nonresponse adjustment, procedure-specific opioid consumption quantiles (medians and 75th percentiles) were estimated and compared to a previous telephone-based reference survey. Results: 6553 patients were included. Opioid consumption was measured in 44% of patients (2868), including 21% (1342) through survey response. Characteristics associated with inability to measure opioid consumption included age, tobacco use, and prescribed opioid dose. Among the 10 most common procedures, median consumption was only 36% of the median prescription size; 64% of prescribed opioids were not consumed. Among those procedures, nonresponse adjustment corrected the median opioid consumption by an average of 37% (IQR: 7, 65%) compared to unadjusted estimates, and corrected the 75th percentile by an average of 5% (IQR: 0, 12%). This brought median estimates for 5/10 procedures closer to telephone survey-based consumption estimates, and 75th percentile estimates for 2/10 procedures closer to telephone survey-based estimates. Conclusions: SMS-recruited online surveying can generate reliable opioid consumption estimates after nonresponse adjustment using patient factors recorded in the electronic health record, protecting patients from the risk of inaccurate prescription guidelines.

TILES-2019: A longitudinal physiologic and behavioral data set of medical residents in an intensive care unit.

The TILES-2019 data set consists of behavioral and physiological data gathered from 57 medical residents (i.e., trainees) working in an intensive care unit (ICU) in the United States. The data set allows for the exploration of longitudinal changes in well-being, teamwork, and job performance in a demanding environment, as residents worked in the ICU for three weeks. Residents wore a Fitbit, a Bluetooth-based proximity sensor, and an audio-feature recorder. They completed daily surveys and interviews at the beginning and end of their rotation. In addition, we collected data from environmental sensors (i.e., Internet-of-Things Bluetooth data hubs) and obtained hospital records (e.g., patient census) and residents' job evaluations. This data set may be may be of interest to researchers interested in workplace stress, group dynamics, social support, the physical and psychological effects of witnessing patient deaths, predicting survey data from sensors, and privacy-aware and privacy-preserving machine learning. Notably, a small subset of the data was collected during the first wave of the COVID-19 pandemic.

A multimodal analysis of physical activity, sleep, and work shift in nurses with wearable sensor data.

Night shift workers are often associated with circadian misalignment and physical discomfort, which may lead to burnout and decreased work performance. Moreover, the irregular work hours can lead to significant negative health outcomes such as poor eating habits, smoking, and being sedentary more often. This paper uses commercial wearable sensors to explore correlates and differences in the level of physical activity, sleep, and circadian misalignment indicators among day shift nurses and night shift nurses. We identify which self-reported assessments of affect, life satisfaction, and sleep quality, are associated with physiological and behavioral signals captured by wearable sensors. The results using data collected from 113 nurses in a large hospital setting, over a period of 10 weeks, indicate that night shift nurses are more sedentary, and report lower levels of life satisfaction than day-shift nurses. Moreover, night shift nurses report poorer sleep quality, which may be correlated with challenges in their attempts to fall asleep on off-days.

Dynamical systems modeling of day-to-day signal-based patterns of emotional self-regulation and stress spillover in highly-demanding health professions.

As hospital workers face a growing number of patients and have to meet increasingly rigorous standards of care, their ability to successfully modulate their emotional reactions and flexibly handle stress presents a significant challenge. This paper examines a multimodal signal-driven way to quantify emotion self-regulation and stress spillover through a dynamical systems model (DSM). The proposed DSM models day-to-day changes of emotional arousal, captured through speech, physiology, and daily activity measures, and its interplay with daily stress. The parameters of the DSM quantify the degree of self-regulation and stress spillover, and are associated with work performance and cognitive ability in a multimodal dataset of 130 full-time hospital workers recorded over a 10-week period. Linear regression experiments indicate the effectiveness of the proposed features to reliably estimate individuals' work performance and cognitive ability, providing significantly higher Pearson's correlations compared to aggregate measures of emotional arousal. Results from this study demonstrate the importance of quantifying oscillatory behaviors from longitudinal ambulatory signals and can potentially deepen our understanding of emotion self-regulation and stress spillover using signal-driven measurements, which complement self-reports and provide estimates of the psychological constructs of interest in a fine-grained time resolution.

TILES-2018, a longitudinal physiologic and behavioral data set of hospital workers.

We present a novel longitudinal multimodal corpus of physiological and behavioral data collected from direct clinical providers in a hospital workplace. We designed the study to investigate the use of off-the-shelf wearable and environmental sensors to understand individual-specific constructs such as job performance, interpersonal interaction, and well-being of hospital workers over time in their natural day-to-day job settings. We collected behavioral and physiological data from n = 212 participants through Internet-of-Things Bluetooth data hubs, wearable sensors (including a wristband, a biometrics-tracking garment, a smartphone, and an audio-feature recorder), together with a battery of surveys to assess personality traits, behavioral states, job performance, and well-being over time. Besides the default use of the data set, we envision several novel research opportunities and potential applications, including multi-modal and multi-task behavioral modeling, authentication through biometrics, and privacy-aware and privacy-preserving machine learning.

Multimodal Human and Environmental Sensing for Longitudinal Behavioral Studies in Naturalistic Settings: Framework for Sensor Selection, Deployment, and Management.

BACKGROUND: Recent advances in mobile technologies for sensing human biosignals are empowering researchers to collect real-world data outside of the laboratory, in natural settings where participants can perform their daily activities with minimal disruption. These new sensing opportunities usher a host of challenges and constraints for both researchers and participants. OBJECTIVE: This viewpoint paper aims to provide a comprehensive guide to aid research teams in the selection and management of sensors before beginning and while conducting human behavior studies in the wild. The guide aims to help researchers achieve satisfactory participant compliance and minimize the number of unexpected procedural outcomes. METHODS: This paper presents a collection of challenges, consideration criteria, and potential solutions for enabling researchers to select and manage appropriate sensors for their research studies. It explains a general data collection framework suitable for use with modern consumer sensors, enabling researchers to address many of the described challenges. In addition, it provides a description of the criteria affecting sensor selection, management, and integration that researchers should consider before beginning human behavior studies involving sensors. On the basis of a survey conducted in mid-2018, this paper further illustrates an organized snapshot of consumer-grade human sensing technologies that can be used for human behavior research in natural settings. RESULTS: The research team applied the collection of methods and criteria to a case study aimed at predicting the well-being of nurses and other staff in a hospital. Average daily compliance for sensor usage measured by the presence of data exceeding half the total possible hours each day was about 65%, yielding over 355,000 hours of usable sensor data across 212 participants. A total of 6 notable unexpected events occurred during the data collection period, all of which had minimal impact on the research project. CONCLUSIONS: The satisfactory compliance rates and minimal impact of unexpected events during the case study suggest that the challenges, criteria, methods, and mitigation strategies presented as a guide for researchers are helpful for sensor selection and management in longitudinal human behavior studies in the wild.

An Evaluation of EEG-based Metrics for Engagement Assessment of Distance Learners.

Maintaining students' cognitive engagement in educational settings is crucial to their performance, though quantifying this mental state in real-time for distance learners has not been studied extensively in natural distance learning environments. We record electroencephalographic (EEG) data of students watching online lecture videos and use it to predict engagement rated by human annotators. An evaluation of prior EEG-based engagement metrics that utilize power spectral density (PSD) features is presented. We examine the predictive power of various supervised machine learning approaches with both subject-independent and individualized models when using simple PSD feature functions. Our results show that engagement metrics with few power band variables, including those proposed in prior research, do not produce predictions consistent with human observations. We quantify the performance disparity between cross-subject and per-subject models and demonstrate that individual differences in EEG patterns necessitate a more complex metric for educational engagement assessment in natural distance learning environments.

Tribological durability of silane monolayers on silicon.

We report the frictional performance and long-term tribological stability of various alkyl silane monolayer films on silicon by using pin-on-disk tribometry at ambient conditions. We show that the durability of monolayers derived from n-alkyltrichlorosilanes on silicon increases exponentially with the chain length of the silane precursor, which we relate to the cohesive energy of these monolayers through molecular dynamics simulations. X-ray photoelectron spectroscopy (XPS) was used to show that tribological damage consisted of the loss of molecular components that could be partially replaced upon exposure to a solution containing perfluorinated silane precursors. For monolayers derived from n-octadecyltrichlorosilane, a critical load was identified to be approximately 250 mN (200 MPa), above which failure of films occurred within 100 cycles of testing. Monolayers with hydroxyl surfaces exhibited reduced stabilities due to stronger tip-surface interactions. Monolayers with the capability for cross-linking exhibited much greater stabilities than monolayers where cross-linking was limited or prevented. Collectively, these results demonstrate that the mechanical durability of monolayers when subjected to a tribological load is greatly enhanced by maximizing dispersional interactions and cross-linking and minimizing tip-surface interactions.

Tribology of monolayer films: comparison between n-alkanethiols on gold and n-alkyl trichlorosilanes on silicon.

This Article presents a quantitative comparison of the frictional performance for monolayers derived from n-alkanethiolates on gold and n-alkyl trichlorosilanes on silicon. Monolayers were characterized by pin-on-disk tribometry, contact angle analysis, ellipsometry, and electrochemical impedance spectroscopy (EIS). Pin-on-disk microtribometry provided frictional analysis at applied normal loads from 10 to 1000 mN at a speed of 0.1 mm/s. At low loads (10 mN), methyl-terminated n-alkanethiolate self-assembled monolayers (SAMs) exhibited a 3-fold improvement in coefficient of friction over SAMs with hydroxyl- or carboxylic-acid-terminated surfaces. For monolayers prepared from both n-alkanethiols on gold and n-alkyl trichlorosilanes on silicon, a critical chain length of at least eight carbons is required for beneficial tribological performance at an applied load of 9.8 mN. Evidence for disruption of chemisorbed alkanethiolate SAMs with chain lengths n

Inkjet-printed monolayers as platforms for tethered polymers.

Combining inkjet printing and atom-transfer radical polymerization (ATRP) provides a straightforward and versatile method for producing patterned polymer surfaces that may serve as platforms for a variety of applications. We report the use of drop-on-demand technology to print binary chemical gradients and simple patterns onto solid substrates and, by using surface-confined ATRP, amplify these patterns and gradients. Chemically graded monolayers prepared by inkjet printing dodecanethiol and backfilling with 11-mercaptoundecanol showed continuous changes in the water contact angle along the gradient. These samples also exhibited a distinct change in the intensity of methyl group and C-O stretching modes along the gradient. Graded or patterned polymer layers were produced by growing, with ATRP, tethered poly(methyl methacrylate) (PMMA) layers from gradient or patterned printed monolayers that contained a bromo-capped initiator. Atomic force microscopy and optical microscopy confirmed that the PMMA layers amplified the underlying printed initiator layer with remarkable fidelity.