ABSTRACT
The current study is the first to document the real-time association between phone use and speech to infants in extended real-world interactions. N= 16 predominantly White (75%) mother-infant dyads (infants aged M = 4.1 months, SD = 2.3; 63% female) shared 16,673 min of synchronized real-world phone use and Language Environment Analysis audio data over the course of 1 week (collected 2017-2020) for our analyses. Maternal phone use was associated with a 16% decrease in infants' speech input, with shorter intervals of phone use (1-2 min) associated with a greater 26% decrease in speech input relative to longer periods. This work highlights the value of multimodal sensing to access dynamic, within-person, and context-specific predictors of speech to infants in real-world settings.
ABSTRACT
Depression in mothers is consistently associated with reduced caregiving sensitivity and greater infant negative affect expression. The current article examined the real-time behavioral mechanisms underlying these associations using Granger causality time series analyses in a sample of mothers (N = 194; 86.60% White) at elevated risk for depression and their 3-month-old infants (46.40% female) living in a major metropolitan area in the United States. Overall, mothers contingently responded to infant distress, and mothers' responses to infant distress increased the likelihood of infant soothing in real time. However, there was no evidence for maternal contingent responding or facilitation of infant soothing in subsamples of mothers who were currently experiencing elevated depression symptoms or in mothers of highly negative infants. These findings suggest real-time behavioral mechanisms by which risks for maladaptive self-regulation may develop. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Subject(s)
Maternal Behavior , Mother-Child Relations , Infant , Humans , Female , Male , Maternal Behavior/physiology , Depression , MothersABSTRACT
Porges' polyvagal theory (1991) proposes that the activity of the vagal nerve modulates moment-by-moment changes in adaptive behavior during stress. However, most work, including research with infants, has only examined vagal changes at low temporal resolutions, averaging 30+ s across phases of structured stressor paradigms. Thus, the true timescale of vagal regulation-and the extent to which it can be observed during unprompted crying-is unknown. The current study utilized a recently validated method to calculate respiratory sinus arrhythmia (RSA) dynamically at a high resolution of 5 Hz (updated every 200 ms) in a home-based infant study. Using an event-related analysis, we calculated the relative change in RSA around the onset of naturally occurring unprompted instances of n = 41 infants' 180 crying events. As predicted, RSA significantly decreased after the onset of crying compared to non-crying chance changes in RSA. Decreasing trends in RSA were driven by infants with higher pre-cry RSA values, infants rated lower in Negative Affectivity, and those rated both high and low in Orienting by their mothers. Our results display the timescale of RSA in spontaneous and naturalistic episodes of infant crying and that these dynamic RSA patterns are aligned with real-time levels of RSA and also caregiver-reported temperament.
Subject(s)
Crying , Respiratory Sinus Arrhythmia , Female , Humans , Infant , Crying/physiology , Vagus Nerve/physiology , Mothers , Respiratory Sinus Arrhythmia/physiology , Arrhythmia, SinusABSTRACT
Exposure to infant crying is a well-established predictor of mothers' mental health. However, this association may reflect many potential mechanisms. Capturing dynamic fluctuations in mothers' states simultaneously with caregiving experiences is necessary to identify the real-time processes influencing mental health. In this study, we leveraged ecological momentary assessments (EMAs) and infant-worn audio recorders to capture variability in mothers' mental health symptoms and their exposure to infant crying over one week in a racially and socio-economically diverse urban North-American sample (N = 53). We use multilevel modeling to characterize within- and between-person effects of crying on maternal negative affect and symptoms of depression and anxiety. Within participants, when infants cried more than average in the 10 min, 1 hr, and 8 hr prior to an EMA report, mothers' negative affect subsequently increased, controlling for mean levels of infant crying. In contrast to findings from laboratory studies, in everyday settings crying exposure did not immediately increase feelings of depression. Only when crying was above average for 8 hr prior to EMA did mothers report increases in subsequent depression symptoms, suggesting that the effects of crying on maternal mental health take hours to unfold in ecologically valid home settings. Between participants, mothers of infants who cried more on average did not report higher negative affect or symptoms of depression or anxiety. Overall, our findings reveal that crying exposure dynamically influences maternal negative affect and depression but not anxiety in ecologically valid real-world settings. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Subject(s)
Crying , Mother-Child Relations , Female , Infant , Humans , Crying/psychology , Mother-Child Relations/psychology , Mental Health , Emotions , Mothers/psychologyABSTRACT
Human infant crying evolved as a signal to elicit parental care and actively influences caregiving behaviors as well as infant-caregiver interactions. Automated cry detection algorithms have become more popular in recent decades, and while some models exist, they have not been evaluated thoroughly on daylong naturalistic audio recordings. Here, we validate a novel deep learning cry detection model by testing it in assessment scenarios important to developmental researchers. We also evaluate the deep learning model's performance relative to LENA's cry classifier, one of the most commonly used commercial software systems for quantifying child crying. Broadly, we found that both deep learning and LENA model outputs showed convergent validity with human annotations of infant crying. However, the deep learning model had substantially higher accuracy metrics (recall, F1, kappa) and stronger correlations with human annotations at all timescales tested (24 h, 1 h, and 5 min) relative to LENA. On average, LENA underestimated infant crying by 50 min every 24 h relative to human annotations and the deep learning model. Additionally, daily infant crying times detected by both automated models were lower than parent-report estimates in the literature. We provide recommendations and solutions for leveraging automated algorithms to detect infant crying in the home and make our training data and model code open source and publicly available.
Subject(s)
Algorithms , Crying , Humans , Infant , Parents , SoftwareABSTRACT
Background & motivation: Household chaos is an established risk factor for child development. However, current methods for measuring household chaos rely on parent surveys, meaning existing research efforts cannot disentangle potentially dynamic bidirectional relations between high chaos environments and child behavior problems. Proposed approach: We train and make publicly available a classifier to provide objective, high-resolution predictions of household chaos from real-world child-worn audio recordings. To do so, we collect and annotate a novel dataset of ground-truth auditory chaos labels compiled from over 411 h of daylong recordings collected via audio recorders worn by N=22 infants in their homes. We leverage an existing sound event classifier to identify candidate high chaos segments, increasing annotation efficiency 8.32× relative to random sampling. Result: Our best-performing model successfully classifies four levels of real-world household auditory chaos with a macro F1 score of 0.701 (Precision: 0.705, Recall: 0.702) and a weighted F1 score of 0.679 (Precision: 0.685, Recall: 0.680). Significance: In future work, high-resolution objective chaos predictions from our model can be leveraged for basic science and intervention, including testing theorized mechanisms by which chaos affects children's cognition and behavior. Additionally, to facilitate further model development we make publicly available the first and largest balanced annotated audio dataset of real-world household chaos.
ABSTRACT
Most existing cry detection models have been tested with data collected in controlled settings. Thus, the extent to which they generalize to noisy and lived environments is unclear. In this paper, we evaluate several established machine learning approaches including a model leveraging both deep spectrum and acoustic features. This model was able to recognize crying events with F1 score 0.613 (Precision: 0.672, Recall: 0.552), showing improved external validity over existing methods at cry detection in everyday real-world settings. As part of our evaluation, we collect and annotate a novel dataset of infant crying compiled from over 780 hours of labeled real-world audio data, captured via recorders worn by infants in their homes, which we make publicly available. Our findings confirm that a cry detection model trained on in-lab data underperforms when presented with real-world data (in-lab test F1: 0.656, real-world test F1: 0.236), highlighting the value of our new dataset and model.
ABSTRACT
Audio recorders, accelerometers, and cameras that infants wear throughout their everyday lives capture the experiences that are available to shape development. Everyday sensing in infancy reveals patterns within the everyday hubbub that are unknowable using methods that capture shorter, more isolated, or more planned slices of behavior. Here, we review ten lessons learned from recent endeavors that removed researchers from designing or participating in infants' experiences and instead quantified patterns that arose within infants' own spontaneously arising everyday experiences. The striking heterogeneity of experiences - there is no meaningfully "representative" hour of a day, instance of a category, interaction context, or infant - inspires next steps in theory and practice that embrace the complex, dynamic, and multiple pathways of human development.
ABSTRACT
BACKGROUND: As mobile technologies become ever more sensor-rich, portable, and ubiquitous, data captured by smart devices are lending rich insights into users' daily lives with unprecedented comprehensiveness and ecological validity. A number of human-subject studies have been conducted to examine the use of mobile sensing to uncover individual behavioral patterns and health outcomes, yet minimal attention has been placed on measuring living environments together with other human-centered sensing data. Moreover, the participant sample size in most existing studies falls well below a few hundred, leaving questions open about the reliability of findings on the relations between mobile sensing signals and human outcomes. RESULTS: To address these limitations, we developed a home environment sensor kit for continuous indoor air quality tracking and deployed it in conjunction with smartphones, Fitbits, and ecological momentary assessments in a cohort study of up to 1,584 college student participants per data type for 3 weeks. We propose a conceptual framework that systematically organizes human-centric data modalities by their temporal coverage and spatial freedom. Then we report our study procedure, technologies and methods deployed, and descriptive statistics of the collected data that reflect the participants' mood, sleep, behavior, and living environment. CONCLUSIONS: We were able to collect from a large participant cohort satisfactorily complete multi-modal sensing and survey data in terms of both data continuity and participant adherence. Our novel data and conceptual development provide important guidance for data collection and hypothesis generation in future human-centered sensing studies.
Subject(s)
Smartphone , Cohort Studies , Home Environment , Humans , Reproducibility of Results , Surveys and QuestionnairesABSTRACT
Mobile and wearable sensors provide a unique opportunity to capture the daily activities and interactions that shape developmental trajectories, with potential to revolutionize the study of development (de Barbaro, 2019). However, developmental research employing sensors is still in its infancy, and parents' comfort using these devices is uncertain. This exploratory report assesses parent willingness to participate in sensor studies via a nationally representative survey (N = 210) and live recruitment of a low-income, minority population for an ongoing study (N = 359). The survey allowed us to assess how protocol design influences acceptability, including various options for devices and datastream resolution, conditions of data sharing, and feedback. By contrast, our recruitment data provided insight into parents' true willingness to participate in a sensor study, with a protocol including 72 h of continuous audio, motion, and physiological data. Our results indicate that parents are relatively conservative when considering participation in sensing studies. However, nearly 41 % of surveyed parents reported that they would be at least somewhat willing to participate in studies with audio or video recordings, 26 % were willing or extremely willing, and 14 % reported being extremely willing. These results roughly paralleled our recruitment results, where 58 % of parents indicated interest, 29 % of parents scheduled to participate, and 10 % ultimately participated. Additionally, 70 % of caregivers stated their reason for not participating in the study was due to barriers unrelated to sensing while about 25 % noted barriers due to either privacy concerns or the physical sensors themselves. Parents' willingness to collect sensitive datastreams increased if data stayed within the household for individual use only, are shared anonymously with researchers, or if parents receive feedback from devices. Overall, our findings suggest that given the correct circumstances, mobile sensors are a feasible and promising tool for characterizing children's daily interactions and their role in development.
Subject(s)
Caregivers , Parents , Child , Feasibility Studies , Humans , Perception , Surveys and QuestionnairesABSTRACT
BACKGROUND: Eating behavior has a high impact on the well-being of an individual. Such behavior involves not only when an individual is eating, but also various contextual factors such as with whom and where an individual is eating and what kind of food the individual is eating. Despite the relevance of such factors, most automated eating detection systems are not designed to capture contextual factors. OBJECTIVE: The aims of this study were to (1) design and build a smartwatch-based eating detection system that can detect meal episodes based on dominant hand movements, (2) design ecological momentary assessment (EMA) questions to capture meal contexts upon detection of a meal by the eating detection system, and (3) validate the meal detection system that triggers EMA questions upon passive detection of meal episodes. METHODS: The meal detection system was deployed among 28 college students at a US institution over a period of 3 weeks. The participants reported various contextual data through EMAs triggered when the eating detection system correctly detected a meal episode. The EMA questions were designed after conducting a survey study with 162 students from the same campus. Responses from EMAs were used to define exclusion criteria. RESULTS: Among the total consumed meals, 89.8% (264/294) of breakfast, 99.0% (406/410) of lunch, and 98.0% (589/601) of dinner episodes were detected by our novel meal detection system. The eating detection system showed a high accuracy by capturing 96.48% (1259/1305) of the meals consumed by the participants. The meal detection classifier showed a precision of 80%, recall of 96%, and F1 of 87.3%. We found that over 99% (1248/1259) of the detected meals were consumed with distractions. Such eating behavior is considered "unhealthy" and can lead to overeating and uncontrolled weight gain. A high proportion of meals was consumed alone (680/1259, 54.01%). Our participants self-reported 62.98% (793/1259) of their meals as healthy. Together, these results have implications for designing technologies to encourage healthy eating behavior. CONCLUSIONS: The presented eating detection system is the first of its kind to leverage EMAs to capture the eating context, which has strong implications for well-being research. We reflected on the contextual data gathered by our system and discussed how these insights can be used to design individual-specific interventions.
Subject(s)
Ecological Momentary Assessment , Meals , Feeding Behavior , Humans , Surveys and QuestionnairesABSTRACT
The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the non-linear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique's application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a "Programming Basics" module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data.
ABSTRACT
Advances in mobile and wearable technologies mean it is now feasible to record hours to days of participant behavior in its naturalistic context, a great boon for psychologists interested in family processes and development. While automated activity recognition algorithms exist for a limited set of behaviors, time-consuming human annotations are still required to robustly characterize the vast majority of behavioral and affective markers of interest. This report is the first to date which systematically tests the efficacy of different sampling strategies for characterizing behavior from audio recordings to provide practical guidelines for researchers. Using continuous audio recordings of the daily lives of 11 preschool-aged children, we compared sampling techniques to determine the most accurate and efficient approach. Results suggest that sampling both low and high frequency verbal and overt behaviors is best if samples are short in duration, systematically rather than randomly selected, and sampled to cover at least 12.5% of recordings. Implications for assessment of real-world behavior are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Subject(s)
Child Behavior , Ecological Momentary Assessment/standards , Verbal Behavior , Child, Preschool , Female , Humans , MaleABSTRACT
Recent advances in large-scale data storage and processing offer unprecedented opportunities for behavioral scientists to collect and analyze naturalistic data, including from underrepresented groups. Audio data, particularly real-world audio recordings, are of particular interest to behavioral scientists because they provide high-fidelity access to subtle aspects of daily life and social interactions. However, these methodological advances pose novel risks to research participants and communities. In this article, we outline the benefits and challenges associated with collecting, analyzing, and sharing multi-hour audio recording data. Guided by the principles of autonomy, privacy, beneficence, and justice, we propose a set of ethical guidelines for the use of longform audio recordings in behavioral research. This article is also accompanied by an Open Science Framework Ethics Repository that includes informed consent resources such as frequent participant concerns and sample consent forms.
Subject(s)
Confidentiality , Privacy , Video Recording , Behavioral Research , Data Collection , Ethics, Research , Humans , Informed ConsentABSTRACT
Rapidly maturing technologies for sensing and activity recognition can provide unprecedented access to the complex structure daily activity and interaction, promising new insight into the mechanisms by which experience shapes developmental outcomes. Motion data, autonomic activity, and "snippets" of audio and video recordings can be conveniently logged by wearable sensors (Lazer et al., 2009). Machine learning algorithms can process these signals into meaningful markers, from child and parent behavior to outcomes such as depression or teenage drinking. Theoretically motivated aspects of daily activity can be combined and synchronized to examine reciprocal effects between children's behaviors and their environments or internal processes. Captured over longitudinal time, such data provide a new opportunity to study the processes by which individual differences emerge and stabilize. This paper introduces the reader to developments in sensing and activity recognition with implications for developmental phenomena across the lifespan, sketching a framework for leveraging mobile sensors for transactional analyses that bridge micro- and longitudinal- timescales of development. It finishes by detailing resources and best practices to facilitate the next generation of developmentalists to contribute to this emerging area.
Subject(s)
Activities of Daily Living , Affect/physiology , Human Development/physiology , Interpersonal Relations , Monitoring, Ambulatory , Motor Activity/physiology , Wearable Electronic Devices , HumansABSTRACT
The advent of mobile health (mHealth) technologies challenges the capabilities of current visualizations, interactive tools, and algorithms. We present Chronodes, an interactive system that unifies data mining and human-centric visualization techniques to support explorative analysis of longitudinal mHealth data. Chronodes extracts and visualizes frequent event sequences that reveal chronological patterns across multiple participant timelines of mHealth data. It then combines novel interaction and visualization techniques to enable multifocus event sequence analysis, which allows health researchers to interactively define, explore, and compare groups of participant behaviors using event sequence combinations. Through summarizing insights gained from a pilot study with 20 behavioral and biomedical health experts, we discuss Chronodes's efficacy and potential impact in the mHealth domain. Ultimately, we outline important open challenges in mHealth, and offer recommendations and design guidelines for future research.
ABSTRACT
Previous research is inconsistent as to whether a more labile (faster-changing) autonomic system confers performance advantages, or disadvantages, in infants and children. To examine this, we presented a stimulus battery consisting of mixed static and dynamic viewing materials to a cohort of 63 typical 12-month-old infants. While viewing the battery, infants' spontaneous visual attention (looks to and away from the screen) was measured. Concurrently, arousal was recorded via heart rate (HR), electrodermal activity, head velocity, and peripheral movement levels. In addition, stress reactivity was assessed using a mild behavioral stressor (watching a video of another infant crying). We found that infants who were generally more attentive showed smaller HR increases to the stressor. However, they also showed greater phasic autonomic changes to attractive, attention-getting stimulus events, a faster rate of change of both look duration and of arousal, and more general oscillatory activity in arousal. Finally, 4 sessions of attention training were applied to a subset of the infants (24 trained, 24 active controls), which had the effect of increasing visual sustained attention. No changes in HR responses to stressor were observed as a result of training, but concomitant increases in arousal lability were observed. Our results point to 2 contrasting autonomic profiles: infants with high autonomic reactivity to stressors show short attention durations, whereas infants with lower autonomic reactivity show longer attention durations and greater arousal lability. (PsycINFO Database Record
Subject(s)
Arousal/physiology , Attention/physiology , Stress, Physiological/physiology , Autonomic Nervous System , Child Development/physiology , Cohort Studies , Cross-Sectional Studies , Female , Heart Rate/physiology , Humans , Infant , MaleABSTRACT
Traditional accounts of developing attention and cognition emphasize static individual differences in information encoding; however, work from Aston-Jones et al. suggests that looking behavior may be dynamically influenced by autonomic arousal. To test this model, a 20-min testing battery constituting mixed photos and cartoon clips was shown to 53 typical 12-month-olds. Look duration was recorded to index attention, and continuous changes in arousal were tracked by measuring heart rate, electrodermal activity, and movement levels. Across three analyses, we found that continuous changes in arousal tracked simultaneous changes in attention measures, as predicted by the Aston-Jones model. It was also found that changes in arousal tended to precede (occur before) subsequent changes in attention. Implications of these findings are discussed.
Subject(s)
Attention/physiology , Child Development/physiology , Galvanic Skin Response/physiology , Heart Rate/physiology , Movement/physiology , Female , Humans , Infant , MaleABSTRACT
Acute stress attenuates frontal lobe functioning and increases distractibility while enhancing subcortical processes in both human and nonhuman animals (reviewed by Arnsten [2009] Nature Reviews Neuroscience, 10(6):410-422). To date however these relations have not been examined for their potential effects in developing populations. Here, we examined the relationship between stress reactivity (infants' heart rate response to watching videos of another child crying) and infant performance on measures of looking duration and visual recognition memory. Our findings indicate that infants with increased stress reactivity showed shorter look durations and more novelty preference. Thus, stress appears to lead to a faster, more stimulus-ready attentional profile in infants. Additional work is required to assess potential negative consequences of stimulus-responsivity, such as decreased focus or distractibility. © 2016 Wiley Periodicals, Inc. Dev Psychobiol 58: 546-555, 2016.
Subject(s)
Attention/physiology , Heart Rate/physiology , Infant Behavior/physiology , Pattern Recognition, Visual/physiology , Recognition, Psychology/physiology , Stress, Psychological , Choice Behavior/physiology , Female , Humans , Infant , Male , Time FactorsABSTRACT
Previous developmental accounts of joint object activity identify a qualitative "shift" around 9-12 months. In a longitudinal study of 26 dyads, videos of joint object interactions at 4, 6, 9, and 12 months were coded for all targets of gaze and manual activity (at 10 Hz). At 12 months, infants distribute their sensorimotor modalities between objects handled by the parent and others controlled by the infant. Analyses reveal novel trajectories in distributed joint object activity across the 1st year. At 4 months, infants predominantly look at and manipulate a single object, typically held by their mothers. Between 6 and 9 months, infants increasingly decouple their visual and haptic modalities and distribute their attention between objects held by their mothers and by themselves. These previously unreported developments in the distribution of multimodal object activity might "bridge the gap" to coordinated joint activity between 6 and 12 months.