Mothers speak less to infants during detected real-world phone use.

The current study is the first to document the real-time association between phone use and speech to infants in extended real-world interactions. N= 16 predominantly White (75%) mother-infant dyads (infants aged M = 4.1 months, SD = 2.3; 63% female) shared 16,673 min of synchronized real-world phone use and Language Environment Analysis audio data over the course of 1 week (collected 2017-2020) for our analyses. Maternal phone use was associated with a 16% decrease in infants' speech input, with shorter intervals of phone use (1-2 min) associated with a greater 26% decrease in speech input relative to longer periods. This work highlights the value of multimodal sensing to access dynamic, within-person, and context-specific predictors of speech to infants in real-world settings.

Spontaneous infant crying modulates vagal activity in real time.

Porges' polyvagal theory (1991) proposes that the activity of the vagal nerve modulates moment-by-moment changes in adaptive behavior during stress. However, most work, including research with infants, has only examined vagal changes at low temporal resolutions, averaging 30+ s across phases of structured stressor paradigms. Thus, the true timescale of vagal regulation-and the extent to which it can be observed during unprompted crying-is unknown. The current study utilized a recently validated method to calculate respiratory sinus arrhythmia (RSA) dynamically at a high resolution of 5 Hz (updated every 200 ms) in a home-based infant study. Using an event-related analysis, we calculated the relative change in RSA around the onset of naturally occurring unprompted instances of n = 41 infants' 180 crying events. As predicted, RSA significantly decreased after the onset of crying compared to non-crying chance changes in RSA. Decreasing trends in RSA were driven by infants with higher pre-cry RSA values, infants rated lower in Negative Affectivity, and those rated both high and low in Orienting by their mothers. Our results display the timescale of RSA in spontaneous and naturalistic episodes of infant crying and that these dynamic RSA patterns are aligned with real-time levels of RSA and also caregiver-reported temperament.

Validating a model to detect infant crying from naturalistic audio.

Human infant crying evolved as a signal to elicit parental care and actively influences caregiving behaviors as well as infant-caregiver interactions. Automated cry detection algorithms have become more popular in recent decades, and while some models exist, they have not been evaluated thoroughly on daylong naturalistic audio recordings. Here, we validate a novel deep learning cry detection model by testing it in assessment scenarios important to developmental researchers. We also evaluate the deep learning model's performance relative to LENA's cry classifier, one of the most commonly used commercial software systems for quantifying child crying. Broadly, we found that both deep learning and LENA model outputs showed convergent validity with human annotations of infant crying. However, the deep learning model had substantially higher accuracy metrics (recall, F1, kappa) and stronger correlations with human annotations at all timescales tested (24 h, 1 h, and 5 min) relative to LENA. On average, LENA underestimated infant crying by 50 min every 24 h relative to human annotations and the deep learning model. Additionally, daily infant crying times detected by both automated models were lower than parent-report estimates in the literature. We provide recommendations and solutions for leveraging automated algorithms to detect infant crying in the home and make our training data and model code open source and publicly available.

Longform recordings of everyday life: Ethics for best practices.

Recent advances in large-scale data storage and processing offer unprecedented opportunities for behavioral scientists to collect and analyze naturalistic data, including from underrepresented groups. Audio data, particularly real-world audio recordings, are of particular interest to behavioral scientists because they provide high-fidelity access to subtle aspects of daily life and social interactions. However, these methodological advances pose novel risks to research participants and communities. In this article, we outline the benefits and challenges associated with collecting, analyzing, and sharing multi-hour audio recording data. Guided by the principles of autonomy, privacy, beneficence, and justice, we propose a set of ethical guidelines for the use of longform audio recordings in behavioral research. This article is also accompanied by an Open Science Framework Ethics Repository that includes informed consent resources such as frequent participant concerns and sample consent forms.

Automated sensing of daily activity: A new lens into development.

Rapidly maturing technologies for sensing and activity recognition can provide unprecedented access to the complex structure daily activity and interaction, promising new insight into the mechanisms by which experience shapes developmental outcomes. Motion data, autonomic activity, and "snippets" of audio and video recordings can be conveniently logged by wearable sensors (Lazer et al., 2009). Machine learning algorithms can process these signals into meaningful markers, from child and parent behavior to outcomes such as depression or teenage drinking. Theoretically motivated aspects of daily activity can be combined and synchronized to examine reciprocal effects between children's behaviors and their environments or internal processes. Captured over longitudinal time, such data provide a new opportunity to study the processes by which individual differences emerge and stabilize. This paper introduces the reader to developments in sensing and activity recognition with implications for developmental phenomena across the lifespan, sketching a framework for leveraging mobile sensors for transactional analyses that bridge micro- and longitudinal- timescales of development. It finishes by detailing resources and best practices to facilitate the next generation of developmentalists to contribute to this emerging area.

Infant Attention Is Dynamically Modulated With Changing Arousal Levels.

Traditional accounts of developing attention and cognition emphasize static individual differences in information encoding; however, work from Aston-Jones et al. suggests that looking behavior may be dynamically influenced by autonomic arousal. To test this model, a 20-min testing battery constituting mixed photos and cartoon clips was shown to 53 typical 12-month-olds. Look duration was recorded to index attention, and continuous changes in arousal were tracked by measuring heart rate, electrodermal activity, and movement levels. Across three analyses, we found that continuous changes in arousal tracked simultaneous changes in attention measures, as predicted by the Aston-Jones model. It was also found that changes in arousal tended to precede (occur before) subsequent changes in attention. Implications of these findings are discussed.

Sensorimotor Decoupling Contributes to Triadic Attention: A Longitudinal Investigation of Mother-Infant-Object Interactions.

Previous developmental accounts of joint object activity identify a qualitative "shift" around 9-12 months. In a longitudinal study of 26 dyads, videos of joint object interactions at 4, 6, 9, and 12 months were coded for all targets of gaze and manual activity (at 10 Hz). At 12 months, infants distribute their sensorimotor modalities between objects handled by the parent and others controlled by the infant. Analyses reveal novel trajectories in distributed joint object activity across the 1st year. At 4 months, infants predominantly look at and manipulate a single object, typically held by their mothers. Between 6 and 9 months, infants increasingly decouple their visual and haptic modalities and distribute their attention between objects held by their mothers and by themselves. These previously unreported developments in the distribution of multimodal object activity might "bridge the gap" to coordinated joint activity between 6 and 12 months.

Stress reactivity speeds basic encoding processes in infants.

Acute stress attenuates frontal lobe functioning and increases distractibility while enhancing subcortical processes in both human and nonhuman animals (reviewed by Arnsten [2009] Nature Reviews Neuroscience, 10(6):410-422). To date however these relations have not been examined for their potential effects in developing populations. Here, we examined the relationship between stress reactivity (infants' heart rate response to watching videos of another child crying) and infant performance on measures of looking duration and visual recognition memory. Our findings indicate that infants with increased stress reactivity showed shorter look durations and more novelty preference. Thus, stress appears to lead to a faster, more stimulus-ready attentional profile in infants. Additional work is required to assess potential negative consequences of stimulus-responsivity, such as decreased focus or distractibility. © 2016 Wiley Periodicals, Inc. Dev Psychobiol 58: 546-555, 2016.

Maternal contingent responses to distress facilitate infant soothing but not in mothers with depression or infants high in negative affect.

Depression in mothers is consistently associated with reduced caregiving sensitivity and greater infant negative affect expression. The current article examined the real-time behavioral mechanisms underlying these associations using Granger causality time series analyses in a sample of mothers (N = 194; 86.60% White) at elevated risk for depression and their 3-month-old infants (46.40% female) living in a major metropolitan area in the United States. Overall, mothers contingently responded to infant distress, and mothers' responses to infant distress increased the likelihood of infant soothing in real time. However, there was no evidence for maternal contingent responding or facilitation of infant soothing in subsamples of mothers who were currently experiencing elevated depression symptoms or in mothers of highly negative infants. These findings suggest real-time behavioral mechanisms by which risks for maladaptive self-regulation may develop. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Auditory chaos classification in real-world environments.

Background & motivation: Household chaos is an established risk factor for child development. However, current methods for measuring household chaos rely on parent surveys, meaning existing research efforts cannot disentangle potentially dynamic bidirectional relations between high chaos environments and child behavior problems. Proposed approach: We train and make publicly available a classifier to provide objective, high-resolution predictions of household chaos from real-world child-worn audio recordings. To do so, we collect and annotate a novel dataset of ground-truth auditory chaos labels compiled from over 411 h of daylong recordings collected via audio recorders worn by N=22 infants in their homes. We leverage an existing sound event classifier to identify candidate high chaos segments, increasing annotation efficiency 8.32× relative to random sampling. Result: Our best-performing model successfully classifies four levels of real-world household auditory chaos with a macro F1 score of 0.701 (Precision: 0.705, Recall: 0.702) and a weighted F1 score of 0.679 (Precision: 0.685, Recall: 0.680). Significance: In future work, high-resolution objective chaos predictions from our model can be leveraged for basic science and intervention, including testing theorized mechanisms by which chaos affects children's cognition and behavior. Additionally, to facilitate further model development we make publicly available the first and largest balanced annotated audio dataset of real-world household chaos.

Infant crying predicts real-time fluctuations in maternal mental health in ecologically valid home settings.

Exposure to infant crying is a well-established predictor of mothers' mental health. However, this association may reflect many potential mechanisms. Capturing dynamic fluctuations in mothers' states simultaneously with caregiving experiences is necessary to identify the real-time processes influencing mental health. In this study, we leveraged ecological momentary assessments (EMAs) and infant-worn audio recorders to capture variability in mothers' mental health symptoms and their exposure to infant crying over one week in a racially and socio-economically diverse urban North-American sample (N = 53). We use multilevel modeling to characterize within- and between-person effects of crying on maternal negative affect and symptoms of depression and anxiety. Within participants, when infants cried more than average in the 10 min, 1 hr, and 8 hr prior to an EMA report, mothers' negative affect subsequently increased, controlling for mean levels of infant crying. In contrast to findings from laboratory studies, in everyday settings crying exposure did not immediately increase feelings of depression. Only when crying was above average for 8 hr prior to EMA did mothers report increases in subsequent depression symptoms, suggesting that the effects of crying on maternal mental health take hours to unfold in ecologically valid home settings. Between participants, mothers of infants who cried more on average did not report higher negative affect or symptoms of depression or anxiety. Overall, our findings reveal that crying exposure dynamically influences maternal negative affect and depression but not anxiety in ecologically valid real-world settings. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Ten lessons about infants' everyday experiences.

Audio recorders, accelerometers, and cameras that infants wear throughout their everyday lives capture the experiences that are available to shape development. Everyday sensing in infancy reveals patterns within the everyday hubbub that are unknowable using methods that capture shorter, more isolated, or more planned slices of behavior. Here, we review ten lessons learned from recent endeavors that removed researchers from designing or participating in infants' experiences and instead quantified patterns that arose within infants' own spontaneously arising everyday experiences. The striking heterogeneity of experiences - there is no meaningfully "representative" hour of a day, instance of a category, interaction context, or infant - inspires next steps in theory and practice that embrace the complex, dynamic, and multiple pathways of human development.

INFANT CRYING DETECTION IN REAL-WORLD ENVIRONMENTS.

Most existing cry detection models have been tested with data collected in controlled settings. Thus, the extent to which they generalize to noisy and lived environments is unclear. In this paper, we evaluate several established machine learning approaches including a model leveraging both deep spectrum and acoustic features. This model was able to recognize crying events with F1 score 0.613 (Precision: 0.672, Recall: 0.552), showing improved external validity over existing methods at cry detection in everyday real-world settings. As part of our evaluation, we collect and annotate a novel dataset of infant crying compiled from over 780 hours of labeled real-world audio data, captured via recorders worn by infants in their homes, which we make publicly available. Our findings confirm that a cry detection model trained on in-lab data underperforms when presented with real-world data (in-lab test F1: 0.656, real-world test F1: 0.236), highlighting the value of our new dataset and model.

Micro-analysis of infant looking in a naturalistic social setting: insights from biologically based models of attention.

A current theory of attention posits that several micro-indices of attentional vigilance are dependent on activation of the locus coeruleus, a brainstem nucleus that regulates cortical norepinephrine activity (Aston-Jones et al., 1999). This theory may account for many findings in the infant literature, while highlighting important new areas for research and theory on infant attention. We examined the visual behaviors of n = 16 infants (6-7 months) while they attended to multiple spatially distributed targets in a naturalistic environment. We coded four measures of attentional vigilance, adapted from studies of norepinergic modulation of animal attention: rate of fixations, duration of fixations, latency to reorientation, and target 'hits'. These measures showed a high degree of coherence in individual infants, in parallel with findings from animal studies. Results also suggest that less vigilant infants showed greater habituation to the trial structure and more attentiveness to less salient stimuli during periods of high attentional competition. This pattern of results is predicted by the Aston-Jones model of attention, but could not be explained by the standard information processing model.

Sensing everyday activity: Parent perceptions and feasibility.

Mobile and wearable sensors provide a unique opportunity to capture the daily activities and interactions that shape developmental trajectories, with potential to revolutionize the study of development (de Barbaro, 2019). However, developmental research employing sensors is still in its infancy, and parents' comfort using these devices is uncertain. This exploratory report assesses parent willingness to participate in sensor studies via a nationally representative survey (N = 210) and live recruitment of a low-income, minority population for an ongoing study (N = 359). The survey allowed us to assess how protocol design influences acceptability, including various options for devices and datastream resolution, conditions of data sharing, and feedback. By contrast, our recruitment data provided insight into parents' true willingness to participate in a sensor study, with a protocol including 72 h of continuous audio, motion, and physiological data. Our results indicate that parents are relatively conservative when considering participation in sensing studies. However, nearly 41 % of surveyed parents reported that they would be at least somewhat willing to participate in studies with audio or video recordings, 26 % were willing or extremely willing, and 14 % reported being extremely willing. These results roughly paralleled our recruitment results, where 58 % of parents indicated interest, 29 % of parents scheduled to participate, and 10 % ultimately participated. Additionally, 70 % of caregivers stated their reason for not participating in the study was due to barriers unrelated to sensing while about 25 % noted barriers due to either privacy concerns or the physical sensors themselves. Parents' willingness to collect sensitive datastreams increased if data stayed within the household for individual use only, are shared anonymously with researchers, or if parents receive feedback from devices. Overall, our findings suggest that given the correct circumstances, mobile sensors are a feasible and promising tool for characterizing children's daily interactions and their role in development.

Multi-modal data collection for measuring health, behavior, and living environment of large-scale participant cohorts.

BACKGROUND: As mobile technologies become ever more sensor-rich, portable, and ubiquitous, data captured by smart devices are lending rich insights into users' daily lives with unprecedented comprehensiveness and ecological validity. A number of human-subject studies have been conducted to examine the use of mobile sensing to uncover individual behavioral patterns and health outcomes, yet minimal attention has been placed on measuring living environments together with other human-centered sensing data. Moreover, the participant sample size in most existing studies falls well below a few hundred, leaving questions open about the reliability of findings on the relations between mobile sensing signals and human outcomes. RESULTS: To address these limitations, we developed a home environment sensor kit for continuous indoor air quality tracking and deployed it in conjunction with smartphones, Fitbits, and ecological momentary assessments in a cohort study of up to 1,584 college student participants per data type for 3 weeks. We propose a conceptual framework that systematically organizes human-centric data modalities by their temporal coverage and spatial freedom. Then we report our study procedure, technologies and methods deployed, and descriptive statistics of the collected data that reflect the participants' mood, sleep, behavior, and living environment. CONCLUSIONS: We were able to collect from a large participant cohort satisfactorily complete multi-modal sensing and survey data in terms of both data continuity and participant adherence. Our novel data and conceptual development provide important guidance for data collection and hypothesis generation in future human-centered sensing studies.

Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series.

The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the non-linear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique's application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a "Programming Basics" module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data.

Optimal sampling strategies for characterizing behavior and affect from ambulatory audio recordings.

Advances in mobile and wearable technologies mean it is now feasible to record hours to days of participant behavior in its naturalistic context, a great boon for psychologists interested in family processes and development. While automated activity recognition algorithms exist for a limited set of behaviors, time-consuming human annotations are still required to robustly characterize the vast majority of behavioral and affective markers of interest. This report is the first to date which systematically tests the efficacy of different sampling strategies for characterizing behavior from audio recordings to provide practical guidelines for researchers. Using continuous audio recordings of the daily lives of 11 preschool-aged children, we compared sampling techniques to determine the most accurate and efficient approach. Results suggest that sampling both low and high frequency verbal and overt behaviors is best if samples are short in duration, systematically rather than randomly selected, and sampled to cover at least 12.5% of recordings. Implications for assessment of real-world behavior are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

A Real-Time Eating Detection System for Capturing Eating Moments and Triggering Ecological Momentary Assessments to Obtain Further Context: System Development and Validation Study.

BACKGROUND: Eating behavior has a high impact on the well-being of an individual. Such behavior involves not only when an individual is eating, but also various contextual factors such as with whom and where an individual is eating and what kind of food the individual is eating. Despite the relevance of such factors, most automated eating detection systems are not designed to capture contextual factors. OBJECTIVE: The aims of this study were to (1) design and build a smartwatch-based eating detection system that can detect meal episodes based on dominant hand movements, (2) design ecological momentary assessment (EMA) questions to capture meal contexts upon detection of a meal by the eating detection system, and (3) validate the meal detection system that triggers EMA questions upon passive detection of meal episodes. METHODS: The meal detection system was deployed among 28 college students at a US institution over a period of 3 weeks. The participants reported various contextual data through EMAs triggered when the eating detection system correctly detected a meal episode. The EMA questions were designed after conducting a survey study with 162 students from the same campus. Responses from EMAs were used to define exclusion criteria. RESULTS: Among the total consumed meals, 89.8% (264/294) of breakfast, 99.0% (406/410) of lunch, and 98.0% (589/601) of dinner episodes were detected by our novel meal detection system. The eating detection system showed a high accuracy by capturing 96.48% (1259/1305) of the meals consumed by the participants. The meal detection classifier showed a precision of 80%, recall of 96%, and F1 of 87.3%. We found that over 99% (1248/1259) of the detected meals were consumed with distractions. Such eating behavior is considered "unhealthy" and can lead to overeating and uncontrolled weight gain. A high proportion of meals was consumed alone (680/1259, 54.01%). Our participants self-reported 62.98% (793/1259) of their meals as healthy. Together, these results have implications for designing technologies to encourage healthy eating behavior. CONCLUSIONS: The presented eating detection system is the first of its kind to leverage EMAs to capture the eating context, which has strong implications for well-being research. We reflected on the contextual data gathered by our system and discussed how these insights can be used to design individual-specific interventions.

New meanings of thin-skinned: The contrasting attentional profiles of typical 12-month-olds who show high, and low, stress reactivity.

Previous research is inconsistent as to whether a more labile (faster-changing) autonomic system confers performance advantages, or disadvantages, in infants and children. To examine this, we presented a stimulus battery consisting of mixed static and dynamic viewing materials to a cohort of 63 typical 12-month-old infants. While viewing the battery, infants' spontaneous visual attention (looks to and away from the screen) was measured. Concurrently, arousal was recorded via heart rate (HR), electrodermal activity, head velocity, and peripheral movement levels. In addition, stress reactivity was assessed using a mild behavioral stressor (watching a video of another infant crying). We found that infants who were generally more attentive showed smaller HR increases to the stressor. However, they also showed greater phasic autonomic changes to attractive, attention-getting stimulus events, a faster rate of change of both look duration and of arousal, and more general oscillatory activity in arousal. Finally, 4 sessions of attention training were applied to a subset of the infants (24 trained, 24 active controls), which had the effect of increasing visual sustained attention. No changes in HR responses to stressor were observed as a result of training, but concomitant increases in arousal lability were observed. Our results point to 2 contrasting autonomic profiles: infants with high autonomic reactivity to stressors show short attention durations, whereas infants with lower autonomic reactivity show longer attention durations and greater arousal lability. (PsycINFO Database Record