Automated facial expression measurement in a longitudinal sample of 4- and 8-month-olds: Baby FaceReader 9 and manual coding of affective expressions.

Facial expressions are among the earliest behaviors infants use to express emotional states, and are crucial to preverbal social interaction. Manual coding of infant facial expressions, however, is laborious and poses limitations to replicability. Recent developments in computer vision have advanced automated facial expression analyses in adults, providing reproducible results at lower time investment. Baby FaceReader 9 is commercially available software for automated measurement of infant facial expressions, but has received little validation. We compared Baby FaceReader 9 output to manual micro-coding of positive, negative, or neutral facial expressions in a longitudinal dataset of 58 infants at 4 and 8 months of age during naturalistic face-to-face interactions with the mother, father, and an unfamiliar adult. Baby FaceReader 9's global emotional valence formula yielded reasonable classification accuracy (AUC = .81) for discriminating manually coded positive from negative/neutral facial expressions; however, the discrimination of negative from neutral facial expressions was not reliable (AUC = .58). Automatically detected a priori action unit (AU) configurations for distinguishing positive from negative facial expressions based on existing literature were also not reliable. A parsimonious approach using only automatically detected smiling (AU12) yielded good performance for discriminating positive from negative/neutral facial expressions (AUC = .86). Likewise, automatically detected brow lowering (AU3+AU4) reliably distinguished neutral from negative facial expressions (AUC = .79). These results provide initial support for the use of selected automatically detected individual facial actions to index positive and negative affect in young infants, but shed doubt on the accuracy of complex a priori formulas.

Predicting pointing from early socioemotional communication with mothers, fathers, and strangers through the lens of temperamental reactivity.

Socioemotional and referential communication are primary expressions of interpersonal engagement in infancy and beyond. Early socioemotional communication in dyadic interactions may form a foundation for triadic referential communication and gesture production, yet the role of temperament in moderating their association has not been examined. We investigated whether early socioemotional communication behaviors, and infant temperamental reactivity, were associated with later pointing production. Participants were 51 infants (45% girls) and both their parents (86.5% Dutch). Early infant socioemotional communication (production of smile, vocalizations, and gaze) was observed during separate home-based face-to-face interactions with mothers, fathers, and strangers at 4 and 8 months. At both ages, mothers and fathers reported on infant temperamental surgency and negative affectivity, and overall means were calculated. Referential communication (declarative pointing) was measured during structured lab-based observations at 12 and 15 months. Socioemotional and referential communication behaviors were microanalytically coded second by second. Poisson multilevel regression analyses indicated interaction effects between temperament and smile, vocalizations, and gazes to the adult's face with each partner in predicting pointing. High levels of infant temperamental surgency tended to enhance positive associations between early socioemotional communication behaviors with mothers and fathers and pointing. By contrast, high levels of negative affectivity tended to dampen associations between early communication behaviors with strangers and pointing. Results highlight the importance of infant socioemotional communication with diverse partners and the moderating role of temperamental reactivity in predicting referential communication. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Cultural facial expressions dynamically convey emotion category and intensity information.

Communicating emotional intensity plays a vital ecological role because it provides valuable information about the nature and likelihood of the sender's behavior.1,2,3 For example, attack often follows signals of intense aggression if receivers fail to retreat.4,5 Humans regularly use facial expressions to communicate such information.6,7,8,9,10,11 Yet how this complex signaling task is achieved remains unknown. We addressed this question using a perception-based, data-driven method to mathematically model the specific facial movements that receivers use to classify the six basic emotions-"happy," "surprise," "fear," "disgust," "anger," and "sad"-and judge their intensity in two distinct cultures (East Asian, Western European; total n = 120). In both cultures, receivers expected facial expressions to dynamically represent emotion category and intensity information over time, using a multi-component compositional signaling structure. Specifically, emotion intensifiers peaked earlier or later than emotion classifiers and represented intensity using amplitude variations. Emotion intensifiers are also more similar across emotions than classifiers are, suggesting a latent broad-plus-specific signaling structure. Cross-cultural analysis further revealed similarities and differences in expectations that could impact cross-cultural communication. Specifically, East Asian and Western European receivers have similar expectations about which facial movements represent high intensity for threat-related emotions, such as "anger," "disgust," and "fear," but differ on those that represent low threat emotions, such as happiness and sadness. Together, our results provide new insights into the intricate processes by which facial expressions can achieve complex dynamic signaling tasks by revealing the rich information embedded in facial expressions.

Automated measures of vocal interactions and engagement in inclusive preschool classrooms.

Classroom engagement plays a crucial role in preschoolers' development, yet the correlates of engagement, especially among children with autism spectrum disorder (ASD) and developmental delays (DD), remains unknown. This study examines levels of engagement with classroom social partners and tasks among children in three groups ASD, DD, and typical development (TD). Here, we asked whether children's vocal interactions (vocalizations to and from peers and teachers) were associated with their classroom engagement with social partners (peers and teachers) and with tasks, and whether the association between classroom engagement and vocal interactions differed between children in the ASD group and their peers in the DD and TD groups. Automated measures of vocalizations and location quantified children's vocal interactions with peers and teachers over the course of the school year. Automated location and vocalization data were used to capture both (1) children's vocal output to specific peers and teachers, and (2) the vocal input they received from those peers and teachers. Participants were 72 3-5-year-olds (Mage = 48.6 months, SD = 7.0, 43% girls) and their teachers. Children in the ASD group displayed lower engagement with peers, teachers, and tasks than children in the TD group; they also showed lower engagement with peers than children in the DD group. Overall, children's own vocalizations were positively associated with engagement with social partners. Thus, although children in the ASD group tend to have lower engagement scores than children in the TD group, active participation in vocal interactions appears to support their classroom engagement with teachers and peers.

A Comparative Analysis of the Full and Short Versions of the Social Responsiveness Scale in Estimating an Established Autism Risk Factor Association in ECHO: Do we Get the Same Estimates?

PURPOSE: Prior work developed a shortened 16-item version of the Social Responsiveness Scale (SRS), a quantitative measure of social communication and autism spectrum disorder (ASD)-related traits. However, its properties for use in risk factor estimation have not been fully tested compared to the full SRS. We compared the associations between gestational age (previously established risk factor for ASD) and the 65-item "full" and 16-item "short" versions of the SRS to test the shortened version's ability to capture associations in epidemiologic analyses of ASD risk factors. METHODS: We used data from participants in the Environmental influences on Child Health Outcomes (ECHO) Program (n = 2,760). SRS scores were collected via maternal/caregiver report when children were aged 2.5-18 years. We compared estimates of associations between gestational age and preterm birth between the full and short SRS using multivariable linear regression, quantile regression, and prediction methods. RESULTS: Overall, associations based on full and short SRS scores were highly comparable. For example, we observed positive associations between preterm birth with both full ([Formula: see text]=2.8; 95% CI [1.7, 4.0]) and short ([Formula: see text]=2.9; 95% CI [1.6, 4.3]) SRS scores. Quantile regression analyses indicated similar direction and magnitude of associations across the distribution of SRS scores between gestational age with both short and full SRS scores. CONCLUSION: The comparability in estimates obtained for full and short SRS scores with an "established" ASD risk factor suggests ability of the shortened SRS in assessing associations with potential ASD-related risk factors and has implications for large-scale research studies seeking to reduce participant burden.

Automated measurement of infant and mother Duchenne facial expressions in the Face-to-Face/Still-Face.

Although still-face effects are well-studied, little is known about the degree to which the Face-to-Face/Still-Face (FFSF) is associated with the production of intense affective displays. Duchenne smiling expresses more intense positive affect than non-Duchenne smiling, while Duchenne cry-faces express more intense negative affect than non-Duchenne cry-faces. Forty 4-month-old infants and their mothers completed the FFSF, and key affect-indexing facial Action Units (AUs) were coded by expert Facial Action Coding System coders for the first 30 s of each FFSF episode. Computer vision software, automated facial affect recognition (AFAR), identified AUs for the entire 2-min episodes. Expert coding and AFAR produced similar infant and mother Duchenne and non-Duchenne FFSF effects, highlighting the convergent validity of automated measurement. Substantive AFAR analyses indicated that both infant Duchenne and non-Duchenne smiling declined from the FF to the SF, but only Duchenne smiling increased from the SF to the RE. In similar fashion, the magnitude of mother Duchenne smiling changes over the FFSF were 2-4 times greater than non-Duchenne smiling changes. Duchenne expressions appear to be a sensitive index of intense infant and mother affective valence that are accessible to automated measurement and may be a target for future FFSF research.

Understanding speech and language in tuberous sclerosis complex.

Tuberous Sclerosis Complex (TSC), is a neurocutaneous disorder, associated with a high prevalence of autism spectrum disorder (ASD; ∼50% of individuals). As TSC is a leading cause of syndromic ASD, understanding language development in this population would not only be important for individuals with TSC but may also have implications for those with other causes of syndromic and idiopathic ASD. In this mini review, we consider what is known about language development in this population and how speech and language in TSC are related to ASD. Although up to 70% of individuals with TSC report language difficulties, much of the limited research to date on language in TSC has been based on summary scores from standardized assessments. Missing is a detailed understanding of the mechanisms driving speech and language in TSC and how they relate to ASD. Here, we review recent work suggesting that canonical babbling and volubility-two precursors of language development that predict the emergence of speech and are delayed in infants with idiopathic ASD-are also delayed in infants with TSC. We then look to the broader literature on language development to identify other early precursors of language development that tend to be delayed in children with autism as a guide for future research on speech and language in TSC. We argue that vocal turn-taking, shared attention, and fast mapping are three such skills that can provide important information about how speech and language develop in TSC and where potential delays come from. The overall goal of this line of research is to not only illuminate the trajectory of language in TSC with and without ASD, but to ultimately find strategies for earlier recognition and treatment of the pervasive language difficulties in this population.

Social motivation predicts gaze following between 6 and 14 months.

Infants vary in their ability to follow others' gazes, but it is unclear how these individual differences emerge. We tested whether social motivation levels in early infancy predict later gaze following skills. We longitudinally tracked infants' (N = 82) gazes and pupil dilation while they observed videos of a woman looking into the camera simulating eye contact (i.e., mutual gaze) and then gazing toward one of two objects, at 2, 4, 6, 8, and 14 months of age. To improve measurement validity, we used confirmatory factor analysis to combine multiple observed measures to index the underlying constructs of social motivation and gaze following. Infants' social motivation-indexed by their speed of social orienting, duration of mutual gaze, and degree of pupil dilation during mutual gaze-was developmentally stable and positively predicted the development of gaze following-indexed by their proportion of time looking to the target object, first object look difference scores, and first face-to-object saccade difference scores-from 6 to 14 months of age. These findings suggest that infants' social motivation likely plays a role in the development of gaze following and highlight the use of a multi-measure approach to improve measurement sensitivity and validity in infancy research.

Objective Measurement of Social Gaze and Smile Behaviors in Children with Suspected Autism Spectrum Disorder During Administration of the Autism Diagnostic Observation Schedule, 2nd Edition.

Best practice for the assessment of autism spectrum disorder (ASD) symptom severity relies on clinician ratings of the Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2), but the association of these ratings with objective measures of children's social gaze and smiling is unknown. Sixty-six preschool-age children (49 boys, M = 39.97 months, SD = 10.58) with suspected ASD (61 confirmed ASD) were administered the ADOS-2 and provided social affect calibrated severity scores (SA CSS). Children's social gaze and smiling during the ADOS-2, captured with a camera contained in eyeglasses worn by the examiner and parent, were obtained via a computer vision processing pipeline. Children who gazed more at their parents (p = .04) and whose gaze at their parents involved more smiling (p = .02) received lower social affect severity scores, indicating fewer social affect symptoms, adjusted R2 = .15, p = .003.

Objective quantification of homophily in children with and without disabilities in naturalistic contexts.

Homophily, the tendency for individuals to preferentially interact with others similar to themselves is typically documented via self-report and, for children, adult report. Few studies have investigated homophily directly using objective measures of social movement. We quantified homophily in children with developmental disabilities (DD) and typical development (TD) using objective measures of position/orientation in preschool inclusion classrooms, designed to promote interaction between these groups of children. Objective measurements were collected using ultra-wideband radio-frequency tracking to determine social approach and social contact, measures of social movement and interaction. Observations of 77 preschoolers (47 with DD, and 30 TD) were conducted in eight inclusion classrooms on a total of 26 days. We compared DD and TD groups with respect to how children approached and shared time in social contact with peers using mixed-effects models. Children in concordant dyads (DD-DD and TD-TD) both moved toward each other at higher velocities and spent greater time in social contact than discordant dyads (DD-TD), evidencing homophily. DD-DD dyads spent less time in social contact than TD-TD dyads but were comparable to TD-TD dyads in their social approach velocities. Children's preference for similar peers appears to be a pervasive feature of their naturalistic interactions.

Infant AFAR: Automated facial action recognition in infants.

Automated detection of facial action units in infants is challenging. Infant faces have different proportions, less texture, fewer wrinkles and furrows, and unique facial actions relative to adults. For these and related reasons, action unit (AU) detectors that are trained on adult faces may generalize poorly to infant faces. To train and test AU detectors for infant faces, we trained convolutional neural networks (CNN) in adult video databases and fine-tuned these networks in two large, manually annotated, infant video databases that differ in context, head pose, illumination, video resolution, and infant age. AUs were those central to expression of positive and negative emotion. AU detectors trained in infants greatly outperformed ones trained previously in adults. Training AU detectors across infant databases afforded greater robustness to between-database differences than did training database specific AU detectors and outperformed previous state-of-the-art in infant AU detection. The resulting AU detection system, which we refer to as Infant AFAR (Automated Facial Action Recognition), is available to the research community for further testing and applications in infant emotion, social interaction, and related topics.

Classroom language during COVID-19: Associations between mask-wearing and objectively measured teacher and preschooler vocalizations.

During the COVID-19 pandemic, mask-wearing in classrooms has become commonplace. However, there are little data on the effect of face-masks on children's language input and production in educational contexts, like preschool classrooms which over half of United States children attend. Leveraging repeated objective measurements, we longitudinally examined child and teacher speech-related vocalizations in two cohorts of 3.5-4.5-year-old children enrolled in the same oral language classroom that included children with and without hearing loss. Cohort 1 was observed before COVID-19 (no face-masks, N = 20) and Cohort 2 was observed during COVID-19 (with face-masks; N = 15). Vocalization data were collected using child-worn audio recorders over 12 observations spanning two successive school years, yielding 9.09 mean hours of audio recording per child. During COVID-19 teachers produced a higher number of words per minute than teachers observed prior to COVID-19. However, teacher vocalizations during COVID-19 contained fewer unique phonemes than teacher vocalizations prior to COVID-19. Children observed during COVID-19 did not exhibit deficits in the duration, rate, or phonemic diversity of their vocalizations compared to children observed prior to COVID-19. Children observed during COVID-19 produced vocalizations that were longer in duration than vocalizations of children observed prior to COVID-19. During COVID-19 (but not before), children who were exposed to a higher number of words per minute from teachers produced more speech-related vocalizations per minute themselves. Overall, children with hearing loss were exposed to teacher vocalizations that were longer in duration, more teacher words per minute, and more phonemically diverse teacher speech than children with typical hearing. In terms of production, children with hearing loss produced vocalizations that were longer in duration than the vocalizations of children with typical hearing. Among children observed during COVID-19, children with hearing loss exhibited a higher vocalization rate than children with typical hearing. These results suggest that children's language production is largely unaffected by mask use in the classroom and that children can benefit from the language they are exposed to despite teacher mask-wearing.

Infant temperamental fear, pupil dilation, and gaze aversion from smiling strangers.

In childhood, higher levels of temperamental fear-an early-emerging proclivity to distress in the face of novelty-are associated with lower social responsivity and greater social anxiety. While the early emergence of temperamental fear in infancy is poorly understood, it is theorized to be driven by individual differences in reactivity and self-regulation to novel stimuli. The current study used eye tracking to capture infants' (N = 124) reactions to a video of a smiling stranger-a common social encounter-including infant gaze aversions from the stranger's face (indexing arousal regulation) and pupil dilation (indexing physiological reactivity), longitudinally at 2, 4, 6, and 8 months of age. Multilevel mixed-effects models indicated that more fearful infants took more time to look away from a smiling stranger's face than less fearful infants, suggesting that high-fear infants may have slower arousal regulation. At 2 and 4 months, more fearful infants also exhibited greater and faster pupil dilation before gaze aversions, consistent with greater physiological reactivity. Together, these findings suggest that individual differences in infants' gaze aversions and pupil dilation can index the development of fearful temperament in early infancy, facilitating the identification of, and interventions for, risk factors to social disruptions.

Early predictors of language skills at 3 years of age vary based on diagnostic outcome: A baby siblings research consortium study.

While previous work has identified the early predictors of language skills in infants at elevated familial risk (ER) and low familial risk (LR) for autism spectrum disorder (ASD), no studies to date have explored whether these predictors vary based on diagnostic outcome of ASD or no ASD. The present study used a large, multisite dataset to examine associations between a set of commonly studied predictor variables (infant gesture abilities, fine motor skills, nonverbal cognition, and maternal education level), measured at 12 months, and language skills, measured at 3 years, across three diagnostic outcome groups-infants with ASD ("ASD"), ER infants without ASD ("ER-no ASD"), and LR infants without ASD ("LR-no ASD"). Findings revealed that the predictors of language skills differed across groups, as gesture abilities were positively associated with language skills in the ER-no ASD group but negatively associated with language skills in the ASD group. Furthermore, maternal education level was positively associated with language skills in the ASD and LR-no ASD groups only. Variability in these early predictors may help explain why language skills are heterogeneous across the autism spectrum, and, with further study, may help clinicians identify those in need of additional and/or specialized intervention services that support language development. LAY SUMMARY: The present study identified predictors of language skills in infants with and without autism spectrum disorder (ASD). Maternal education level and 12-month gesture abilities predicted 3-year language skills in infants with ASD. Measuring these predictors early in life may help identify infants and families in need of additional and/or specialized intervention services that support language development.

Objective measurement of vocalizations in the assessment of autism spectrum disorder symptoms in preschool age children.

Assessment of autism spectrum disorder (ASD) relies on expert clinician observation and judgment, but objective measurement tools have the potential to provide additional information on ASD symptom severity. Diagnostic evaluations for ASD typically include the autism diagnostic observation schedule (ADOS-2), a semi-structured assessment composed of a series of social presses. The current study examined associations between concurrent objective features of child vocalizations during the ADOS-2 and examiner-rated autism symptom severity. The sample included 66 children (49 male; M = 40 months, SD = 10.58) evaluated in a university-based clinic, 61 of whom received an ASD diagnosis. Research reliable administration of the ADOS-2 provided social affect (SA) and restricted and repetitive behavior (RRB) calibrated severity scores (CSS). Audio was recorded from examiner-worn eyeglasses during the ADOS-2 and child and adult speech were differentiated with LENA SP Hub. PRAAT was used to ascertain acoustic features of the audio signal, specifically the mean fundamental vocal frequency (F0) of LENA-identified child speech-like vocalizations (those with phonemic content), child cry vocalizations, and adult speech. Sphinx-4 was employed to estimate child and adult phonological features indexed by the average consonant and vowel count per vocalization. More than a quarter of the variance in ADOS-2 RRB CSS was predicted by the combination of child phoneme count per vocalization and child vocalization F0. Findings indicate that both acoustic and phonological features of child vocalizations are associated with expert clinician ratings of autism symptom severity. LAY SUMMARY: Determination of the severity of autism spectrum disorder is based in part on expert (but subjective) clinician observations during the ADOS-2. Two characteristics of child vocalizations-a smaller number of speech-like sounds per vocalization and higher pitched vocalizations (including cries)-were associated with greater autism symptom severity. The results suggest that objectively ascertained characteristics of children's vocalizations capture variance in children's restricted and repetitive behaviors that are reflected in clinician severity indices.

Beyond the familial: The development of emotional communication with mothers, fathers, and strangers.

Interaction with unfamiliar partners is a component of social life from infancy onward. Yet little is known about preverbal communication with strangers. This study compared the development of infant communication with strangers to communication with mothers and fathers and examined the contribution of temperament to partner-specific communication patterns. A sample of 58 infants was observed at four and eight months during separate home-based face-to-face interactions with three partners (mother, father, and stranger). Infant visual, facial, and vocal communication behaviors were coded microanalytically. Each parent reported on infant temperament at both ages. Multilevel regression analyses indicated that infants gazed longer at strangers than at fathers, exhibited less smiling to strangers than to mothers, and produced fewer vocalizations with strangers than with either parent. Both age and temperament moderated these differences. Vocal communication with fathers became more frequent at eight months; smiling to mothers was accentuated among infants with higher levels of temperamental surgency. Importantly, levels of communication behaviors with strangers were concurrently and longitudinally associated with those with mothers and fathers. Overall, findings suggest that infant emotional communication patterns are modulated by individual temperamental differences and are reproduced in and over time, though at different levels, when interacting with novel partners.

Reciprocal Patterns of Peer Speech in Preschoolers with and without Hearing Loss.

Children with hearing loss often attend inclusive preschool classrooms aimed at improving their spoken language skills. Although preschool classrooms are fertile environments for vocal interaction with peers, little is known about the dyadic processes that influence children's speech to one another and foster their language abilities and how these processes may vary in children with hearing loss. We used new objective measurement approaches to identify and quantify children's vocalizations during social contact, as determined by children's proximity and mutual orientation. The contributions of peer vocalizations to children's future vocalizations and language abilities were examined in oral language inclusion classrooms containing children with hearing loss who use hearing aids or cochlear implants and their typically hearing peers. Across over 600 hours of recorded vocal interactions of twenty-nine 2.5-3.5 year olds (16 girls) in three cohorts of children in a classroom, we found that vocalizations from each peer on a given observation predicted a child's vocalizations to that same peer on the subsequent observation. Children who produced more vocalizations to their peers had higher receptive and expressive language abilities, as measured by a standardized end-of-year language assessment. In fact, vocalizations from peers had an indirect association with end-of-year language abilities as mediated by children's vocalizations to peers. These findings did not vary as a function of hearing status. Overall, then, the results demonstrate the importance of dyadic peer vocal interactions for children's language use and abilities.

Simulating COVID19 transmission from observed movement.

Current models of COVID-19 transmission predict infection from reported or assumed interactions. Here we leverage high-resolution observations of interaction to simulate infectious processes. Ultra-Wide Radio Frequency Identification (RFID) systems were employed to track the real-time physical movements and directional orientation of children and their teachers in 4 preschool classes over a total of 34 observations. An agent-based transmission model combined observed interaction patterns (individual distance and orientation) with CDC-published risk guidelines to estimate the transmission impact of an infected patient zero attending class on the proportion of overall infections, the average transmission rate, and the time lag to the appearance of symptomatic individuals. These metrics highlighted the prophylactic role of decreased classroom density and teacher vaccinations. Reduction of classroom density to half capacity was associated with an 18.2% drop in overall infection proportion while teacher vaccination receipt was associated with a 25.3% drop. Simulation results of classroom transmission dynamics may inform public policy in the face of COVID-19 and similar infectious threats.