Intersensory processing of faces and voices at 6 months predicts language outcomes at 18, 24, and 36 months of age.

Intersensory processing of social events (e.g., matching sights and sounds of audiovisual speech) is a critical foundation for language development. Two recently developed protocols, the Multisensory Attention Assessment Protocol (MAAP) and the Intersensory Processing Efficiency Protocol (IPEP), assess individual differences in intersensory processing at a sufficiently fine-grained level for predicting developmental outcomes. Recent research using the MAAP demonstrates 12-month intersensory processing of face-voice synchrony predicts language outcomes at 18- and 24-months, holding traditional predictors (parent language input, SES) constant. Here, we build on these findings testing younger infants using the IPEP, a more comprehensive, fine-grained index of intersensory processing. Using a longitudinal sample of 103 infants, we tested whether intersensory processing (speed, accuracy) of faces and voices at 3- and 6-months predicts language outcomes at 12-, 18-, 24-, and 36-months, holding traditional predictors constant. Results demonstrate intersensory processing of faces and voices at 6-months (but not 3-months) accounted for significant unique variance in language outcomes at 18-, 24-, and 36-months, beyond that of traditional predictors. Findings highlight the importance of intersensory processing of face-voice synchrony as a foundation for language development as early as 6-months and reveal that individual differences assessed by the IPEP predict language outcomes even 2.5-years later.

Building language learning: Relations between infant attention and social contingency in the first year of life.

In Western societies, social contingency, or prompt and meaningful back-and-forth exchanges between infant and caregiver, is a powerful feature of the early language environment. Research suggests that infants with better attentional skills engage in more social contingency during interactions with adults and, in turn, social contingency supports infant attention. This reciprocity is theorized to build infant language skills as the adult capitalizes on and extends the infant's attention during socially contingent interactions. Using data from 104 infants and caregivers, this paper tests reciprocal relations between infant attention and social contingency at 6- and 12-months and the implications for infant vocabulary at 18-months. Infant attentional skills to social (women speaking) and nonsocial (objects dropping) events were assessed, and social contingency was examined during an 8-minute toy play interaction with a caregiver. Child receptive and expressive vocabulary was measured by caregiver-report. Both social and nonsocial attentional skills related to engagement in social contingency during caregiver-infant interaction, though only models that included social attention and social contingency predicted vocabulary. These findings provide empirical evidence for the proposed reciprocal relations between infant attention and social contingency as well as how they relate to later language.

Individual Differences in Multisensory Attention Skills in Children with Autism Spectrum Disorder Predict Language and Symptom Severity: Evidence from the Multisensory Attention Assessment Protocol (MAAP).

Children with autism spectrum disorders (ASD) show atypical attention, particularly for social events. The new Multisensory Attention Assessment Protocol (MAAP) assesses fine-grained individual differences in attention disengagement, maintenance, and audiovisual matching for social and nonsocial events. We investigated the role of competing stimulation on attention, and relations with language and symptomatology in children with ASD and typical controls. Findings revealed: (1) the MAAP differentiated children with ASD from controls, (2) greater attention to social events predicted better language for both groups and lower symptom severity in children with ASD, (3) different pathways from attention to language were evident in children with ASD versus controls. The MAAP provides an ideal attention assessment for revealing diagnostic group differences and relations with outcomes.

Effects of English versus Spanish language exposure on basic multisensory attention skills across 3 to 36 months of age.

Recent research has demonstrated that individual differences in infant attention to faces and voices of women speaking predict language outcomes in childhood. These findings have been generated using two new audiovisual attention assessments appropriate for infants and young children, the Multisensory Attention Assessment Protocol (MAAP) and the Intersensory Processing Efficiency Protocol (IPEP). The MAAP and IPEP assess three basic attention skills (sustaining attention, shifting/disengaging, intersensory matching), as well as distractibility, deployed in the context of naturalistic audiovisual social (women speaking English) and nonsocial events (objects impacting a surface). Might children with differential exposure to Spanish versus English show different patterns of attention to social events on these protocols as a function of language familiarity? We addressed this question in several ways using children (n = 81 dual-language learners; n = 23 monolingual-language learners) from South Florida, tested longitudinally across 3-36 months. Surprisingly, results indicated no significant English language advantage on any attention measure for children from monolingual English versus dual English-Spanish language environments. Second, for dual-language learners, exposure to English changed across age, decreasing slightly from 3-12 months and then increasing considerably by 36 months. Furthermore, for dual-language learners, structural equation modeling analyses revealed no English language advantage on the MAAP or IPEP as a function of degree of English language exposure. The few relations found were in the direction of greater performance for children with greater Spanish exposure. Together, findings indicate no English language advantage for basic multisensory attention skills assessed by the MAAP or IPEP between the ages of 3 to 36 months. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The effects of socioeconomic status on working memory in childhood are partially mediated by intersensory processing of audiovisual events in infancy.

Socioeconomic status (SES) is a well-established predictor of individual differences in childhood language and cognitive functioning, including executive functions such as working memory. In infancy, intersensory processing-selectively attending to properties of events that are redundantly specified across the senses at the expense of non-redundant, irrelevant properties-also predicts language development. Our recent research demonstrates that individual differences in intersensory processing in infancy predict a variety of language outcomes in childhood, even after controlling for SES. However, relations among intersensory processing and cognitive outcomes such as working memory have not yet been investigated. Thus, the present study examines relations between intersensory processing in infancy and working memory in early childhood, and the role of SES in this relation. Children (N = 101) received the Multisensory Attention Assessment Protocol at 12-months to assess intersensory processing (face-voice and object-sound matching) and received the WPPSI at 36-months to assess working memory. SES was indexed by maternal education, paternal education, and income. A variety of novel findings emerged. 1) Individual differences in intersensory processing at 12-months predicted working memory at 36-months of age even after controlling for SES. 2) Individual differences in SES predicted intersensory processing at 12-months of age. 3) The well-established relation between SES and working memory was partially mediated by intersensory processing. Children from families of higher-SES have better intersensory processing skills at 12-months and this combination of factors predicts greater working memory two years later at 36-months. Together these findings reveal the role of intersensory processing in cognitive functioning.

Infant distractibility from social events mediates the relation between maternal responsiveness and infant language outcomes.

Research demonstrates that contingent and appropriate maternal responsiveness to infant requests and bids for attention leads to better language outcomes. Research also indicates that infants who are less distracted by irrelevant competing stimulation and attend efficiently to audiovisual social events (e.g., faces and voices) show better language outcomes. However, few studies have assessed relations between maternal responsiveness, infant attention to faces and voices, and distractibility, and how together these factors lead to early language outcomes. A newly developed audiovisual protocol, the Multisensory Attention Assessment Protocol (MAAP; Bahrick et al., 2018), allows researchers to examine individual differences in attention to faces and voices and distractibility, and to assess relations with other variables. At 12 months, infants (n = 79) in an ongoing longitudinal study participated in the MAAP to assess intersensory matching of synchronous faces and voices and attention to an irrelevant competing visual distractor event. They also were observed in a brief play interaction to assess infant bids for attention and maternal responsiveness (accept, redirect, or ignore). At 18 months, receptive and expressive language were assessed using the Mullen Scales of Early Learning. Several noteworthy findings emerged: 1) mothers were generally responsive, accepting 74% and redirecting 14% of infant bids, 2) infants who had a greater number of their bids redirected by mothers, and who had better intersensory matching of synchronous faces and voices, showed less attention to the distractor, and 3) infants who showed less attention to the distractor had better receptive language. Findings demonstrate that maternal redirecting of infant attention by mothers who are generally responsive may promote better infant attentional control (lower distractibility) which in turn predicts better receptive language in toddlers.

Intersensory matching of faces and voices in infancy predicts language outcomes in young children.

Parent language input is a well-established predictor of child language development. Multisensory attention skills (MASks; intersensory matching, shifting and sustaining attention to audiovisual speech) are also known to be foundations for language development. However, due to a lack of appropriate measures, individual differences in these skills have received little research focus. A newly established measure, the Multisensory Attention Assessment Protocol (MAAP), allows researchers to examine predictive relations between early MASks and later outcomes. We hypothesized that, along with parent language input, multisensory attention to social events (faces and voices) in infancy would predict later language outcomes. We collected data from 97 children (predominantly White and Hispanic, 48 males) participating in an ongoing longitudinal study assessing 12-, 18-, and 24-month MASks (MAAP) and parent language input (quality, quantity), and 18- and 24-month language outcomes (child speech production, vocabulary size). Results revealed 12-month intersensory matching (but not maintaining or shifting attention) of faces and voices in the presence of a distractor was a strong predictor of language. It predicted a variety of 18- and 24-month child language outcomes (expressive vocabulary, child speech production), even when holding traditional predictors constant: parent language input and SES (maternal education: 52% bachelor's degree or higher). Further, at each age, parent language input predicted just one outcome, expressive vocabulary, and SES predicted child speech production. These novel findings reveal infant intersensory matching of faces and voices in the presence of a distractor can predict which children might benefit most from parent language input and show better language outcomes. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Remote Data Collection During a Pandemic: A New Approach for Assessing and Coding Multisensory Attention Skills in Infants and Young Children.

In early 2020, in-person data collection dramatically slowed or was completely halted across the world as many labs were forced to close due to the COVID-19 pandemic. Developmental researchers who assess looking time (especially those who rely heavily on in-lab eye-tracking or live coding techniques) were forced to re-think their methods of data collection. While a variety of remote or online platforms are available for gathering behavioral data outside of the typical lab setting, few are specifically designed for collecting and processing looking time data in infants and young children. To address these challenges, our lab developed several novel approaches for continuing data collection and coding for a remotely administered audiovisual looking time protocol. First, we detail a comprehensive approach for successfully administering the Multisensory Attention Assessment Protocol (MAAP), developed by our lab to assess multisensory attention skills (MASks; duration of looking, speed of shifting/disengaging, accuracy of audiovisual matching). The MAAP is administered from a distance (remotely) by using Zoom, Gorilla Experiment Builder, an internet connection, and a home computer. This new data collection approach has the advantage that participants can be tested in their homes. We discuss challenges and successes in implementing our approach for remote testing and data collection during an ongoing longitudinal project. Second, we detail an approach for estimating gaze direction and duration collected remotely from webcam recordings using a post processing toolkit (OpenFace) and demonstrate its effectiveness and precision. However, because OpenFace derives gaze estimates without translating them to an external frame of reference (i.e., the participant's screen), we developed a machine learning (ML) approach to overcome this limitation. Thus, third, we trained a ML algorithm [(artificial neural network (ANN)] to classify gaze estimates from OpenFace with respect to areas of interest (AOI) on the participant's screen (i.e., left, right, and center). We then demonstrate reliability between this approach and traditional coding approaches (e.g., coding gaze live). The combination of OpenFace and ML will provide a method to automate the coding of looking time for data collected remotely. Finally, we outline a series of best practices for developmental researchers conducting remote data collection for looking time studies.

The Multisensory Attention Assessment Protocol (MAAP): Characterizing individual differences in multisensory attention skills in infants and children and relations with language and cognition.

Multisensory attention skills provide a crucial foundation for early cognitive, social, and language development, yet there are no fine-grained, individual difference measures of these skills appropriate for preverbal children. The Multisensory Attention Assessment Protocol (MAAP) fills this need. In a single video-based protocol requiring no language skills, the MAAP assesses individual differences in three fundamental building blocks of attention to multisensory events-the duration of attention maintenance, the accuracy of intersensory (audiovisual) matching, and the speed of shifting-for both social and nonsocial events, in the context of high and low competing visual stimulation. In Experiment 1, 2- to 5-year-old children (N = 36) received the MAAP and assessments of language and cognitive functioning. In Experiment 2 the procedure was streamlined and presented to 12-month-olds (N = 48). Both infants and children showed high levels of attention maintenance to social and nonsocial events, impaired attention maintenance and speed of shifting when competing stimulation was high, and significant intersensory matching. Children showed longer maintenance, faster shifting, and less impairment from competing stimulation than infants. In 2- to 5-year-old children, duration and accuracy were intercorrelated, showed increases with age, and predicted cognitive and language functioning. The MAAP opens the door to assessing developmental pathways between early attention patterns to audiovisual events and language, cognitive, and social development. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Assessing individual differences in the speed and accuracy of intersensory processing in young children: The intersensory processing efficiency protocol.

Detecting intersensory redundancy guides cognitive, social, and language development. Yet, researchers lack fine-grained, individual difference measures needed for studying how early intersensory skills lead to later outcomes. The intersensory processing efficiency protocol (IPEP) addresses this need. Across a number of brief trials, participants must find a sound-synchronized visual target event (social, nonsocial) amid five visual distractor events, simulating the "noisiness" of natural environments. Sixty-four 3- to 5-year-old children were tested using remote eye-tracking. Children showed intersensory processing by attending to the sound-synchronous event more frequently and longer than in a silent visual control, and more frequently than expected by chance. The IPEP provides a fine-grained, nonverbal method for characterizing individual differences in intersensory processing appropriate for infants and children. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Temporal Dependency and the Structure of Early Looking.

Although looking time is used to assess infant perceptual and cognitive processing, little is known about the temporal structure of infant looking. To shed light on this temporal structure, 127 three-month-olds were assessed in an infant-controlled habituation procedure and presented with a pre-recorded display of a woman addressing the infant using infant-directed speech. Previous individual look durations positively predicted subsequent look durations over a six look window, suggesting a temporal dependency between successive infant looks. The previous look duration continued to predict the subsequent look duration after accounting for habituation-linked declines in look duration, and when looks were separated by an inter-trial interval in which no stimulus was displayed. Individual differences in temporal dependency, the strength of associations between consecutive look durations, are distinct from individual differences in mean infant look duration. Nevertheless, infants with stronger temporal dependency had briefer mean look durations, a potential index of stimulus processing. Temporal dependency was evident not only between individual infant looks but between the durations of successive habituation trials (total looking within a trial). Finally, temporal dependency was evident in associations between the last look at the habituation stimulus and the first look at a novel test stimulus. Thus temporal dependency was evident across multiple timescales (individual looks and trials comprised of multiple individual looks) and persisted across conditions including brief periods of no stimulus presentation and changes from a familiar to novel stimulus. Associations between consecutive look durations over multiple timescales and stimuli suggest a temporal structure of infant attention that has been largely ignored in previous work on infant looking.

Enhanced attention to speaking faces versus other event types emerges gradually across infancy.

The development of attention to dynamic faces versus objects providing synchronous audiovisual versus silent visual stimulation was assessed in a large sample of infants. Maintaining attention to the faces and voices of people speaking is critical for perceptual, cognitive, social, and language development. However, no studies have systematically assessed when, if, or how attention to speaking faces emerges and changes across infancy. Two measures of attention maintenance, habituation time (HT) and look-away rate (LAR), were derived from cross-sectional data of 2- to 8-month-old infants (N = 801). Results indicated that attention to audiovisual faces and voices was maintained across age, whereas attention to each of the other event types (audiovisual objects, silent dynamic faces, silent dynamic objects) declined across age. This reveals a gradually emerging advantage in attention maintenance (longer HTs, lower LARs) for audiovisual speaking faces compared with the other 3 event types. At 2 months, infants showed no attentional advantage for faces (with greater attention to audiovisual than to visual events); at 3 months, they attended more to dynamic faces than objects (in the presence or absence of voices), and by 4 to 5 and 6 to 8 months, significantly greater attention emerged to temporally coordinated faces and voices of people speaking compared with all other event types. Our results indicate that selective attention to coordinated faces and voices over other event types emerges gradually across infancy, likely as a function of experience with multimodal, redundant stimulation from person and object events. (PsycINFO Database Record

Intrasensory Redundancy Facilitates Infant Detection of Tempo: Extending Predictions of the Intersensory Redundancy Hypothesis.

Research has demonstrated that intersensory redundancy (stimulation synchronized across multiple senses) is highly salient and facilitates processing of amodal properties in multimodal events, bootstrapping early perceptual development. The present study is the first to extend this central principle of the intersensory redundancy hypothesis (IRH) to certain types of intrasensory redundancy (stimulation synchronized within a single sense). Infants were habituated to videos of a toy hammer tapping silently (unimodal control), depicting intersensory redundancy (synchronized with a soundtrack) or intrasensory redundancy (synchronized with another visual event; light flashing or bat tapping). In Experiment 1, 2-month-olds showed both intersensory and intrasensory facilitation (with respect to the unimodal control) for detecting a change in tempo. However, intrasensory facilitation was found when the hammer was synchronized with the light flashing (different motion) but not with the bat tapping (same motion). Experiment 2 tested 3-month-olds using a somewhat easier tempo contrast. Results supported a similarity hypothesis: intrasensory redundancy between two dissimilar events was more effective than that between two similar events for promoting processing of amodal properties. These findings extend the IRH and indicate that in addition to intersensory redundancy, intrasensory redundancy between two synchronized dissimilar visual events is also effective in promoting perceptual processing of amodal event properties.