Neural correlates of audiovisual narrative speech perception in children and adults on the autism spectrum: A functional magnetic resonance imaging study.

Autistic individuals show substantially reduced benefit from observing visual articulations during audiovisual speech perception, a multisensory integration deficit that is particularly relevant to social communication. This has mostly been studied using simple syllabic or word-level stimuli and it remains unclear how altered lower-level multisensory integration translates to the processing of more complex natural multisensory stimulus environments in autism. Here, functional neuroimaging was used to examine neural correlates of audiovisual gain (AV-gain) in 41 autistic individuals to those of 41 age-matched non-autistic controls when presented with a complex audiovisual narrative. Participants were presented with continuous narration of a story in auditory-alone, visual-alone, and both synchronous and asynchronous audiovisual speech conditions. We hypothesized that previously identified differences in audiovisual speech processing in autism would be characterized by activation differences in brain regions well known to be associated with audiovisual enhancement in neurotypicals. However, our results did not provide evidence for altered processing of auditory alone, visual alone, audiovisual conditions or AV- gain in regions associated with the respective task when comparing activation patterns between groups. Instead, we found that autistic individuals responded with higher activations in mostly frontal regions where the activation to the experimental conditions was below baseline (de-activations) in the control group. These frontal effects were observed in both unisensory and audiovisual conditions, suggesting that these altered activations were not specific to multisensory processing but reflective of more general mechanisms such as an altered disengagement of Default Mode Network processes during the observation of the language stimulus across conditions.

Neural correlates of multisensory enhancement in audiovisual narrative speech perception: A fMRI investigation.

This fMRI study investigated the effect of seeing articulatory movements of a speaker while listening to a naturalistic narrative stimulus. It had the goal to identify regions of the language network showing multisensory enhancement under synchronous audiovisual conditions. We expected this enhancement to emerge in regions known to underlie the integration of auditory and visual information such as the posterior superior temporal gyrus as well as parts of the broader language network, including the semantic system. To this end we presented 53 participants with a continuous narration of a story in auditory alone, visual alone, and both synchronous and asynchronous audiovisual speech conditions while recording brain activity using BOLD fMRI. We found multisensory enhancement in an extensive network of regions underlying multisensory integration and parts of the semantic network as well as extralinguistic regions not usually associated with multisensory integration, namely the primary visual cortex and the bilateral amygdala. Analysis also revealed involvement of thalamic brain regions along the visual and auditory pathways more commonly associated with early sensory processing. We conclude that under natural listening conditions, multisensory enhancement not only involves sites of multisensory integration but many regions of the wider semantic network and includes regions associated with extralinguistic sensory, perceptual and cognitive processing.

Sensory and semantic category subdivisions within the anterior temporal lobes.

In the semantic memory literature the anterior temporal lobe (ATL) is frequently discussed as one homogeneous region when in fact, anatomical studies indicate that it is likely that there are discrete subregions within this area. Indeed, the influential Hub Account of semantic memory has proposed that this region is a sensory-amodal, general-purpose semantic processing region. However review of the literature suggested two potential demarcations: sensory subdivisions and a social/nonsocial subdivision. To test this, participants were trained to associate social or non-social words with novel auditory, visual, or audiovisual stimuli. Later, study participants underwent an fMRI scan where they were presented with the sensory stimuli and the task was to recall the semantic associate. The results showed that there were sensory specific subdivisions within the ATL - that the perceptual encoding of auditory stimuli preferentially activated the superior ATL, visual stimuli the inferior ATL, and multisensory stimuli the polar ATL. Moreover, our data showed that there is stimulus-specific sensitivity within the ATL - the superior and polar ATLs were more sensitive to the retrieval of social knowledge as compared to non-social knowledge. No ATL regions were more sensitive to the retrieval of non-social knowledge. These findings indicate that the retrieval of newly learned semantic associations activates the ATL. In addition, superior and polar aspects of the ATL are sensitive to social stimuli but relatively insensitive to non-social stimuli, a finding that is predicted by anatomical connectivity and single-unit studies in non-human primates. And lastly, the ATL contains sensory processing subdivisions that fall along superior (auditory), inferior (visual), polar (audiovisual) subdivisions.