Neural correlates of audiovisual narrative speech perception in children and adults on the autism spectrum: A functional magnetic resonance imaging study.

Autistic individuals show substantially reduced benefit from observing visual articulations during audiovisual speech perception, a multisensory integration deficit that is particularly relevant to social communication. This has mostly been studied using simple syllabic or word-level stimuli and it remains unclear how altered lower-level multisensory integration translates to the processing of more complex natural multisensory stimulus environments in autism. Here, functional neuroimaging was used to examine neural correlates of audiovisual gain (AV-gain) in 41 autistic individuals to those of 41 age-matched non-autistic controls when presented with a complex audiovisual narrative. Participants were presented with continuous narration of a story in auditory-alone, visual-alone, and both synchronous and asynchronous audiovisual speech conditions. We hypothesized that previously identified differences in audiovisual speech processing in autism would be characterized by activation differences in brain regions well known to be associated with audiovisual enhancement in neurotypicals. However, our results did not provide evidence for altered processing of auditory alone, visual alone, audiovisual conditions or AV- gain in regions associated with the respective task when comparing activation patterns between groups. Instead, we found that autistic individuals responded with higher activations in mostly frontal regions where the activation to the experimental conditions was below baseline (de-activations) in the control group. These frontal effects were observed in both unisensory and audiovisual conditions, suggesting that these altered activations were not specific to multisensory processing but reflective of more general mechanisms such as an altered disengagement of Default Mode Network processes during the observation of the language stimulus across conditions.

Neural correlates of multisensory enhancement in audiovisual narrative speech perception: A fMRI investigation.

This fMRI study investigated the effect of seeing articulatory movements of a speaker while listening to a naturalistic narrative stimulus. It had the goal to identify regions of the language network showing multisensory enhancement under synchronous audiovisual conditions. We expected this enhancement to emerge in regions known to underlie the integration of auditory and visual information such as the posterior superior temporal gyrus as well as parts of the broader language network, including the semantic system. To this end we presented 53 participants with a continuous narration of a story in auditory alone, visual alone, and both synchronous and asynchronous audiovisual speech conditions while recording brain activity using BOLD fMRI. We found multisensory enhancement in an extensive network of regions underlying multisensory integration and parts of the semantic network as well as extralinguistic regions not usually associated with multisensory integration, namely the primary visual cortex and the bilateral amygdala. Analysis also revealed involvement of thalamic brain regions along the visual and auditory pathways more commonly associated with early sensory processing. We conclude that under natural listening conditions, multisensory enhancement not only involves sites of multisensory integration but many regions of the wider semantic network and includes regions associated with extralinguistic sensory, perceptual and cognitive processing.

Multisensory Audiovisual Processing in Children With a Sensory Processing Disorder (II): Speech Integration Under Noisy Environmental Conditions.

Background: There exists a cohort of children and adults who exhibit an inordinately high degree of discomfort when experiencing what would be considered moderate and manageable levels of sensory input. That is, they show over-responsivity in the face of entirely typical sound, light, touch, taste, or smell inputs, and this occurs to such an extent that it interferes with their daily functioning and reaches clinical levels of dysfunction. What marks these individuals apart is that this sensory processing disorder (SPD) is observed in the absence of other symptom clusters that would result in a diagnosis of Autism, ADHD, or other neurodevelopmental disorders more typically associated with sensory processing difficulties. One major theory forwarded to account for these SPDs posits a deficit in multisensory integration, such that the various sensory inputs are not appropriately integrated into the central nervous system, leading to an overwhelming sensory-perceptual environment, and in turn to the sensory-defensive phenotype observed in these individuals. Methods: We tested whether children (6-16 years) with an over-responsive SPD phenotype (N = 12) integrated multisensory speech differently from age-matched typically-developing controls (TD: N = 12). Participants identified monosyllabic words while background noise level and sensory modality (auditory-alone, visual-alone, audiovisual) were varied in pseudorandom order. Improved word identification when speech was both seen and heard compared to when it was simply heard served to index multisensory speech integration. Results: School-aged children with an SPD show a deficit in the ability to benefit from the combination of both seen and heard speech inputs under noisy environmental conditions, suggesting that these children do not benefit from multisensory integrative processing to the same extent as their typically developing peers. In contrast, auditory-alone performance did not differ between the groups, signifying that this multisensory deficit is not simply due to impaired processing of auditory speech. Conclusions: Children with an over-responsive SPD show a substantial reduction in their ability to benefit from complementary audiovisual speech, to enhance speech perception in a noisy environment. This has clear implications for performance in the classroom and other learning environments. Impaired multisensory integration may contribute to sensory over-reactivity that is the definitional of SPD.

A Computational Analysis of Neural Mechanisms Underlying the Maturation of Multisensory Speech Integration in Neurotypical Children and Those on the Autism Spectrum.

Failure to appropriately develop multisensory integration (MSI) of audiovisual speech may affect a child's ability to attain optimal communication. Studies have shown protracted development of MSI into late-childhood and identified deficits in MSI in children with an autism spectrum disorder (ASD). Currently, the neural basis of acquisition of this ability is not well understood. Here, we developed a computational model informed by neurophysiology to analyze possible mechanisms underlying MSI maturation, and its delayed development in ASD. The model posits that strengthening of feedforward and cross-sensory connections, responsible for the alignment of auditory and visual speech sound representations in posterior superior temporal gyrus/sulcus, can explain behavioral data on the acquisition of MSI. This was simulated by a training phase during which the network was exposed to unisensory and multisensory stimuli, and projections were crafted by Hebbian rules of potentiation and depression. In its mature architecture, the network also reproduced the well-known multisensory McGurk speech effect. Deficits in audiovisual speech perception in ASD were well accounted for by fewer multisensory exposures, compatible with a lack of attention, but not by reduced synaptic connectivity or synaptic plasticity.

White Matter Changes in HIV+ Women with a History of Cocaine Dependence.

Cocaine use is associated with the transmission of human immunodeficiency (HIV) virus through risky sexual behavior. In HIV+ individuals, cocaine use is linked with poor health outcomes, including HIV-medication non-adherence and faster disease progression. Both HIV and cocaine dependence are associated with reduced integrity of cerebral white matter (WM), but the effects of HIV during cocaine abstinence have not yet been explored. We used diffusion tensor imaging (DTI) to understand the effect of combined HIV+ serostatus and former cocaine dependence on cerebral WM integrity. DTI data obtained from 15 HIV+ women with a history of cocaine dependence (COC+/HIV+) and 21 healthy females were included in the analysis. Diffusion-based measures [fractional anisotropy (FA), radial diffusivity (RD), mean diffusivity, and axial diffusivity] were examined using tract-based spatial statistics and region-of-interest analyses. In a whole-brain analysis, COC+/HIV+ women showed significantly reduced FA and increased RD in all major WM tracts, except the left corticospinal tract for RD. The tract with greatest percentage of voxels showing significant between-group differences was the forceps minor (FA: 75.6%, RD: 59.7%). These widespread changes in diffusion measures indicate an extensive neuropathological effect of HIV and former cocaine dependence on WM.

Common variation in the autism risk gene CNTNAP2, brain structural connectivity and multisensory speech integration.

Three lines of evidence motivated this study. 1) CNTNAP2 variation is associated with autism risk and speech-language development. 2) CNTNAP2 variations are associated with differences in white matter (WM) tracts comprising the speech-language circuitry. 3) Children with autism show impairment in multisensory speech perception. Here, we asked whether an autism risk-associated CNTNAP2 single nucleotide polymorphism in neurotypical adults was associated with multisensory speech perception performance, and whether such a genotype-phenotype association was mediated through white matter tract integrity in speech-language circuitry. Risk genotype at rs7794745 was associated with decreased benefit from visual speech and lower fractional anisotropy (FA) in several WM tracts (right precentral gyrus, left anterior corona radiata, right retrolenticular internal capsule). These structural connectivity differences were found to mediate the effect of genotype on audiovisual speech perception, shedding light on possible pathogenic pathways in autism and biological sources of inter-individual variation in audiovisual speech processing in neurotypicals.

Neuroanatomical Abnormalities in Violent Individuals with and without a Diagnosis of Schizophrenia.

Several structural brain abnormalities have been associated with aggression in patients with schizophrenia. However, little is known about shared and distinct abnormalities underlying aggression in these subjects and non-psychotic violent individuals. We applied a region-of-interest volumetric analysis of the amygdala, hippocampus, and thalamus bilaterally, as well as whole brain and ventricular volumes to investigate violent (n = 37) and non-violent chronic patients (n = 26) with schizophrenia, non-psychotic violent (n = 24) as well as healthy control subjects (n = 24). Shared and distinct volumetric abnormalities were probed by analysis of variance with the factors violence (non-violent versus violent) and diagnosis (non-psychotic versus psychotic), adjusted for substance abuse, age, academic achievement and negative psychotic symptoms. Patients showed elevated vCSF volume, smaller left hippocampus and smaller left thalamus volumes. This was particularly the case for non-violent individuals diagnosed with schizophrenia. Furthermore, patients had reduction in right thalamus size. With regard to left amygdala, we found an interaction between violence and diagnosis. More specifically, we report a double dissociation with smaller amygdala size linked to violence in non-psychotic individuals, while for psychotic patients smaller size was linked to non-violence. Importantly, the double dissociation appeared to be mostly driven by substance abuse. Overall, we found widespread morphometric abnormalities in subcortical regions in schizophrenia. No evidence for shared volumetric abnormalities in individuals with a history of violence was found. Finally, left amygdala abnormalities in non-psychotic violent individuals were largely accounted for by substance abuse. This might be an indication that the association between amygdala reduction and violence is mediated by substance abuse. Our results indicate the importance of structural abnormalities in aggressive individuals.

A Common CYFIP1 Variant at the 15q11.2 Disease Locus Is Associated with Structural Variation at the Language-Related Left Supramarginal Gyrus.

Copy number variants (CNVs) at the Breakpoint 1 to Breakpoint 2 region at 15q11.2 (BP1-2) are associated with language-related difficulties and increased risk for developmental disorders in which language is compromised. Towards underlying mechanisms, we investigated relationships between single nucleotide polymorphisms (SNPs) across the region and quantitative measures of human brain structure obtained by magnetic resonance imaging of healthy subjects. We report an association between rs4778298, a common variant at CYFIP1, and inter-individual variation in surface area across the left supramarginal gyrus (lh.SMG), a cortical structure implicated in speech and language in independent discovery (n = 100) and validation cohorts (n = 2621). In silico analyses determined that this same variant, and others nearby, is also associated with differences in levels of CYFIP1 mRNA in human brain. One of these nearby polymorphisms is predicted to disrupt a consensus binding site for FOXP2, a transcription factor implicated in speech and language. Consistent with a model where FOXP2 regulates CYFIP1 levels and in turn influences lh.SMG surface area, analysis of publically available expression data identified a relationship between expression of FOXP2 and CYFIP1 mRNA in human brain. We propose that altered CYFIP1 dosage, through aberrant patterning of the lh.SMG, may contribute to language-related difficulties associated with BP1-2 CNVs. More generally, this approach may be useful in clarifying the contribution of individual genes at CNV risk loci.

Sex differences in multisensory speech processing in both typically developing children and those on the autism spectrum.

BACKGROUND: Previous work has revealed sizeable deficits in the abilities of children with an autism spectrum disorder (ASD) to integrate auditory and visual speech signals, with clear implications for social communication in this population. There is a strong male preponderance in ASD, with approximately four affected males for every female. The presence of sex differences in ASD symptoms suggests a sexual dimorphism in the ASD phenotype, and raises the question of whether this dimorphism extends to ASD traits in the neurotypical population. Here, we investigated possible sexual dimorphism in multisensory speech integration in both ASD and neurotypical individuals. METHODS: We assessed whether males and females differed in their ability to benefit from visual speech when target words were presented under varying levels of signal-to-noise, in samples of neurotypical children and adults, and in children diagnosed with an ASD. RESULTS: In typically developing (TD) children and children with ASD, females (n = 47 and n = 15, respectively) were significantly superior in their ability to recognize words under audiovisual listening conditions compared to males (n = 55 and n = 58, respectively). This sex difference was absent in our sample of neurotypical adults (n = 28 females; n = 28 males). CONCLUSIONS: We propose that the development of audiovisual integration is delayed in male relative to female children, a delay that is also observed in ASD. In neurotypicals, these sex differences disappear in early adulthood when females approach their performance maximum and males "catch up." Our findings underline the importance of considering sex differences in the search for autism endophenotypes and strongly encourage increased efforts to study the underrepresented population of females within ASD.

Severe multisensory speech integration deficits in high-functioning school-aged children with Autism Spectrum Disorder (ASD) and their resolution during early adolescence.

Under noisy listening conditions, visualizing a speaker's articulations substantially improves speech intelligibility. This multisensory speech integration ability is crucial to effective communication, and the appropriate development of this capacity greatly impacts a child's ability to successfully navigate educational and social settings. Research shows that multisensory integration abilities continue developing late into childhood. The primary aim here was to track the development of these abilities in children with autism, since multisensory deficits are increasingly recognized as a component of the autism spectrum disorder (ASD) phenotype. The abilities of high-functioning ASD children (n = 84) to integrate seen and heard speech were assessed cross-sectionally, while environmental noise levels were systematically manipulated, comparing them with age-matched neurotypical children (n = 142). Severe integration deficits were uncovered in ASD, which were increasingly pronounced as background noise increased. These deficits were evident in school-aged ASD children (5-12 year olds), but were fully ameliorated in ASD children entering adolescence (13-15 year olds). The severity of multisensory deficits uncovered has important implications for educators and clinicians working in ASD. We consider the observation that the multisensory speech system recovers substantially in adolescence as an indication that it is likely amenable to intervention during earlier childhood, with potentially profound implications for the development of social communication abilities in ASD children.

Intact inhibitory control processes in abstinent drug abusers (I): a functional neuroimaging study in former cocaine addicts.

Neuroimaging studies in current cocaine dependent (CD) individuals consistently reveal cortical hypoactivity across regions of the response inhibition circuit (RIC). Dysregulation of this critical executive network is hypothesized to account for the lack of inhibitory control that is a hallmark of the addictive phenotype, and chronic abuse is believed to compound the issue. A crucial question is whether deficits in this circuit persist after drug cessation, and whether recovery of this system will be seen after extended periods of abstinence, a question with implications for treatment course and outcome. Utilizing functional magnetic resonance imaging (fMRI), we examined activation in nodes of the RIC in abstinent CD individuals (n = 27) and non-using controls (n = 45) while they performed a motor response inhibition task. In contrast to current users, these abstinent individuals, despite extended histories of chronic cocaine-abuse (average duration of use = 8.2 years), performed the task just as efficiently as non-users. In line with these behavioral findings, no evidence for between-group differences in activation of the RIC was found and instead, robust activations were apparent in both groups within the well-characterized nodes of the RIC. Similarly, our complementary Electroencephalography (EEG) investigation also showed an absence of behavioral and electrophysiological deficits in abstinent drug abusers. These results are consistent with an amelioration of neurobiological deficits in inhibitory circuitry following drug cessation, and could help explain how long-term abstinence is maintained. Finally, regression analyses revealed a significant association between level of activation in the right insula with inhibition success and increased abstinence duration in the CD cohort suggesting that this region may be integral to successful recovery from cocaine addiction.

Social cognition and the anterior temporal lobes: a review and theoretical framework.

Memory for people and their relationships, along with memory for social language and social behaviors, constitutes a specific type of semantic memory termed social knowledge. This review focuses on how and where social knowledge is represented in the brain. We propose that portions of the anterior temporal lobe (ATL) play a critical role in representing and retrieving social knowledge. This includes memory about people, their names and biographies and more abstract forms of social memory such as memory for traits and social concepts. This hypothesis is based on the convergence of several lines of research including anatomical findings, lesion evidence from both humans and non-human primates and neuroimaging evidence. Moreover, the ATL is closely interconnected with cortical nuclei of the amygdala and orbitofrontal cortex via the uncinate fasciculus. We propose that this pattern of connectivity underlies the function of the ATL in encoding and storing emotionally tagged knowledge that is used to guide orbitofrontal-based decision processes.

What's unique about unique entities? An fMRI investigation of the semantics of famous faces and landmarks.

Famous people and artifacts are referred to as "unique entities" (UEs) due to the unique nature of the knowledge we have about them. Past imaging and lesion experiments have indicated that the anterior temporal lobes (ATLs) as having a special role in the processing of UEs. It has remained unclear which attributes of UEs were responsible for the observed effects in imaging experiments. In this study, we investigated what factors of UEs influence brain activity. In a training paradigm, we systematically varied the uniqueness of semantic associations, the presence/absence of a proper name, and the number of semantic associations to determine factors modulating activity in regions subserving the processing of UEs. We found that a conjunction of unique semantic information and proper names modulated activity within a section of the left ATL. Overall, the processing of UEs involved a wider left-hemispheric cortical network. Within these regions, brain activity was significantly affected by the unique semantic attributes especially in the presence of a proper name, but we could not find evidence for an effect of the number of semantic associations. Findings are discussed in regard to current models of ATL function, the neurophysiology of semantics, and social cognitive processing.

Improved proper name recall in aging after electrical stimulation of the anterior temporal lobes.

Evidence from neuroimaging and neuropsychology suggests that portions of the anterior temporal lobes (ATLs) play a critical role in proper name retrieval. We previously found that anodal transcranial direct current stimulation (tDCS) to the ATLs improved retrieval of proper names in young adults (Ross et al., 2010). Here we extend that finding to older adults who tend to experience greater proper-naming deficits than young adults. The task was to look at pictures of famous faces or landmarks and verbally recall the associated proper name. Our results show a numerical improvement in face naming after left or right ATL stimulation, but a statistically significant effect only after left-lateralized stimulation. The magnitude of the enhancing effect was similar in older and younger adults but the lateralization of the effect differed depending on age. The implications of these findings for the use of tDCS as tool for rehabilitation of age-related loss of name recall are discussed.

Sensory and semantic category subdivisions within the anterior temporal lobes.

In the semantic memory literature the anterior temporal lobe (ATL) is frequently discussed as one homogeneous region when in fact, anatomical studies indicate that it is likely that there are discrete subregions within this area. Indeed, the influential Hub Account of semantic memory has proposed that this region is a sensory-amodal, general-purpose semantic processing region. However review of the literature suggested two potential demarcations: sensory subdivisions and a social/nonsocial subdivision. To test this, participants were trained to associate social or non-social words with novel auditory, visual, or audiovisual stimuli. Later, study participants underwent an fMRI scan where they were presented with the sensory stimuli and the task was to recall the semantic associate. The results showed that there were sensory specific subdivisions within the ATL - that the perceptual encoding of auditory stimuli preferentially activated the superior ATL, visual stimuli the inferior ATL, and multisensory stimuli the polar ATL. Moreover, our data showed that there is stimulus-specific sensitivity within the ATL - the superior and polar ATLs were more sensitive to the retrieval of social knowledge as compared to non-social knowledge. No ATL regions were more sensitive to the retrieval of non-social knowledge. These findings indicate that the retrieval of newly learned semantic associations activates the ATL. In addition, superior and polar aspects of the ATL are sensitive to social stimuli but relatively insensitive to non-social stimuli, a finding that is predicted by anatomical connectivity and single-unit studies in non-human primates. And lastly, the ATL contains sensory processing subdivisions that fall along superior (auditory), inferior (visual), polar (audiovisual) subdivisions.

The development of multisensory speech perception continues into the late childhood years.

Observing a speaker's articulations substantially improves the intelligibility of spoken speech, especially under noisy listening conditions. This multisensory integration of speech inputs is crucial to effective communication. Appropriate development of this ability has major implications for children in classroom and social settings, and deficits in it have been linked to a number of neurodevelopmental disorders, especially autism. It is clear from structural imaging studies that there is a prolonged maturational course within regions of the perisylvian cortex that persists into late childhood, and these regions have been firmly established as being crucial to speech and language functions. Given this protracted maturational timeframe, we reasoned that multisensory speech processing might well show a similarly protracted developmental course. Previous work in adults has shown that audiovisual enhancement in word recognition is most apparent within a restricted range of signal-to-noise ratios (SNRs). Here, we investigated when these properties emerge during childhood by testing multisensory speech recognition abilities in typically developing children aged between 5 and 14 years, and comparing them with those of adults. By parametrically varying SNRs, we found that children benefited significantly less from observing visual articulations, displaying considerably less audiovisual enhancement. The findings suggest that improvement in the ability to recognize speech-in-noise and in audiovisual integration during speech perception continues quite late into the childhood years. The implication is that a considerable amount of multisensory learning remains to be achieved during the later schooling years, and that explicit efforts to accommodate this learning may well be warranted.

The neurophysiology of human biological motion processing: a high-density electrical mapping study.

The neural processing of biological motion (BM) is of profound experimental interest since it is often through the movement of another that we interpret their immediate intentions. Neuroimaging points to a specialized cortical network for processing biological motion. Here, high-density electrical mapping and source-analysis techniques were employed to interrogate the timing of information processing across this network. Participants viewed point-light-displays depicting standard body movements (e.g. jumping), while event-related potentials (ERPs) were recorded and compared to ERPs to scrambled motion control stimuli. In a pair of experiments, three major phases of BM-specific processing were identified: 1) The earliest phase of BM-sensitive modulation was characterized by a positive shift of the ERP between 100 and 200 ms after stimulus onset. This modulation was observed exclusively over the right hemisphere and source-analysis suggested a likely generator in close proximity to regions associated with general motion processing (KO/hMT). 2) The second phase of BM-sensitivity occurred from 200 to 350 ms, characterized by a robust negative-going ERP modulation over posterior middle temporal regions bilaterally. Source-analysis pointed to bilateral generators at or near the posterior superior temporal sulcus (STS). 3) A third phase of processing was evident only in our second experiment, where participants actively attended the BM aspect of the stimuli, and was manifest as a centro-parietal positive ERP deflection, likely related to later cognitive processes. These results point to very early sensory registration of biological motion, and highlight the interactive role of the posterior STS in analyzing the movements of other living organisms.

Improved proper name recall by electrical stimulation of the anterior temporal lobes.

People's names have an embarrassing propensity to be forgotten. This problem is exacerbated by normal aging and by some kinds of dementia. As evidence from neuroimaging and neuropsychology suggest that portions of the anterior temporal lobes play a role in proper name retrieval, we hypothesized that transcranial direct current stimulation (tDCS), a technique that modulates neural transmission, to the anterior temporal lobes would alter the retrieval of proper names. Fifteen young adults received left anodal, right anodal, or sham stimulation of the anterior temporal lobes while naming pictures of famous individuals and landmarks. Right anterior temporal lobe stimulation significantly improved naming for people but not landmarks. These findings are consistent with the notion that the anterior temporal lobes are critically involved in the retrieval of people's names.

Social cognition and the anterior temporal lobes.

Two distinct literatures have emerged on the functionality of the anterior temporal lobes (ATL): in one field, the ATLs are conceived of as a repository for semantic or conceptual knowledge. In another field, the ATLs are thought to play some undetermined role in social-emotional functions such as Theory of Mind. Here we attempted to reconcile these distinct functions by assessing whether social semantic processing can explain ATL activation in other social cognitive tasks. Social semantic functions refer to knowledge about social concepts and rules. In a first experiment we tested the idea that social semantic representations can account for activations in the ATL to social attribution stimuli such as Heider and Simmel animations. Left ATL activations to Heider and Simmel stimuli overlapped with activations to social words. In a second experiment we assessed the putative roles of the ATLs in the processing of narratives and theory of mind content and found evidence for a role of the ATLs in the processing of theory of mind but not narrative per se. These findings indicate that the ATLs are part of a neuronal network supporting social cognition and that they are engaged when tasks demand access to social conceptual knowledge.

Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space.

Watching a speaker's facial movements can dramatically enhance our ability to comprehend words, especially in noisy environments. From a general doctrine of combining information from different sensory modalities (the principle of inverse effectiveness), one would expect that the visual signals would be most effective at the highest levels of auditory noise. In contrast, we find, in accord with a recent paper, that visual information improves performance more at intermediate levels of auditory noise than at the highest levels, and we show that a novel visual stimulus containing only temporal information does the same. We present a Bayesian model of optimal cue integration that can explain these conflicts. In this model, words are regarded as points in a multidimensional space and word recognition is a probabilistic inference process. When the dimensionality of the feature space is low, the Bayesian model predicts inverse effectiveness; when the dimensionality is high, the enhancement is maximal at intermediate auditory noise levels. When the auditory and visual stimuli differ slightly in high noise, the model makes a counterintuitive prediction: as sound quality increases, the proportion of reported words corresponding to the visual stimulus should first increase and then decrease. We confirm this prediction in a behavioral experiment. We conclude that auditory-visual speech perception obeys the same notion of optimality previously observed only for simple multisensory stimuli.