Audiovisual integration of speech is disturbed in schizophrenia: an fMRI study.

Speech perception is an essential part of social interaction. Visual information (lip movements, facial expression) may supplement auditory information in particular under inadvertent listening situations. Schizophrenia patients have been shown to have a deficit in integrating articulatory motions with the auditory speech input. The goal of this study was to investigate the neural basis of this deficit in audiovisual speech processing in schizophrenia patients by using fMRI. Disyllabic nouns were presented in congruent (audio matches visual information) and incongruent conditions in a slow event related fMRI design. Schizophrenia patients (n=15) were compared to age and gender matched control participants. The statistical examination was conducted by analysis of variance with main factors: audiovisual congruency and group membership. The patients' brain activity differed from the control group as evidenced by congruency by group interaction effects. The pertinent brain sites were located predominantly in the right hemisphere and comprised the pars opercularis, middle frontal sulcus, and superior temporal gyrus. In addition, we observed interactions bilaterally in the fusiform gyrus and the nucleus accumbens. We suggest that schizophrenia patients' deficits in audiovisual integration during speech perception are due to a dysfunction of the speech motor system in the right hemisphere. Furthermore the results can be also seen as a reflection of reduced lateralization of language functions to the left hemisphere in schizophrenia.

Preattentive mechanisms of change detection in early auditory cortex: a 7 Tesla fMRI study.

The auditory system continuously monitors the environment for irregularities in an automatic, preattentive fashion. This is presumably accomplished by two mechanisms: a sensory mechanism detects a deviant sound on the basis of differential refractoriness of neural populations sensitive to the standard and deviant sounds, whereas the cognitive mechanism reveals deviance by comparing incoming auditory information with a template derived from previous input. Using fast event-related high-resolution functional magnetic resonance imaging at 7 Tesla we show that both mechanisms can be mapped to different parts of the auditory cortex both at the group level and the single-subject level. The sensory mechanism is supported by primary auditory areas in Heschl's gyrus whereas the cognitive mechanism is implemented in more anterior secondary auditory areas. Both mechanisms are equally engaged by simple sine-wave tones and speech-related phonemes indicating that streams of speech and non-speech stimuli are processed in a similar fashion.

Maladaptive connectivity of Broca's area in schizophrenia during audiovisual speech perception: an fMRI study.

Speech comprehension relies on auditory as well as visual information, and is enhanced in healthy subjects, when audiovisual (AV) information is present. Patients with schizophrenia have been reported to have problems regarding this AV integration process, but little is known about which underlying neural processes are altered. Functional magnetic resonance imaging was performed in 15 schizophrenia patients (SP) and 15 healthy controls (HC) to study functional connectivity of Broca's area by means of a beta series correlation method during perception of audiovisually presented bisyllabic German nouns, in which audio and video either matched or did not match. Broca's area of SP showed stronger connectivity with supplementary motor cortex for incongruent trials whereas HC connectivity was stronger for congruent trials. The right posterior superior temporal sulcus (RpSTS) area showed differences in connectivity for congruent and incongruent trials in HC in contrast to SP where the connectivity was similar for both conditions. These smaller differences in connectivity in SP suggest a less adaptive processing of audiovisually congruent and incongruent speech. The findings imply that AV integration problems in schizophrenia are associated with maladaptive connectivity of Broca's and RpSTS area in particular when confronted with incongruent stimuli. Results are discussed in light of recent AV speech perception models.

Inside a synesthete's head: a functional connectivity analysis with grapheme-color synesthetes.

Grapheme-color synesthesia is a condition in which letters are perceived with an additional color dimension. To identify brain regions involved in this type of synesthesia and to analyze functional connectivity of these areas, 18 grapheme-color synesthetes and 18 matched controls were stimulated with letters and pseudo-letters presented in black and color in an event-related fMRI experiment. Based on the activation-differences between synesthetes and non-synesthetic controls regions of interest were defined. In a second analysis step functional connectivity was calculated using beta series correlation analysis for these seed regions. First we identified one seed region in the left inferior parietal (IPL) cortex (BA7) showing activation differences between grapheme-color synesthetes and controls. Furthermore, we found activation differences in brain areas involved in processing of letters and pseudo-letters, in particular the right IPL cortex (BA7), but also two more clusters in the right hemispheric BA 18 and BA 40. Functional connectivity analysis revealed an increased connectivity between the left IPL seed region and primary/secondary visual areas (BA 18) in synesthetes. Also the right BA 7 showed a stronger connectivity with primary/secondary visual areas (BA 18) in grapheme-color synesthetes. The results of this study support the idea that the parietal lobe plays an important role in synesthetic experience. The data suggest furthermore that the information flow in grapheme-color synesthetes was already modulated at the level of the primary visual cortex which is different than previously thought. Therefore, the current models of grapheme-color synesthesia have to be refined as the unusual communication flow in synesthetes is not restricted to V4, fusiform cortex and the parietal lobe but rather involves a more extended network.

Disinhibited feedback as a cause of synesthesia: evidence from a functional connectivity study on auditory-visual synesthetes.

In synesthesia, certain stimuli to one sensory modality lead to sensory perception in another unstimulated modality. In addition to other models, a two-stage model is discussed to explain this phenomenon, which combines two previously formulated hypotheses regarding synesthesia: direct cross-activation and hyperbinding. The direct cross-activation model postulates that direct connections between sensory-specific areas are responsible for co-activation and synesthetic perception. The hyperbinding hypothesis suggests that the inducing stimulus and the synesthetic sensation are coupled by a sensory nexus area, which may be located in the parietal cortex. This latter hypothesis is compatible with the disinhibited feedback model, which suggests unusual feedback from multimodal convergence areas as the cause of synesthesia. In this study, the relevance of these models was tested in a group (n=14) of auditory-visual synesthetes by performing a functional connectivity analysis on functional magnetic resonance imaging (fMRI) data. Different simple and complex sounds were used as stimuli, and functionally defined seed areas in the bilateral auditory cortex (AC) and the left inferior parietal cortex (IPC) were used for the connectivity calculations. We found no differences in the connectivity of the AC and the visual areas between synesthetes and controls. The main finding of the study was stronger connectivity of the left IPC with the left primary auditory and right primary visual cortex in the group of auditory-visual synesthetes. The results support the model of disinhibited feedback as a cause of synesthetic perception but do not suggest direct cross-activation.

The neural correlates of coloured music: a functional MRI investigation of auditory-visual synaesthesia.

In auditory-visual synaesthesia, all kinds of sound can induce additional visual experiences. To identify the brain regions mainly involved in this form of synaesthesia, functional magnetic resonance imaging (fMRI) has been used during non-linguistic sound perception (chords and pure tones) in synaesthetes and non-synaesthetes. Synaesthetes showed increased activation in the left inferior parietal cortex (IPC), an area involved in multimodal integration, feature binding and attention guidance. No significant group-differences could be detected in area V4, which is known to be related to colour vision and form processing. The results support the idea of the parietal cortex acting as sensory nexus area in auditory-visual synaesthesia, and as a common neural correlate for different types of synaesthesia.