Auditory-visual processing represented in the human superior temporal gyrus.

ABSTRACT

In natural face-to-face communication, speech perception utilizes both auditory and visual information. We described previously an acoustically responsive area on the posterior lateral surface of the superior temporal gyrus (field PLST) that is distinguishable on physiological grounds from other auditory fields located within the superior temporal plane. Considering the empirical findings in humans and non-human primates of cortical locations responsive to heard sounds and/or seen sound-sources, we reasoned that area PLST would also contain neural signals reflecting audiovisual speech interactions. To test this hypothesis, event related potentials (ERPs) were recorded from area PLST using chronically implanted multi-contact subdural surface-recording electrodes in patient-subjects undergoing diagnosis and treatment of medically intractable epilepsy, and cortical ERP maps were acquired during five contrasting auditory, visual and bimodal speech conditions. Stimulus conditions included consonant-vowel (CV) syllable sounds alone, silent seen speech or CV sounds paired with a female face articulating matched or mismatched syllables. Data were analyzed using a MANOVA framework, with the results from planned comparisons used to construct cortical significance maps. Our findings indicate that evoked responses recorded from area PLST to auditory speech stimuli are influenced significantly by the addition of visual images of the moving lower face and lips, either articulating the audible syllable or carrying out a meaningless (gurning) motion. The area of cortex exhibiting this audiovisual influence was demonstrably greater in the speech-dominant hemisphere.