Generalization of the generation of an MMN by illusory McGurk percepts: voiceless consonants.

OBJECTIVE: The existence of a Mismatch Negativity (MMN) evoked by McGurk percepts elicited by audiovisual syllables with constant auditory components has been previously demonstrated with voiced consonants [Clin. Neurophysiol. 113 (2002) 495]. The present study aimed at generalizing such results with voiceless consonants. In a first experiment, the MMN was computed using the classical subtraction method (standard minus deviant). Since results showed a possible contamination by exogenous visual components, a technique preventing from including those components in the differential waveform was used in a second experiment (deviant in sequence minus deviant presented alone). METHODS: Cortical evoked potentials were recorded using the oddball paradigm on eight adults in three experimental conditions (auditory alone, visual alone and audiovisual) for experiment one and in two conditions (visual alone and audiovisual) for experiment two. Obtaining illusory percepts was confirmed in additional psychophysical experiments. RESULTS: Significant MMNs were recorded in the three conditions of experiment one, whereas only the audiovisual condition of experiment two gave rise to a significant MMN. CONCLUSIONS: Provided that the MMN is computed with deviant stimuli only, the present results confirm the elicitation of genuine audiovisual MMN. Possible refractoriness effects and N2b confound have, however, to be controlled for in further studies.

Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion.

OBJECTIVES: The ventriloquism effect is the tendency to underestimate the spatial separation between synchronous auditory and visual signals moderately separated in space. If, as it is thought, this effect is pre-attentive, it could modulate the mismatch negativity (MMN) that indexes the automatic, pre-attentive detection of deviant auditory stimuli rarely occurring in a sequence of standard stimuli. We assessed the existence of an MMN evoked by auditory and visual signals made up of standard sounds coming from the same location as the visual signal and deviant sounds coming from lateral deviations (20 or 60 degrees). As first observed in a behavioral study, a ventriloquism effect occurred for 20 degrees spatial separation but not for 60 degrees. METHODS: Cortical evoked potentials were recorded using the oddball paradigm on 8 adults in auditory alone and audiovisual conditions. RESULTS: In the auditory alone condition, each spatial localization contrast elicited a significant MMN. In the audiovisual condition, a significant MMN was only evoked for the 60 degrees contrasts. CONCLUSIONS: The MMN evoked by spatial separation contrasts (20 and 60 degrees) in the auditory alone condition was suppressed by the corresponding audiovisual condition only when the latter yielded a ventriloquism effect, suggesting that this effect occurs at an early perceptual level.

Mismatch negativity evoked by the McGurk-MacDonald effect: a phonetic representation within short-term memory.

OBJECTIVES: The McGurk-MacDonald illusory percept is obtained by dubbing an incongruent articulatory movement on an auditory phoneme. This type of audiovisual speech perception contributes to the assessment of theories of speech perception. The mismatch negativity (MMN) reflects the detection of a deviant stimulus within the auditory short-term memory and besides an acoustic component, possesses, under certain conditions, a phonetic one. The present study assessed the existence of an MMN evoked by McGurk-MacDonald percepts elicited by audiovisual stimuli with constant auditory components. METHODS: Cortical evoked potentials were recorded using the oddball paradigm on 8 adults in 3 experimental conditions: auditory alone, visual alone and audiovisual stimulation. Obtaining illusory percepts was confirmed in an additional psychophysical condition. RESULTS: The auditory deviant syllables and the audiovisual incongruent syllables elicited a significant MMN at Fz. In the visual condition, no negativity was observed either at Fz, or at O(z). CONCLUSIONS: An MMN can be evoked by visual articulatory deviants, provided they are presented in a suitable auditory context leading to a phonetically significant interaction. The recording of an MMN elicited by illusory McGurk percepts suggests that audiovisual integration mechanisms in speech take place rather early during the perceptual processes.