Audiovisual integration in the human brain: a coordinate-based meta-analysis.

People can seamlessly integrate a vast array of information from what they see and hear in the noisy and uncertain world. However, the neural underpinnings of audiovisual integration continue to be a topic of debate. Using strict inclusion criteria, we performed an activation likelihood estimation meta-analysis on 121 neuroimaging experiments with a total of 2,092 participants. We found that audiovisual integration is linked with the coexistence of multiple integration sites, including early cortical, subcortical, and higher association areas. Although activity was consistently found within the superior temporal cortex, different portions of this cortical region were identified depending on the analytical contrast used, complexity of the stimuli, and modality within which attention was directed. The context-dependent neural activity related to audiovisual integration suggests a flexible rather than fixed neural pathway for audiovisual integration. Together, our findings highlight a flexible multiple pathways model for audiovisual integration, with superior temporal cortex as the central node in these neural assemblies.

Evoked and induced power oscillations linked to audiovisual integration of affect.

Our affective experiences are influenced by combined multisensory information. Although the enhanced effects of congruent audiovisual information on our affective experiences have been well documented, the role of neural oscillations in the audiovisual integration of affective signals remains unclear. First, it is unclear whether oscillatory activity changes as a function of valence. Second, the function of phase-locked and non-phase-locked power changes in audiovisual integration of affect has not yet been clearly distinguished. To fill this gap, the present study performed time-frequency analyses on EEG data acquired while participants perceived positive, neutral and negative naturalistic video and music clips. A comparison between the congruent audiovisual condition and the sum of unimodal conditions was used to identify supra-additive (Audiovisual > Visual + Auditory) or sub-additive (Audiovisual < Visual + Auditory) integration effects. The results showed that early evoked sub-additive theta and sustained induced supra-additive delta and beta activities are linked to audiovisual integration of affect regardless of affective content.

Electrophysiological evidence of an attentional bias in crossmodal inhibition of return.

Inhibition of return (IOR) refers to a delay in responding to targets when they appear at recently attended locations, relative to unattended locations. Within the visual modality, this attentional bias has been associated with a reduction in the N2pc event-related potential (ERP) component at previously attended locations. The present study examined whether a similar attentional bias was observed in crossmodal audio-visual IOR. Our results demonstrate that for visual targets, the attentional component of IOR behaves similarly for both unimodal and crossmodal target pairs, as indexed by a reduction in the N2pc component for targets appearing at previously attended locations. Further, similar IOR-related modulations on the auditory-evoked N2ac indicated that an attentional bias can be observed for auditory targets as well. Finally, we identified two additional ERP components - the ACOP and VCAN - that appear to reflect biasing of attention in the currently unattended sensory modality. These results suggest that the inhibitory attentional bias that underlies the IOR effect may be supramodal and bias attention away from previously attended locations regardless of sensory modality.

Theta modulation of inter-regional gamma synchronization during auditory attention control.

Synchronization of gamma oscillations among brain regions is relevant for dynamically organizing communication among neurons to support cognitive and perceptual processing, including attention orienting. Recent research has demonstrated that inter-regional synchronization in the gamma-band is modulated by theta rhythms during cortical processing. It has been proposed that such cross-frequency dynamics underlie the integration of local processes into large-scale functional networks. To investigate the potential role of theta-gamma mechanisms during auditory attention control, we localized activated regions using EEG beamformer analysis, and calculated inter-regional gamma-band synchronization between activated regions as well as modulation of inter-regional gamma-band synchronization by the phase of cortical theta rhythms. Abundant synchronization of gamma-band oscillations among regions comprising the auditory attention control network was observed. This inter-regional gamma synchronization was modulated by theta phase. These results provide further evidence implicating inter-regional gamma-band synchronization, and theta-gamma interactions, in task-dependent communication among cortical regions, and provide the first evidence that such mechanisms are relevant for auditory attention control.

Electrical neuroimaging of voluntary audiospatial attention: evidence for a supramodal attention control network.

Previous attempts to investigate the supramodal nature of attentional control have focused primarily on identifying neuroanatomical overlap in the frontoparietal systems activated during voluntary shifts of spatial attention in different sensory modalities. However, the activation of the same neural structures is insufficient evidence for a supramodal system, as the same brain regions could interact with one another in very different ways during shifts of attention in different modalities. Thus, to explore the similarity of the functional networks, it is necessary to identify the neural structures involved and to examine the timing and sequence of activities within the network. To this end, we used an electrical neuroimaging technique to localize the neural sources of electroencephalographic signals recorded from human subjects during audiospatial shifts of attention and to examine the timing and sequence of activities within several regions of interest. We then compared the results to an analogous study of visuospatial attention shifts. Similar frontal and parietal regions were activated during visual and auditory shifts of attention, and the timing of activities within these regions was nearly identical. Following this modality-independent sequence of attention-control activity, activity in the relevant sensory cortex was enhanced in anticipation of the response-relevant target. These results are consistent with the hypothesis that a single supramodal network of frontal and parietal regions mediates voluntary shifts of spatial attention and controls the flow of sensory information in modality-specific sensory pathways.

Tracking the voluntary control of auditory spatial attention with event-related brain potentials.

A lateralized event-related potential (ERP) component elicited by attention-directing cues (ADAN) has been linked to frontal-lobe control but is often absent when spatial attention is deployed in the auditory modality. Here, we tested the hypothesis that ERP activity associated with frontal-lobe control of auditory spatial attention is distributed bilaterally by comparing ERPs elicited by attention-directing cues and neutral cues in a unimodal auditory task. This revealed an initial ERP positivity over the anterior scalp and a later ERP negativity over the parietal scalp. Distributed source analysis indicated that the anterior positivity was generated primarily in bilateral prefrontal cortices, whereas the more posterior negativity was generated in parietal and temporal cortices. The anterior ERP positivity likely reflects frontal-lobe attentional control, whereas the subsequent ERP negativity likely reflects anticipatory biasing of activity in auditory cortex.

Lateralized frontal activity elicited by attention-directing visual and auditory cues.

In event-related potential studies of voluntary spatial attention, lateralized activity observed over anterior scalp sites prior to an impending target has been interpreted as the activity of a supramodal attentional control mechanism in the frontal lobes. However, variability in the scalp topography and presence of this activity across studies suggests that multiple neural generators contribute to the lateralized activity recorded at the scalp. Using distributed source modeling we found two distinct frontal lobe sources following attention-directing cues, one dependent on the sensory modality of the eliciting stimulus and one dependent on the response requirements of the task. Differential activity of these sources depending on task parameters suggests that neither source reflects activity necessary for controlling attention.

An event-related potential study of supramodal attentional control and crossmodal attention effects.

We conducted two audiovisual experiments to determine whether event-related potential (ERP) components elicited by attention-directing cues reflect supramodal attentional control. Symbolic visual cues were used to direct attention prior to auditory targets in Experiment 1, and symbolic auditory cues were used to direct attention prior to visual targets in Experiment 2. Different patterns of cue ERPs were found in the two experiments. A frontal negativity called the ADAN was absent in Experiment 2, which indicates that this component does not reflect supramodal attentional control. A posterior positivity called the LDAP was observed in both experiments but was focused more posteriorly over the occipital scalp in Experiment 2. This component appears to reflect multiple processes, including visual processes involved in location marking and target preparation as well as supramodal processes involved in attentional control.

Control mechanisms mediating shifts of attention in auditory and visual space: a spatio-temporal ERP analysis.

The neural systems that mediate voluntary shifts of attention to visual and auditory stimuli were investigated by examining the patterns of human brain electricity elicited by attention-directing cues in auditory and visual tasks. Several lateralized event-related potential (ERP) components were observed when participants shifted attention in expectation of visual targets (experiment 1). One component was focused over frontal cortex and a second was focused primarily over the occipital-temporal cortex but also spread to parietal regions of the scalp. Previous work has indicated that the frontal component reflects supramodal processes involved in the executive control of attention and that the posterior component reflects either spatial attentional control processes in the posterior parietal lobe or modulation of processes in visual cortex. Here, the posterior component was observed when participants shifted attention in expectation of auditory targets (experiments 2-4), but the frontal component was found only in the visual task. The posterior component seemed to be generated in parietal and occipital areas even when there was no visual information about the to-be-attended locations. These results are consistent with the view that voluntary shifts of attention are mediated by supramodal processes in the parietal lobe and that the spatial coordinates of the to-be-attended location are based on visual representations of space.