Audiovisual integration in the human brain: a coordinate-based meta-analysis.

People can seamlessly integrate a vast array of information from what they see and hear in the noisy and uncertain world. However, the neural underpinnings of audiovisual integration continue to be a topic of debate. Using strict inclusion criteria, we performed an activation likelihood estimation meta-analysis on 121 neuroimaging experiments with a total of 2,092 participants. We found that audiovisual integration is linked with the coexistence of multiple integration sites, including early cortical, subcortical, and higher association areas. Although activity was consistently found within the superior temporal cortex, different portions of this cortical region were identified depending on the analytical contrast used, complexity of the stimuli, and modality within which attention was directed. The context-dependent neural activity related to audiovisual integration suggests a flexible rather than fixed neural pathway for audiovisual integration. Together, our findings highlight a flexible multiple pathways model for audiovisual integration, with superior temporal cortex as the central node in these neural assemblies.

Cortical and Subcortical Coordination of Visual Spatial Attention Revealed by Simultaneous EEG-fMRI Recording.

Visual spatial attention has been studied in humans with both electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) individually. However, due to the intrinsic limitations of each of these methods used alone, our understanding of the systems-level mechanisms underlying attentional control remains limited. Here, we examined trial-to-trial covariations of concurrently recorded EEG and fMRI in a cued visual spatial attention task in humans, which allowed delineation of both the generators and modulators of the cue-triggered event-related oscillatory brain activity underlying attentional control function. The fMRI activity in visual cortical regions contralateral to the cued direction of attention covaried positively with occipital gamma-band EEG, consistent with activation of cortical regions representing attended locations in space. In contrast, fMRI activity in ipsilateral visual cortical regions covaried inversely with occipital alpha-band oscillations, consistent with attention-related suppression of the irrelevant hemispace. Moreover, the pulvinar nucleus of the thalamus covaried with both of these spatially specific, attention-related, oscillatory EEG modulations. Because the pulvinar's neuroanatomical geometry makes it unlikely to be a direct generator of the scalp-recorded EEG, these covariational patterns appear to reflect the pulvinar's role as a regulatory control structure, sending spatially specific signals to modulate visual cortex excitability proactively. Together, these combined EEG/fMRI results illuminate the dynamically interacting cortical and subcortical processes underlying spatial attention, providing important insight not realizable using either method alone.SIGNIFICANCE STATEMENT Noninvasive recordings of changes in the brain's blood flow using functional magnetic resonance imaging and electrical activity using electroencephalography in humans have individually shown that shifting attention to a location in space produces spatially specific changes in visual cortex activity in anticipation of a stimulus. The mechanisms controlling these attention-related modulations of sensory cortex, however, are poorly understood. Here, we recorded these two complementary measures of brain activity simultaneously and examined their trial-to-trial covariations to gain insight into these attentional control mechanisms. This multi-methodological approach revealed the attention-related coordination of visual cortex modulation by the subcortical pulvinar nucleus of the thalamus while also disentangling the mechanisms underlying the attentional enhancement of relevant stimulus input and those underlying the concurrent suppression of irrelevant input.

Multinational Pharmaceutical Companies Shortchange Canada in Research and Development Investments: Is It Time to Pursue Other Options?

In 1987, the government passed legislation to protect brand-name pharmaceutical firms against competition from generic drug brands in exchange for economic investment in Canadian pharmaceutical research and development (R&D). Since 2002, brand-name pharmaceutical companies' R&D investments have fallen short of their commitment, while Canadians now pay the fourth highest drug prices of all the Organisation for Economic Co-operation and Development member countries. In this article, we examine the degree to which brand-name pharmaceutical companies have fallen short of their promises, discuss whether a patent policy is the best strategy to secure Canadian pharmaceutical R&D funding and propose practical alternatives to this arrangement.

Electrical neuroimaging of voluntary audiospatial attention: evidence for a supramodal attention control network.

Previous attempts to investigate the supramodal nature of attentional control have focused primarily on identifying neuroanatomical overlap in the frontoparietal systems activated during voluntary shifts of spatial attention in different sensory modalities. However, the activation of the same neural structures is insufficient evidence for a supramodal system, as the same brain regions could interact with one another in very different ways during shifts of attention in different modalities. Thus, to explore the similarity of the functional networks, it is necessary to identify the neural structures involved and to examine the timing and sequence of activities within the network. To this end, we used an electrical neuroimaging technique to localize the neural sources of electroencephalographic signals recorded from human subjects during audiospatial shifts of attention and to examine the timing and sequence of activities within several regions of interest. We then compared the results to an analogous study of visuospatial attention shifts. Similar frontal and parietal regions were activated during visual and auditory shifts of attention, and the timing of activities within these regions was nearly identical. Following this modality-independent sequence of attention-control activity, activity in the relevant sensory cortex was enhanced in anticipation of the response-relevant target. These results are consistent with the hypothesis that a single supramodal network of frontal and parietal regions mediates voluntary shifts of spatial attention and controls the flow of sensory information in modality-specific sensory pathways.

QEEG coherence patterns related to mathematics ability in children.

The current study investigates the utility of resting-state EEG coherence values in predicting standardized math scores in children. Quantitative EEG and standardized academic achievement analyses were performed on 60 school-aged children. Analyses assessing intrahemispheric coherence at rest were conducted across the sample of participants and several coherence networks were extracted and compared to standardized math achievement values. Specifically, networks that included Brodmann area 40 (a brain region involved in the cognitive processes responsible for mathematics performance) and whose coherence values were significantly correlated with standardized math scores were examined. Results indicate a total of four coherence networks, two in each hemisphere, that have utility in predicting general math skills in children. In addition to BA 40, these coherence networks include BAs in the right temporal lobe, right frontoparietal lobe, left superior temporal lobe, and the left medial prefrontal cortex. These findings address the current dearth of research on the neurological connectivity patterns that are foundational for mathematics abilities in children. Further, these results lay a foundation for the supplementary use of EEG in the assessment and identification practices surrounding math learning disabilities in children and additionally provide a neurocognitive framework upon which intervention research may be targeted.

Dynamic inhibitory control prevents salience-driven capture of visual attention.

The salience-driven selection theory is comprised of three main tenets: (a) the most salient stimulus within a monitored region of the visual field captures attention, (b) the only way to prevent salience-driven distraction is by narrowly focusing attention elsewhere, and (c) all other goal-driven processes are possible only after the most salient item has been attended. Evidence for and against this theory has been provided from two experimental paradigms. Here, event-related potentials (ERPs) recorded in a novel Go/No-Go paradigm disconfirmed all three of tenets of the theory. Participants were instructed to search cyan-item displays for a salient orientation singleton (Go trials) and to ignore randomly intermixed yellow-item displays that could also contain an orientation singleton (No-Go trials). ERP components associated with attentional orienting (posterior contralateral N2; N2pc), distractor suppression (distractor positivity; PD), and stimulus relevance (P2a) were isolated to test predictions stemming from the salience-driven selection theory. On No-Go trials, the salient oddball elicited a PD rather than an N2pc, indicating that it was suppressed, not attended. Moreover, a P2a emerged before the N2pc on Go trials, demonstrating that observers first evaluated the global color of each display and then decided to search for the oddball (Go trials) or to ignore it (No-Go trials). We conclude that goal-driven processes can lead to the prevention of salience-driven attention capture by salient visual objects within the attentional window. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Evoked and induced power oscillations linked to audiovisual integration of affect.

Our affective experiences are influenced by combined multisensory information. Although the enhanced effects of congruent audiovisual information on our affective experiences have been well documented, the role of neural oscillations in the audiovisual integration of affective signals remains unclear. First, it is unclear whether oscillatory activity changes as a function of valence. Second, the function of phase-locked and non-phase-locked power changes in audiovisual integration of affect has not yet been clearly distinguished. To fill this gap, the present study performed time-frequency analyses on EEG data acquired while participants perceived positive, neutral and negative naturalistic video and music clips. A comparison between the congruent audiovisual condition and the sum of unimodal conditions was used to identify supra-additive (Audiovisual > Visual + Auditory) or sub-additive (Audiovisual < Visual + Auditory) integration effects. The results showed that early evoked sub-additive theta and sustained induced supra-additive delta and beta activities are linked to audiovisual integration of affect regardless of affective content.

From alternation to repetition: Spatial attention biases contribute to sequential effects in a choice reaction-time task.

Observers often take longer to respond to a visual target when it appears at a recently stimulated location than when it appears at a new location in the visual field. This behavioral impairment - known as inhibition of return (IOR) - is mirrored by a reduction of an event-related potential (ERP) component called the N2pc that has been associated with attentional selection. Together, these findings indicate that the mechanism underlying IOR operates to bias covert attention against re-visiting the most recently attended location. The goal of the present study was to determine how this inhibitory attention bias evolves across successive trials of a two-item search task. Initially, targets appearing at previously attended locations were associated with behavioral IOR and a concomitant reduction of the N2pc. After several successive trials, this initial inhibitory bias was superseded by expectancy-based biases associated with "predictable" inter-trial patterns of location repeats or location changes, in some cases leading to faster responses and a larger N2pc when the target location repeated (facilitation of return).  These results provide evidence that biases in the covert deployment of attention are updated dynamically according to the recent selection history and contribute to well-known sequential effects in serial choice reaction-time tasks.

Multisensory Integration Is Modulated by Auditory Sound Frequency and Visual Spatial Frequency.

Accurate integration of auditory and visual information is essential for our ability to communicate with others. Previous studies have shown that the temporal discrepancies over which audiovisual speech stimuli will be integrated into a coherent percept are much wider than those typically observed for simple stimuli like beeps and flashes of light. However, our sensitivity to the low-level features of simple stimuli is not constant. We hypothesized that part of the enhanced integration of audiovisual speech may be due to it consisting predominantly of the sound frequencies and visual spatial frequencies that humans are most sensitive to. Here, we examined integration behaviors for pure tones across the sound frequency spectrum and visual gratings across the spatial frequency spectrum to examine how these low-level features modulate integration. The temporal window of integration was modulated by both sound frequency and visual spatial frequency, with the widest integration window occurring when both stimuli fell within their respective peak sensitivity ranges. These results suggest that part of the increased tolerance for temporal asynchrony typically observed for audiovisual speech may be due to the differential integration of low-level stimulus features that are dominant within complex audiovisual speech.

Isolating event-related potential components associated with voluntary control of visuo-spatial attention.

Attention-directing cues presented at fixation evoke several lateralized event-related potential (ERP) components prior to the onset of visual targets. These components have been associated with the control of visuo-spatial attention, but the neuro-cognitive operations and neural generators of the components are still largely unknown. Here, we isolated cue-elicited ERP activity in different ways to home in on different neuro-cognitive operations and to gain a better understanding about the possible neuroanatomical sources of the cue-elicited ERP activities. To isolate lateralized cue-ERP activity, we compared shift-left and shift-right cue ERPs to shift-up cue ERPs. To measure all of the ERP activity related to attentional control, including spatially nonspecific activity that is removed in the process of isolating lateralized cue-ERP components, we compared shift-cue ERPs to neutral-cue (i.e., no-shift) ERPs. Isolated lateralized-ERP activity was seen in the contralateral-occipital lobe in the early phase of the cue-target interval and in the ipsilateral-occipital lobe in the late phase. The later, ipsilateral activity indicates that the late directing attention positivity (LDAP) reflected processing of the to-be-ignored location. The neutral-cue isolation revealed a shift-related positivity over posterior scalp regions and a shift-related negativity over more anterior scalp regions. The spatio-temporal sequence of shift-related activity observed on the scalp, together with estimates of distributed source activity underlying the shift-related ERP components, indicated that frontal and parietal regions of cortex participated in the control of attention and led to pre-target biasing in visual cortical areas.

Electrophysiological evidence of an attentional bias in crossmodal inhibition of return.

Inhibition of return (IOR) refers to a delay in responding to targets when they appear at recently attended locations, relative to unattended locations. Within the visual modality, this attentional bias has been associated with a reduction in the N2pc event-related potential (ERP) component at previously attended locations. The present study examined whether a similar attentional bias was observed in crossmodal audio-visual IOR. Our results demonstrate that for visual targets, the attentional component of IOR behaves similarly for both unimodal and crossmodal target pairs, as indexed by a reduction in the N2pc component for targets appearing at previously attended locations. Further, similar IOR-related modulations on the auditory-evoked N2ac indicated that an attentional bias can be observed for auditory targets as well. Finally, we identified two additional ERP components - the ACOP and VCAN - that appear to reflect biasing of attention in the currently unattended sensory modality. These results suggest that the inhibitory attentional bias that underlies the IOR effect may be supramodal and bias attention away from previously attended locations regardless of sensory modality.

Temporal dynamics of audiovisual affective processing.

This study used event-related potentials (ERPs) to investigate the time course of auditory, visual, and audiovisual affective processing. Stimuli consisted of naturalistic silent videos, instrumental music clips, or combination of the two, with valence varied at three levels for each modality and arousal matched across valence conditions. Affective ratings of the unimodal and multimodal stimuli showed evidence of visual dominance, congruency, and negativity dominance effects. ERP results for unimodal presentations revealed valence effects in early components for both modalities, but only for the visual condition in a late positive potential. The ERP results for multimodal presentations showed effects for both visual valence and auditory valence in three components, early N200, P300 and LPP. A modeling analysis of the N200 component suggested its role in the visual dominance effect, which was further supported by a correlation between behavioral visual dominance scores and the early ERP components. Significant congruency comparisons were also found for N200 amplitudes, suggesting that congruency effects may occur early. Consistent differences between negative and positive valence were found for both visual and auditory modalities in the P300 at anterior electrode clusters, suggesting a potential source for the negativity dominance effect observed behaviorally. The separation between negative and positive valence also occurred at LPP for the visual modality. Significant auditory valence modulation was found for the LPP, implying an integration effect in which valence sensitivity of the LPP emerged for the audiovisual condition. These results provide a basis for mapping out the temporal dynamics of audiovisual affective processing.

Evidence for an attentional component of inhibition of return in visual search.

Inhibition of return (IOR) is typically described as an inhibitory bias against returning attention to a recently attended location as a means of promoting efficient visual search. Most studies examining IOR, however, either do not use visual search paradigms or do not effectively isolate attentional processes, making it difficult to conclusively link IOR to a bias in attention. Here, we recorded ERPs during a simple visual search task designed to isolate the attentional component of IOR to examine whether an inhibitory bias of attention is observed and, if so, how it influences visual search behavior. Across successive visual search displays, we found evidence of both a broad, hemisphere-wide inhibitory bias of attention along with a focal, target location-specific facilitation. When the target appeared in the same visual hemifield in successive searches, responses were slower and the N2pc component was reduced, reflecting a bias of attention away from the previously attended side of space. When the target occurred at the same location in successive searches, responses were facilitated and the P1 component was enhanced, likely reflecting spatial priming of the target. These two effects are combined in the response times, leading to a reduction in the IOR effect for repeated target locations. Using ERPs, however, these two opposing effects can be isolated in time, demonstrating that the inhibitory biasing of attention still occurs even when response-time slowing is ameliorated by spatial priming.

The effects of attention on the temporal integration of multisensory stimuli.

In unisensory contexts, spatially-focused attention tends to enhance perceptual processing. How attention influences the processing of multisensory stimuli, however, has been of much debate. In some cases, attention has been shown to be important for processes related to the integration of audio-visual stimuli, but in other cases such processes have been reported to occur independently of attention. To address these conflicting results, we performed three experiments to examine how attention interacts with a key facet of multisensory processing: the temporal window of integration (TWI). The first two experiments used a novel cued-spatial-attention version of the bounce/stream illusion, wherein two moving visual stimuli with intersecting paths tend to be perceived as bouncing off rather than streaming through each other when a brief sound occurs near in time. When the task was to report whether the visual stimuli appeared to bounce or stream, attention served to narrow this measure of the TWI and bias perception toward "streaming". When the participants' task was to explicitly judge the simultaneity of the sound with the intersection of the moving visual stimuli, however, the results were quite different. Specifically, attention served to mainly widen the TWI, increasing the likelihood of simultaneity perception, while also substantially increasing the simultaneity judgment accuracy when the stimuli were actually physically simultaneous. Finally, in Experiment 3, where the task was to judge the simultaneity of a simple, temporally discrete, flashed visual stimulus and the same brief tone pip, attention had no effect on the measured TWI. These results highlight the flexibility of attention in enhancing multisensory perception and show that the effects of attention on multisensory processing are highly dependent on the task demands and observer goals.

Resolving conflicting views: Gaze and arrow cues do not trigger rapid reflexive shifts of attention.

It has become widely accepted that the direction of another individual's eye gaze induces rapid, automatic, attentional orienting, due to it being such a vital cue as to where in our environment we should attend. This automatic orienting has also been associated with the directional-arrow cues used in studies of spatial attention. Here, we present evidence that the response-time cueing effects reported for spatially non-predictive gaze and arrow cues are not the result of rapid, automatic shifts of attention. For both cue types, response-time effects were observed only for long-duration cue and target stimuli that overlapped temporally, were largest when the cues were presented simultaneously with the response-relevant target, and were driven by a slowing of responses for invalidly cued targets rather than speeding for validly cued ones. These results argue against automatic attention-orienting accounts and support a novel spatial-incongruency explanation for a whole class of rapid behavioral cueing effects.

On the electrophysiological evidence for the capture of visual attention.

The presence of a salient distractor interferes with visual search. According to the salience-driven selection hypothesis, this interference is because of an initial deployment of attention to the distractor. Three event-related potential (ERP) findings have been regarded as evidence for this hypothesis: (a) salient distractors were found to elicit an ERP component called N2pc, which reflects attentional selection; (b) with target and distractor on opposite sides, a distractor N2pc was reported to precede the target N2pc (N2pc flip); (c) the distractor N2pc on slow-response trials was reported to occur particularly early, suggesting that the fastest shifts of attention were driven by salience. This evidence is equivocal, however, because the ERPs were noisy (b, c) and were averaged across all trials, thereby making it difficult to know whether attention was deployed directly to the target on some trials (a, b). We reevaluated this evidence using a larger sample size to reduce noise and by analyzing ERPs separately for fast- and slow-response trials. On fast-response trials, the distractor elicited a contralateral positivity (PD)-an index of attentional suppression-instead of an N2pc. There was no N2pc flip or early distractor N2pc. As it stands, then, there is no ERP evidence for the salience-driven selection hypothesis.

Arrow-elicited cueing effects at short intervals: Rapid attentional orienting or cue-target stimulus conflict?

The observation of cueing effects (faster responses for cued than uncued targets) rapidly following centrally-presented arrows has led to the suggestion that arrows trigger rapid automatic shifts of spatial attention. However, these effects have primarily been observed during easy target-detection tasks when both cue and target remain on the screen until the behavioral response. We manipulated stimulus duration and task difficulty in an attention-cueing experiment to explore non-attentional explanations for rapid cueing effects. Contrary to attention-based predictions, short-interval cueing effects were observed only for long-duration cue and target stimuli, occurred even when the cue and target were presented simultaneously, and were driven by slowing of the uncued-target responses, rather than any facilitation for cued targets. We propose that, under these long-duration, short-interval conditions, the processing of the cue and target interact more extensively in the brain, and that when the cue and target convey incongruent spatial information (i.e., on invalidly cued trials) it leads to conflict-related slowing of responses.

Theta modulation of inter-regional gamma synchronization during auditory attention control.

Synchronization of gamma oscillations among brain regions is relevant for dynamically organizing communication among neurons to support cognitive and perceptual processing, including attention orienting. Recent research has demonstrated that inter-regional synchronization in the gamma-band is modulated by theta rhythms during cortical processing. It has been proposed that such cross-frequency dynamics underlie the integration of local processes into large-scale functional networks. To investigate the potential role of theta-gamma mechanisms during auditory attention control, we localized activated regions using EEG beamformer analysis, and calculated inter-regional gamma-band synchronization between activated regions as well as modulation of inter-regional gamma-band synchronization by the phase of cortical theta rhythms. Abundant synchronization of gamma-band oscillations among regions comprising the auditory attention control network was observed. This inter-regional gamma synchronization was modulated by theta phase. These results provide further evidence implicating inter-regional gamma-band synchronization, and theta-gamma interactions, in task-dependent communication among cortical regions, and provide the first evidence that such mechanisms are relevant for auditory attention control.

The role of temporal predictability in the anticipatory biasing of sensory cortex during visuospatial shifts of attention.

The presentation of an attention-directing cue elicits a lateralized ERP deflection called the late directing attention positivity (LDAP) and lateralized changes in alpha-band elelctroencephalogram oscillations. Both of these electrophysiological responses have been independently linked to biasing of visual cortex in anticipation of an impending target. However, the LDAP is not always observed, and the link between the ERP and alpha-band modulations remains unclear. Here, we examined the effect of advance knowledge of the time of target onset on the ERP and alpha-band responses to cues. The LDAP was present only when the attention-directing cues accurately indicated the time of target appearance, whereas two sequential attention-related alpha-band modulations were observed regardless of the temporal information provided by the cues. Thus, alpha-band activity may be a more reliable index of pretarget biasing of visual cortical activity than lateralized ERP effects.

From local inhibition to long-range integration: a functional dissociation of alpha-band synchronization across cortical scales in visuospatial attention.

When attention is allocated to one visual hemifield, increased alpha power is observed in ipsilateral visual cortex. This has been attributed to synchronization of alpha-band oscillations within cortical regions which reflects inhibitory processing. Recent results, however, indicate that synchronization of alpha oscillations between cortical regions is relevant for transient functional coupling. Such coupling is thought to be involved in orienting attention to a specific region of the visual field. We thus hypothesized that alpha-band synchronization between low-level visual cortex and higher-level visual brain regions would be increased in the hemisphere contralateral to an attended location. To test this hypothesis we calculated phase synchronization between attention-related EEG source activations occurring between predictive directional cues and expected visual targets. Alpha amplitude (understood as an index of local synchronization) within low-level visual cortex was increased ipsilateral to attended locations and decreased contralateral to attended locations, consistent with alpha-band scalp topography and previous research relating local alpha power to active inhibition. Increased long-range alpha-band synchronization between low-level visual cortex and parietal cortex, however, was observed contralateral to the attended visual hemifield, whereas decreased synchronization (phase scattering) was observed in the ipsilateral hemisphere. These results identify a potential mechanism for the enhanced processing of stimuli appearing at attended locations, as long-range synchronization is thought to increase the fidelity and effectiveness of communication between brain areas. Our observation of inhibitory amplitude changes, interpreted as increased local-area synchronization, and facilitatory long-range synchronization demonstrates a functional dissociation for alpha-band synchronization across cortical scales.