Propagation and update of auditory perceptual priors through alpha and theta rhythms.

To maintain a continuous and coherent percept over time, the brain makes use of past sensory information to anticipate forthcoming stimuli. We recently showed that auditory experience of the immediate past is propagated through ear-specific reverberations, manifested as rhythmic fluctuations of decision bias at alpha frequencies. Here, we apply the same time-resolved behavioural method to investigate how perceptual performance changes over time under conditions of stimulus expectation and to examine the effect of unexpected events on behaviour. As in our previous study, participants were required to discriminate the ear-of-origin of a brief monaural pure tone embedded in uncorrelated dichotic white noise. We manipulated stimulus expectation by increasing the target probability in one ear to 80%. Consistent with our earlier findings, performance did not remain constant across trials, but varied rhythmically with delay from noise onset. Specifically, decision bias showed a similar oscillation at ~9 Hz, which depended on ear congruency between successive targets. This suggests rhythmic communication of auditory perceptual history occurs early and is not readily influenced by top-down expectations. In addition, we report a novel observation specific to infrequent, unexpected stimuli that gave rise to oscillations in accuracy at ~7.6 Hz one trial after the target occurred in the non-anticipated ear. This new behavioural oscillation may reflect a mechanism for updating the sensory representation once a prediction error has been detected.

Auditory Perceptual History Is Propagated through Alpha Oscillations.

Perception is a proactive, "predictive" process, in which the brain relies, at least in part, on accumulated experience to make best guesses about the world to test against sensory data, updating the guesses as new experience is acquired. Using novel behavioral methods, the present study demonstrates the role of alpha rhythms in communicating past perceptual experience. Participants were required to discriminate the ear of origin of brief sinusoidal tones that were presented monaurally at random times within a burst of uncorrelated dichotic white noise masks. Performance was not constant but varied with delay after noise onset in an oscillatory manner at about 9 Hz (alpha rhythm). Importantly, oscillations occurred only for trials preceded by a target tone to the same ear, either on the previous trial or two trials back. These results suggest that communication of perceptual history generates neural oscillations within specific perceptual circuits, strongly implicating behavioral oscillations in predictive perception and with formation of working memory.

Auditory frequency perception adapts rapidly to the immediate past.

Frequency modulation is critical to human speech. Evidence from psychophysics, neurophysiology, and neuroimaging suggests that there are neuronal populations tuned to this property of speech. Consistent with this, extended exposure to frequency change produces direction specific aftereffects in frequency change detection. We show that this aftereffect occurs extremely rapidly, requiring only a single trial of just 100-ms duration. We demonstrate this using a long, randomized series of frequency sweeps (both upward and downward, by varying amounts) and analyzing intertrial adaptation effects. We show the point of constant frequency is shifted systematically towards the previous trial's sweep direction (i.e., a frequency sweep aftereffect). Furthermore, the perception of glide direction is also independently influenced by the glide presented two trials previously. The aftereffect is frequency tuned, as exposure to a frequency sweep from a set centered on 1,000 Hz does not influence a subsequent trial drawn from a set centered on 400 Hz. More generally, the rapidity of adaptation suggests the auditory system is constantly adapting and "tuning" itself to the most recent environmental conditions.

Auditory and tactile signals combine to influence vision during binocular rivalry.

Resolution of perceptual ambiguity is one function of cross-modal interactions. Here we investigate whether auditory and tactile stimuli can influence binocular rivalry generated by interocular temporal conflict in human subjects. Using dichoptic visual stimuli modulating at different temporal frequencies, we added modulating sounds or vibrations congruent with one or the other visual temporal frequency. Auditory and tactile stimulation both interacted with binocular rivalry by promoting dominance of the congruent visual stimulus. This effect depended on the cross-modal modulation strength and was absent when modulation depth declined to 33%. However, when auditory and tactile stimuli that were too weak on their own to bias binocular rivalry were combined, their influence over vision was very strong, suggesting the auditory and tactile temporal signals combined to influence vision. Similarly, interleaving discrete pulses of auditory and tactile stimuli also promoted dominance of the visual stimulus congruent with the supramodal frequency. When auditory and tactile stimuli were presented at maximum strength, but in antiphase, they had no influence over vision for low temporal frequencies, a null effect again suggesting audio-tactile combination. We also found that the cross-modal interaction was frequency-sensitive at low temporal frequencies, when information about temporal phase alignment can be perceptually tracked. These results show that auditory and tactile temporal processing is functionally linked, suggesting a common neural substrate for the two sensory modalities and that at low temporal frequencies visual activity can be synchronized by a congruent cross-modal signal in a frequency-selective way, suggesting the existence of a supramodal temporal binding mechanism.

Masker location uncertainty reveals evidence for suppression of maskers in two-talker contexts.

In many natural settings, spatial release from masking aids speech intelligibility, especially when there are competing talkers. This paper describes a series of three experiments that investigate the role of prior knowledge of masker location on phoneme identification and spatial release from masking. In contrast to previous work, these experiments use initial stop-consonant identification as a test of target intelligibility to ensure that listeners had little time to switch the focus of spatial attention during the task. The first experiment shows that target phoneme identification was worse when a masker played from an unexpected location (increasing the consonant identification threshold by 2.6 dB) compared to when an energetically very similar and symmetrically located masker came from an expected location. In the second and third experiments, target phoneme identification was worse (increasing target threshold levels by 2.0 and 2.6 dB, respectively) when the target was played unexpectedly on the side from which the masker was expected compared to when the target came from an unexpected, symmetrical location in the hemifield opposite the expected location of the masker. These results support the idea that listeners modulate spatial attention by both focusing resources on the expected target location and withdrawing attentional resources from expected locations of interfering sources.

Multisensory congruency as a mechanism for attentional control over perceptual selection.

The neural mechanisms underlying attentional selection of competing neural signals for awareness remains an unresolved issue. We studied attentional selection, using perceptually ambiguous stimuli in a novel multisensory paradigm that combined competing auditory and competing visual stimuli. We demonstrate that the ability to select, and attentively hold, one of the competing alternatives in either sensory modality is greatly enhanced when there is a matching cross-modal stimulus. Intriguingly, this multimodal enhancement of attentional selection seems to require a conscious act of attention, as passively experiencing the multisensory stimuli did not enhance control over the stimulus. We also demonstrate that congruent auditory or tactile information, and combined auditory-tactile information, aids attentional control over competing visual stimuli and visa versa. Our data suggest a functional role for recently found neurons that combine voluntarily initiated attentional functions across sensory modalities. We argue that these units provide a mechanism for structuring multisensory inputs that are then used to selectively modulate early (unimodal) cortical processing, boosting the gain of task-relevant features for willful control over perceptual awareness.

Retinotopic and non-retinotopic stimulus encoding in binocular rivalry and the involvement of feedback.

Adaptation is one of the key constituents of the perceptual alternation process during binocular rivalry, as it has been shown that preadapting one of the rivaling pairs before rivalry onset biases perception away from the adapted stimulus during rivalry. We investigated the influence of retinotopic and spatiotopic preadaptation on binocular rivalry. We show that for grating stimuli, preadaptation only influences rivalry when adaptation and rivalry locations are retinotopically matched. With more complex house and face stimuli, effects of preadaptation are found for both retinotopic and spatiotopic preadaptation, showing the importance of spatiotopic encoding in binocular rivalry. We show, furthermore, that adaptation to phase-scrambled faces results in retinotopic effects only, demonstrating the importance of form content for spatiotopic adaptation effects, as opposed to spatial frequency content. Are the spatiotopic adaptation influences on rivalry caused by direct spatiotopic stimulus interactions, or instead are they due to altered feedback from the adapted spatiotopic representations to the retinotopic representations that are involved in rivalry? By using rivaling face and grating stimuli that minimize rivalry between spatiotopic representations while still engaging these representations in stimulus encoding, we show that at least part of the preadaptation effects with face stimuli depend on feedback information.

Combining visual and auditory information.

Robust perception requires that information from by our five different senses be combined at some central level to produce a single unified percept of the world. Recent theory and evidence from many laboratories suggests that the combination does not occur in a rigid, hardwired fashion, but follows flexible situation-dependent rules that allow information to be combined with maximal efficiency. In this review we discuss recent evidence from our laboratories investigating how information from auditory and visual modalities is combined. The results support the notion of Bayesian combination. We also examine temporal alignment of auditory and visual signals, and show that perceived simultaneity does not depend solely on neural latencies, but involves active processes that compensate, for example, for the physical delay introduced by the relatively slow speed of sound. Finally, we go on to show that although visual and auditory information is combined to maximize efficiency, attentional resources for the two modalities are largely independent.

Separate attentional resources for vision and audition.

Current models of attention, typically claim that vision and audition are limited by a common attentional resource which means that visual performance should be adversely affected by a concurrent auditory task and vice versa. Here, we test this implication by measuring auditory (pitch) and visual (contrast) thresholds in conjunction with cross-modal secondary tasks and find that no such interference occurs. Visual contrast discrimination thresholds were unaffected by a concurrent chord or pitch discrimination, and pitch-discrimination thresholds were virtually unaffected by a concurrent visual search or contrast discrimination task. However, if the dual tasks were presented within the same modality, thresholds were raised by a factor of between two (for visual discrimination) and four (for auditory discrimination). These results suggest that at least for low-level tasks such as discriminations of pitch and contrast, each sensory modality is under separate attentional control, rather than being limited by a supramodal attentional resource. This has implications for current theories of attention as well as for the use of multi-sensory media for efficient informational transmission.

Perceptual synchrony of audiovisual streams for natural and artificial motion sequences.

We investigated the conditions necessary for perceptual simultaneity of visual and auditory stimuli under natural conditions: video sequences of conga drumming at various rhythms. Under most conditions, the auditory stream needs to be delayed for sight and sound to be perceived simultaneously. The size of delay for maximum perceived simultaneity varied inversely with drumming tempo, from about 100 ms at 1 Hz to 30 ms at 4 Hz. Random drumming motion produced similar results, with higher random tempos requiring less delay. Video sequences of disk stimuli moving along a motion profile matched to the drummer produced near-identical results. When the disks oscillated at constant speed rather than following "biological" speed variations, the delays necessary for perceptual synchrony were systematically less. The results are discussed in terms of real-world constraints for perceptual synchrony and possible neural mechanisms.

Perceived timing of first- and second-order changes in vision and hearing.

Simultaneous changes in visual stimulus attributes (such as motion or color) are often perceived to occur at different times, a fact usually attributed to differences in neural processing times of those attributes. However, other studies suggest that perceptual misalignments are not due to stimulus attributes, but to the type of change, first- or second-order. To test whether this idea generalizes across modalities, we studied perceptual synchrony of acoustic and of audiovisual cross-modal stimuli, which varied in a first- or second-order fashion. First-order changes were abrupt changes in tone intensity or frequency (auditory), or spatial position (visual), while second-order changes were an inversion of the direction of change, such as a turning point when a rising tone starts falling or a translating visual blob reverses. For both pure acoustic and cross-modal stimuli, first-order changes were systematically perceived before second-order changes. However, when both changes were first-order, or both were second-order, little or no difference in perceptual delay was found between them, regardless of attribute or modality. This shows that the type of attribute change, as well as latency differences, is a strong determinant of subjective temporal alignments. We also performed an analysis of reaction times (RTs) to the first- and second-order attribute changes used in these temporal alignment experiments. RT differences between these stimuli did not correspond with our temporal alignment data, suggesting that subjective alignments cannot be accounted for by a simple latency-based explanation.

No direction-specific bimodal facilitation for audiovisual motion detection.

After several decades of unimodal perceptual research, interest is turning increasingly to cross-modal interactions. At a physiological level, the existence of bimodal cells is well documented and it is known that correlated audiovisual input enhances localisation and orienting behaviours. Audiovisual perceptual interactions have also been demonstrated (e.g., the well-known McGurk effect). The present study explores motion perception and asks whether correlated audiovisual motion signals would be better detected than unimodal motions or bimodal motions in opposing directions. Using a dynamic random-dot field with variable motion coherence as a visual stimulus, together with an auditory motion defined by a stereo noise source smoothly translating along a horizontal trajectory, we find that correlated bimodal motion yields only a slight improvement (approximately a square root of two advantage) in detection threshold relative to unimodal detection. The size of this benefit is consistent with a statistical advantage rather than a bimodal facilitation account. Moreover, anticorrelated bimodal motion showed the same modest improvement, again speaking against linear summation but consistent with statistical combination of visual and auditory signals. These findings were replicated in peripheral as well as in central vision, and with translating visual objects as well as with spatially distributed visual motion. The superadditivity observed neurally (especially in deep-layer superior collicular cells), when weak unimodal signals are combined in bimodal cells does not apply to the detection of linear translational motion.