Sound reduces saccadic chronostasis illusion.

The saccadic chronostasis illusion refers to the duration overestimation of the first visual stimulation after saccadic eye movement, which is also known as "stopped clock illusion." The present study investigated whether saccadic chronostasis would be observed in the auditory modality and whether the saccade-induced time dilation in the visual modality would be reduced by a synchronously presented sound. In each trial, a unisensory visual stimulus, unisensory sound, or bimodal audio-visual stimulus with a duration of 200-800 ms (probe stimulus) was presented at the saccade target location and temporally around the offset of the saccade, followed by a unisensory visual or auditory standard stimulus for a fixed 500 ms. Participants were required to identify which of the two stimuli (probe or standard) presented in the target modality (visual or auditory) was perceived as longer. The results showed that no saccadic chronostasis was observed in the auditory modality, regardless of whether the sound was presented alone or synchronously accompanied by a visual stimulus. Interestingly, the magnitude of the saccadic chronostasis illusion was reduced by the synchronously presented sound. Moreover, the combined effect of the saccade and sound on visual time perception fits well with the standard scalar model, and the weight of the cross-modal effect was higher than that of saccadic visual time dilation. These results suggest that sound dominates vision in time processing during saccades and linearly modulates saccadic chronostasis, which follows the Scalar Expectancy Theory.

Cross-modal enhancement of spatially unpredictable visual target discrimination during the attentional blink.

The attentional blink can be substantially reduced by delivering a task-irrelevant sound synchronously with the second target (T2) embedded in a rapid serial visual presentation stream, which is further modulated by the semantic congruency between the sound and T2. The present study extended the cross-modal boost during attentional blink and the modulation of audiovisual semantic congruency in the spatial domain by showing that a spatially uninformative, semantically congruent (but not incongruent) sound could even improve the discrimination of spatially unpredictable T2 during attentional blink. T2-locked event-related potential (ERP) data yielded that the early cross-modal P195 difference component (184-234 ms) over the occipital scalp contralateral to the T2 location was larger preceding accurate than inaccurate discriminations of semantically congruent, but not incongruent, audiovisual T2s. Interestingly, the N2pc component (194-244 ms) associated with visual-spatial attentional allocation was enlarged for incongruent audiovisual T2s relative to congruent audiovisual and unisensory visual T2s only when they were accurately discriminated. These ERP findings suggest that the spatially extended cross-modal boost during attentional blink involves an early cross-modal interaction strengthening the perceptual processing of T2, without any sound-induced enhancement of visual-spatial attentional allocation toward T2. In contrast, the absence of an accuracy decrease in response to semantically incongruent audiovisual T2s may originate from the semantic mismatch capturing extra visual-spatial attentional resources toward T2.

Audiovisual Emotional Congruency Modulates the Stimulus-Driven Cross-Modal Spread of Attention.

It has been reported that attending to stimuli in visual modality can spread to task-irrelevant but synchronously presented stimuli in auditory modality, a phenomenon termed the cross-modal spread of attention, which could be either stimulus-driven or representation-driven depending on whether the visual constituent of an audiovisual object is further selected based on the object representation. The stimulus-driven spread of attention occurs whenever a task-irrelevant sound synchronizes with an attended visual stimulus, regardless of the cross-modal semantic congruency. The present study recorded event-related potentials (ERPs) to investigate whether the stimulus-driven cross-modal spread of attention could be modulated by audio-visual emotional congruency in a visual oddball task where emotion (positive/negative) was task-irrelevant. The results first demonstrated a prominent stimulus-driven spread of attention regardless of audio-visual emotional congruency by showing that for all audiovisual pairs, the extracted ERPs to the auditory constituents of audiovisual stimuli within the time window of 200-300 ms were significantly larger than ERPs to the same auditory stimuli delivered alone. However, the amplitude of this stimulus-driven auditory Nd component during 200-300 ms was significantly larger for emotionally incongruent than congruent audiovisual stimuli when their visual constituents' emotional valences were negative. Moreover, the Nd was sustained during 300-400 ms only for the incongruent audiovisual stimuli with emotionally negative visual constituents. These findings suggest that although the occurrence of the stimulus-driven cross-modal spread of attention is independent of audio-visual emotional congruency, its magnitude is nevertheless modulated even when emotion is task-irrelevant.