The auditory stimulus reduced the visual inhibition of return: Evidence from psychophysiological interaction analysis.

Visual inhibition of return (IOR) is a mechanism for preventing attention from returning to previously examined spatial locations. Previous studies have found that auditory stimuli presented simultaneously with a visual target can reduce or even eliminate the visual IOR. However, the mechanism responsible for decreased visual IOR accompanied by auditory stimuli is unclear. Using functional magnetic resonance imaging, we aimed to investigate how auditory stimuli reduce visual IOR. Behaviorally, we found that the visual IOR accompanying auditory stimuli was significant but smaller than the visual IOR. Neurally, only in the validly cued trials, the superior temporal gyrus showed increased neural coupling with the intraparietal sulcus, presupplementary motor area, and some other areas in audiovisual conditions compared with visual conditions. These results suggest that the reduction in visual IOR by the simultaneous auditory stimuli may be due to a dual mechanism: rescuing the suppressed visual salience and facilitating response initiation. Our results support crossmodal interactions can occur across multiple neural levels and cognitive processing stages. This study provides a new perspective for understanding attention-orienting networks and response initiation based on crossmodal information.

Acoustic Stimuli Can Improve and Impair Somatosensory Perception.

The integration of stimuli from different sensory modalities forms the basis for human perception. While the relevant impact of visual stimuli on the perception of other sensory modalities is recognized, much less is known about the impact of auditory stimuli on general sensory processing. This study aims to investigate the effect of acoustic stimuli on the processing of somatosensory stimuli using real noise (i.e., unpleasant everyday noise, RN) and neutral white noise (WN). To this purpose, we studied 20 healthy human subjects between 20 and 29 years of age (mean: 24, SD: ±1.9 years sex ratio 1:1). Somatosensory perception was evaluated using mechanical detection threshold (MDT) of the skin on the back of the dominant hand. To investigate the underlying mechanisms in the brain, fMRI was performed while applying acoustic stimulation (RN and WN) and tactile stimulation of the dominant hand. Here we show that acoustic stimulation with noise alters the perception of touch on the skin. We found that the effect of RN and WN differed. RN leads to an improved tactile perception, whereas WN impaired tactile perception. These changes go along with significant differences in brain activity and connectivity. WN is associated with a significant increase in brain activity in multiple brain areas such as the auditory and somatosensory cortex, parietal association cortex, and the thalamus compared to RN. With tactile stimulation of the skin, the flow of information in these brain areas is altered. While with RN the information flow from the thalamus to the somatosensory cortex is prominent, the network activity pattern changes under WN revealing an increase in interaction between multiple networks. Unpleasant noise inhibits the multisensory integration and enables a more efficient unimodal perception in the somatosensory system, improving perception. Whether this is to be interpreted as a temporary increase in phasic alertness or by a stronger filter function of the thalamus with a preference for unimodal stimuli is still open for debate.

Gender-Dependent Crossmodal Interactions Between Olfactory and Tactile Stimulation Revealed Using the Unimodal Tactile Stimulation Device (UniTaSD).

Due to the complex stimulation methods required, olfaction and touch are 2 relatively understudied senses in the field of perceptual (neuro-)science. In order to establish a consistent presentation method for the bimodal stimulation of these senses, we combined an olfactometer with the newly developed Unimodal Tactile Stimulation Device. This setup allowed us to study the influence of olfaction on tactile perception and opened up an unexplored field of research by examining the crossmodal influence of tactile stimuli on olfaction. Using a pseudorandomized design, we analyzed how positive or negative tactile and olfactory stimuli influenced the opposing modality's perceived intensity and pleasantness. By asking participants to rate tactile stimuli, we were able to reproduce previously reported differences indicating that bimodal presentation with an olfactory stimulus increases or reduces perceived tactile pleasantness in an odor-dependent manner while highlighting that this effect appears unique to women. Furthermore, we found the first evidence for the influence of tactile stimuli on perceived odor pleasantness, an effect that is also driven primarily by women in our study. Based on these findings we believe that future neurophysiological studies, using controlled stimulus presentation can help unravel how and why olfactory and tactile perception interact in the human brain.

Visuotactile interaction even in far sagittal space in older adults with decreased gait and balance functions.

Spatial proximity of signals from different sensory modalities is known to be a crucial factor in facilitating efficient multisensory processing in young adults. However, recent studies have demonstrated that older adults exhibit strong visuotactile interactions even when the visual stimuli were presented in a spatially disparate position from a tactile stimulus. This suggests that visuotactile peripersonal space differs between older and younger adults. In the present study, we investigated to what extent peripersonal space expands in the sagittal direction and whether this expansion was linked to the decline in sensorimotor functions in older adults. Vibrotactile stimuli were delivered either to the left or right index finger, while visual stimuli were presented at a distance of 5 cm (near), 37.5 cm (middle), or 70 cm (far) from each finger. The participants had to respond rapidly to a randomized sequence of unimodal (visual or tactile) and simultaneous visuotactile targets (i.e., a redundant target paradigm). Sensorimotor functions were independently assessed by the Timed Up and Go (TUG) and postural stability tests. Results showed that reaction times to the visuotactile bimodal stimuli were significantly faster than those to the unimodal stimuli, irrespective of age group [younger adults: 22.0 ± 0.6 years, older adults: 75.0 ± 3.3 years (mean ± SD)] and target distance. Of importance, a race model analysis revealed that the co-activation model (i.e., visuotactile multisensory integrative process) is supported in the far condition especially for older adults with relatively poor performance on the TUG or postural stability tests. These results suggest that aging can change visuotactile peripersonal space and that it may be closely linked to declines in sensorimotor functions related to gait and balance in older adults.

Watching what's coming near increases tactile sensitivity: An experimental investigation.

During medical examinations, doctors regularly investigate a patient's somatosensory system by approaching the patient with a medical device (e.g. Von Frey hairs, algometer) or with their hands. It is assumed that the obtained results reflect the true capacities of the somatosensory system. However, evidence from crossmodal spatial research suggests that sensory experiences in one modality (e.g. touch) can be influenced by concurrent information from other modalities (e.g. vision), especially near the body (i.e. in peripersonal space). Hence, we hypothesized that seeing someone approaching your body could alter tactile sensitivity in that body-part. In the In Vivo Approaching Object (IVAO) paradigm, participants detected and localized threshold-level vibrotactile stimuli administered on the left of right hand (=tactile targets). In Experiment 1, this was always preceded by the experimenter approaching the same (congruent trials) or the other (incongruent trials) hand with a pen (=visual cue). In Experiment 2, a condition was added in which a point further away from the hands (also left vs. right) was approached. Response Accuracy was calculated for congruent and incongruent trials (Experiment 1 & 2) and compared between the close and far condition (Experiment 2). As expected, Response Accuracy was higher in congruent trials compared to incongruent trials, but only near the body. As a result, evidence was found for a crossmodal interaction effect between visual and tactile information in peripersonal space. These results suggest that somatosensory evaluations-both medical or research-based-may be biased by viewing an object approaching the body.

Watching a real moving object expands tactile duration: the role of task-irrelevant action context for subjective time.

Although it is well established that action contexts can expand the perceived durations of action-related events, whether action contexts also impact the subjective duration of events unrelated to the action remains an open issue. Here we examined how the automatic implicit reactions induced by viewing task-irrelevant, real moving objects influence tactile duration judgments. Participants were asked to make temporal bisection judgments of a tactile event while seeing a potentially catchable swinging ball. Approaching movement induced a tactile-duration overestimation relative to lateral movement and to a static baseline, and receding movement produced an expansion similar in duration to that from approaching movement. Interestingly, the effect of approaching movement on the subjective tactile duration was greatly reduced when participants held lightweight objects in their hands, relative to a hands-free condition, whereas no difference was obtained in the tactile-duration estimates between static hands-free and static hands-occupied conditions. The results indicate that duration perception is determined by internal bodily states as well as by sensory evidence.

Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review.

Research regarding crossmodal interactions has garnered much interest in the last few decades. A variety of studies have demonstrated that multisensory information (vision, audition, tactile sensation, and so on) can perceptually interact with each other in the spatial and temporal domains. Findings regarding crossmodal interactions in the spatiotemporal domain (i.e., motion processing) have also been reported, with updates in the last few years. In this review, we summarize past and recent findings on spatiotemporal processing in crossmodal interactions regarding perception of the external world. A traditional view regarding crossmodal interactions holds that vision is superior to audition in spatial processing, but audition is dominant over vision in temporal processing. Similarly, vision is considered to have dominant effects over the other sensory modalities (i.e., visual capture) in spatiotemporal processing. However, recent findings demonstrate that sound could have a driving effect on visual motion perception. Moreover, studies regarding perceptual associative learning reported that, after association is established between a sound sequence without spatial information and visual motion information, the sound sequence could trigger visual motion perception. Other sensory information, such as motor action or smell, has also exhibited similar driving effects on visual motion perception. Additionally, recent brain imaging studies demonstrate that similar activation patterns could be observed in several brain areas, including the motion processing areas, between spatiotemporal information from different sensory modalities. Based on these findings, we suggest that multimodal information could mutually interact in spatiotemporal processing in the percept of the external world and that common perceptual and neural underlying mechanisms would exist for spatiotemporal processing.

Multisensory perception as an associative learning process.

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O'Callaghan (forthcoming) calls this awareness "intermodal feature binding awareness." Psychologists have long assumed that multimodal perceptions such as this one are the result of a automatic feature binding mechanism (see Pourtois et al., 2000; Vatakis and Spence, 2007; Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal's jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called "unitization," whereby we come to "chunk" the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux's question to the question of how active or passive we consider perception to be.

A single auditory tone alters the perception of multiple visual events.

We aimed to show that a single auditory tone crossmodally affects multiple visual events using a multiple stream/bounce display (SBD), consisting of two disk pairs moving toward each other at equal speeds, coinciding, and then moving apart in a two-dimensional (2-D) display. The temporal offsets were manipulated between the coincidences of the disk pairs (0 to ±240 ms) by staggering motion onset between the pairs. A tone was presented at the coincidence timing of one of the disk pairs on half of the trials. Participants judged whether the disks in each of two pairs appeared to stream through or bounce off each other. Results show that a tone presented at either of the disk pairs' coincidence points promoted bouncing percepts in both disk pairs compared to no-tone trials. Perceived bouncing persisted in the disk-pair whose coincidence was offset 60 ms before and up to more than 120 ms after the audiovisual coincidence timing of the other disk-pair. The temporal window of bounce promotion was comparable to that obtained with a conventional SBD. The interaction of a single auditory event and multiple visual events was also modulated by the kind of experimental task (the stream/bounce or simultaneity judgments). These findings suggest that, using a single auditory cue, the perceptual system resolves the ambiguity of the motion of multiple disk pairs presented within the conventional temporal window of crossmodal interaction.