Involvement of the superior colliculi in crossmodal correspondences.

There is an increasing body of evidence suggesting that there are low-level perceptual processes involved in crossmodal correspondences. In this study, we investigate the involvement of the superior colliculi in three basic crossmodal correspondences: elevation/pitch, lightness/pitch, and size/pitch. Using a psychophysical design, we modulate visual input to the superior colliculus to test whether the superior colliculus is required for behavioural crossmodal congruency effects to manifest in an unspeeded multisensory discrimination task. In the elevation/pitch task, superior colliculus involvement is required for a behavioural elevation/pitch congruency effect to manifest in the task. In the lightness/pitch and size/pitch task, we observed a behavioural elevation/pitch congruency effect regardless of superior colliculus involvement. These results suggest that the elevation/pitch correspondence may be processed differently to other low-level crossmodal correspondences. The implications of a distributed model of crossmodal correspondence processing in the brain are discussed.

Implicit expectation modulates multisensory perception.

Stimulus statistics can induce expectations that in turn can influence multisensory perception. In three experiments, we manipulate perceptual history by biasing stimulus statistics and examined the effect of implicit expectations on the perceptual resolution of a bistable visual stimulus that is modulated by sound. First, we found a general effect of expectation such that responses were biased in line with the biased statistics and interpret this as a bias towards an implicitly expected outcome. Second, expectation did not influence the perception of all types of stimuli. In both Experiment 1 and Experiment 2, integrated audio-visual stimuli were affected by expectation but visual-only and unintegrated audio-visual stimuli were not. In Experiment 3 we examined the sensory versus interpretational effects of expectation and found that contrary to our predictions, an expectation of audio-visually integrated stimuli was associated with impaired multisensory integration compared to visual-only or unintegrated audio-visual stimuli. Our findings suggest that perceptual experience implicitly creates expectations that influence multisensory perception, which appear to be about perceptual outcomes rather than sensory stimuli. Finally, in the case of resolving perceptual ambiguity, the expectation effect is an effect on cognitive rather than sensory processes.

The effects of recent perceptual history on stream-bounce perception.

We examine perceptual disambiguation and crossmodal interactions by considering the effect of recent perceptual history on stream-bounce perception. First, we tested the assumption that the audio-visual stream-bounce effect (visual-only trials mostly stream, whereas audio-visual trials mostly bounce) reflects some intrinsic preference for streaming that is broken by sound. Instead, we found that for naïve observers, visual-only stimuli are bistable and bias free. In intermixed trials, sound acts as a polarizing factor (rather than a bounce-inducing factor) and has as much effect on visual-only trials as it does on audio-visual trials. Second, temporal context exerts a comparative influence and exposure to audio-visual stimuli influences responses to visual-only stimuli and vice-versa. Finally, there is a serial dependence in responses and the current stimulus (unisensory or multisensory) is interpreted with a bias to recent interpretations. Recent perceptual history exerts a substantial influence on the perception of stream-bounce stimuli. This influence occurs in the unisensory case and is in line with an extensive literature on visual bistability; it extends to the multisensory case, and there are interactions between the 2 cases. Our combined findings support a role for top-down interpretational influences in the stream-bounce effect and stream-bounce perception more generally. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The Lightness/Pitch Crossmodal Correspondence Modulates the Rubin Face/Vase Perception.

We examine whether crossmodal correspondences (CMCs) modulate perceptual disambiguation by considering the influence of lightness/pitch congruency on the perceptual resolution of the Rubin face/vase (RFV). We randomly paired a black-and-white RFV (black faces and white vase, or vice versa) with either a high or low pitch tone and found that CMC congruency biases the dominant visual percept. The perceptual option that was CMC-congruent with the tone (white/high pitch or black/low pitch) was reported significantly more often than the perceptual option CMC-incongruent with the tone (white/low pitch or black/high pitch). However, the effect was only observed for stimuli presented for longer and not shorter durations suggesting a perceptual effect rather than a response bias, and moreover, we infer an effect on perceptual reversals rather than initial percepts. We found that the CMC congruency effect for longer-duration stimuli only occurred after prior exposure to the stimuli of several minutes, suggesting that the CMC congruency develops over time. These findings extend the observed effects of CMCs from relatively low-level feature-based effects to higher-level object-based perceptual effects (specifically, resolving ambiguity) and demonstrate that an entirely new category of crossmodal factors (CMC congruency) influence perceptual disambiguation in bistability.

Temporal dynamics of a perceptual decision.

Previous research suggests that cognitive factors acting in a top-down manner influence the perceptual interpretation of ambiguous stimuli. To examine the temporal unfolding of these influences as a perceptual decision evolves, we have implemented a modified version of the stream-bounce display. Our novel approach allows us to track responses to stream-bounce stimuli dynamically over the entire course of the motion sequence rather than collecting a subjective report after the fact. Using a trackpad, we had participants control a cursor to track a stream-bounce target actively from start to end and measured tracking speed throughout as the dependent variable. Our paradigm replicated the typical effect of visual-only displays being associated with a streaming bias and audiovisual displays with a bouncing bias. Our main finding is a significant behavioral change preceding a perceptual decision that then predicts that decision. Specifically, for trials in which the sound was presented, tracking speeds were significantly slower starting 500 ms before the point of coincidence and presentation of the sound for bounce compared to stream responses. We suggest that behavioral response may reflect a cognitive expectation of a perceptual outcome that then biases action and the interpretation of sensory input to favor that forthcoming percept in a manner consistent with both the predictive-coding and common-coding theoretical frameworks. Our approach provides a novel behavioral corroboration of recent imaging studies that are suggestive of early brain activity in perception and action.

Lightness/pitch and elevation/pitch crossmodal correspondences are low-level sensory effects.

We tested the sensory versus decisional origins of two established audiovisual crossmodal correspondences (CMCs; lightness/pitch and elevation/pitch), applying a signal discrimination paradigm to low-level stimulus features and controlling for attentional cueing. An audiovisual stimulus randomly varied along two visual dimensions (lightness: black/white; elevation: high/low) and one auditory dimension (pitch: high/low), and participants discriminated either only lightness, only elevation, or both lightness and elevation. The discrimination task and the stimulus duration varied between subjects. To investigate the influence of crossmodal congruency, we considered the effect of each CMC (lightness/pitch and elevation/pitch) on the sensitivity and criterion of each discrimination as a function of stimulus duration. There were three main findings. First, discrimination sensitivity was significantly higher for visual targets paired congruently (compared with incongruently) with tones while criterion was unaffected. Second, the sensitivity increase occurred for all stimulus durations, ruling out attentional cueing effects. Third, the sensitivity increase was feature specific such that only the CMC that related to the feature being discriminated influenced sensitivity (i.e. lightness congruency only influenced lightness discrimination and elevation congruency only influenced elevation discrimination in the single and dual task conditions). We suggest that these congruency effects reflect low-level sensory processes.

Sensitivity and Bias in the Resolution of Stream-Bounce Stimuli.

The audiovisual stream-bounce effect refers to the resolution of ambiguous motion sequences as streaming or bouncing depending on the presence or absence of a sound. We used a novel experimental design and signal detection theory (SDT) to determine its sensory or decisional origins. To account for issues raised by Witt et al. on the interpretation of SDT results, we devised a pure signal detection (as opposed to signal discrimination) paradigm and measured participants' sensitivity and criterion when detecting a weak tone concurrent with objectively streaming or bouncing visual displays. We observed no change in sensitivity but a significant change in criterion with participants' criterion more liberal with bouncing targets than for streaming targets with. In a second experiment, we tasked participants with detecting a weak tone in noise while viewing an ambiguous motion sequence. They also indicated whether the targets appeared to stream or bounce. Participants' reported equivalent, mostly bouncing responses for hit and false alarm trials, and equivalent, mostly streaming responses for correct rejection and miss trials. Further, differences in participants' sensitivity and criterion measures for detecting tones in subjectively streaming compared to subjectively bouncing targets were inconsistent with sensory factors. These results support a decisional account of the sound-induced switch from mostly streaming to mostly bouncing responses in audiovisual stream-bounce displays.

Low-Level Motion Characteristics Do Not Account for Perceptions of Stream-Bounce Stimuli.

The stream-bounce effect refers to a bistable motion stimulus that is interpreted as two targets either "streaming" past or "bouncing" off one another, and the manipulations that bias responses. Directional bias, according to Bertenthal et al., is an account of the effect proposing that low-level motion integration promotes streaming, and its disruption leads to bouncing, and it is sometimes cited either directly in a bottom-up fashion or indirectly under top-down control despite Sekuler and Sekuler finding evidence inconsistent with it. We tested two key aspects of the hypothesis: (a) comparable changes in speed should produce comparable disruptions and lead to similar effects; and (b) speed changes alone should disrupt integration without the need for additional more complex changes of motion. We found that target motion influences stream-bounce perception, but not as directional bias predicts. Our results support Sekuler and Sekuler and argue against the low-level motion signals driving perceptual outcomes in stream-bounce displays (directly or indirectly) and point to higher level inferential processes involving perceptual history and expectation. Directional bias as a mechanism should be abandoned and either another specific bottom-up process must be proposed and tested or consideration should be given to top-down factors alone driving the effect.