Multisensory enhancement of attention depends on whether you are already paying attention.

Multisensory stimuli are argued to capture attention more effectively than unisensory stimuli due to their ability to elicit a super-additive neuronal response. However, behavioural evidence for enhanced multisensory attentional capture is mixed. Furthermore, the notion of multisensory enhancement of attention conflicts with findings suggesting that multisensory integration may itself be dependent upon top-down attention. The present research resolves this discrepancy by examining how both endogenous attentional settings and the availability of attentional capacity modulate capture by multisensory stimuli. Across a series of four studies, two measures of attentional capture were used which vary in their reliance on endogenous attention: facilitation and distraction. Perceptual load was additionally manipulated to determine whether multisensory stimuli are still able to capture attention when attention is occupied by a demanding primary task. Multisensory stimuli presented as search targets were consistently detected faster than unisensory stimuli regardless of perceptual load, although they are nevertheless subject to load modulation. In contrast, task irrelevant multisensory stimuli did not cause greater distraction than unisensory stimuli, suggesting that the enhanced attentional status of multisensory stimuli may be mediated by the availability of endogenous attention. Implications for multisensory alerts in practical settings such as driving and aviation are discussed, namely that these may be advantageous during demanding tasks, but may be less suitable to signaling unexpected events.

Watching Subtitled Films Can Help Learning Foreign Languages.

Watching English-spoken films with subtitles is becoming increasingly popular throughout the world. One reason for this trend is the assumption that perceptual learning of the sounds of a foreign language, English, will improve perception skills in non-English speakers. Yet, solid proof for this is scarce. In order to test the potential learning effects derived from watching subtitled media, a group of intermediate Spanish students of English as a foreign language watched a 1h-long episode of a TV drama in its original English version, with English, Spanish or no subtitles overlaid. Before and after the viewing, participants took a listening and vocabulary test to evaluate their speech perception and vocabulary acquisition in English, plus a final plot comprehension test. The results of the listening skills tests revealed that after watching the English subtitled version, participants improved these skills significantly more than after watching the Spanish subtitled or no-subtitles versions. The vocabulary test showed no reliable differences between subtitled conditions. Finally, as one could expect, plot comprehension was best under native, Spanish subtitles. These learning effects with just 1 hour exposure might have major implications with longer exposure times.

Somatosensory saccades reveal the timing of tactile spatial remapping.

Remapping tactile events from skin to external space is an essential process for human behaviour. It allows us to refer tactile sensations to their actual externally based location, by combining anatomically based somatosensory information with proprioceptive information about the current body posture. We examined the time course of tactile remapping by recording speeded saccadic responses to somatosensory stimuli delivered to the hands. We conducted two experiments in which arm posture varied (crossed or uncrossed), so that anatomical and external frames of reference were either put in spatial conflict or were aligned. The data showed that saccade onset latencies in the crossed hands conditions were slower than in the uncrossed hands condition, suggesting that, in the crossed hands condition, remapping had to be completed before a correct saccade could be executed. Saccades to tactile stimuli when the hands were crossed were sometimes initiated to the wrong direction and then corrected in-flight, resulting in a turn-around saccade. These turn-around saccades were more likely to occur in short-latency responses, compared to onset latencies of saccades that went straight to target. The latter suggests that participants were postponing their saccade until the time the tactile event was represented according to the current body posture. We propose that the difference between saccade onset latencies of crossed and uncrossed hand postures, and between the onset of a turn-around saccade and a straight saccade in the crossed hand posture, reveal the timing of tactile spatial remapping.

Acoustic facilitation of object movement detection during self-motion.

In humans, as well as most animal species, perception of object motion is critical to successful interaction with the surrounding environment. Yet, as the observer also moves, the retinal projections of the various motion components add to each other and extracting accurate object motion becomes computationally challenging. Recent psychophysical studies have demonstrated that observers use a flow-parsing mechanism to estimate and subtract self-motion from the optic flow field. We investigated whether concurrent acoustic cues for motion can facilitate visual flow parsing, thereby enhancing the detection of moving objects during simulated self-motion. Participants identified an object (the target) that moved either forward or backward within a visual scene containing nine identical textured objects simulating forward observer translation. We found that spatially co-localized, directionally congruent, moving auditory stimuli enhanced object motion detection. Interestingly, subjects who performed poorly on the visual-only task benefited more from the addition of moving auditory stimuli. When auditory stimuli were not co-localized to the visual target, improvements in detection rates were weak. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow can operate on multisensory object representations.

Filling-in visual motion with sounds.

Information about the motion of objects can be extracted by multiple sensory modalities, and, as a consequence, object motion perception typically involves the integration of multi-sensory information. Often, in naturalistic settings, the flow of such information can be rather discontinuous (e.g. a cat racing through the furniture in a cluttered room is partly seen and partly heard). This study addressed audio-visual interactions in the perception of time-sampled object motion by measuring adaptation after-effects. We found significant auditory after-effects following adaptation to unisensory auditory and visual motion in depth, sampled at 12.5 Hz. The visually induced (cross-modal) auditory motion after-effect was eliminated if visual adaptors flashed at half of the rate (6.25 Hz). Remarkably, the addition of the high-rate acoustic flutter (12.5 Hz) to this ineffective, sparsely time-sampled, visual adaptor restored the auditory after-effect to a level comparable to what was seen with high-rate bimodal adaptors (flashes and beeps). Our results suggest that this auditory-induced reinstatement of the motion after-effect from the poor visual signals resulted from the occurrence of sound-induced illusory flashes. This effect was found to be dependent both on the directional congruency between modalities and on the rate of auditory flutter. The auditory filling-in of time-sampled visual motion supports the feasibility of using reduced frame rate visual content in multisensory broadcasting and virtual reality applications.

Failure to remap visuotactile space across the midline in the split-brain.

We examined the effect of posture change on the representation of visuotactile space in a split-brain patient using a cross-modal congruency task. Split-brain patient J.W. made speeded elevation discrimination responses (up versus down) to a series of tactile targets presented to the index finger or thumb of his right hand. We report congruency effects elicited by irrelevant visual distractors placed either close to, or far from, the stimulated hand. These cross-modal congruency effects followed the right hand as it moved within the right hemispace, but failed to do so when the hand crossed the midline into left hemispace. These results support recent claims that interhemispheric connections are required to maintain an accurate representation of visuotactile space.

The effects of acoustic mismatch and selective listening on repetition deafness.

Previous research (i.e., M. Miller & D. O. MacKay, 1994, 1996) has suggested that repetition deafness (RD), like repetition blindness, is robust to physical identity and that it consists in a failure to recall specifically the 2nd of the 2 critical targets (C1 and C2). However, some confounds due to memory load and response biases make available evidence inconclusive. Experiment 1 provided a strong test of RD between physically mismatching stimuli using a low memory load methodology. In Experiment 2, the same presentation method was combined with a selective recall task to find that RD is specific of C2. Experiments 3A and 3B showed, through an attentional manipulation, that RD is eliminated when people can successfully ignore C1 but not otherwise. It is argued that present data favor a perceptual interpretation of the RD. Furthermore, the present results support the hypothesis of recognition failure as opposed to the alternative token individuation failure hypothesis.

Spatial modulation of repetition blindness and repetition deafness.

When two identical visual items are presented in rapid succession, people often fail to report the second instance when trying to recall both (e.g., Kanwisher, 1987). We investigated whether this temporal processing deficit is modulated by the spatial separation between the repeated stimuli within both audition and vision. In Experiment 1, lists of one to three digits were rapidly presented from loudspeaker cones arranged in a semicircle around the participant. Recall accuracy was lower when repeated digits were presented from different positions rather than from the same position, as compared to unrepeated control pairs, demonstrating that auditory repetition deafness (RD) is modulated by the spatial displacement between repeated items. A similar spatial modulation of visual repetition blindness (RB) was reported when pairs of masked letters were presented visually from either the same or different positions arranged on a semicircle around fixation (Experiment 2). These results cannot easily be accounted for by the token individuation hypothesis of RB (Kanwisher, 1987; Park & Kanwisher, 1994) and instead support a recognition failure account (Hochhaus & Johnston, 1996; Luo & Caramazza, 1995, 1996).

An auditory repetition deficit under low memory load.

Previous studies of the auditory analogue of repetition blindness have led to different conclusions regarding the nature of the effect (e.g., N. Kanwisher & M. C. Potter, 1989; M. Miller & D. MacKay, 1994). In the present study, recall accuracy for repeated elements was examined with lists of 2 or 3 items presented dichotically under high temporal pressure. When this procedure was used, a repetition deficit in recall was obtained for both vowels (Experiment 1) and consonant-vowel syllables (Experiment 2). Further experiments demonstrated that this deficit decreases as the stimulus onset asynchrony between the 2 critical elements increases (Experiment 3) and showed that the effect also occurs for words and not just nonsense syllables (Experiment 4). In all 4 experiments, estimations of guessing biases showed that responses to unrepeated lists were not artificially favored over responses to repeated lists.

Online processing of native and non-native phonemic contrasts in early bilinguals.

There is considerable debate about whether bilinguals can distinguish L2 phonemic contrasts as efficiently as first language speakers can. To test this issue, a group of highly proficient Spanish-dominant Catalan-Spanish bilinguals (who had been exposed to Catalan between the ages of 3 and 4, but who, previous to this age, had been exposed only to Spanish) and another group of Catalan-dominant bilinguals (who had been exposed to Catalan from birth) were compared in a gating task. We developed a variation of the gating procedure that included a two-alternative forced choice test after each fragment was played. The differences between the two alternatives consisted of phonemic contrasts existing in Catalan but not in Spanish. Four contrasts were tested: two vocalic contrasts [symbols: see text] and two consonantal contrasts [symbols: see text]. The results showed that Spanish-dominant bilinguals, even the subset who were able to make correct identifications at the last gate, systematically performed worse than the group of Catalan-dominant bilinguals, needing longer portions of the signal to be able to correctly identify the stimuli. We argue that these results support the hypothesis that L1 shapes the perceptual system at early stages of development in such a way that it will determine the perception of non-native phonemic contrasts, even if there is extensive and early exposure to L2.