The perception of visible speech: estimation of speech rate and detection of time reversals.

Four experiments investigated the perception of visible speech. Experiment 1 addressed the perception of speech rate. Observers were shown video-clips of the lower face of actors speaking at their spontaneous rate. Then, they were shown muted versions of the video-clips, which were either accelerated or decelerated. The task (scaling) was to compare visually the speech rate of the stimulus to the spontaneous rate of the actor being shown. Rate estimates were accurate when the video-clips were shown in the normal direction (forward mode). In contrast, speech rate was underestimated when the video-clips were shown in reverse (backward mode). Experiments 2-4 (2AFC) investigated how accurately one discriminates forward and backward speech movements. Unlike in Experiment 1, observers were never exposed to the sound track of the video-clips. Performance was well above chance when playback mode was crossed with rate modulation, and the number of repetitions of the stimuli allowed some amount of speechreading to take place in forward mode (Experiment 2). In Experiment 3, speechreading was made much more difficult by using a different and larger set of muted video-clips. Yet, accuracy decreased only slightly with respect to Experiment 2. Thus, kinematic rather then speechreading cues are most important for discriminating movement direction. Performance worsened, but remained above chance level when the same stimuli of Experiment 3 were rotated upside down (Experiment 4). We argue that the results are in keeping with the hypothesis that visual perception taps into implicit motor competence. Thus, lawful instances of biological movements (forward stimuli) are processed differently from backward stimuli representing movements that the observer cannot perform.

Detecting temporal reversals in human locomotion.

An experiment investigated the ability by human observers to detect temporal reversals in dynamic displays of human locomotion. We video-taped the lower portion of the body of actors walking at their preferred speed either in the normal, forward direction (FW) or in the backward direction (BW). The videos were presented in a random order either as recorded (N) or in reverse (R). In one session, we presented both normal and time-reversed stimuli in the original upright orientation. In a second session, the stimuli were rotated by 180° around the horizontal axis. Observers were informed that the real recorded movement was either forward or backward and were asked to decide whether or not the movement had been time-reversed prior to the presentation. Although the kinematics of forward and backward human locomotion is quite similar, the detection of temporal reversals followed a consistent pattern showing a good accuracy in condition FW-N and a reduced but still above-chance performance in condition BW-R (by design, in both conditions actors appeared to walk forward). Performance was instead at chance level in the other two conditions where the apparent direction of the movement was backward. Inverting the spatial orientation of the stimuli reduced but did not suppress the ability to detect temporal reversals in the two conditions with apparent forward direction of movement. It is argued that implicit motor competence is at least in part instrumental for extracting the subtle discriminal information from the stimuli.

The influence of distracters, stimulus duration and hemianopia on first saccade in patients with unilateral neglect.

The aim of the present study was to assess the influence of visual distracters, stimulus duration and the presence of contralesional hemianopia on direction of the first saccade in right brain damaged (RBD) patients affected by left unilateral neglect (UN). During a visual search task we recorded eye movements in five RBD patients with UN and hemianopia (N+H+), nine RBD patients with UN but no hemianopia (N+H-), four RBD patients with neither neglect nor hemianopia and four normal controls. Two task variables were orthogonally manipulated: (a) presence or absence of distracters and (b) short or long stimulus duration. A significant interaction was found between groups, presence of distracters, stimulus duration and the direction of the first saccade made in the search. Independently of the temporal duration of targets, in N+H+ patients the presence of distracters enhanced the frequency of saccades directed ipsilesionally (i.e., rightward). In N+H- patients, distracters biased the first saccade toward the right side only at short stimulus duration. These data show that bias of attentional orienting toward stimuli in the right half of space is specific of UN. This pathological mechanism (a) is enhanced and prolonged, over the period of exploration, by concomitant complete contralateral hemianopia and (b) is most evident, even in the absence of concomitant visual field defects, when voluntary planning of attention and eye movements are precluded by the short duration of stimuli to be inspected.

Determining priority between attentional and referential-coding sources of the Simon effect through optokinetic stimulation.

The "Simon effect" is the performance advantage for spatially corresponding target-response ensembles that is observed when coding of target position is irrelevant for the selection of motor responses. The "attentional-shift" account of the Simon effect holds that it arises from the congruency between response location and the direction of the last shift of attention toward the target. The "referential-coding" account traces the origin of the Simon effect back to the congruency between the response location and the position of the target with respect to a spatial reference frame. We were able to contrast these two hypotheses using full-field horizontal optokinetic stimulation (OKS). It was shown that OKS moving in one horizontal direction drives covert orienting of attention toward the side of arrival of OKS, i.e. the "In-coming" side, which is opposed to the direction of OKS motion toward the "Out-going" side (Teramoto et al., 2004; Watanabe, 2001). We therefore asked healthy participants to discriminate between slow and fast velocities of leftward or rightward OKS. "Fast" and "slow" responses were associated to response buttons positioned in the left or right side of space. The "attentional-shift" account of the Simon effect predicts that response compatibility should be related to the direction of the attentional shift induced by OKS, i.e. in the direction opposite to OKS motion. By contrast, the "referential-coding" hypothesis predicts that response compatibility should be related to the direction of OKS displacement with respect to its starting position. We observed faster RTs when the response button was on the "In-coming" side of space, opposite to the direction of OKS motion. This result supports priority of attentional over referential-coding factors in the genesis of the Simon effect.

The "ways" we look at dreams: evidence from unilateral spatial neglect (with an evolutionary account of dream bizarreness).

Despite decades of research, the question of whether the rapid eye movements (REMs) of paradoxical sleep (PS) are equivalent to waking saccades and whether their direction is congruent with visual spatial events in the dream scene is still very controversial. We gained an insight into these questions through the study of a right brain damaged patient suffering attentional neglect for the left side of space and drop of the optokinetic nystagmus (OKN) with alternating rightward slow/leftward fast phases evoked by rightward optic flow. During PS the patient had frequent Nystagmoid REMs with alternating leftward slow/rightward fast phases and reported dreams with visual events evoking corresponding OKN such as a train running leftward. By contrast, just as in waking OKN, Nystagmoid REMs with alternating rightward slow/leftward fast phases were virtually absent. REMs followed by staring eye position or by consecutive REMs were also observed: these showed no asymmetry comparable to that of Nystagmoid ones. The selective disappearance of Nystagmoid REMs in one horizontal direction proves, for the first time, that in humans different types of REMs exists and that these are driven by different premotor mechanisms. Concomitant drop of OKN and Nystagmoid REMs toward the same horizontal direction demonstrates that phylogenetically ancient oculomotor mechanisms, such as the OKN, are shared by waking and PS. On this evidence and converging findings from animal, neuropsychological and brain imaging studies, a new evolutionary account of dream bizarreness is proposed. Classification and labelling of the different types of REMs are also provided.

Effects of vestibular rotatory accelerations on covert attentional orienting in vision and touch.

Peripheral vestibular organs feed the central nervous system with inputs favoring the correct perception of space during head and body motion. Applying temporal order judgments (TOJs) to pairs of simultaneous or asynchronous stimuli presented in the left and right egocentric space, we evaluated the influence of leftward and rightward vestibular rotatory accelerations given around the vertical head-body axis on covert attentional orienting. In a first experiment, we presented visual stimuli in the left and right hemifield. In a second experiment, tactile stimuli were presented to hands lying on their anatomical side or in a crossed position across the sagittal body midline. In both experiments, stimuli were presented while normal subjects suppressed or did not suppress the vestibulo-ocular response (VOR) evoked by head-body rotation. Independently of VOR suppression, visual and tactile stimuli presented on the side of rotation were judged to precede simultaneous stimuli presented on the side opposite the rotation. When limbs were crossed, attentional facilitatory effects were only observed for stimuli presented to the right hand lying in the left hemispace during leftward rotatory trials with VOR suppression. This result points to spatiotopic rather than somatotopic influences of vestibular inputs, suggesting that cross-modal effects of these inputs on tactile ones operate on a representation of space that is updated following arm crossing. In a third control experiment, we demonstrated that temporal prioritization of stimuli presented on the side of rotation was not determined by response bias linked to spatial compatibility between the directions of rotation and the directional labels used in TOJs (i.e., "left" or "right" first). These findings suggest that during passive rotatory head-body accelerations, covert attention is shifted toward the direction of rotation and the direction of the fast phases of the VOR.

Underestimation of contralateral space in neglect: a deficit in the "where" task.

Space perception was investigated in right brain damaged patients with ( N=13) and without neglect ( N=5; control group). Patients were requested to localise a target tachistoscopically flashed at various eccentricities along the horizontal meridian. All patients had an intact visual field and spared ability to manually point to a target. To segregate magno- and parvo-pathway activity, stimuli modulated in either luminance or chromatic contrast were used. Patients were required to verbally report the stimulus position (verbal task) or to manually point to the stimulus (pointing task). Neglect patients reported the stimuli in the left visual field closer to the centre than they actually were. In the verbal test, underestimation was about 7 deg at the most eccentric position tested (20 deg), and decreased linearly for smaller eccentricities. The effect was similar but less marked in the pointing task. No difference was found for stimuli with luminance or chromatic contrast. Space underestimation was confined to the contralesional space; no evidence of misperception was detected in the ipsilesional hemifield. The present findings are consistent with the view that contralesional space representation is compressed in neglect patients.