Socially Communicative Eye Contact and Gender Affect Memory.

Because of their value as a socially communicative cue, researchers have strived to understand how the gaze of other people influences a variety of cognitive processes. Recent work in social attention suggests that the use of images of people in laboratory studies, as a substitute for real people, may not effectively test socially communicative aspects of eye gaze. As attention affects many other cognitive processes, it is likely that social attention between real individuals could also affect other cognitive processes, such as memory. However, from previous work alone, it is unclear whether, and if so how, socially communicative eye gaze affects memory. The present studies test the assumption that socially communicative aspects of eye gaze may impact memory by manipulating the eye gaze of a live speaker in the context of a traditional recognition paradigm used frequently in the laboratory. A female (Experiment 1) or male (Experiment 2) investigator read words aloud and varied whether eye contact was, or was not, made with a participant. With both female and male investigators, eye contact improved word recognition only for female participants and hindered word recognition in male participants. When a female investigator prolonged their eye contact (Experiment 3) to provide a longer opportunity to both observe and process the investigator's eye gaze, the results replicated the findings from Experiments 1 and 2. The findings from Experiments 1-3 suggest that females interpret and use the investigator's eye gaze differently than males. When key aspects from the previous experiments were replicated in a noncommunicative situation (i.e., when a video of a speaker is used instead of a live speaker; Experiment 4), the memory effects observed previously in response to eye gaze were eliminated. Together, these studies suggest that it is the socially communicative aspects of eye gaze from a real person that influence memory. The findings reveal the importance of using social cues that are communicative in nature (e.g., real people) when studying the relationship between social attention and memory.

Eye Contact Is a Two-Way Street: Arousal Is Elicited by the Sending and Receiving of Eye Gaze Information.

Research shows that arousal is significantly enhanced while participants make eye contact with a live person compared to viewing a picture of direct or averted gaze. Recent research has pointed toward the potential for social interaction as a possible driving force behind the arousal enhancement. That is, eye gaze is not only a signal perceived but also a signal sent out in order to communicate with others. This study aimed to test this by having dyads engage in eye contact and averted gaze naturally, while wearing sunglasses, and while blindfolded; such that the gaze signals were clear, degraded, and blocked, respectively. Autonomic nervous system arousal was measured via skin conductance response and level. The results showed that dyads exhibited the highest degree of arousal (increased skin conductance) while making eye contact (send/receive) compared to send-only or receive-only gaze trials; however, this was only the case if eye contact was clear. Once gaze information became degraded (by sunglasses or blindfold), arousal significantly decreased and was no longer modulated by the sending and receiving of gaze. Therefore, the arousal enhancement observed during eye contact is not only caused by receiving gaze signals (the focus of previous research) and should be more accurately attributed to the subtle interplay between sending and receiving gaze signals.

Eye contact affects attention more than arousal as revealed by prospective time estimation.

Eye contact can both increase arousal and engage attention. Because these two changes impact time estimation differently, we were able to use a prospective time estimation task to assess the relative changes in arousal and attention during eye contact. Pairs of participants made a 1-minute prospective time estimate while sitting side-by-side and performing three different gaze trials: looking at the wall away from their partner (baseline/away trials), looking at their partner's profile (profile trials), or making eye contact with their partner (eye contact trials). We found that participants produced significantly longer estimates when they were engaged in eye contact, more so than when they looked at another person's profile or baseline. As research has shown that people produce shorter estimates during arousing events and longer estimates when attention is captured, we attribute this difference to the attention-demanding process of interacting with another person, via mutual eye contact, over and above any changes in arousal.

The duality of gaze: eyes extract and signal social information during sustained cooperative and competitive dyadic gaze.

In contrast to non-human primate eyes, which have a dark sclera surrounding a dark iris, human eyes have a white sclera that surrounds a dark iris. This high contrast morphology allows humans to determine quickly and easily where others are looking and infer what they are attending to. In recent years an enormous body of work has used photos and schematic images of faces to study these aspects of social attention, e.g., the selection of the eyes of others and the shift of attention to where those eyes are directed. However, evolutionary theory holds that humans did not develop a high contrast morphology simply to use the eyes of others as attentional cues; rather they sacrificed camouflage for communication, that is, to signal their thoughts and intentions to others. In the present study we demonstrate the importance of this by taking as our starting point the hypothesis that a cornerstone of non-verbal communication is the eye contact between individuals and the time that it is held. In a single simple study we show experimentally that the effect of eye contact can be quickly and profoundly altered merely by having participants, who had never met before, play a game in a cooperative or competitive manner. After the game participants were asked to make eye contact for a prolonged period of time (10 min). Those who had played the game cooperatively found this terribly difficult to do, repeatedly talking and breaking gaze. In contrast, those who had played the game competitively were able to stare quietly at each other for a sustained period. Collectively these data demonstrate that when looking at the eyes of a real person one both acquires and signals information to the other person. This duality of gaze is critical to non-verbal communication, with the nature of that communication shaped by the relationship between individuals, e.g., cooperative or competitive.

Do you see what I hear? Vantage point preference and visual dominance in a time-space synaesthete.

Time-space synaesthetes "see" time units organized in a spatial form. While the structure might be invariant for most synaesthetes, the perspective by which some view their calendar is somewhat flexible. One well-studied synaesthete L adopts different viewpoints for months seen vs. heard. Interestingly, L claims to prefer her auditory perspective, even though the month names are represented visually upside down. To verify this, we used a spatial-cueing task that included audiovisual month cues. These cues were either congruent with L's preferred "auditory" viewpoint (auditory-only and auditory + month inverted) or incongruent (upright visual-only and auditory + month upright). Our prediction was that L would show enhanced cueing effects (larger response time difference between valid and invalid targets) following the audiovisual congruent cues since both elicit the "preferred" auditory perspective. Also, when faced with conflicting cues, we predicted L would choose the preferred auditory perspective over the visual perspective. As we expected, L did show enhanced cueing effects following the audiovisual congruent cues that corresponded with her preferred auditory perspective, but that the visual perspective dominated when L was faced with both viewpoints simultaneously. The results are discussed with relation to the reification hypothesis of sequence space synaesthesia (Eagleman, 2009).

The frustrating effects of just missing the jackpot: slot machine near-misses trigger large skin conductance responses, but no post-reinforcement pauses.

Near-misses in slot machines resemble jackpot wins but fall just short. Previous research has demonstrated that near-misses are behaviorally reinforcing despite the absence of monetary reward. We assessed the hedonic properties of near-misses by measuring the time between outcome delivery and the initiation of the next spin-the post-reinforcement pause (PRP) and skin conductance responses (SCRs) for losses, near-misses, and a range of wins (5, 15, 25, 50 or 250 credits) while participants (N = 122) played a slot machine simulator. PRPs and SCRs were compared for 40 low frequency and 22 high frequency slots players who were non-problem gamblers, 37 at risk players, and 23 problem gamblers. For winning outcomes, PRPs and SCRs tracked monotonically with win size such that progressively larger wins were associated with progressively larger PRPs and SCRs. Near-misses with jackpot symbols landing on the first two reels had significantly larger SCRs than regular losses, and other types of near misses. Crucially, PRPs for this kind of near-miss were significantly smaller than all wins, and when non-parametric statistics were used, significantly smaller than regular losses. This pattern of large SCRs and small PRPs suggest that these are highly frustrating outcomes that stimulate appetitive components of the reward system to promote continued gambling.

The automaticity of vantage point shifts within a synaesthetes' spatial calendar.

Time-space synaesthetes report that time units (e.g., months, days, hours) occupy idiosyncratic spatial locations. For the synaesthete (L), the months of the year are projected out in external space in the shape of a 'scoreboard 7', where January to July extend across the top from left to right and August to December make up the vertical segment from top to bottom. Interestingly, L can change the mental vantage point (MVP) from where she views her month-space depending on whether she sees or hears the month name. We used a spatial cueing task to demonstrate that L's attention could be directed to locations within her time-space and change vantage points automatically - from trial to trial. We also sought to eliminate any influence of strategy on L's performance by shortening the interval between the cue and target onset to only 150 ms, and have the targets fall in synaesthetically cued locations on only 15% of trials. If L's performance was attributable to intentionally using the cue to predict target location, these manipulations should eliminate any cueing effects. In two separate experiments, we found that L still showed an attentional bias consistent with her synaesthesia. Thus, we attribute L's rapid and resilient cueing effects to the automaticity of her spatial forms.

9 is always on top: assessing the automaticity of synaesthetic number-forms.

For number-form synaesthetes, digits occupy idiosyncratic spatial locations. Atypical to the mental number line that extends horizontally, the synaesthete (L) experiences the numbers 1-10 vertically. We used a spatial cueing task to demonstrate that L's attention could be automatically directed to locations within her number-space - being faster to detect targets appearing in synaesthetically cued locations. We sought to eliminate any influence of strategy on L's performance by: (a) shortening the cue-target onset to 150 ms, (b) making the cues counterpredictive, and (c) instructing L to use an opposing strategy. If L's performance was attributable to intentionally using the cue to predict target location, these manipulations should eliminate any cuing effects consistent with her synaesthesia. However, L showed an attentional bias compatible with her number-form, except when explicitly instructed of the opposing strategy and given enough time (800 ms). Therefore, we attribute L's resilient cueing effects to the automaticity of her number-form.

A different outlook on time: visual and auditory month names elicit different mental vantage points for a time-space synaesthete.

Synaesthesia is a fascinating condition whereby individuals report extraordinary experiences when presented with ordinary stimuli. Here we examined an individual (L) who experiences time units (i.e., months of the year and hours of the day) as occupying specific spatial locations (January is 30 degrees to the left of midline). This form of time-space synaesthesia has been recently investigated by Smilek et al. (2007) who demonstrated that synaesthetic time-space associations are highly consistent, occur regardless of intention, and can direct spatial attention. We extended this work by showing that for the synaesthete L, her time-space vantage point changes depending on whether the time units are seen or heard. For example, when L sees the word JANUARY, she reports experiencing January on her left side, however when she hears the word "January" she experiences the month on her right side. L's subjective reports were validated using a spatial cueing paradigm. The names of months were centrally presented followed by targets on the left or right. L was faster at detecting targets in validly cued locations relative to invalidly cued locations both for visually presented cues (January orients attention to the left) and for aurally presented cues (January orients attention to the right). We replicated this difference in visual and aural cueing effects using hour of the day. Our findings support previous research showing that time-space synaesthesia can bias visual spatial attention, and further suggest that for this synaesthete, time-space associations differ depending on whether they are visually or aurally induced.

The ups and downs (and lefts and rights) of synaesthetic number forms: validation from spatial cueing and SNARC-type tasks.

Typically, numbers are spatially represented using a mental 'number line' running from left to right. Individuals with number-form synaesthesia experience numbers as occupying specific spatial coordinates that are much more complex than a typical number line. Two synaesthetes (L and B) describe experiencing the numbers 1 through 10 running vertically from bottom to top, 10-20 horizontally from left to right, 21-40 from right to left, etc. We investigated whether their number forms could bias their spatial attention using a cueing paradigm and a SNARC-type task. In both experiments, the synaesthetes' responses confirmed their synaesthetic number forms. When making odd-even judgments for the numbers 1, 2, 8, and 9, they showed SNARC-compatibility effects for up-down movements (aligned with their number form), but not left-right (misaligned) movements. We conceptually replicated these biases using a spatial cueing paradigm. Both synaesthetes showed significantly faster response times to detect targets on the bottom of the display if preceded by a low number (1, 2), and the top of the display if preceded by a high number (8, 9), whereas they showed no cueing effects when targets appeared on the left or right (misaligned with their number forms). They were however reliably faster to detect left targets following the presentation of numbers 10 and 11, and right targets following numbers 19 and 20 (since 10-20 runs from left to right). In sum, cueing and SNARC tasks can be used to empirically verify synaesthetic number forms, and show that numbers can direct spatial attention to these idiosyncratic locations.

Effects of seeing and hearing speech on speech production: a response time study.

Research demonstrates that listening to and viewing speech excites tongue and lip motor areas involved in speech production. This perceptual-motor relationship was investigated behaviourally by presenting video clips of a speaker producing vowel-consonant-vowel syllables in three conditions: visual-only, audio-only, and audiovisual. Participants identified target letters that were flashed over the mouth during the video, either manually or verbally as quickly as possible. Verbal responses were fastest when the target matched the speech stimuli in all modality conditions, yet optimal facilitation was observed when participants were presented with visual-only stimuli. Critically, no such facilitation occurred when participants were asked to identify the target manually. Our findings support previous research suggesting a close relationship between speech perception and production by demonstrating that viewing speech can 'prime' our motor system for subsequent speech production.

Observation of static gestures influences speech production.

Research investigating 'mirror neurons' has demonstrated the presence of an observation-execution matching system in humans. One hypothesized role for this system might be to aid in action understanding by encoding the underlying intentions of the actor. To investigate this hypothesis, we asked participants to observe photographs of an actor making orofacial gestures (implying verbal or non-verbal acts), and to produce syllables that were compatible or incompatible with the gesture they observed. We predicted that if mirror neurons encode the intentions of an actor, then the pictures implying verbal gestures would affect speech production, whereas the non-verbal gestures would not. Our results showed that the observation of compatible verbal gestures facilitated verbal responses, while incompatible verbal gestures caused interference. Although this compatibility effect did not reach statistical significance when the photographs implied a non-verbal act, responses were faster on average when the gesture implied the use of similar articulators as those involved with the production of the target syllable. Altogether, these behavioral findings compliment previous neuroimaging studies indicating that static pictures portraying gestures activate brain regions associated with an observation-execution matching system.

Multisensory integration of speech signals: the relationship between space and time.

Integrating audiovisual cues for simple events is affected when sources are separated in space and time. By contrast, audiovisual perception of speech appears resilient when either spatial or temporal disparities exist. We investigated whether speech perception is sensitive to the combination of spatial and temporal inconsistencies. Participants heard the bisyllable /aba/ while seeing a face produce the incongruent bisyllable /ava/. We tested the level of visual influence over auditory perception when the sound was asynchronous with respect to facial motion (from -360 to +360 ms) and emanated from five locations equidistant to the participant. Although an interaction was observed, it was not related to participants' perception of synchrony, nor did it indicate a linear relationship between the effect of spatial and temporal discrepancies. We conclude that either the complexity of the signal or the nature of the task reduces reliance on spatial and temporal contiguity for audiovisual speech perception.