The Timing of Gaze Direction Perception: ERP Decoding and Task Modulation.

Distinguishing the direction of another person's eye gaze is extremely important in everyday social interaction, as it provides critical information about people's attention and, therefore, intentions. The temporal dynamics of gaze processing have been investigated using event-related potentials (ERPs) recorded with electroencephalography (EEG). However, the moment at which our brain distinguishes the gaze direction (GD), irrespectively of other facial cues, remains unclear. To solve this question, the present study aimed to investigate the time course of gaze direction processing, using an ERP decoding approach, based on the combination of a support vector machine and error-correcting output codes. We recorded EEG in young healthy subjects, 32 of them performing GD detection and 34 conducting face orientation tasks. Both tasks presented 3D realistic faces with five different head and gaze orientations each: 30°, 15° to the left or right, and 0°. While the classical ERP analyses did not show clear GD effects, ERP decoding analyses revealed that discrimination of GD, irrespective of head orientation, started at 140 ms in the GD task and at 120 ms in the face orientation task. GD decoding accuracy was higher in the GD task than in the face orientation task and was the highest for the direct gaze in both tasks. These findings suggest that the decoding of brain patterns is modified by task relevance, which changes the latency and the accuracy of GD decoding.

Electrophysiological correlates of distance and direction processing during cognitive map retrieval: A source analysis.

Introduction: The cognitive map is an internal representation of the environment and allows us to navigate through familiar environments. It preserves the distances and directions between landmarks which help us orient ourselves in our surroundings. The aim of our task was to understand the role played by theta waves in the cognitive map and especially how the cognitive map is recalled and how the manipulation of distances and directions occurs within the cognitive map. Method: In order to investigate the neural correlates of the cognitive map, we used the Cognitive Map Recall Test, in which 33 participants had to estimate distances and directions between familiar landmarks tailored to their own knowledge. We examined the role of theta waves in the cognitive map, as well as the brain regions that generated them. To that aim, we performed electroencephalographic source imaging while focusing on frequency spectral analysis. Results: We observed increases of theta amplitude in the frontal, temporal, parahippocampal gyri and temporal poles during the recall of the cognitive map. We also found increases of theta amplitude in the temporal pole and retrosplenial cortex during manipulation of directions. Overall, direction processing induces higher theta amplitude than distance processing, especially in the temporal lobe, and higher theta amplitude during recall compared to manipulation, except in the retrosplenial cortex where this pattern was reversed. Discussion: We reveal the role of theta waves as a marker of directional processing in the retrosplenial cortex and the temporal poles during the manipulation of spatial information. Increases in theta waves in frontal, parahippocampal, temporal and temporal pole regions appear to be markers of working memory and cognitive map recall. Therefore, our Cognitive Map Recall Test could be useful for testing directional difficulties in patients. Our work also shows that there are two distinct parts to the cognitive map test: recall and manipulation of spatial information. This is often considered as two similar processes in the literature, but our work demonstrates that these processes could be different, with theta waves from different brain regions contributing to either recall or manipulation; this should be considered in future studies.

Association between attachment anxiety and the gaze direction-related N170.

Attachment theory suggests that interindividual differences in attachment security versus insecurity (anxiety and avoidance) contribute to the ways in which people perceive social emotional signals, particularly from the human face. Among different facial features, eye gaze conveys crucial information for social interaction, with a straight gaze triggering different cognitive and emotional processes as compared to an averted gaze. It remains unknown, however, how interindividual differences in attachment associate with early face encoding in the context of a straight versus averted gaze. Using electroencephalography (EEG) and recording event-related potentials (ERPs), specifically the N170 component, the present study (N = 50 healthy adults) measured how the characteristics of attachment anxiety and avoidance relate to the encoding of faces with respect to gaze direction and head orientation. Our findings reveal a significant relationship between gaze direction (irrespective of head orientation) and attachment anxiety on the interhemispheric (i.e. right) asymmetry of the N170 and thus provide evidence for an association between attachment anxiety and eye gaze processing during early visual face encoding.

Effect of perceived eye gaze on the N170 component - A systematic review.

Direction of another person's eye gaze provides crucial information about their attention and intentions, which is essential for an effective social interaction. Event-related potential (ERP) measures offer precise temporal tracking of neural processes related to gaze perception. While the sensitivity of the ERP component N170 to face processing is principally agreed, the research on gaze direction effect on this component is thus far inconsistent. Here, we systematically reviewed literature on the sensitivity of N170 to gaze direction. We analysed if four factors, known to affect the face N170 (i.e., emotion, face orientation, task demand, and stimuli motion), were modulated by gaze direction. N170 sensitivity to gaze was reported the most in the studies that involved deviated faces, dynamic stimuli, and that used explicit tasks directly related to gaze or face processing. The present review provides a much-needed summary of the literature to date, highlighting the complexity of the effect of gaze direction on the N170 component, and the need of systematic studies investigating the combination of these factors.

Do we need attentional suppression?

Gaspelin and Luck describe the signal suppression hypothesis, which proposes that attentional suppression prevents the capture of visual attention by salient distractors. We will discuss several problems with this proposal. On a theoretical level, we will argue that attentional suppression is a dispensable mechanism. Most effects of attentional suppression can be easily explained by reduced target expectancy at the distractor location. On an empirical level, we will argue that electrophysiological evidence for attentional suppression is spurious because, in key conditions, the PD most likely reflects idiosyncratic scan paths.

Meeting another's gaze shortens subjective time by capturing attention.

Gaze directed at the observer (direct gaze) is an important and highly salient social signal with multiple effects on cognitive processes and behavior. It is disputed whether the effect of direct gaze is caused by attentional capture or increased arousal. Time estimation may provide an answer because attentional capture predicts an underestimation of time whereas arousal predicts an overestimation. In a temporal bisection task, observers were required to classify the duration of a stimulus as short or long. Stimulus duration was selected randomly between 988 and 1479 ms. When gaze was directed at the observer, participants underestimated stimulus duration, suggesting that effects of direct gaze are caused by attentional capture, not increased arousal. Critically, this effect was limited to dynamic stimuli where gaze appeared to move toward the participant. The underestimation was present with stimuli showing a full face, but also with stimuli showing only the eye region, inverted faces and high-contrast eye-like stimuli. However, it was absent with static pictures of full faces and dynamic nonfigurative stimuli. Because the effect of direct gaze depended on motion, which is common in naturalistic scenes, more consideration needs to be given to the ecological validity of stimuli in the study of social attention.

When Left is One and Right is Double: An Experimental Investigation of Visual Allesthesia after Right Parietal Damage.

Illusory visual phenomena, such as palinopsia, polyopsia or allesthesia, are rare manifestations of posterior cortical damage. Symptoms are characterized by illusory perceptions, ranging from isolated stationary objects to scenes and moving persons. Such illusions may appear while the original object is still in view, or become manifest with a delay and last for minutes, hours or even days. Some authors have suggested a disinhibited cortical response underlying visual illusions, but experimental studies supporting this hypothesis are lacking. Here, we examined a rare patient who after focal right parietal injury consistently reported a second stimulus on the left when briefly shown a target in his right hemifield. The patient perceived the illusory stimulus as less intense, and therefore concluded that it must have a different shape than the original stimulus. A masking experiment revealed that the frequency of the illusion was inversely related to the visibility of the original stimulus, suggesting that it depended on early, feedforward visual processing. We propose that illusory perceptions reflect the interplay of two physiological processes: a fast and automatic activation of contralateral, homotopic visual cortex after unilateral stimulation, and the lack of top-down inhibition following damage to the posterior parietal cortex.

Capture by Context Elements, Not Attentional Suppression of Distractors, Explains the PD with Small Search Displays.

Top-down control of attention allows us to resist attentional capture by salient stimuli that are irrelevant to our current goals. Recently, it was proposed that attentional suppression of salient distractors contributes to top-down control by biasing attention away from the distractor. With small search displays, attentional suppression of salient distractors may even result in reduced RTs on distractor-present trials. In support of attentional suppression, electrophysiological measures revealed a positivity between 200 and 300 msec contralateral to the distractor, which has been referred to as distractor positivity (PD). We reexamined distractor benefits with small search displays and found that the positivity to the distractor was followed by a negativity to the distractor. The negativity, referred to as N2pc, is considered an index of attentional selection of the contralateral element. Thus, attentional suppression of the distractor (PD) preceded attentional capture (N2pc) by the distractor, which is at odds with the idea that attentional suppression avoids attentional capture by the distractor. Instead, we suggest that the initial "PD" is not a positivity to the distractor but rather a negativity (N2pc) to the contralateral context element, suggesting that, initially, the context captured attention. Subsequently, the distractor was selected because, paradoxically, participants searched all lateral target positions (even when irrelevant) before they examined the vertical positions. Consistent with this idea, search times were shorter for lateral than vertical targets. In summary, the early voltage difference in small search displays is unrelated to distractor suppression but may reflect capture by the context.

Task Demands Modulate Effects of Threatening Faces on Early Perceptual Encoding.

The threat capture hypothesis states that threatening stimuli are automatically processed with higher priority than non-threatening stimuli, irrespective of observer intentions or focus of attention. We evaluated the threat capture hypothesis with respect to the early perceptual stages of face processing. We focused on an electrophysiological marker of face processing (the lateralized N170) in response to neutral, happy, and angry facial expressions displayed in competition with a non-face stimulus (a house). We evaluated how effects of facial expression on the lateralized N170 were modulated by task demands. In the pixel task, participants were required to identify the gender of the face, which made the face task-relevant and entailed structural encoding of the face stimulus. In the pixel task, participants identified the location of a missing pixel in the fixation cross, which made the face task-irrelevant and placed it outside the focus of attention. When faces were relevant, the lateralized N170 to angry faces was enhanced compared to happy and neutral faces. When faces were irrelevant, facial expression had no effect. These results reveal the critical role of task demands on the preference for threatening faces, indicating that top-down, voluntary processing modulates the prioritization of threat.

The influence of top-down modulation on the processing of direct gaze.

Gaze or eye contact is one of the most important nonverbal social cues, which is fundamental to human social interactions. To achieve real time and dynamic face-to-face communication, our brain needs to process another person's gaze direction rapidly and without explicit instruction. In order to explain the fast and spontaneous processing of direct gaze, the fast-track modulator model was proposed. Here, we review recent developments in gaze processing research in the last decade to extend the fast-track modulator model. In particular, we propose that task demand or top-down modulation could play a more crucial role at gaze processing than formerly assumed. We suggest that under different task demands, top-down modulation can facilitate or interfere with the direct gaze effects for early visual processing. The proposed modification of the model extends the role of task demand and its implication on the direct gaze effect, as well as the need to better control for top-down processing in order to better disentangle the role of top-down and bottom-up processing on the direct gaze effect. This article is categorized under: Cognitive Biology > Evolutionary Roots of Cognition Psychology > Perception and Psychophysics Neuroscience > Cognition.

Differential impact of trait, social, and attachment anxiety on the stare-in-the-crowd effect.

Eye gaze conveys crucial information for social interactions, with straight versus averted gaze triggering distinct emotional and cognitive processes. The "stare-in-the-crowd" effect exemplifies such differential visual processing of gaze direction, in more recent reports also in interaction with head orientation. Besides aiming at replicating the "stare-in-the-crowd" effect by means of an eye gaze by head orientation interaction, the present study intended to for the first time testing its susceptibility to inter-individual differences in trait, social, and attachment anxiety. Our findings reveal a significant relation between the "stare-in-the-crowd" effect and social and attachment, but not trait anxiety, and therefore provide preliminary cues for personality influences on visual processing of eye gaze and head orientation.

Affective blindsight relies on low spatial frequencies.

The human brain can process facial expressions of emotions rapidly and without awareness. Several studies in patients with damage to their primary visual cortices have shown that they may be able to guess the emotional expression on a face despite their cortical blindness. This non-conscious processing, called affective blindsight, may arise through an intact subcortical visual route that leads from the superior colliculus to the pulvinar, and thence to the amygdala. This pathway is thought to process the crude visual information conveyed by the low spatial frequencies of the stimuli. In order to investigate whether this is the case, we studied a patient (TN) with bilateral cortical blindness and affective blindsight. An fMRI paradigm was performed in which fearful and neutral expressions were presented using faces that were either unfiltered, or filtered to remove high or low spatial frequencies. Unfiltered fearful faces produced right amygdala activation although the patient was unaware of the presence of the stimuli. More importantly, the low spatial frequency components of fearful faces continued to produce right amygdala activity while the high spatial frequency components did not. Our findings thus confirm that the visual information present in the low spatial frequencies is sufficient to produce affective blindsight, further suggesting that its existence could rely on the subcortical colliculo-pulvino-amygdalar pathway.

Early spatial attention deployment toward and away from aggressive voices.

Salient vocalizations, especially aggressive voices, are believed to attract attention due to an automatic threat detection system. However, studies assessing the temporal dynamics of auditory spatial attention to aggressive voices are missing. Using event-related potential markers of auditory spatial attention (N2ac and LPCpc), we show that attentional processing of threatening vocal signals is enhanced at two different stages of auditory processing. As early as 200 ms post-stimulus onset, attentional orienting/engagement is enhanced for threatening as compared to happy vocal signals. Subsequently, as early as 400 ms post-stimulus onset, the reorienting of auditory attention to the center of the screen (or disengagement from the target) is enhanced. This latter effect is consistent with the need to optimize perception by balancing the intake of stimulation from left and right auditory space. Our results extend the scope of theories from the visual to the auditory modality by showing that threatening stimuli also bias early spatial attention in the auditory modality. Attentional enhancement was only present in female and not in male participants.

Arithmetic word problems describing discrete quantities: E.E.G evidence for the construction of a situation model.

In this research, university students were asked to solve arithmetic word problems constructed either with discrete quantities, such as apples or marbles, or continuous quantities such as meters of rope or grams of sand. An analysis of their brain activity showed different alpha levels between the two types of problems with, in particular, a lower alpha power in the parieto-occipital area for problems describing discrete quantities. This suggests that processing discrete quantities during problem solving prompts more mental imagery than processing continuous quantities. These results are difficult to reconcile with the schema theory, according to which arithmetic problem solving depends on the activation of ready-made mental frames stored in long-term memory and triggered by the mathematical expression used in the texts. Within the schema framework, the nature of the objects described in the text should be quickly abstracted during problem solving because it cannot impact the semantic structure of the problem. On the contrary, our results support the situation model theory, which places greater emphasis on the problem context in order to account for individuals' behaviour. On a more methodological point of view, this study constitutes the first attempt to infer the characteristics of individual's mental representations of arithmetic text problems from EEG recordings. This opens the door for the application of brain activity measures in the field of arithmetic word problem.

Avoiding sedentary behaviors requires more cortical resources than avoiding physical activity: An EEG study.

Why do individuals fail to exercise regularly despite knowledge of the risks associated with physical inactivity? Automatic processes regulating exercise behaviors may partly explain this paradox. Yet, these processes have only been investigated with behavioral outcomes (i.e., based on reaction times). Here, using electroencephalography, we investigated the cortical activity underlying automatic approach and avoidance tendencies toward stimuli depicting physical activity and sedentary behaviors in 29 young adults who were physically active or physically inactive but with the intention of becoming physically active. Behavioral results showed faster reactions when approaching physical activity compared to sedentary behaviors and when avoiding sedentary behaviors compared to physical activity. These faster reactions were more pronounced in physically active individuals and were associated with changes during sensory integration (earlier onset latency and larger positive deflection of the stimulus-locked lateralized readiness potentials) but not during motor preparation (no effect on the response-locked lateralized readiness potentials). Faster reactions when avoiding sedentary behaviors compared to physical activity were also associated with higher conflict monitoring (larger early and late N1 event-related potentials) and higher inhibition (larger N2 event-related potentials), irrespective of the usual level of physical activity. These results suggest that additional cortical resources were required to counteract an attraction to sedentary behaviors. Data and Materials [https://doi.org/10.5281/zenodo.1169140]. Preprint [https://doi.org/10.1101/277988].

Early and late cortical responses to directly gazing faces are task dependent.

Gender categorisation of human faces is facilitated when gaze is directed toward the observer (i.e., a direct gaze), compared with situations where gaze is averted or the eyes are closed (Macrae, Hood, Milne, Rowe, & Mason, Psychological Science, 13(5), 460-464, 2002). However, the temporal dynamics underlying this phenomenon remain to some extent unknown. Here, we used electroencephalography (EEG) to assess the neural correlates of this effect, focusing on the event-related potential (ERP) components known to be sensitive to gaze perception (i.e., P1, N170, and P3b). We first replicated the seminal findings of Macrae et al. (2002, Experiment 1) regarding facilitated gender discrimination, and subsequently measured the underlying neural responses. Our data revealed an early preferential processing of direct gaze as compared with averted gaze and closed eyes at the P1, which reverberated at the P3b (Experiment 2). Critically, using the same material, we failed to reproduce these effects when gender categorisation was not required (Experiment 3). Taken together, our data confirm that direct gaze enhances the early P1, as well as later cortical responses to face processing, although the effect appears to be task dependent.

ERP responses greater for faces in the temporal compared to the nasal visual field.

The distribution of retino-tectal projections is dissimilar depending on whether the receptors are situated in the nasal and temporal visual hemiretinas. Indeed, it has been claimed that the superior colliculus receives a greater proportion of its input from the temporal visual hemifield (nasal hemi-retina) relative to the nasal hemifield (temporal hemi-retina). In order to investigate whether these subcortical projections influence face processing, we investigated the early cortical ERP responses to faces and houses presented in the temporal and nasal retinas using monocular viewing. Neutral or fearful faces were presented concurrently with houses on either side of a central fixation cross, while participants were asked to discriminate changes in luminance at the center. Results showed that the lateralized N170, computed as the contralateral-ipsilateral electrode difference, was greater for faces appearing in the nasal relative to the temporal visual hemifield. This was due to a greater ipsilateral N170 for temporal relative to nasal presentations. By contrast, no difference was found across emotional expressions. The enhanced ERP response to faces appearing in the temporal visual field, suggests that the retinotectal pathway modulates cortical processing, most likely through activation of a colliculo-pulvino-amygdalar pathway, with subsequent back-projections from the amygdala to visual cortical regions. However, unattended facial expressions do not seem to modulate the response, at least at these angles of eccentricity.

Processing of gaze direction within the N170/M170 time window: A combined EEG/MEG study.

Gaze direction is an important social signal for human beings. Beside the role of gaze in attention orienting, direct gaze (that is, gaze directed toward an observer) is a highly relevant biological stimulus that elicits attention capture and increases face encoding. Brain imaging studies have emphasized the role of the superior temporal sulcus (STS) in the coding of gaze direction and in the integration of gaze and head cues of social attention. The dynamics of the processing and integration of these cues remains, however, unclear. In order to address this question, we used deviated and frontal faces with averted and direct gaze in a combined electro- and magneto- encephalography (EEG-MEG) study. We showed distinct effects of gaze direction on the N170 and M170 responses. There was an interaction between gaze direction and head orientation between 134 and 162ms in MEG and a main effect of gaze direction between 171 and 186ms in EEG. These effects involved the posterior and anterior regions of the STS respectively. Both effects also emphasized the sensitivity to direct gaze. These data highlight the central role of the STS in gaze processing.

Gaze-cueing requires intact face processing - Insights from acquired prosopagnosia.

Gaze-cueing is the automatic spatial orienting of attention in the direction of perceived gaze. Participants respond faster to targets located at positions congruent with the direction of gaze, compared to incongruent ones (gaze cueing effect, GCE). However, it still remains unclear whether its occurrence depends on intact integration of information from the entire eye region or face, rather than simply the presence of the eyes per se. To address this question, we investigated the GCE in PS, an extensively studied case of pure acquired prosopagnosia. In our gaze-cueing paradigm, we manipulated the duration at which cues were presented (70ms vs. 400ms) and the availability of facial information (full-face vs. eyes-only). For 70ms cue duration, we found a context-dependent dissociation between PS and controls: PS showed a GCE for eyes-only stimuli, whereas controls showed a GCE only for full-face stimuli. For 400ms cue duration, PS showed gaze-cueing independently of stimulus context, whereas in healthy controls a GCE again emerged only for full-face stimuli. Our findings suggest that attentional deployment based on the gaze direction of briefly presented faces requires intact processing of facial information, which affords salience to the eye region.

Electrophysiological evidence for attentional capture by irrelevant angry facial expressions: Naturalistic faces.

Recently, research on lateralized event related potentials (ERPs) in response to irrelevant distractors has revealed that angry but not happy schematic distractors capture spatial attention. Whether this effect occurs in the context of the natural expression of emotions is unknown. To fill this gap, observers were asked to judge the gender of a natural face surrounded by a color singleton among five other face identities. In contrast to previous studies, the similarity between the task-relevant feature (color) and the distractor features was low. On some trials, the target was displayed concurrently with an irrelevant angry or happy face. The lateralized ERPs to these distractors were measured as a marker of spatial attention. Our results revealed that angry face distractors, but not happy face distractors, triggered a PD, which is a marker of distractor suppression. Subsequent to the PD, angry distractors elicited a larger N450 component, which is associated with conflict detection. We conclude that threatening expressions have a high attentional priority because of their emotional value, resulting in early suppression and late conflict detection.